HGP-14 [IB] Release Candidate (staging branch) bug hunt

Author: PHBR0 x58B

Original Post Date: 1 Apr 2022

Snapshot Proposal Link

Category

Initiative Budget

Summary

To improve the quality of each Hummingbot Client Release, we propose to allocate 1,000,000 HBOT tokens to reward the comunity for bug hunting and bug fix, during the staging phase of each monthly Release.

Details

As detailed on the Hummingbot Release cycle, Release Candidates will be moved to the staging branch once all approved PRPs for that release are merged into development.

During this phase, the Release Candidate still needs to be tested for bugs not found during the initial merge of Pull Requests into the development branch.

As a way to improve the quality of each release, this proposal establish the Bug Hunt program, and allocate 1,000,000 HBOT tokens to be distributed as rewards to community members that find and/or fix bugs on the staging branch.

Severity Classification

Each bug will be classified according to the following structure:

NOTE: The list above are examples for reference. Once the bug is reported, Hummigbot Foundation team will evaluate the severity of the bug through internal testing.

Rewards allocation Structure

Community members can request a reward as a Bug Finding and/or a Bug Fix.

The rewards will be allocated of the following structure

Participation Rules

After all approved PRPs are merged and the staging branch is updated as the next Release Candidate, Hummingbot Foundation team will create a Pinned Thread on Commonwealth, detailing what Pull Requests are going into the next release.

Example: Release v1.1 Thread

Once the Release Candidate Thread is created, the Bug Hunt for that release will be active, until the new relase deployment on the main branch is done. Then, the thread for that release candidate will be closed.

To report a bug and/or a bug fix on the release candidate staging branch, the community members must reply to the “Client release candidate staging branch review active” thread using the formats below.

Bug Finding Report

Type: Bug Found

Severity:

Bug Description:

How to reproduce: <Describe the steps to reproduce the bug, with as much details as possible?

Bug Fix Report

Type: Bug Fix <Add Bug Found to the type, if the bug wasn’t reported before>

Severity: . NOTE: Hummingbot Foundation reserved the right to modify the Severity.

Bug Description:

Fix PR: <add a link to the Pull Request targeting the staging branch>

How to reproduce: <Describe the steps to reproduce the bug, with as much details as possible?

Reward payment

Rewards approved by the Hummingbot Foundation team will be sent to the address used to report on the respective staging branch review thread on Commonwealth, after every new release is deployed into the master branch.

Important Notes

  • Bug fixes aimed to the staging branch do not need to go through a PRP voting
  • Bug fixes reports will only be rewarded if the Foundantion QA approves and merge the respective PR
  • Bug findings reports will only be rewarded if Foundation QA is able to reproduce the bug and confirm its existance
  • If a bug report/fix is related to a possible leak of users sensitive information we ask the community to reach out the Hummingbot Foundation team directly before posting a report.

Budget Allocation

Allocate 1,000,000 HBOT to reward bug findings and bug fixes for the release candidate

Author: leastchaos 0x46f

  1. Are the rewards for new bugs found during staging release or also include old bugs that had persisted in the previous releases?
  2. How are the rewards determined for bug finding and bug fixing?
  3. For bug fixes, do we have to also do it like hip where we plan and propose a reward to be received or do we just submit and let the foundation determine the value?

Author: PHBR0 x58B

  1. It can be bugs that havent been fixed on the past. The goal is to use the staging process to reduce the amount of existing bugs before a new release is deployed
  2. Rewards/bug severity will be evaluated by Foundation QA team
  3. No. Once this proposal is approved, the fixes need to be submitted to the staging branch, and the information about it must be added to the monthly “release candidate” thread here on Commonwealth.

Author: X Chop 0x873

“It can be bugs that haven’t been fixed on the past. The goal is to use the staging process to reduce the amount of existing bugs before a new release is deployed” - this contradicts calling it a staging process. One of the goals of testing on staging is to reduce the amount of NEW bugs introduced in development and prevent them from going into production.

I voted against this proposal because it’s missing an important step (also practiced in software QA process) for participants to ensure that the bug found was indeed only present in staging and not in the current stable version. Making sure that a bug is only present in staging means that it should be prioritized more than the other bugs in production to be fixed before the upcoming release.

Otherwise, this should be called just a “bug hunt” in general if we also aim to look for bugs including production (bugs that haven’t been fixed in the past).

Author: PHBR0 x58B

@X Chop That’s a fair point.

The reason we contained this proposal to the staging branch is because it is a less mutable branch and it would be easier for Foundation QA team to process the reports that will be submitted.

Since this is going to be the first iteration of this type reward distribution, we wanted to work on a smaller scale to tune up the process.

But eventually we want to have a more fleshed out bug hunt process, that would cover all branches (main, staging, & development).

This proposal will be valid until the end of Epoch 1(ends June, 30 but we want to propose a wider bug hunt reward program for Epoch 2.

Let’s get in touch on our Discord. Want to hear your suggestions on how we could improve the bug find/fix reward program.

Author: fengtality 0x352

I voted YES for this proposal since I think incentivizing the community to find and fix bugs will help to improve the long-term quality of the code base