Recommendations from Giovanni Pagani
Imagine yourself moving in a new application’s dungeon of features, performing your test case when everyone else in your country is sleeping, just to be the first one to find that precious gem, err, bug. You feel that this click will be the right one… Yes! I got it! Let’s rush to report and get some cash, and go on to that “Gold tester” status, where I’ll be able to unlock some fantastic testing opportunities.
Bug warrior’s sleep is sweet after some finding, right? But the morning after you see that instead of “Approved” you get a “Rejected”.
Stop. Don’t do what you’re thinking to do. Don’t “dispute” just now. First of all, you have only one dispute opportunity per bug, and a limited number of disputes per month. You can’t really play to “shoot ‘em all”. And your reputation as a tester could be affected. Surely you are going to waste time and definitely loose an opportunity.
At the same time a few discarded submission are quite normal when working in a testing environment. It’s an opportunity to improve your testing level; it’s not a honor offense to be avoided at all costs.
Let’s follow instead a simple check to see if raising your hand and asking for a debate (forget considering it a contest or an altercation, we are offering our services, even if responding to an invitation) is the best thing you can do, and if you are ready to.
Are you a newbie uTester?
Even if you’ve some silver in your hairs (or lost them all on the way) we’re all newbies sometime: it’s either lack of generic testing experience, or specific experience on that application (or website), or on uTest’s platform and community’s way to perform activities.
If you feel newbie try to slow down your bug-submitting rush, set your pace at a slower speed and watch around, study what other testers are reporting, which submissions are approved and which rejected. Take a look at test cycle’s discussion thread. Don’t be shy: The time and place for asking clarifications is here and before submission.
Being new to a test cycle isn’t something to leave behind, it’s actually what our customers are looking for, since it raises your attention level. Lack of attention towards details when searching and reporting is our worse enemy, and it means you’re ready to start developing your own bugs.
Is the customer new?
Even if you’re the best uTester in our community you can encounter a newbie customer. Sales and marketing did their best job to reach a new customer. Thanks to our platform we can dedicate to testing only, not to self-marketing and invoicing. Maybe the customer is not familiar with uTest platform, maybe that they’re looking only for specific bug types but didn’t specify them into the out of scope section.
Maybe that a new customer forgot to share his internal list of known issues, so is discarding all your findings. Instead of raising a controversy against someone that has just chosen our community to simplify and improve his testing processes, please contact the project manager. This is the way to go for a win-win-win solution. Maybe your bug won’t be approved, but surely your project manager will remember your approach in this kind of troubleshooting.
Is the bug a new one?
Check if that issue is really a new one: this step should be the easier to recognize after the rejection has happened since the customer will answer “known issue.” This may happen, it’s not always easy to check within a long and tedious list of known issues and findings by other testers, especially if there’s a list of several hundreds of uncategorized issues, with no proper keywords to search for.
If this is the case, consider that known issues lists are often made by copying and pasting our “bug reports” text from previous test cycles, but without attachments. Remember this when reporting your bugs, and help other testers avoid rejections or boring searches among meaningless descriptions. Same way, follow test cycle’s specific directions on bug title elements.
If you have checked your submission, and feel that you’ve been misunderstood, be sure to:
- Collect more data. It’s a time consuming activity, it’s easier to simply type a few words about our finding than to search for a video capture utility, learn to use it, capture and upload. But you need to do it, to show that your findings can be helpful: if video attachment is allowed it’s often the preferred and easier way to show your findings.
- Collect significant data. Be specific when detailing your findings, even if you got a “generic error.” Add environment details, and if possible, cross check on alternate platforms or production version.
- Check again if test cycle instructions have been changed. You get an email when objectives are updated, so maybe you missed an important one. Check if your bug was within old scope, if it has been changed. Better to take a copy of a test case description for your reference
- Don’t get angry or frustrated if the customer asks you to verify your information or asks you to repeat your test EVEN IF you’d already entered your results into the system. Instead, verify them once again. In the meantime your knowledge of the testing project could have been improved, so you are able to deliver a better report.
- Consider that your level of testing and the customer’s level of acceptance for the release can be at a different sensitivity. For example, an Alpha or early beta application version usually won’t be hurt by a misspelled word lying on the “terms and conditions” page.
- Put your efforts on describing your rejected bug as if it is a different finding. You need to add details on a different approach, not copy and paste what has already been rejected. Think to your bugs reporting standards it as to the step-by-step user manual, not a brief note for someone that already knows everything about the application being tested
Based on what’s the rejection listed reason and on which of the above checks shows your submission’s weak points you can finally place your request. Each cycle and each submission is different, but please apply this rule to all your disputes: never beg for some sort of gratuitous mercy regarding your findings. It’s a clear indicator of how poor are your arguments. Instead be polite and ask for a bit more of customer’s attention since you have a better explanation for your findings. By showing that you want to better learn what a customer is looking for you can end up as being chosen as a favorite tester even with a discarded bug , rather than having an approved feedback and being discarded as a tester.
Recommendation from Pramod Lumb
The primary responsibility of a tester is to help improve quality of the product being tested. This is best achieved by detecting high-value bugs in the product, before the customer releases it to the public. However, does the responsibility of a tester end with documentation of the defects found? Sometimes yes, but often times, no.
A tester should not only ensure that the defects detected in a product are logged and brought to the notice of development team as soon as possible, but also that they are properly disposed off. In order to ensure this, testers may sometimes be required to sit down with the development team in order to help them understand and reproduce documented defects. Testers may also, sometimes, be required to help developers understand the impact of not fixing or deferring the fix of a defect.
Valid reasons to dispute a bug:
This is where the concept of disputing a bug comes into place. Development teams may often dispute the severity of a bug as pointed out by testers. In fact, they may even dispute (in certain cases) if the incident being reported is a defect in the first place. There may be other cases as well where development team may dispute the observations of testers.
Let us look at some of the areas wherein a defect reported by the test team may be disputed:
- A defect may be considered to be a product design feature by the development team. Hence, they may believe that it does not need to be fixed at all.
- Development teams (or product management teams) may not agree with the stated severity, or implied priority, of a defect logged by the test team. Hence, the defect may get scant and delayed attention, or may never get a chance to be taken up for resolution at all.
- Development teams may not be able to reproduce a bug in their environment, or else, a bug marked as resolved by them may seem fixed in their environment but may not actually turn out to be resolved when verified in test environment.
Besides, there may be disagreements on the way a defect should be documented. Sometimes, a debate may even take place on independent parts of a bug which have been documented as a single defect. At other times, different development teams may not agree on the correct allocation of defects assigned to them for resolution, i.e., there may not be an agreement on which part of an application is defective.
Tips to avoid bugs being disputed:
In order to ensure effective and speedy disposal of bugs, it is essential that certain good practices be followed. First and foremost, the defects should be documented in an unambiguous manner, with the test environment as well as steps to reproduce the bugs being documented clearly. Secondly, supporting screenshots/videos should be provided wherever possible. Finally, the perceived impact of the bug on AUT (Application under Test) as well as business of the customer should be unambiguously documented.
However, in spite of providing supporting documentation and a clean sequence of steps for reproducing the defect, sometimes confusions do creep in. If they do, it is essential that the bugs be disputed in a peaceful and amicable manner.
Tips for effectively disputing logged bugs:
Here are a few tips to help you dispute bugs effectively without leaving a sour taste in the mouths of either development or test team members:
- If the reported bug is considered to be a design feature and not a defect, always provide supporting documentation, e.g., requirements or design documents, to support your viewpoint. In case the supporting documentation is itself ambiguous, seek clarification from the customer before disputing a bug.
- Always understand why a particular severity or priority has been assigned to a defect. There may be cascading impact(s) of the bug unknown to you because of which it has been assigned a higher priority/severity compared to the values you deem fit.
- Do not ever point fingers at a particular individual for logging an invalid defect, or for marking a defect as resolved even though it has not been fixed as such. Never use offensive language in your bug reports, or in related communication.
- If a particular individual seems to be making repeated mistakes in terms of logging invalid defects or marking defects as resolved incorrectly, talk to that individual’s manager. Do not let the ‘Bozo bit’ be flipped on for yourself.
- If you are not able to reproduce a defect in your environment, always ask the author of the defect for help. There may have been certain changes between the time when the defect was documented, and now, because of which you are not able to reproduce the defect now. Application build may have got updated, or else, the defect may have got masked because of some other defect which got fixed.
If you keep these tips in mind, you will always find that you are respected across development and test teams for your inputs, and that your suggestions and inputs are looked forward to, rather than being frowned upon.
Recommendation from Yolanda Solo
As a tester you have the option to dispute a limited number of bugs each month, but this is a feature that should be used wisely and sparingly. Testers can only dispute a limited number of bugs per month and if a dispute is rejected, the decision is final. We have outlined some tips that will help you determine if you should file a dispute and how to do so appropriately.
Many rejections could be avoided by spending more time submitting a bug properly at the first stage. Read the list below and make sure you followed the correct procedure that ensures bugs are valid before you submit the dispute.
- Check the “Scope & Instructions” to make sure that the bug is within the scope of the test cycles.
- Check a Known Bugs List (if applicable) and previously submitted bugs list for duplicates. The most common reason for bugs getting rejected is that they are a duplicate. The similarities may not be obvious from the title and will be found in the description or test case result.
- Check the discussion thread to see if the issue you found has been discussed and clarified. You might find that the project manager has already told testers in the discussion thread that some issues are not within scope.
- Check your rejected bug thoroughly – did you provide a clear, concise description of the bug that will allow the client to reproduce it and illustrate it with evidence including screenshots and/or video captures? If not, you may now understand why your report was rejected. Thus, make sure you add all the necessary information and supporting evidence when you submit your dispute.
Reasons for disputing a bug:
Having checked that you bug submission meets the necessary criteria you may want to dispute a bug rejection for the following reasons:
- You did not provide enough information the first time
If the information you gave in the initial test case result was not clear enough, you can clarify any information in the dispute if you feel it will help the customer understand the issue better.
- You have additional evidence
If the customer was unable to reproduce the bug you can provide screen shots and screen captures showing the bug.
- Your bug was rejected for duplication and want to explain why yours is different
A bug may be similar to others and therefore get rejected by mistake. Explain why your bug is unique to ensure that the dispute is handled correctly.
- All disputes, tester messenger discussions, and discussion thread discussions are monitored by uTest and the customer. Therefore, be professional in your communication at all times.
- Any disputes must be submitted using the dispute feature. Never ask customers not to reject your bug as ‘works as designed’ as this may result in the suspension of your account.
How to dispute a bug rejection
- Click on your rejected bug. Next, click on the Dispute Rejection button as shown on this FAQ.
- Explain why you are disputing the rejected bug
You have a limited amount of room to write your explanation so make sure it is clear and concise. Explain why you are disputing the rejected bug and address the reason it was rejected e.g. if it was rejected as a duplicate bug, explain the differences between your bug and similar bugs.
- Add supporting material
If you feel the customer did not fully understand the bug it is essential you provide supporting evidence. Screenshots are good, video captures are better so that customers can see step-by-step what you are trying to explain. You can upload the screenshots and video captures directly to the uTest Platform.