Exploratory testing with uTest
Introduction: Exploratory Testing is the most effective way of testing being performed by skilled testers. The main advantage of it is that less preparation is needed (i.e., compared with automated testing), important bugs are found quickly, and at execution time the approach tends to be more intellectually stimulating than execution of scripted tests.
Don’t forget that final tester’s goal is to make customer happy by providing testing of the highest quality. And it’s crucial to find and report the most important, highest value bugs as perceived by the customer (put on the hat of a product manager). Also, usually uTest projects are limited by number of bugs or by budget, so it’s necessary for testers to find and report the most important bugs first and then doing the same for subsequent bugs in descending priority/importance to customer.
The most important bugs are those that block end users to execute standard workflow of a given software. Also, sometimes security testing is allowed by customers, thus it may be helpful to look for security issues as well during your exploratory testing phase.
In order to achieve the goals listed above, it is important to execute exploratory testing according to priority:
- Divide application to different end users flows. E.g. if application is a website like a Q&A site, then end user flows would be: find an answer as guest user, register new user, sign in, find an answer as registered user, find a question as registered user, post a question as registered user, reply a question as registered user, share question as registered user, etc.
- Begin by executing sanity testing for the most important workflows first: E.g., registering, sign in, finding question/answer, posting question/answer, replying to questions. Find and report the highest value bugs first. Remember, high value for a customer may not be correlated to bug severity. For example, if a bug causes the application to crash, but it can only be triggered by pressing a button 100 times in one minute, this would be a low value bug (even if it’s high severity). Therefore, make sure the application works as expected at the sanity level.
- Next, execute security testing and report the highest value bugs (again, wearing the hat of a product manager).
- Then, execute sanity testing for other workflows in the application and report high value bugs.
- Lastly, execute extensive positive, negative and security testing for most important flow first and after that cover rest flows.
If you find medium or low value bugs during executing stages from 2nd to 4th, do not report them all at once, but save them in Word document and report during phase #5. Note: uTest projects may become locked due to budget limitation at any stage of your testing. Therefore, we recommend that you report the highest value bugs first.
Lastly, avoid reporting duplicate bugs. Note, that the same error may have different code in different browsers, like in IE8 and Chrome. These are likely the same issue and only one bug should be reported in these cases. Additionally, don’t forget that you are not alone in project testing. Check the bugs already reported by fellow testers. Here are additional tips:
- Cover areas and flows that have not been covered yet. This way you will be rewarded with new bugs and application will be covered more uniformly, which is also important for customer.
- Test using browsers that are not used in testing yet.
If some part of the application is blocked for testing due to a showstopper bug, use Discussion Thread to notify Project Manager and customer about this issue. Thus most likely customer will have chance to fix this issue as soon as possible and in result you will be able to test this area lately, before the testing ends.
