Test Case Completion (Execution) 101

Author: Atul Angra, Ese Oduyoye, Teresa Blevins
Reviewer: uTest

Introduction: The software-testing phase in the software development life cycle is one of the most crucial components of the quality management process.  It requires that testing begin as early as possible in the development process and that it continue all the way through implementation.  The earlier defects are found in development the lower the risk and cost a company has to absorb in the deployment of their software.  As a tester, you will be required to review and evaluate different software and materials as each customer, software, and test scripts will vary from client to client.  Every client runs software projects differently and you will encounter various different approaches and methodologies across the entire spectrum; from prepared customers with full test management processes in place to others that believe defects will be fixed when there is more time and money.   Regardless of where the customer is along this continuum, uTest offers a scalable strategy to fill a void that exists in many organizations by providing services to assist in the testing life cycle where resources and budgets can be very scant and scarce, particularly if the project is already over budget.  Proper execution of test cases is essential in ensuring the success of any testing project, therefore, focusing on best testing practices help minimize rework and reduce time to market for software delivery.  As such, a trained, knowledgeable, informed, and engaged tester provides the best defense against quality and software mishaps, bringing value and operational excellence to a customers’ product. 

A common challenge for customers is to ensure appropriate breadth and depth in the testing of their software applications. Testing breadth speaks to coverage across the application – for example, are the new features properly working, is the uninstall executable removing the correct files from the user’s machine, and are actions performed via the mobile application synching with the web application. Testing depth speaks to exhaustive testing of a particular feature – for example, is the checkout form being tested for potential timing out conditions, pressing the back button on the browser, and are there issues with entering an online coupon after adding a product to the user’s shopping cart. This course specifically addresses how to complete test cases and includes a few scenarios that are often included in test cases. Keep in mind that this course only serves as background information, so please follow instructions on test case completion/execution as highlighted for each and every project you test.

Since every customer utilizes a different approach or methodology towards testing, you will also see different types and styles of test case documentation.  It is not usually recommended to modify a customer’s test case as testers, but it is critical that any columns in the test case be completed fully as you do not know how the customer will compile and analyze the data from the test case.  Since there are different types of testing that can be performed, it is essential to have as much familiarization as possible with the application before you begin executing the test case.  Review all related material provided by the customer, complete any required tutorials, and ensure that you clearly understand the testing objectives.  The more a tester understands what is expected, the better they will be able to execute the test case according to the customer’s specifications and in the process identify potential areas for testing that may not have been included in test case.  This prepares the tester for improved ad hoc or exploratory testing. It is important to mention however, that prior to spending time and energy in testing areas outside of the provided test case, the scope must be clearly understood.  This information is usually provided in the Testing Objectives area for a testing project.  In the unlikely event that information you are seeking is not provided, you can always contact the uTest project manager for additional information.

Test Case Definition: Every test case has series of steps to help the customer determine whether an application is working correctly. In general, most test cases contain several fields that either inform or require testers to complete, such as: Step Description, Pre-Conditions, Steps (input description), Expected Result, Actual Result, Status, and Observations/Comments columns. Bug ID is another field that you may come across, typically at the end of the table.

Sample Test Case:

Test Case Testing Approach and Tips: When a tester is executing test cases, they should follow instructions carefully as every test case is unique in requirements and formatting. Typically, test case submissions come with guaranteed compensation, so as a tester there may be more attraction with test cases versus pure exploratory testing. An important tip is to follow the test case carefully and perform all steps listed. Customers and project managers review all test cases and will be able to spot reports that are not completed accurately and thoroughly.

Pre-Conditions:  When you start executing the test case. First make sure all the pre-conditions are fulfilled. If this precondition is not fulfilled then you will not be able to execute the test case. As shown in the sample test case 1 user should be logged in Gmail to execute this test case. Other examples or pre-conditions are - user should have admin rights, user already logged in etc... .

Completing steps in order: Test cases should be run in the order specified. Test Cases normally have dependencies on the previous test cases. For example, step 2 requires you to execute step 1 first.

Verification: Verify each step of the test case very carefully. When testers execute a test case, they should take care that each verification point mentioned in the test case is verified properly as per the expected results.

Status: If the actual result is matching with the expected result of the test case then the Status column should be updated to PASS and if it is not then the status should be updated to FAIL.

• PASS – If the status for a particular step is PASS, then move to the next test case.
• FAIL – If the status for a particular step is FAIL, verify if this bug has already been reported. If not, report it immediately. After you have reported the bug, continue completing the subsequent steps in the test case. If subsequent steps have dependencies on the failed step and there are no workarounds, then mark them as N/A and move to next step.

Observations: Observations are very important in the test case. Here you can highlight the insight you gathered while completing a particular step. The information provided serves to help the customer troubleshoot an issue or better understand if there is any potential confusion for end users. Observations can be provided for both PASS and FAIL steps. 

Thorough completion of the Test Case: Do not leave any steps blank when you are executing the test case. If for whatever reason you are not able to complete a particular step, please add your comments under observation or actual results and update the status as N/A. Customers and project managers review all test cases and will be able to spot reports that are not completed accurately and thoroughly. Therefore, please be sure to pay close attention when executing every step of the test case.

Defect Reporting: As mentioned above under a failed step, if you encounter a defect first try to reproduce it and understand the root cause*.Then verify if that particular defect has already been logged – duplicate defects should not be reported. Details should include step by step information of what the tester did for ease of reproduction including screenshots and if possible a video and log files if available. Each failed step should have a defect attached to it. Enter the Bug ID in the Bug ID column; e.g. see Bug ID # 451178 for step 2. Do not always assume that the reviewer of your observations/comments or reporting is familiar with the software.  It is best practice to provide observations and comments as though your audience is a not a user of the application you are testing.  This ensures that any recipient will be able to reproduce what you’ve experienced and assist in having your defects approved in a quick, efficient manner.  Remember, clear communication on observations and steps taken when experiencing a defect is key to resolving issues found during testing.  Please refer to the Bug Title Standardization course and the Bug Reporting 102 course for guidelines on how to enter defects.

* Root cause analysis in it of itself requires additional instruction on how to best approach this method of analysis and understand if what you’re experiencing is the root cause of an issue so please refer to TBD for more information on this course.

Exploratory Testing: When you are completing/executing test cases, the customer may ask you to supplement it with exploratory testing. If so, make sure you doing adequate exploratory testing by testing for depth (for testing breadth versus depth, please see the introduction section for this course). Typically, you will be required to complete the test case first before performing extensive exploratory testing. Therefore, please complete and submit the test case before further exploratory testing (also refer to the introduction section for this course).

Test Case File Name: Customers will often ask testers to save their test case files with information to help identify the test case and tester who completed it.  It is best practice to retain the customer’s name for the test case but also add your operating system and browser to the test case name.  For example:  Sample Test Case_Mac OS_Safari.xls.  Sometimes, when multiple testers with the same OS and Browser are involved in a test cycle, it also may be helpful to add the tester’s last name.