Requirements Management for Test Projects 101

Requirements Management for Test Projects

Author: Pramod Lumb
Date: June 24th 2011 

Introduction: Requirements form the basis of any project. The complete life cycle of a project is designed around understanding customer requirements and ensuring that they are met. Irrespective of whether it is a development project or a testing project, the importance of understanding and managing client requirements cannot be over-emphasized.

For a test project, managing customer requirements becomes all the more critical. Why so? Because irrespective of agile development and other test driven development methodologies that we talk about, the truth is that testing teams start playing an active role in a project rather late. Often, it is at a time when the development team has already finalized the software design, if not the code. In this case, it becomes necessary for the test team not only to understand customer requirements firsthand but also to verify that these requirements are correctly understood by other teams, specially the development team. In case of independent test projects, the test manager still has to ensure that his/her understanding of customer requirements is correct. Further, the test manager also needs to guard against incomplete or ambiguous requirements.

So, let us look at what all may get wrong as far as project requirements are concerned, and how the same should be handled.

What may go wrong with documented requirements and how we should take care of such situations: Requirements are thought of and documented by humans, and humans are fallible. Tools may be used in order to help human beings document requirements but the inputs are still being provided by a person. So, let us look at what may go wrong with specification of customer requirements, and how to take care of such situations:

1. Requirements may be incomplete or ambiguous. The customer, to be fair to him/her, may be thinking that all requirements have been documented completely and unambiguously, but that belief may be incorrect. The customer may have indeed documented all that is required out of the project, but it may not have been documented completely or unambiguously. In most of the projects that I have worked on, implicit requirements are left undocumented, and the vendor is assumed to be aware of them.

Let me share a real life example with you. In one of the test automation projects that I worked on, we were told that the execution reports generated by the test automation tool would need to be fed to the customer’s proprietary document management system. This requirement seems to be complete in itself, and is genuinely complete for someone who has worked with this customer’s document management system. But we were working with this customer for the first time. Towards the end of the project, when we had already selected the test automation tool and were almost done with automation of manual scripts did we realize that the reports had to be generated in PDF format in order for them to be accepted into the customer’s system. But the tool chosen by us could generate execution reports only in HTML format. So, we had to put in extra effort for getting all test execution reports converted to PDF format. Had we known beforehand that only reports generated in PDF format would be acceptable, we might have chosen a different automation tool.

Let us take another example. A customer may state that test automation needs to be carried out on the latest stable build of an application. In this case, the customer has not specified his/her definition of a stable build. It is also not known as to when the next stable build of the application is expected. What would happen if we commit to complete test automation within the next six months without knowing that the upcoming stable release is expected only after four months? Also, how do we know the criteria that this customer uses in order to classify a build as a stable build?

How can these situations be avoided? By asking the customer relevant questions. In the first example, the customer should have been asked to specify the formats in which their document management system is able to accept reports, or else they should have been told (before finalizing the test automation tool) that we would be able to provide reports in HTML format only. In the second case, the expected timeline for release of a stable build should be asked.

2. Requirements may be in conflict with other requirements. Sometimes, the SRS (software requirements specification) document is prepared by different departments of a large organization together. However, because of many reasons, one or more departments may be unaware of what other departments are looking out for, or what they have asked for in the requirements document. This often leads to ambiguities and conflicts in specification of requirements.

In this case, the test manager should bring out any inconsistencies in requirements to the notice of the customer, and get them sorted out at the earliest.

3. All impacted stakeholders may not have been consulted when finalizing requirements. This is something that often happens but unfortunately, the test manager has little ammunition available to avoid getting into this situation. Because of many reasons, senior officials from other departments may not be consulted when finalizing the expectations from the new product, or from test automation or performance testing projects. This becomes evident only later in the project life cycle when people from specific departments in customer’s organization respond coldly to requests for help or additional information.

The best option available to the test manager in this case would be to have the customer designate a SPOC (Single Point Of Contact) for the vendor personnel. It should be the SPOC’s responsibility to let the vendor team have access to all requested information and support. If this does not happen, the issue should be escalated to senior management as early as possible.  

Conclusion: We have seen that the requirements document may not contain all the customer requirements, or it may have them documented ambiguously or incompletely. Besides, all stakeholders may not have been consulted at the time of finalizing requirements. These factors may lead to issues with execution of the project as well as with obtaining customer sign off at the end of a project. Hence, the test manager should always ensure that all requirements are clearly understood in the manner in which the customer intends them to be understood, before finalizing on the strategy for a test project. This may be achieved by asking additional questions or by stressing on the need for a POC (Proof Of Concept) phase before starting with the actual project (in case of an independent testing project or a test automation project). This will ensure that customer requirements are understood and managed successfully.