Recording a Typical Load Test Using SOASTA CloudTest

Author: Bill Ricardi – Content Coach - uTest Load Tester of the Year 2010
Date: June 23, 2011

Recording a Typical Load Test Using SOASTA CloudTest

SOASTA's CloudTest Enterprise offers a fully integrated, enterprise level load testing suite that you can access from just about any computer that you can lay your hands on. Because the hard work is done on the back end, you can mastermind a 100,000 virtual user load test from your 5 year old laptop. I would like to show you how to record a typical step by step load test using the world’s largest global test platform.

You will need:

• Access to the SOASTA CloudTest server in question
• A list of test objectives and client demands
• A Windows, Mac, or Linux PC to work on
• Two (preferably more) different web browsers installed

1) Login to Concerto in First Browser:
In your first browser (I suggest the most up to date FireFox) log in to the CloudTest Concerto instance that you've been directed to. Normally, a standard username and password is fine. With certain clients, you might be required to install OpenVPN or some other virtual private networking software to comply with their privacy and security standards.

2) Install Conductor for that Instance:
If you've never worked on this instance, if you've installed a different Conductor for another test, or if the software needs updating, you'll be asked to download and install the Conductor containing the specific settings required for recording on this Concerto instance. It's a rapid process.

3) Start the Conductor Service on your PC:
Run Conductor, the client end recording interface, on your PC. Allow it to auto update if required. On certain Windows installations you'll need Administrator privileges on the computer to run this service (it is a packet sniffer after all).

4) Review Test Document:
Open the test document given to you by the client or SOASTA. In it you will see the vital information that you need, such as which domains to include and exclude, what steps to take during the scenario recording, wait times between pages, authentication information, virtual user resources (mock credit card info, address, etc.), and the like.

5) Sanity Test in the First Browser:
Go through the clip that you're recording using the first browser. If it works as intended, you'll know that any issues that you come up with on the recording run are browser-specific issues. If it does not work as intended, do a sanity test in the second browser. If that doesn't work as intended, you can bring up the test flow and/or browser compatibility issue with the test coordinator or the client if you have been asked to communicate directly with them.

6) Check Second Browser's Proxy Settings:
Start the second browser, and just make sure that the proxy settings match what you selected during your Concerto install (typically use port 4440). This also allows whatever home page your browser uses to load/fail, so that it is not included in the test recording.

7) Start Message Recording in Concerto:
Now that Conductor knows to listen, let Concerto know that you're talking. By starting a new message recording, you can specify where the raw data stream will be recorded within the Concerto directory structure. The standard placement is /Client Name/Month Year/ for your working directory, with any sub directories that you wish for organizational purposes. The final clip should be placed in the /Client Name/Month Year/Final/ directory along with your final scripts and resources.

8) Perform the Test in your Second Browser:
Now go through the steps provided by the client. Depending on how specific they are, follow them exactly or use your load testing instincts if they don't spell out every specific step. Use the SOASTA guides, tutorials, training, and standards that you have been provided to you.

9) Stop Recording and Convert to Clip:
Return to Concerto in the first browser and hit the Stop Recording button. After the raw data compiles, click on the Convert to Clip button. Remember to use the target domains specified by the client. If you're unsure about which domains to include (avoiding production sites unless specified), ask the test coordinator (or the client directly if you're so empowered and you think that's wise). Standards include grouping messages as pages, including wait times, and only using dynamic content if so instructed.

10) Clean and Organize New Clip:
Now take the recorded chaos and make sense of it. Delete unwanted content, fix the delays as specified in the document, and rename the pages so that they're logical when the test is being run. Unless instructed otherwise, page names should be the function, then a dash, then the recorded filename or page title. In the image below, the first page would be changed to 'Walden by Thoreau – books', the delay would be reset as specified (5 seconds being 5000ms for example) and 'FavIcon' would probably be deleted. Often, you might use shorter delays, within reason, to expedite the composition phase, and then change them to the specified delays for the final testing. Requests spawning '404 – File Not Found' responses are often given their own page automatically, which makes deleting them an easy task.

11) Test in Composition:
Do a first run in Composition, with a single virtual user unless otherwise instructed. This will help you find 404 errors, your upcoming scripting challenges (errors caused by authentication token mismatch, form ID issues, etc.), and you can see firsthand if your naming conventions make sense. Just click the Composition icon and then the Play button unless you want to mess around with the tracks and virtual users.

12) Note and Delete / Exclude 404 Errors:
Most often, the test site will be a work in progress and there will be some 404 errors. The client understands that you will not be including requests that spawn 404's unless they specifically request it. For static clips, you can just remove the offending requests directly in the clip editor. For dynamic clips (often required in Drupal implementations), you can do a glob exclude of the filenames, as shown in this screenshot.

13) Parametrize the Clip in the Session Template Package Wizard:
Here's where your SOASTA training and technical knowledge comes in handy. Running the Session Template Package Wizard, find the key parametizations that will change from test to test. Login tokens, form ID's, session trackers, and page target data that will shift because of a script you plan to include. Then use substrings, X-Path, RegEx, and the like to pick out the unique data for special treatment. Save this template for application to this and possibly later clips.

14) Add Custom Properties, Validations, and Scripts:
Now you can add your custom properties, validations, and scripts to meet other client needs. For example, you might add a script that randomizes the login name based on a .CSV file hosted on the web. Then you might add the 'username' and 'password' properties to the clip, and replace the form data in key messages to reflect the parametrization. Finally, you might add validation that fails the clip if, for any reason, the successful login isn't announced on the page after the post data is submitted. Remember to adjust scopes (private, local, or public) for scripts and requests as appropriate.

15) Re-Test in Composition and Check Responses:
It is critical that you continue to re-run the test in Composition until you've discovered all of the errors and sticking points. To this end, manually check each critical response to make sure that the result of a request is what you expect. Don't just rely on validations, as often this can produce false positives.

16) Fix All Remaining Errors:
Continue the bug hunt and script tweaking process until the clip is performing properly in Composition.

17) Double Check Delays and Page Naming:
Remember to change all of the delays and naming conventions back to whatever the client has requested, or reasonable standards otherwise. This is where you undo the shorter delays that made your testing easier, and replace them with the delays that will be used in the actual load test.

18) Move to /Final Directory, Log Hours, and E-Mail Lead:
After moving all critical resources to the appropriate /final directory, put your time spent and activity list in an Hours text file. Each one hour block should have a one line overview of what you did. This is how you get paid! Finally, E-mail your lead letting them know that the clip is done, outline any problems or specifications you need to, and get ready for your next assignment.

That is the basic outline for recording a SOASTA clip. The specific steps vary from test to test of course, and new SOASTA CloudTest users will rely heavily on the documentation, examples, and their mentors. But this guide should keep you on the right track.