Click here to view the complete list of archived articles
This article was originally published in the Summer 2003 issue of Methods & Tools
Modeling the Real World for Load Testing Web Sites
Steven Splaine, www.splaine.net
Introduction
If you want to get an accurate idea of how your Web site is going to perform in the real world, it pays to create a load profile that closely models conditions your site will experience. This article addresses nine elements that can affect Web load.
Requesting your Web site’s home page 100 times per minute is not going to give you a very accurate idea of how your Web site is actually going to perform in the real world. This article seeks to provide an overview of some of the load parameters that you should consider when designing the test load to be used for performance testing a Web site. The more accurate the load profile can be made, the closer the performance tests will be to modeling the real world conditions that your Web site will ultimately have to survive. This, in turn, will lead to more reliable test results.
1. User Activities
Many transactions occur more frequently than others, and should therefore comprise a larger proportion of the test data and scripts used for performance testing.
In order to simulate the real workload that a Web site is likely to experience, it’s important that the test data and scripts are a realistic representation of the types of transactions that the Web site can be expected to handle. If your organization typically only sells one product for every fifty visitors, it would be unrealistic to weigh two test scripts evenly (one mimicking a browser customer, the other an actual buyer). A better approach would be to use a 50:1 weight (fifty browsers for every buyer).
2. Think Times
The time that it takes for a client (or virtual client) to respond to a Web site has a significant impact on the number of clients that the site can support. People, like computers, have a wide range of different processing levels and abilities. Different users require various amounts of time to think about the information that they have received. Some users race from one Web page to another, barely pausing long enough to comprehend what they’ve seen; others need some time to contemplate what they’ve read before moving on to the next page. The length of this pause is called think time. Think time is generally considered to be the length of time from when a client receives the last data packet for the Web page currently being viewed to the moment that the client requests a new page.
In theory, a Web site that can support a maximum of 100 "10-second aggressive" clients should be able to support 300 "30-second casual" clients because both types of clients result in 600 transactions per minute. Unfortunately, this theory only holds true for the most rudimentary of Web sites. Web sites that are more interactive require resources for both active and dormant clients, meaning that the 300 casual clients are likely to require more resources than the 100 aggressive clients.
When recording or writing test scripts for performance testing, you should consider how much time each of the various types of clients might spend thinking about each page. From this data, you can create a reasonable distribution of timings and, possibly, a randomizing algorithm. Web logs can be a good source for estimating think times for Web pages that have already been hosted in production.
3. Site Abandonment
Basic test scripts/tools assume that a client will wait until a Web page has been completely downloaded before requesting the subsequent page. Unfortunately in real life, some users may be able to select their next page before the previous page has been completely downloaded (e.g., they may know where they want to go as soon as the Navigation bar appears).
Alternatively, some users will surf off to another Web site if they have to wait too long (i.e., terminate the test script before reaching the end of the script). The percentage of users that abandoned a Web site will vary from site to site depending on how vital the content is to the user (e.g., a user may only be able to get their bank balance from their bank’s Web site, but could get a stock quote from numerous competitors). One thing is certain; the longer a user has to wait, the more likely it is that they will abandon the Web site. Therefore, in order to model the real world, test scripts should include a randomized event that will terminate a test script execution for a particular client if the client is forced to wait too long for the page to download. The longer the delay, the more likely that the client will be terminated.