SPEED SCALABILITY STABILITY
Performance Testing is done to provide stakeholders with information about their application regarding speed, stability, and scalability. More importantly, Performance Testing uncovers what needs to be improved before the product goes to market. Without Performance Testing, software is likely to suffer from issues such as: running slow while several users use it simultaneously, inconsistencies across different operating systems and poor usability. P2S also guides in performance engineering in addition to doing performance testing.
Performance testing gathers all the tests that verify an application’s speed, robustness, reliability, and correct sizing. It examines several indicators such as a browser, page and network response times, server query processing time, number of acceptable concurrent users architected, CPU memory consumption, and number/type of errors which may be encountered when using an application.
Types of Performance Testing performed by P2S
- Load testing – checks the application’s ability to perform under anticipated user loads. The objective is to identify performance bottlenecks before the software application goes live.
- Stress testing – involves testing an application under extreme workloads to see how it handles high traffic or data processing. The objective is to identify the breaking point of an application.
- Endurance testing – is done to make sure the software can handle the expected load over a long period of time.
- Spike testing – tests the software’s reaction to sudden large spikes in the load generated by users.
- Volume testing – Under Volume Testing large no. of. Data is populated in a database and the overall software system’s behavior is monitored. The objective is to check software application’s performance under varying database volumes.
- Scalability testing – The objective of scalability testing is to determine the software application’s effectiveness in “scaling up” to support an increase in user load. It helps plan capacity addition to your software system.
P2S 9 step approach for Performance Testing
1. Identify the testing environment.
Identifying the hardware, software, network configurations and tools available allows the testing team to design the test and identify performance testing challenges early on. Performance testing environment options include:
- Subset of production system with fewer servers of lower specification
- Subset of production system with fewer servers of the same specification
- Replica of productions system
- Actual production system
2. Identify performance metrics.
In addition to identifying metrics such as response time, throughput and constraints, identify what are the success criteria for performance testing.
3. Plan and design performance tests.
Identify performance test scenarios that take into account user variability, test data, and target metrics. This will create one or two models.
4. Configure the test environment.
Prepare the elements of the test environment and instruments needed to monitor resources.
5. Implement your test design.
Develop the tests.
6. Execute tests.
In addition to running the performance tests, monitor and capture the data generated.
7. Analyze, report.
Analyze the data and share the findings.
8. Performance Engineering
Guide developers to fix issues
9. Retest
Run the performance tests again using the same parameters and different parameters.
Performance Testing Metrics produce by P2S
The basic parameters monitored during performance testing include:
- Response time – time from when a user enters a request until the first character of the response is received.
- Throughput – rate a computer or network receives requests per second.
- Maximum active sessions – the maximum number of sessions that can be active at once.
- Hit ratios – This has to do with the number of SQL statements that are handled by cached data instead of expensive I/O operations. This is a good place to start for solving bottlenecking issues.
- Hits per second – the no. of hits on a web server during each second of a load test.
- Processor Usage – an amount of time processor spends executing non-idle threads.
- Memory use – amount of physical memory available to processes on a computer.
- Disk time – amount of time disk is busy executing a read or write request.
- Bandwidth – shows the bits per second used by a network interface.
- Private bytes – number of bytes a process has allocated that can’t be shared amongst other processes. These are used to measure memory leaks and usage.
- Committed memory – amount of virtual memory used.
- Memory pages/second – number of pages written to or read from the disk in order to resolve hard page faults. Hard page faults are when code not from the current working set is called up from elsewhere and retrieved from a disk.
- Page faults/second – the overall rate in which fault pages are processed by the processor. This again occurs when a process requires code from outside its working set.
- CPU interrupts per second – is the avg. number of hardware interrupts a processor is receiving and processing each second.
- Disk queue length – is the avg. no. of read and write requests queued for the selected disk during a sample interval.
- Network output queue length – length of the output packet queue in packets. Anything more than two means a delay and bottlenecking needs to be stopped.
- Network bytes total per second – rate which bytes are sent and received on the interface including framing characters.
- Amount of connection pooling – the number of user requests that are met by pooled connections. The more requests met by connections in the pool, the better the performance will be.
Example Performance Test Cases
- Verify response time is not more than 4 secs when 1000 users access the website simultaneously.
- Verify response time of the Application Under Load is within an acceptable range when the network connectivity is slow
- Check the maximum number of users that the application can handle before it crashes.
- Check database execution time when 500 records are read/written simultaneously.
- Check CPU and memory usage of the application and the database server under peak load conditions
- Verify response time of the application under low, normal, moderate and heavy load conditions.