How to Use Performance Testing Reports to Improve Performance

1. Statistics Summary

• Maximum running concurrent users
• Total throughput
• Average throughput
• Average hits per second
• HTTP responses summary

2. Transactions summary

• Total passed transactions
• Total failed transactions

3. HTTP responses summary

• The total amount of HTTP 2XX responses
• The total amount of HTTP 4XX responses
• The total amount of HTTP 5XX responses

4. Running concurrent users graph

5. Response times graph

• The reports contain metrics and statistics described above, a list of issues (with description and links to the statistics section) that occurred during test execution, and a general conclusion about the application's performance.
• The testing team prepares the reports after each iteration of the performance script execution and sends them to the Customer.

Impact of slow load time on website performance

An estimated 4.4 billion in revenue loss is recorded annually due to poor web performance.

Impact on Revenue / Sales:

• If an e-commerce site makes $100,000 per day, a 1-second page delay could cost you $2.5 million in lost sales every year.
• 1 second of load lag time would cost Amazon $1.6 billion in sales per year.

Main Issues:

Consequences of slow page load:

Performance Testing Real Time Examples

 Performance Testing:

Types of Performance Testing:

1. Load Test

2. Stress Test

3. Scalability Test

4. Soak Test

5. Load vs Stress

JMeter Works:

Thread Group

Thread count vs Loop count

Duration Assertion

The Essential Performance Testing Metrics Every Developer Should Know

1.  Processor Usage: Time spent by the processor to execute non-idle threads.
2. Memory use: The available physical memory to process on a system.
3. Disk time: It is the time taken by the disk to read or write a request.
4. Bandwidth: Bits per second used by a network interface.
5. Private bytes: A specific number of bytes allocated to a particular process.
6. Response time: The time between the user’s request and the first response character.
7. Throughput: Rate of requests received per second by a network. (The formula is Throughput = (number of requests) / (total time). )
8. Maximum active sessions: Maximum number of sessions that may stay active at once.
9. Thread (user) counts: Determining the well-being of the application by checking the number of running and active threads.
10. Latency - JMeter measures the latency from just before sending the request to just after the first response has been received.
11. Think Time - the time between two transactions of user actions. 
    (Login -> Think Time -> Search -> Think Time -> Logout). 
    The average thinking time could be in the range of 3 to 10 seconds for a normal user.
12. Error rate - Measures the number of errors that occur in each request.
13. Network latency - Measures the time it takes for a request to travel from the client to the server.
14. Transaction time: Measures the time it takes for a transaction to complete.
15. Resource utilization: Measures the number of resources used by the server to handle requests.
16. Memory usage: Measures the amount of memory used by the server to process requests.
17. Load test scalability: Measures how well the server can handle an increasing amount of requests.
18. Page load time: Measures the time it takes for a web page to load.

Basic concepts of Performance Testing

 Performance Testing

• Performance testing is a type of software testing that is used to evaluate the speed, scalability, and stability of a system. It is used to ensure that the system can handle the expected load and volume of traffic.

Types:

1. Load Testing
2. Stress Testing
3. Data/Volume Testing
4. Scalability testing
5. Endurance testing
6. Spike testing

Load Testing:

• Load testing is a generic term covering Performance Testing and Stress Testing.
• Testing the app with the maximum number of users.
• To measure the performance under the expected load.

Stress Testing:

• The system is under extreme load conditions, such as peak user activity or maximum transaction throughput.
• Testing the application with MORE than the maximum number of users.
• To measure performance under a load much higher than expected.

Endurance Testing:

• Endurance Testing is done to ensure the software can handle the EXPECTED load over a long period.

Spike Testing:

• System by SUDDENLY increasing the load (e.g. the number of users or transactions) to check how it responds.

Data/Volume Testing:

• System by increasing the number of users or transactions over a PERIOD OF TIME.

Scalability testing:

• The system performs when the workload increases or decreases.

Common Performance Problem:

• Most performance problems revolve around speed, response time, load time, and poor scalability.
• A slow-running application will lose potential users.
• Long Load time - While some applications are impossible to make load in under a minute, Load time should be kept under a few seconds if possible.
• Poor response time - Response time is the time it takes from when a user inputs data into the application until the application outputs a response to that input. Generally, this should be very quick. Again if a user has to wait too long, they lose interest.
• Poor scalability - A software product suffers from poor scalability when it cannot handle the expected number of users or when it does not accommodate a wide enough range of users.
• Bottlenecking  - Bottlenecking is when either coding errors or hardware issues cause a decrease in throughput under certain loads.

Common performance bottlenecks are:

• CPU utilization
• Memory utilization
• Network utilization
• Operating System limitations
• Disk usage

Introduction to JMETER and Performance Testing

 JMeter

• JMeter is an open-source load-testing tool used by developers and performance engineers to measure the performance of web applications.
• It can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types.
• Measure the application performance and response times.

Test Plan: (Top level directory)

• A complete test plan will consist of one or more Thread Groups, logic controllers, sample-generating controllers, Listeners, Timers, Assertions, and configuration elements.

1. Adding elements.
2. Removing elements.
3. Saving test plans.
4. Running test plans.
5. Stopping a test plan(immediate shutdown), Shutdown(graceful shutdown).
6. Logging the info and errors.

Important components of a Test Plan:

1. Thread Group
2. Listeners
3. Timers
4. Assertions

Thread Group:

• The beginning point of any test plan.
• The thread group element controls the number of threads Jmeter will use to execute your test.
• All controllers and samplers must be under a thread group.

Main properties of thread group:

• Set the number of threads (simulation of a number of concurrent users).
• Set the ramp-up period (time taken to threads up and running = ramp-up period/Thread 

Samplers:

• Samplers send requests to the server and collect the response.
• JMeter supports several types of samplers, including HTTP, FTP, JDBC, LDAP and SOAP.

Listeners:

• Listeners can be added to test plans or thread group level.
• Listeners provide a way to view the results of a test in JMeter.
• Listener options - Graph result, view results tree, view result table, aggregate report, aggregate graph, response time graph.