Ensuring optimal functioning with fast response for times of excessive stress
A well-functioning application may come across times when it may see a huge surge in user interaction, putting stress on an otherwise smooth operation. Common examples of this are a festive surge witnessed by a shopping application, a multi-player event set up by a gaming application, or academic year closure times for an educational application. In each of these instances, the application might witness an exponential increase for a certain period of time, putting extra load on an otherwise functioning system.
Performance testing assures the speed, scalability, and reliability of the application, and assures it is ready to withhold the additional market load expectations.
The Need
Qualitrix was approached for a performance testing project by a client having an application with a pan-Japan presence. This application was built for conducting a ‘Centralized Training Management Program’ providing live and pre-recorded training material to healthcare service professionals. Since it is a training platform, the objective was to ensure optimal application performance under all usage scenarios like multiple customer registration, simultaneous multi-user login, and accessibility for both live and pre-recorded sessions. The expected load was said to be about 1800–3000 concurrent users attending the training.
Kickstarting the project
The first preliminary step was to understand the application and identify the key touchpoints which will surely experience the added load. Seven critical business scenarios were identified where performance impact is expected; scenarios where benchmark testing, load testing, and stress testing would need to be performed with concurrent 1800–3000 users to find out the performance bottlenecks. Here are those seven critical scenarios along with the number of respective concurrent users:
Execution and observation
Depending on the scenario to be tested, the relevant test was picked up. Not only was a stand-alone load test implemented per scenario, but another combined load test was also executed for 1800 concurrent users.
The intent of this activity was to identify any places where the pre-defined SLA and benchmarks are not met, or where the application’s response is sluggish or crashes entirely in a simulated environment. Prior knowledge of the same will help the team ideate and arrive at resolutions so that at the time of actual market usage, the same issues won’t occur.
During the load testing, a high response time of the application was observed across many of the scenarios breaching the defined SLA. While the defined SLA was 3 seconds, some scenarios took around 5-10 times of the defined SLA. Some of the observations are available below:
Further observations include:
- It was observed that CPU utilization reached around 98% during the load, which was much higher when compared to the defined threshold.
- Login failures were observed due to database unresponsiveness.
Outcome and Closure
A project of performance testing is like a diagnostic check-up to know whether all parts are functioning optimally or not. Further, if sections are identified which need tweaking, then these tests should help identify the areas where work is needed. Thus, in this scenario, when it was observed that application response was slow in many scenarios, further analyses were carried out and recommendations were provided.
For instance, login page data structuring was recommended and it was trying to fetch lots of data from the Database which was impacting application performance. This type of database tuning was critical for optimal user experience. Other scenario-specific recommendations were made and shared with the client for further enhancement of the application.
The entire performance testing was performed over a duration of 1 month and results were published to the client along with recommendations.
This case study is to bring to light the benefit and need of performance testing in addition to the manual and automation testing being conducted in the day-to-day functioning of the application.