Mobile App Testing in 2020 – Does It Really Matter

Mobile app testing may not make your app super hit but it can certainly help you avoid failure by identifying potential problems early.

“Who is not using Smartphones and Mobile Apps in today’s world?”

“There are 2.6 billion smartphone subscriptions globally. By 2020, globally there will be 6.1 billion smartphone users led by huge growth in less mature markets. And with 6.1 billion smartphones in circulation, we will see a tipping point of sorts: smartphones will finally overtake the number of active fixed line subscriptions worldwide in 2020.”, according to the Annual Mobility Report from Ericsson published in 2015.

The growth of Mobile apps in Playstore.

The exponential growth in mobile phone and tablet usage has led to a large number of mobile apps being created that impact our personal and professional lives. It is easy to create a simple, low-cost mobile app. However, even a simple app is used on different devices, operating systems and by a diverse user base. This creates a complicated situation where even a simple mobile app needs comprehensive testing to attract and retain users and build your brand.

A few months ago, a friend came up with an interesting app idea. He was able to raise funds and engage a top class development team to build the app. The app was released after intensive feature testing by the developer.

Since the app uses an attractive concept, it attracted a large number of users immediately after launch. However, within two weeks, the user throughput started declining and dropped to near zero.

This friend asked me for an outsider’s opinion as he was not able to pinpoint why users were abandoning the app. He told me, “Vikas, I set up the best product team, had a winning idea and the app was downloaded by hundreds of users within a week. But I don’t know what happened. My user base has stopped growing for the last 6 weeks.”

The first thing I did, as I normally do to in such a situation, was to look at user ratings and reviews of the app posted on the app store.

These reviews and ratings give the genuine user feedback that helps me understand the usefulness and reliability of the app. Reviews also help me assess how good or bad the app is before I install it on my mobile.

While analyzing the reviews, I realized that many users had rated the app very low.

Most users loved the concept and thought the app would be useful. But the app had installation problems, the crash rate was high and had features that did not work on many devices.

I explained the situation to my friend and asked him to focus on usability and compatibility testing of the app to improve the user experience.

Together, we developed a testing plan covering all aspects of app testing. With the 3600 testings, the app quality improved dramatically. We managed to turn around the app within a quarter. It currently has a growing user base of more than 300K with 4+ rating on the app store.

This case clearly highlights the most common problem with the mobile apps today. The app owners use the same approach they would use to test a web application. They do not realize the need for a holistic testing model for mobile apps. The mobile app testing plan needs to focus equally on compatibility, usability, and functionality.

A poor quality mobile app not only hampers user adoption but can also cause revenue loss and damage the brand image. With millions of apps competing for user attention, ensuring high quality of your app is essential to its success.

Let me share another story about the “Big Billion Day” launched by Flipkart, the largest online store in India, in October 2014. While millions of users were attracted by the hype, users faced several problems. They complained about misplaced product views, server crashes, and slow orders processing. Flipkart took a hit on its brand name and also lost significant revenue. Why did this happen?

Flipkart must have spent millions on product development and testing. But they missed testing the app for compatibility on low-cost devices. They did not test the server and back up the infrastructure for peak load conditions. FlipKart learned from this and had a very successful “Big Billion Day” in 2015 after doing thorough testing to avoid these problems.

Flipkart learned its lesson the hard way but was able to come back. However, most mobile app companies do not have enough resources and brand strength to survive poor user experience. I always recommend a 3600 mobile app testing to ensure it becomes a winner.

Dimensions of Mobile App Testing

In order to understand the complexity of testing mobile apps, it is important to understand various aspects of the app to be tested.

A 3600 mobile app testing plan gives the confidence that the app will function as intended in both functional and non-functional areas. It ensures that your app works on devices with different screen sizes, resolutions, operating systems and across telecom networks.

The most important test for any software is its functionality and behaviour as per the user requirement. For example, an eCommerce app needs to test if the user is unable to make payment in the application. Functional testing is the basic necessity which is covered by all conventional testing models.

The second most important thing is to focus on the usability testing of the app. Usability is the most important, yet in most cases, neglected aspect of the app. It dictates user stickiness for your app against your competition. A good user experience is the entire ecosystem created by your app and its surrounding systems. It includes feature relevance, ease of navigating the app, consistency across devices, UI layout and overall design.

It is critical that your app behaves consistently across all operating systems, device models, languages and networks in your target market. You must perform compatibility tests for your app on different mobile devices, operating systems, browsers, screen sizes, interfaces and network operators.

When a user installs an app, she looks watches for app responsiveness and its impact on her phone. Most mobile users are sensitive to an app’s impact on battery consumption, data usage, heat generation and speed. Therefore, performance impact of your app on phone is very important. It can be tested using various tools and as well as manually. In absence of this test, a user might uninstall your app very quickly.

Security is another important aspect of your app that needs to be tested well. The contacts, photos, messages are something very personal to a mobile user. Security checks are important for mobile app testing strategy to check against vulnerabilities to hacking, authentication and authorization policies, data security, session management and other security standards. Security Testing can be performed using various tools such as Debian linux, Genymotion, Android Tamer, APKTOOL, Dex2JAR, JD-GUI, BURP Suite etc.

There are many tools available in market to automate mobile app testing. Appium, Calabash, Monkey Talk, Robotium, Selendriod, UI Automator are some of the commonly used tools in the market.

Increasingly, the app companies are looking beyond from Internal QA testing teams and using crowdsourced testing. Crowdsourced testing is one of the emerging trends in the software testing industry. It enables app testing by hundreds of testers and users under real-world conditions across multiple device models, network conditions and languages. This form of software testing meets the scale and agility needed to test mobile apps faster and in a cost-effective manner. The crowd diversity enhances test coverage which is not possible through a stand-alone QA team.

If the projections in Ericsson’s Annual Mobility Report for 2020 materialize, smartphone users and number of apps the app stores will increase exponentially. App testing will become even more important in future.

We cannot ignore the role mobile app testing plays in a mobile app’s success. With thousands of mobile apps being added to app stores every day, each app owner dreams that his app will become the next big hit. But only a few succeed, and they are the ones who focus on holistic quality in all aspects of app usage. A user uninstalls a new app within a few minutes if she does not like something. She will also leave a negative review for other users, discouraging them from installing the app.

You need to maintain app quality from the start in order to stay ahead of the competition. The users will love your app and spread positive feedback, benefiting your business.

ABOUT THE AUTHOR

VIKAS KANAGO
Senior Test Engineer, Qualitrix
B. E (Electronics & Communication)