Background:
Client is a leading International Telecom Operator based in Europe, providing mobile services to millions of customers across Europe, US, Australia, Asia. The company embarked on a comprehensive digital transformation initiative to modernize its operations and enhance customer experiences, it has recently revamped its mobile app and website to enhance the user experience and introduce new features for more than 16 countries. The Qualitrix Quality Assurance (QA) team is tasked with ensuring a seamless and bug-free experience for users across both platforms with aggressive deadlines.
Objective:
To conduct comprehensive QA testing for the Lycamobile mobile app and website for 16 countries, ensuring functionality, performance, content, and usability meet the highest standards before customer release.
Key Features and Tech stack :
- Mobile App:
- Customer acquisition journey
- Top up and bundle Payment
- Renew journey
- Local language support
- Setting and account information
- Dashboard
- Website:
- New user journey
- Existing user journey
- International calls journey
- Live Chat Support
- Payment journey
- Homepage
- Tech stack
- Before migration
- Frameworks: PHP framework
- Databases: MySQL
- front-end Frameworks: PHP
- Back-end tools: WordPress
- APIs: CodeIgniter
- After Migration
- Frontend Frameworks: React
- Backend for frontend: Node (Fastify)
- Mediation layer: Java-JSP
- Data storage and querying: PostgreSQL & MySQL
- Content management: PIM & Drupal
QA Testing Strategy:
Prioritization and Risk-Based Testing:
- Identified critical and high-priority features that directly impacted the app’s core functionality and user experience.
- Focused on risk-based testing to allocate resources where the highest risks were identified.
Parallel Testing:
- Split the testing team into parallel streams to simultaneously test different modules and functionalities.
- Parallel testing allowed for faster coverage and identification of critical issues.
- Assigning country wise responsibility to small groups of QA team.
Exploratory Testing:
- Supplement scripted test cases with exploratory testing to uncover unforeseen issues quickly.
- Leverage the experience and creativity of testers to simulate real-world usage scenarios.
Test Automation:
- Leveraged test automation tools to rapidly execute repetitive and regression test cases.
- Automated tests for key functionalities, such as Payment, dashboard, footer, menu options to expedite testing cycles.
- Quickly automated sanity suite for android and website for all 16 countries
Real-Time Communication:
- Established real-time communication channels between developers and QA testers to quickly address and resolve issues.
- Used collaboration tools for instant feedback and quick decision-making.
- Implemented war rooms and collaborated with multiple teams and escalated the blockers immediately.
- Establish a streamlined feedback loop to ensure rapid iteration and improvement
Testing Phases:
- Functional Testing:
- Validate all features and functionalities across the mobile app and website.
- Verified all the journeys, login, and logout processes.
- Test payment transactions, ensuring accuracy and security.
- Compatibility Testing:
- Ensure the mobile app is compatible with various devices (iOS and Android) and screen sizes.
- Test the website on different browsers (Chrome, Firefox, Safari, Edge) and devices to ensure a consistent user experience.
- Performance Testing:
- Evaluate website response times under different conditions
- Check for scalability by simulating a high number of concurrent users.
- Assess the loading times of key pages and transactions.
- Content Testing:
- Conduct comparison of content from old websites to new website using open-source tools
- Validate all the contact numbers, links and grammatical errors in the website
- Usability Testing:
- Evaluate the user interface (UI) and user experience (UX) of both the mobile app and website.
- Test navigation flows and check for intuitive design.
- Verify accessibility features and compliance with usability standards.
- Regression Testing:
- Perform regression testing after each code change to ensure new updates do not introduce new issues.
- Confirm that existing functionalities still work as expected after updates.
- Cross-platform Testing:
- Validate data synchronization between the mobile app and website.
- Check for consistency in user data, account information, and transaction history.
- Localization Testing:
- Validate the user journeys for the local languages for all the countries which was part of the launch
Tools used:
| Tools | Description |
| PIM | Plan Bundle management tool – Mainly used for configuration of plans / price etc. |
| CMS | Content management took – Mainly used for configuring content of the homepage, FAQs, settings menu, links of the website |
| AWS-CloudWatch | To check or monitor logs. |
| Jira | To create and track the defects and user stories of respective projects |
| G-sheets | To create and maintain the testcases for mobile app and website. |
| Confluence | To create and store bug reports and sign off reports |
| Lambda test | To execute automation parallel execution of website and mobile application of all 16+ countries |
| Browser stack | To test device compatibility of the apps for different android and IOS versions |
Challenges Faced:
- Third-party Integration:
- Ensuring seamless integration with third-party services for payment journey.
- Ensuring the correct content is being fed through the backend systems through PIM and CMS.
- Network Variability
- Addressing issues related to varying network conditions that may impact app and website performance. since the launch countries were non-Asian countries, few functionalities related to call, message, and data were challenging to test.
- Limited Timeframe:
- The development team was working on an expedited timeline, leaving a limited window for thorough QA testing.
- The project has ambitious timelines to meet market demands and outpace competitors.
- Dynamic Requirements:
- Owing to the fast-paced nature of the telecom industry there are frequent changes in requirements and feature enhancements.
- Complex Feature Set:
- The software update included multiple features, integrations with multiple backend systems, and bug fixes, increasing the complexity of testing.
- Release process
- The release process for launch of 15 countries were not streamlined as per usual agile process, it was dynamic, and it was difficult to track and update the features respective to particular countries.
Outcome:
After rigorous testing and addressing identified issues, Lycadigital successfully launched the updated mobile app and website in the period of 6 months for 16+ countries. The QA process significantly contributed to a positive user experience, reduced the likelihood of post-release issues, and enhanced the overall quality and reliability of Lycamobile’sdigital platforms.
Key Performance Metrics:
Knowledge gathering process: Qualitrix team were able to quickly collaborate with the developers / Business head / other PMs to understand the application, we were able to quickly gain in detail knowledge of the application and implemented the reverse KT process to analyse the application understanding.
Test case creation: Qualitrix QA team were able to create testcases by splitting up the modules to different team members for mobile app and we also re used some of the mobile common testcases to website app.
Defect Detection Rate: The QA team were able to raise more than 100 bugs during the initial release of QA build, which gave the business to get more insight into app’s stability.QA team has raised overall 1000+ bugs for mobile application for 16 countries, and 800+ bugs for website application.
Mobile app breakup
Website issues breakdown
Test Case Execution: The QA team executed 2 kind of testcases,
- High level check – For each build, which was majorly focused on verification of major flows for complete application / this check was done pre and post release.
- Sanity check – For the final build which was majorly focused on verification of backend API calls, Logs check, UI / UX along with functionality of Priority 1 testcases of entire application.
Automation Test Coverage:
- Once the launch of mobile and web application, our QA team automated top 100 E2E flows which was focused on browser, mobile devices, Msite
- Team was able to integrate to third party tools to execute automation scripts and schedule it run after each release.
Release process:
- QA team were able to streamline process for app and website release
- QA team implemented Bug tracking via Jira, Release notes after each build, Bug life cycle process, Release cycle timeline, testing timelines for each builds, testcase management through g-sheet, daily bug report status etc. This helped the team in achieving better quality of apps.
- Implemented Test signoff from Qualitrix team for each country release.
Conclusion:
Effective QA testing played a crucial role in ensuring the successful launch of Lycadigital’supdated mobile app and website. The company’s commitment to delivering a high-quality user experience and addressing potential issues proactively has contributed to increased customer satisfaction and engagement with their digital platforms. The case study highlights the importance of a thorough QA process in achieving a reliable and efficient telecom mobile app and website.