Health plan provider executes a shift-left in mobile app testing
One of the nation’s largest not-for-profit health plan providers is focused on delivering the highest quality of care in the most efficient and effective manner possible. With more than 9 million members, nearly 180,000 employees and 18,000 physicians, and with a 2015 operating revenue of more than $62 billion, the organization is committed to continually reinventing patient care with the latest research and technology. Among its initiatives are apps that allow members to connect with their doctors, other care providers and their health information via their mobile devices. Its flagship app has been downloaded more than a million times and is in wide use by its members.
To ensure a good experience for users of two of its mobile apps, the organization wanted to set in place a unified automation framework. Compared to websites or traditional client/server applications, building mobile apps requires development processes that are faster and produce higher-quality apps that can meet higher user expectations. Mobile app performance is complex, with more than 100 million permutations of variables to manage.
Recent trends have caused mobile app testing to shift-left, with activities happening earlier in the development process, which puts more of the burden on mobile developers. A 2015 study of developers around the world found that 69 percent are doing more automated testing, and 61 percent are testing earlier in the development cycle than ever before. New research reveals 94 percent of developers have changed their testing approach since adopting Agile/Continuous integration.
The increased focus on mobile is changing how enterprises create software –and amplifying the need for more efficient and effective testing practices due to the variety of devices, operating systems and front-end development technologies involved. Enterprises are turning more and more to automation to improve time-to-market and test coverage while reducing the manual labor cost of testing.
To meet these needs, the organization turned to PK to automate regression testing and validate the user experience and all key business workflows related to two mobile native apps developed on the iOS and Android platforms. The organization also needed a highly flexible, easy-to-maintain test suite certified over a large set of device configurations across both platforms in addition to the ability to quality-certify the apps for every release, with test reports generated in the Rally tool.
Automated tests can easily execute thousands of complex test cases during every test run, providing coverage that is simply not possible with manual testing. PK’s solutions allowed the organization to easily and quickly simulate and manage test environments to bring higher quality apps to the market faster and at a lower cost.
PK designed and built a smart, unified automation framework that cut the regression cycle from three weeks to one week. With a grid infrastructure that included more than 25 devices, PK deployed a test suite handling more than 1,000 test cases, with 70 percent reusability.
PK utilized Appium WebDriver to automate mobile testing across devices, regardless of the operating system, and Appium Grids to enable cross-device testing. PK’s team of mobile experts turned to Page Object model design to write maintainable and reusable test routines and built annotation-driven Page Objects to make the code less verbose. The organization also was able to realize reduced defects and greater cost efficiencies by adopting a testing framework able to integrate with Rally through REST APIs to upload test results and automate the process of defect-raising, and with Jenkins for automated test execution with every release.
New Research Reveals 94 Percent of Developers Have Changed Their Testing Approach Since Adopting Agile/Continuous Integration. Reuters. Feb. 11, 2015. http://www.reuters.com/article/idUSnMKWk9xJha+1c4+MKW20150211
The Basics of Test Automation for Apps, Games and the Mobile Web. Smashing Magazine. Jan. 14, 2015. https://www.smashingmagazine.com/2015/01/basic-test-automation-for-apps-games-and-mobile web/