By: Matthew Seese
Twelve years ago, the world was catching it’s breath after the fears of a “Y2K freak-out” subsided. A lot of people were focused on the older machines that ran the world and what the implications would be if they suddenly stopped working. A few folks, myself included, were looking towards the future. A future that was “mobile” and “wireless”. I joined a fledgling company that was making software for “wireless devices” that was started up by some very bright people who cut their teeth at a popular cellphone company. I was to be the sole QA person and build a process around testing their software. I was so excited I could hardly stand it! Then it hit me “How do I test these mobile applications?” Up until that point, I had been testing web applications and some client/server apps. There just weren’t that many mobile apps around. What I found was that there were a lot of similarities in the approach to testing the software for the mobile devices, but there were some key differences.
Fast forward to today, where mobile devices and the software that run them are ubiquitous. There are literally thousands of apps on the various mobile platforms and hundreds of devices with new ones being released weekly. In looking at the changes over the past twelve years I see a lot of the testing similarities and again, some key differences.
At a very high level, the approach for testing mobile applications is the same in regards to Expected Results and Actual results. Testers can utilize the same toolsets to create, manage and execute their manual tests against the mobile application. The software development lifecycle and project management principles still apply to the overall project. The tester still has positive and negative test cases and static testing can be applied early on in the project.
The changes that have happened over the past twelve years include things like fragmentation, faster rates of change and some operational logistics. As stated, there are numerous platforms and hundreds of devices in the marketplace. The rapid expansion of platforms and devices has led to fragmentation and subsequently the need to develop and manage applications across the mobile landscape. The rates of change have also increased causing an increased need for flexibility, modularity and scalability. If your apps are written in stone, they won’t migrate. Finally, organizations are having to monitor and regulate mobile application usage by their employees and provide mobile solutions to their customers. This leads to an increased operational cost and eventually a dependence on mobile accessibility. The fewer problems here, the better.
The challenges facing mobile application testing are due to the numerous platforms and device differences, there are new usability issues and user interface concerns, network latency issues and interaction with the device’s primary functions such as receiving phone calls or incoming text messages. And let’s not forget battery consumption, data plan usage concerns and security for things like password management and saved or cached data. These additional considerations must be accounted for in the specifications and design and then verified in testing. It would do no good to have the killer app that broke your phone when you got a phone call or have an app that looks fantastic on an iPhone but won’t render on an Android phone, for example. The application designers and business analysts also need to decide early on which platforms and devices they will support. Another important consideration is whether it will be a native app or browser-based. Forrester research says to do both and that means double your testing effort. Then there are the performance considerations: response time, bandwidth utilization and additional load on backend servers. Once again, you can have the greatest app but if it’s slow you customers will go elsewhere.
So what’s the good news? The good news is that there are solutions to the aforementioned problems. Test cases can be written in the traditional method and requirements based testing carried out on scheduled releases. If you chose the Agile methodology, your teams can operate in the same ways as they have before with non-mobile applications. There are emulators that which closely mimic the devices and testers can use them at different levels of testing to help detect and eliminate defects. There are performance testing tools specifically designed to simulate mobile application testing scenarios. There are also vendors who are providing real device testing platforms as a service, such as Perfecto Mobile. Testing organizations can “rent” a combination of devices that will meet their requirements for a specified testing period. The devices are hosted and testers can access them remotely but manipulate the devices as if they were holding it in their hands (for more info check out Perfecto Mobile – The Mobile Cloud Platform).
With the widespread use of mobile devices and the applications that run them, the ever increasing speed of the changes, and the complexity of the user experience so critical for mobile application success it is good to know that proper testing can help mitigate these risks. Stay tuned for more as we discuss mobile application testing strategies in detail and explain ways to reduce your cost of quality while increasing your mobile application user satisfaction.