How device coverage should influence your mobile app testing strategy ?

Brief Intro: There was a time when everything was manual, then came automation and now the device farms. I am talking in the context of Mobile app testing. When you change focus from Web to Mobile, one of the thing which ideally should get detailed attention from you is “device coverage”.
Why device coverage matters: Let me quote some numbers for Android OS first to give you a rough idea. Below is a snapshot from OpenSignal. According to the report, android device fragmentation has increased 28% over last year.

When it comes to iOS the scene is relatively better. Below is the screenshot of iOS to Android comparison in terms of fragmentation.
Probable solutions to device Coverage:  The good news is there are lot of vendors offering device farm and to name the prominent ones:
  • Xamarin Test Cloud Service
  • Cloud Test Lab by Google
  • AWS Device Farm
  • Testdroid

But the not so good news is most of these are paid service and the charges for engaging a device could go up to $240 per device per month (on AWS device farm for example). Till device farm became easily accessible and popular, folks were majorly testing on emulator and some handful of devices. Beta testing phase used to be a little longer and most of the bugs were found during that and field testing of app.
Now that device farms brings you plethora of devices but the concern is every minute of a device in a farm costs you.  What should be your strategy for mobile testing and automation given the limited testing time you have and the money to spend on?
Contextual information about the app:  A lot in your strategy would be shaped by what kind of app it is ? Is it a B2B or B2C ? Is it a hybrid app or native app? Do you know your targeted customer segment ? Do you know the age segment of those customers ? Possible usage patterns  of your app?
Once you have these info, you could potentially decide what are the typical devices which might get used.  There are other set of questions which can definitely lead you to some pointers to test strategy for mobile apps.
  • Should you invest in devices locally ?
  • What should be your emulator configuration to run test?
  • Can your functional test run in parallel ?
  • Which section of functional test you should run for device coverage?
  • What devices you should opt when using device farm ?
  • When should you run your tests on Device Farm ?
Probable approach:  We have tried to diffuse how you can help shape your mobile testing strategy in the below picture. It’s more to prepare you and open you up for discussion which should help you arrive at your device testing strategy.
Apart from the test strategy you also need to have a highly optimised code where you are handling “wait” smartly in your test code. Remember that with each minute of your test execution on actual device, there is a cost associated.
Want us to help you with your mobile testing strategy and automation, reach out to us.

Popular posts from this blog

The 3 qualities that will always help you become better at almost anything…

Image
Have we ever noticed how excited we are when we start anything? But it’s a sad fact that 70% of people abandon what they have started with so much energy and love. It could be as small as building a hobby or starting a pet project or even an activity or a side learning we started. “ Well begun is half done .” Aristotle did say this but we are judged for how we have finished something. You separate yourself fro m  the masses when you choose to give that task a finish.  The first quality is finishing off what you started and it helps develop mental stamina or willpower.   It builds confidence and inculcates as a habit in a long run. You don’t need to sign up for the mammoth activity to build this habit, even finishing mundane day-to-day routine tasks will help. Diligence — the quality of working carefully and with a lot of effort Diligence is a quality that almost never will get unnoticed. It’s one of the underlying traits of grea...
Read More »

Embracing the Journey: Finding Joy in the Process

Image
 In our fast-paced world, it’s easy to get caught up in chasing our goals. Whether it's a career milestone, a fitness target, or a financial aim, we often fixate on the end result. But recently, I came across a quote by Sal Di Stefano that struck a chord with me: “The man who loves walking will walk further than the man who loves the destination.” Rediscovering the Joy of the Journey This quote reminded me of something profound – the importance of loving the journey, not just the destination. When we enjoy the process itself, we find a deeper, more sustainable source of motivation. This mindset can be a game-changer, not just professionally but personally as well. A Personal Anecdote: My Professional Journey Let me share a bit of my own experience. As a software engineer turned entrepreneur, I initially focused heavily on the end goals: launching the company, acquiring clients, and hitting revenue targets. But over time, I realized that what kept me going during tough times was not...
Read More »

How to install Android Emulator on Windows

Image
As part of my testing work, I am currently working on using webdriver to test on Android. So I had to install Android Emulator on Windows 7. Below are the sequential steps and I hope it will ease some pain on getting Android emulator on your Windows 7 box. Download the installer from http://developer.android.com/sdk/index.html Install it to a location (like: C:\Program Files\Android) Launch “SDK Manager.exe” It will launch a “Android SDK and AVD Manager”. By default “Installed packages” will be highlighted. On the pop up “Choose Packages to Install”, there will be some packages selected already. Go ahead with the “Install”. 4.    Once it is done, select “Virtual Devices” to create one for you. Click on “New” and enter some name. Target as “Android 2.2 – API Level 8”. Select Size as “512” and “ Built-in ” to be “WVGA854” and click on “ Create AVD ”. 5. Once done select the Virtual device you created and click on Start . You can find the “Start” button on t...
Read More »