Sunday 13 April 2014

Learn Mobile application testing in 10 minutes.

Can you imagine a day without mobile phones? Mobile phones have become an integral part of our daily lives. Initially, only Symbian phones were available in the market. After which, came the iPhone , Windows 6, Blackberry  and Android 1.5 phones which took over the market . Recently OS has been updated in iPhone 5 with iOS 6, Android 4 +, Windows 8 and Blackberry 7 + OS. This was followed by Pad1, 2, 3, iPad mini, Galaxy Tabs, Nexus, Kindle Fire, Xoom, BlackBerry playbooks and Android tabs to name just a few.

Mobile Comparison 

What is mobile application testing ?
Mobile application testing is the process of testing the functionality, usability and consistency of an application for hand-held mobile devices, from pre-installed or can-be-installed mobile application software platforms like iTunes, Google Play, etc.

Why is Mobile Application Testing Necessary
With so many mobile applications in the market today, it is important that providers ensure that the quality of their apps is high in order not to lose out to competition. The quality of the apps brought out can make or break the product or business completely. With competition getting hotter by the day, the pressure to build, test and launch apps has never been greater. Companies need to test their apps across models, carriers, OS, browsers and location. It is a job that should be performed by experienced mobile application testers. Testing ensures that companies streamline their testing, save time and money and deliver great user experience through their mobile channels.

The reasons why mobile application testing is required :
  1. Many types of mobile devices
  2. Different types of mobile platforms/operating systems  from various companies
  3. Different mobile carriers
  4. Many types of mobile screens
Type of Mobile Applications
Native Application: A native application (native app) is an application program that has been developed for use on a particular mobile device i.e smartphones or tablets. It is installed directly onto the device or acquire these apps through an online store or marketplace such as iPhone on  App Store or Androids on Google Play. Native Apps divided into pre-installed applications and installable applications.
Browser Based Mobile Application (Mobile Web) : Applications which we can acess through the mobile browser by hitting the url of the web. This type of application includes HTML 5 also.  Example – m.qburst.com, www.google.com, m.redbus.in, etc

Types of mobile application testing
  1. Functional Testing
  2. User Interface Testing
  3. Usability Testing
  4. Performance Testing
  5. Stress Testing
  6. Compatibility Testing
  7. Interruption Testing
General testing scenarios in mobile application testing
  • Connectivity testing- Identify the network connection. 3G/Wifi/Edge
  • Compatibility testing- Check the application with different OS and devices
  • Cross platform testing -Identify the OS and also identify if the device has any cross platform OS installed. Example- 3Gs phone with 4 iOS
  • Orientation testing- Test the Application with landscape and portrait mode
  • Interruption testing- Test the application with following interruption
    • Incoming call
    • Receiving incoming call
    • Receiving message
    • Device shutdown
    • Remove battery
    • Camera activated
    • Lose network connectivity and then regain it
  • Installation / Un-installation
  • Test the application with/without network connection
  • Check whether proper error/success messages are displayed under the following scenarios
    • Network connection not available
    • Weak signal or unable to complete the task
    • Posts to any social media network or send email
  • Navigation testing- Check the navigation of the application. User should not get stuck anywhere within the application
  • Web service testing
    • Identify the web service language
    • Identify the web server location (for checking time zones and so on)
  • Idle testing – Test the application in phone’s idle mode
  • Share/Post testing – Try to send/post the email/post/comment without configuring Gmail/FB/Twitter account. Try the same after configuring the accounts too
  • Volume testing – Test the application with large amounts of data
  • Stress Testing – Test the application continuously over sufficiently long periods of time
  • Load Testing – Check the application with load (web services). Various tools are available online for serving this purpose
  • Sync testing – Check the application in Sync intervals
  • Usability Testing – Testing usability aspects of mobile application
  • Universal application in testing – Testing same build in different devices such us tablets and phones
  • Simulator testing – Test the app in both simulator and device
  • Performance testing – Check application performance and phone memory using performance test tool, memory leak tool etc. from Xcode
  • GPS testing – Set up mock GPS for GPS related functionalities
  • Monitoring battery consumption –  Check battery consumption over long term usage in background/foreground applications
  • Application side effects – Ensure that installation of a new app does not damage already existing applications or the device itself
  • Security – Ensure security of the application
Some Challenges in Mobile Application Testing
Today, mobile applications deliver complex functionalities on different platforms that have limited resources for computing. Mobile applications can either be standalone applications or web based mobile applications.

Screen Size – Screen size is one of the biggest limitations for mobile devices. The screen size varies from 128 X 128 to 800 X 480. Smaller screens have a portrait orientation and larger screens have a landscape orientation. There are than phones that can change their orientation – meaning they are capable of working in both landscape and portrait modes. 240 x 320 is the overall dominant screen size so far. Small screen resolutions tend to makes web pages almost illegible.

Display Resolutions – Different mobiles have different resolutions. A low resolution can degrade the quality of multimedia displayed on the screen of mobile device.

Processing capability  - Limitations in processing speed and memory size of mobile devices is a major issue faced during testing. Phones sometimes allow only a single active process in them. Your applications may fail if something like opera-mini is running in the background and you are trying to communicate out.

Diversified platform and devices – An application might work in one phone model and may not in the very next model from the same company. Just because you’ve tested in one phone model does not mean it will work on other available models. The jar size limits on these phone models can vary. Some popular models don’t really allow big downloads. Space is always at a premium so you will have to be at a constant lookout for the download size of your application and flag it when it increases.

Type of  content – Content delivery bothers both mobile user and carrier. For example, when page size is large, the device will not be able to handle it. Page should be cropped or re-sized and delivered without losing relevant information. The page can also be divided into fragments and displayed along multiple pages. Scrolling to read documents or web page will not be user friendly.

Connectivity – Network connectivity largely decides the data download time and also the quality of streaming media. Slow and unreliable wireless network connection with low bandwidth is a common hindrance for mobile applications. Carrier or device should be GPRS, Edge, 3G or 4G compatible.

Data Input Methods – Mobile devices come in two flavors – soft keyboards/touch screens and physical keyboards. Small buttons and labels limit the user’s effectiveness and efficiency in entering data. This results in slowing down the input speeds and increasing the chances of error.

Device screen flip capabilities – Page layout or content will change when user access page in different screen modes.
  Usability factors - User friendliness, self explanatory features, browser compatibility, help, etc
Challenges to Mobile Testing
Tools Available for testing
Along with manual testing, we need to think about automation testing in mobile application. Since mobile testing is a recent scenario, very few automation tools are currently available. However,  many more automation tools are expected to appear in the market.  Some of the popular tools currently available in the market are EggPlant,  Phonemonkey and Selenium- Robotium.
Here are some automation tools available for mobile apps testing:-
  • TestComplete
  • Test Quest
  • Robotium
  • VNC
  • Sikuli
  • Deviceanywhere
  • Keynote
  • FoneMonkey (iPhone)
  • Eggplant
  • TestiPhone( For iPhone Mobile Web)
  • IBM® Rational® Performance Tester (RPT)
  • 3P Mobile
  • Expertest
  • Mobile Interactive Testing Environment (MITE) for  Mobile Web app
  • Utest
Simulators and Remote Device Access (RDA) Services
Due to the lack of device shortage, a lot of simulators are available for testing. It represents more of the end-user scenario, which makes the use of these simulators important.
We need not purchase a new device with every new update or new device launch. This is where simulators become reliable. RDA’s are good solutions for application testing.
Some popular RDA Services are:
  • Device Anywhere (Featured)
  • Perfecto Mobile (Featured)
  • Nokia RDA (Free, For Symbian Phones)
Advantages
  • User can select different carriers  e.g.  Verizon, T-Mobile, AT & T.
  • RDAs are more reliable than simulators as they are real devices.
  • Some RDA Service have automation capabilities (Device anywhere)
Disadvantages
  • Long time for actions
  • High cost
  • Lack availability of RDA’s
Online Simulators for mobile web
iPad Simulators
  • Alexw.me
  • iPadPeek.com
  • iPad-emulator.org
iPhone Simulators
  • TestiPhone.com
  • InteractiveiPhone.com
  • iPhone-Emulator.org
  • iPhone4Simulator.com
  • Transmog.net
  • iPhoneTester.com (use safari for better results)
General Mobile based Opera Mini Browser Simulator
  • Opera Mini Simulator (not specifically for an iPhone or iPad)
Just like personal computer and internet, mobile devices have become a “game changer” in this technical era. This article attempts to focus on certain aspects of mobile application testing, which is a must follow scenario in Mobile application development process.

Source - http://blog.qburst.com/2013/01/mobile-application-testing/

The Ins & Outs of Mobile App Testing

Over the last decade, application testing has continually proved itself to be an important concern. When done well, testing can drastically reduce the number of bugs that make it into your release code (and thus actually affect your users). In addition, good testing approaches will help your team catch bugs earlier in the development lifecycle – resulting in a savings of both time and money (not to mention reputation with your users). Code that has good test coverage enables you and your team to make changes and introduce new features to your app without the fear of it breaking existing functionality.
shutterstock_131210207
The word “Testing” is a large umbrella, and is usually better understood when you break it down to specific types of testing. For example:
  • Unit Testing – automated tests written by developers, with each test targeting a narrow slice of application behavior.
  • Functional & Acceptance Testing – typically performed by QA personnel or automated UI testing frameworks.
  • Performance Testing – often performed manually with profiling tools (for example – heap and CPU profiling tools), though many mobile app developers are moving towards integrating app analytics to gather this data from real usage as well.
That’s certainly not an exhaustive list of the types of testing, but enough to make an important point: mobile applications face several challenges when it comes to testing. Key among those challenges are:
  • Platform & Device Diversity
  • Immature Tooling
  • Lack of Awareness
If you opt to write native applications for each target platform, then any code-level testing (i.e. – Unit Tests) will not be transportable as you move from Objective-C (iOS) to Java (Android). In addition, any scripted UI-Automation testing tools may not work for multiple platforms (or at the very least require separate scripts for each platform). Hybrid solutions like PhoneGap, or cross-compiled solutions like Xamarin can offer a single approach to unit testing (given a single codebase for multiple platforms) – but do not always offer the same level of quality as native tooling when it comes to performance profiling. Despite the trade offs involved, I’ve found in my own experience that the biggest barrier to entry in mobile app testing is often a lack of awareness of what tools are availableThat is the barrier which I hope to address in this post.

Unit Testing

Unit testing for specific platforms or cross-platform tools is not difficult, and your options abound. Let’s look at a sample of some of these choices.

iOS

iOS developers who’ve been writing Objective-C for a while may be familiar with OCUnit, which shipped with XCode prior to the XCTest framework. It’s still supported in XCode 5, but the understanding is that new and future projects should focus on using XCTest.
Don’t let Apple’s sparse documentation on unit testing deter you from checking out the XCTest framework. If you’re running an OS X Server, you can also take advantage of theXCode service’s continuous integration features. As part of a continuous integration workflow, you can create “bots”, which can continually build and test your app.
Many developers prefer a Behavior-Driven-Development (BDD) style syntax for unit testing. If this describes you, be sure to check out Kiwi – a BDD style unit testing framework for iOS.
One other important mention is OCMock a mocking framework for iOS. Mocks are an indispensable part of writing adequate tests around your application’s behavior.

Android

JUnit is perhaps by far the most well known (and officially recommended by Google) testing framework for Android. The JUnit Android extensions allow you to mock Android components, but I’ve also seen quite a number of Android developers use JUnit withMockito, another Android mocking framework.
Robolectric takes a different – and very interesting – approach by allowing you to run your Android unit tests in the normal JVM (Java Virtual Machine), without the need for an emulator. This enables your tests to not only run from within your IDE, but also as part of a continuous integration workflow.

Qt

Qt made the top 5 most used CPTs in 2013. If you’re building mobile applications with Qt, you’ll be happy to know about QTestLib, a unit testing framework built by Nokia. Based on my research, it appears that QTestLib can be integrated with a 3rd party continuous integration workflow – enabling very helpful testing automation.

PhoneGap/Apache Cordova

Web-based hybrid approaches to mobile apps can take advantage of a host of testing and mocking frameworks, not to mention scripted UI/acceptance testing tools as well (more on that in a moment). When it comes to unit testing JavaScript, three of the biggest names are QUnitMocha and Jasmine. I’ve personally used all three, with my favorite setup including Mocha and expect.js (which provides a BDD style test syntax). Mocking and “spy” frameworks abound in JavaScript as well, with Sinon.js and JsMockito among the more popular stand-alone mocking options.
Many PhoneGap developers take advantage of tools like PhantomJS – which is a “headless” (no UI) WebKit browser, with a JavaScript API. PhantomJS can be easily integrated into a continuous integration workflow to automatically run unit tests against your hybrid mobile application’s codebase.

Xamarin

Xamarin uses a customized version of NUnit (ported from JUnit), called NUnitLite which enables you to write unit tests against your Xamarin iOS & Android projects. For any shared codebase, you can use the unit testing framework of your choice.

Scripted UI Testing

Not every team can afford to hire an army of manual QA testers, despite how valuable that can be. Automated tooling can bridge the gap.
If you’re writing native iOS and Android apps, you’re in luck. Apple provides an“Automation instrument” that will automate UI tests against your iOS mobile application. The Android SDK provides the “uiautomator” library, described as “A Java library containing APIs to create customized functional UI tests, and an execution engine to automate and run the tests.” In addition to these, you can use third party tools likeSquishRanorex and Perfecto Mobile’s MobileCloud Automation to automate UI tests against Android and iOS apps, web apps and more. It’s worth mentioning that Perfecto Mobile’s MobileCloud Automation exposes an API to better facilitate integration with existing build/continuous integration tools. Perfecto Mobile also offers MobileCloud Interactive, which enables you to “perform remote manual testing on real smartphones and tablets regardless of where you are” – who wouldn’t want to have a “testing army” of real mobile device users at their disposal?
Among the more interesting developments in mobile UI automated testing is the emergence of an open source project named Appium. Appium uses the WebDriverJsonWireProtocol to interact with iOS, Android and Firefox OS apps and gives you the choice of writing your UI tests in any WebDriver-compatible language (Java, Objective-C, JavaScript, PHP, Python, Ruby, C#, Clojure, Perl and others).

Performance & Profiling

Apple’s Instruments is one of the more impressive native toolsets I’ve seen recently. With Instruments, it’s possible to profile how your app executes, run stress tests, record and replay user actions, create custom instruments and a lot more. If you’re writing native iOS apps & not using Instruments, I recommend reading through the Quick Start to get up to speed.
With Android apps, you have several (albeit, lower-level) tools available: Systrace &Traceview. You can also use the Device Monitor to view memory usage based on logcatmessages.
For hybrid mobile apps, you have a host of mature desktop browser tools (Chrome Developer ToolsFirefox/Firebug, etc.), which you can bring to bear on your app to profile CPU usage, memory, DOM manipulation and much more.
Many mobile developers have started taking advantage of third party analytics services such as Google Mobile AnalyticsCountlyEQATECFlurryPerfecto Mobile’s MobileCloud Monitoring and many others. The focus of these kind of analytics is usually more about how your app is actually used, user engagement, demographics, feature popularity, etc. However, it provides an opportunity to measure certain pieces of application performance from within real-world usage. While I wouldn’t recommend this being your first line-of-defense in performance testing, having the ability to track real world performance metrics can be a powerful tool in tuning your application to your users’ needs.
We’ve only scratched the surface of the various testing options available for mobile app development. What testing approaches & tools are you using when writing mobile apps? If you’re not currently testing your application, what are some factors that would change your mind?
Source - http://www.developereconomics.com/testing-mobile-applications/