Mobile Web/App Testing Tools

AWS Device Farm – App testing service from Amazon Web Services that lets you test and interact with Android, iOS, and web apps on many physical devices at once, or reproduce issues on a device in real time. View video, screenshots, logs, and performance data. Gesture, swipe, and interact with devices in real time, directly from your web browser. Run the AWS built-in test suite (no scripting required) or customize your tests by selecting from open-source test frameworks like Appium, Calabash, Espresso, etc. Manual testing also.

Automate-On – Automate functional, visual and performance testing of web, mobile applications in multiple Browsers, Devices, OS and Screen Resolutions. Executes test scripts on cloud machines, mobile devices and removes the need for customers to procure, maintain computer systems and mobile devices for testing. Provides comprehensive reporting; Targeted Platforms/OS: Windows, Mac OS, IOS, and Android.

Selendroid – Open source test automation framework which drives off the UI of Android native and hybrid apps and the mobile web. Tests are written using the Selenium 2 client API. Can be used on emulators and real devices and can be integrated as a node into the Selenium Grid for scaling and parallel testing. Gestures are supported via the Advanced User Interactions API. Includes a built in Inspector to simplify test case development.

ios-driver – Open source tool for testing IOS native, hybrid, or mobile web apps using the Selenium/Webdriver API. Includes ios-driver Inspector to examine native app elements, similar to Firebug. Can be used as a Selenium grid node – run tests in parallel on the same architecture as for the web.

KIF – KIF (Keep It Functional) is an open source iOS integration test framework. Enables automation by leveraging the iOS accessibility attributes. Builds and performs the tests using a standard XCTest testing target. Like some other iOS test tools, uses undocumented Apple APIs, thus is appropriate for testing but not prod code. Integrates directly into your Xcode project; tests for KIF are written in Objective C.

Mobile-Friendly Test – Free online test tool from Google analyzes a URL and reports if the page has a mobile-friendly/responsive design. Reports on issues found and suggest mitigation approaches. Extensive mobile/responsive advice and documentation is available via the site.

Remote TestKit – Real device test cloud from NTT Resonant Inc. Enables manual and automated testing on hundreds of real iOS and Android smartphones and tablets. Integrates with Xcode for iOS and Eclipse for Android. After each test run all apps and data are wiped from devices and they are automatically re-initialized. Supports Selenium Webdriver, Jenkins. Image and video capture functions include Simultaneous Capture Function to enable simultaneously viewing of multiple URLs specified in advance on multiple rented devices. Real-time Clipboard Synchronization capability enables copying and pasting text that has been prepared in advance on your computer, or copying text from device to local. Can operate test device using your smartphone as a ‘remote control’ controller.

TestObject – Real device test cloud from TestObject GmbH, with more than 150 devices, primarily Android but some iOS also. Access real devices from your browser for manual and automated testing. Native, web and hybrid apps are supported. Other capabilities include screenshots, switching system language to test internationalization, support for testing frameworks including Robotium, Espresso and Appium; a REST API, Ant and Gradle plugins. Reports include device specifications, logs and screenshots. Free and $ plans.

Frankk – Open source framework from Thoughtworks for writing structured text tests (using Cucumber) and have them execute against your iOS application. ‘Selenium for native iOS apps’ (not for pure web apps). Requires installion of a ruby gem, and compiling/linking the Frank server into the app being tested. Includes a powerful “app inspector” (called Symbiote) that can be used to get detailed information on the running app. Run tests on both the Simulator and Device.

TestFire – SDK for untethered video and app log recording on iOS and Android, from Testfire Inc. Install SDK in your build, SDK installs in minutes with CocoaPods or Gradle. Distribute app for UAT, beta, dogfooding etc, then let users record experience, feedback, and issues. Get video, app logs, and device details with every recording, including gesture recording, screenshot annotation, system log capture (logcat), custom log capture (Lumberjack, Logger, etc) synchronized with screen recordings. Includes Device Detail Capture with Model, OS, Carrier, Battery, Location. Video replay with synchronized log streaming. Free and $paid options available.

Monkey – Free command-line tool from Google that runs on your emulator or device and generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as a number of system-level events. Enables stress-testing of apps that you are developing, in a random yet repeatable manner. If the app crashes or receives any sort of unhandled exception, or if it generates an application not responding error, the Monkey will stop and report the error;

MonkeyRunner – Free tool from Google provides a python API for writing programs that control an Android device or emulator from outside of Android code. Can write a Python program that installs an Android application or test package, runs it, sends keystrokes and touch events to it, takes screenshots of its user interface, and stores screenshots on the workstation. Oriented to testing of apps and devices at the functional/framework level and for running unit test suites. Can apply one or more test suites across multiple devices or emulators. You can physically attach all the devices or start up all the emulators (or both) at once. Can be extended with plugins.

Android Testing Support Library – Framework from Google for testing Android apps. A set of Java APIs available through the Android SDK Manager that enables buidling and running app test code, including JUnit 4 and functional UI tests. Tests can run from the Android Studio IDE or from the command line. Includes AndroidJUnitRunner: JUnit 4-compatible test runner for Android; Espresso: UI testing framework for white box functional UI testing within an app, and UI Automator: UI testing framework for black box functional UI testing across system and installed apps.

Android Lint – Free downloadable static code analysis tool from Google that checks your Android project source files for potential bugs and optimization improvements for correctness, security, performance, usability, accessibility, and internationalization. Runs from command line or Android Studio.

Calabash – Free open source framework anables writing and executing automated acceptance tests of mobile apps using Cucumber and Ruby; from Xamarin Inc. Cross-platform, supporting Android and iOS native apps. consists of libraries that enable test code to interact programmatically with native and hybrid apps. Actions can be gestures, assertions, screen shots

Xamarin Test Cloud – Provides a locally executed, powerful object-based scripting environment for imitating and automating actions a real user would take through a mobile app on iOS or Android. Provides test device cloud, with over 1,800 real, non-jailbroken devices to test on. Test scripts can run in parallel on hundreds of devices at a time. Test Scripts can be built in Calabash (Ruby) and C# using Xamarin Studio or Visual Studio. Share code for cross-platform tests between iOS and Android. Screenshots and video playback for every step of every test; performance data (memory, CPU, duration, etc.). Integrates with any CI systems.

TestMunk – On-the-web mobile app test automation runner enables running BDD-style testcases on virtual emulator, your plugged in device, or on a large pool of Testmunk Android and iOS versions and devices; native and hybrid apps supported. High-resolution screenshots of each teststep are taken across all devices; image-comparison capabilities. Upload your app file and testcases from Jenkins, Travis or CircleCI and start testruns automatically with the Testmunk API. Provides over 50 BDD-style teststeps available from the test step library for both iOS and Android; can extend and write custom teststeps. Uses Ruby. Can run your tests locally against an emulator or on one device at no cost or on various devices in the Testmunk cloud. Requires the Calabash gem. Free basic or $ plans.

Crashlytics – Free set of tools from Twitter, implemented by instrumenting your iOS or Android app, enables reporting of aggregated as well as detailed information on app crashes in the field during beta testing etc. Includes details about the devices that your apps run on including information such as whether a crash only happens on a specific model or generation of a device, if app only crashes in landscape mode, whether the proximity sensor is always on, is it a memory issue, an issue with specific versions, etc. as well as reports on deep analysis of stack traces. Tool set includes ‘Beta by Crashlytics’ for managing and distributing beta apps via a single, cross-platform toolset for iOS and Android, including tracking testers progress and issues. Also includes ‘Answers’ kit which provides critical performance metrics on your app, detailed growth and engagement indicators, etc based on the set of core events and actions of most interest. Included as part of the ‘Fabric’ toolset.

UberTesters – Free set of tools from UberTesters Inc. for managing/reporting of iOS/Android app crowdsourced/beta testing. Requires the Ubertesters SDK which can be integrated with many frameworks used for cross-platform development. Once the app is built with the integrated UberTester SDK, distribute your builds/revisions to specified testers using the Ubertesters app that is installed already on the tester’s device. Capabilities include in-app bug editing, marking, reporting and user feedback; trcking of all testing sessions in real-time sorted by devices or testers; real time reporting of status of a device and whether it is active, closed or suspended, along with additional device-specific session information and logs; configurable as to which data is captured and conditional logic for filtering. Can integrate with many 3rd party bug tracking systems or use the included bug tracker.

TestFairy – Set of tools from TestFairy for managing/reporting of iOS/Android app crowdsourced/beta testing. Does not require an SDK or code changes to the tested app. Provides video that shows everything that happened on the mobile device, including detailed internal metrics such as CPU, memory, GPS, logs, and crash reports as well as detecting ANRs and alerting when exactly apps behave abnormally. Can integrate directly with Jenkins / Gradle, or write your own plugin with the open API. Catches crashes and posts information directly to your bug system together with a video showing what happened prior to the crash. Testers can submit bug reports via multiple methods including from within the app by shaking their phone. Includes Android Heatmaps that show via color heatmaps exactly how users interact with the app on Android.

HockeyApp – Set of tools from Microsoft for managing/reporting of iOS/Android/WinPhone/OSX app crowdsourced/beta testing. Requires integratiing the open source HockeyApp SDK for Android, iOS, Mac OS X, Win, Unity, Xamarin; then uploading your app build to HockeyApp. Testers can then download and install the app and then create support requests, file bugs, or post ideas right from within your app; test team management features; can send crash reports automatically or with user interaction or add custom metadata and log files. Reports include metrics showing which devices were tested, which testers used the app for how long, which language was tested, etc. Data export API for connecting to your own or third-party services.

TestDroid – Suite of Android application testing tools for test automation and manual testing, from Bitbar Inc. Testdroid Enterprise is an in-house test server, implemented as a Jenkins plugin, for executing your tests internally. Testdroid Cloud is an online service providing thousands of different real Android and iOS devices on which to run your tests. All test runs start with a device reboot, ensuring tests are executed on clean devices with no interfering processes running. Your application and test package are installed onto the device and tests can then be executed. Includes a web service available through a web browser or as a REST API. Testdroid Recorder is an Eclipse Plug-in record/playback tool for creation of automated test cases for Android.

eggPlant Mobile – An image-based, low-overhead, non-invasive black-box test automation tool for mobile. eggPlant Mobile is the combination of eggPlant Functional and eggOn – the mobile agent that enables eggPlant Functional to fully test mobile apps on iOS, Android, Windows Phone, BlackBerry, and other mobile devices.The image-based UI testing approach can reduce the effort of cross-device and cross-platform mobile software testing.

Monkop – A cloud-based automated Android testing service using real devices, provides insights about performance, security, usability, and functionality over a large device lab containing representative brands, OS versions, screen sizes, and configurations. Requires only an upload of your app .apk. Utilizes automatic learning, monkey testing and application disassembly techniques in order to run different levels of tests on different devices. Reports include response time and resource consumption (cpu, memory, data transfer, battery, etc.), security scan results and suggestions, results of exerciing app looking for crashes, compatibility and layout issues. Verifies that your .apk follows Android and Google’s best practices, indicating issues and how to mitigate. Can also run your own automation scripts for each device. Has APIs to integrate with CI.

Maveryx – Test automation tool for Android and Java apps from Maveryx. Utilizes ‘GUI Objects Finder’ that recognizes GUI test objects with text at execution time. The object search engine supports exact and fuzzy matching algorithms to identify test objects in the UI, even in case of partial or approximate matching, misspellings and synonyms or if the objects have changed since test creation. Supports data-driven tests via Excel, CSV and XML files and data retrieval from any database and from INI, TXT, HTML files and others. Support for keyword-driven testing through Excel spreadsheets and XML files; offers a rich set of built-in keywords to rapidly develop robust test scripts. Supports JUnit assertion-based verification points, distributed testing, plugin for Eclipse, creation of custom extensions/plugins. Free and $paid versions. Open source version written in java from 2011 is available on Sourceforge.

Mobitaz – An Android/iOS test automation tool from MSys Technologies; features include native app and Web test automation, end-to-end automation, etc. Test cases can be easily created, without needing the source code.

Clang Static Analyzer – Open-source source code analysis tool that finds bugs in C, C++, and Objective-C programs. Can be run either as a standalone tool or within Xcode; intended to be run in tandem with a build of a codebase. Implemented as a C++ library that can be used by other tools and applications.

Roboelectric – An open source unit test framework that modifies Android SDK classes so you can test your Android app inside the JVM on your workstation in seconds.

TestArchitect Mobile Plus – Test automation tool from LogiGear for native/hybrid apps on Android/iOS, and Web, Windows and Java applications. Tests can be executed on multiple local devices via USB and Wi-Fi, or on devices hosted by cloud-based mobile testing partners. Multi-touch gestures, access to the physical device buttons, and command-line execution are fully supported. Image recognition allows for testing of standard apps as well as games with fast, 3D, interactive graphics. A small footprint communication client is placed on the mobile device. Create objective-driven test flows by utilizing either the action recorder, by action drag & drop from the built-in library, or by simply typing in actions into the easy-to-use spreadsheet-like test editor. Supports IE, Firefox, Chrome, Safari, WebView, HTML5. Custom actions can be created using C#, C++, JAVA, or Python.

Appium – An open source test automation framework for driving native and hybrid iOS and Android mobile apps using the WebDriver JSON wire protocol. Supports a sub-set of the Selenium WebDriver JSON Wire Protocol, and has extensions for automating mobile gestures like tap, flick, and swipe. Write tests in Ruby, Python, Java, etc. As of mid-2013, requires Mac OS X 10.7 or higher, Linux OK for Android-only; Win support is in “beta”; Node and npm (brew install node). For iOS automation requires XCode and Apple Developer Tools (iPhone simulator SDK, command line tools). For Android automation requires Android SDK API >= 17.

Android/iOS Source Code Analysis – Static code scanning tool from Checkmarx provides the ability to find vulnerable lines of code and learn how to fix them, for Android and iOS app source code (also capable of analyzing many other code languages).

Soasta TouchTest – Functional test automation from Soasta, includes capabilities for multi-touch, gesture-based mobile apps. Capture/playback: captures the start and end points of each gesture, the journey between and the speed with which the gesture is performed. Can validate tests using internal variable values and conditionally wait for internal application state changes. Comes with rich set of test validations and object location strategies that can be extended via JavaScript. All actions and gestures captured at object level. Controls mobile devices through a lightweight software agent and accesses them using their IP address, eliminating need for tethering, jailbreaking or contracting with external device testing services. Enables any mobile device at any location, to be used for testing. Devices can be dedicated to testing in lab, used as part of a short external test, or crowd-sourced as part of globally distributed test.

Adobe Edge Inspect – Preview content across multiple mobile devices. Wirelessly pair multiple iOS and Android devices to your computer, grab screenshots from any connected device, and see real-time results from changes to HTML, CSS, and JavaScript. For Android, iOS, Kindle Fire. For Adobe Creative Cloud.

SeeTestMobile – Mobile application test tool from Experitest Ltd., for iOS, Android, Blackberry and WindowsPhone. Can be used on both emulators as well as real devices and covers visual testing, functionality testing and speed performance. Incorporates image recognition and self-learning algorithms. Test recording can take place utilizing real devices – plug real device in to desktop via USB. Utilizes self-learning diagnostic and matching algorithms and a modular self-enhancing image recognition technology. Editable scripts using included IDE; scripts can also be exported and edited in QTP, TestComplete, VBScript, C#, Java, Perl, Python. Runs on Windows, Linux, or Mac. Other mobile-related tools available include SeeTest Cloud, SeeTest Network Virtualization, Mobile Addon for UFT.

Ranorex Cross-Device Mobile App Testing – A Windows GUI test automation framework from Ranorex GmbH for testing many different application types including mobile (iOS, Android, Windows8 UI) test automation as well as mobile web test automation (iOS). Directly record tests on your device. Does not require you to jailbreak, unlock or root your device – just execute the recorded tests on your device or emulator. The IDE includes test project management, integration of all Ranorex tools (Recorder, Repository, Spy), intuitive code editor, code completion, debugging, and watch monitor.

M-eux – Mobile apps test automation software from Jamo Solutions; recognizes the GUI elements of the mobile device resulting in test cases that are device model independent and re-useable across different versions of the service/application under test. Integrates with existing environments such as Eclipse, QuickTest Professional and Visual Studio.

KIF iOS Integration Testing Framework – Open source iOS integration test framework enables automation of iOS apps by leveraging accessibility attributes that the OS makes available for those with visual disabilities; uses undocumented Apple APIs (like many iOS testing frameworks). Tests are written in Objective C, allowing for maximum integration with code while minimizing layers to build. Integrates directly into iOS app, so no need to run additional web server or install additional packages. Automation done using tap events where possible.

UI Automation – iOS Developer Library – An Apple JavaScript library to write test scripts that exercise an iOS app’s UI elements while the app runs on a connected device. Write tests in JavaScript, calling the UI Automation API to simulate user interaction; returns log information to host computer. Also see the UI Automation Javascript Reference.

GSAutomation – An open source Javascript extension/wrapper on iOS UIAutomation to make test scripts more robust and easier to write and maintain; contributed by Hulu. Can define test tasks in simple javascript arrays, and have them execute with helper methods.

Instruments for Xcode 4 – Part of Apple’s XCode developer toolkit for building Mac iOS (and OS X) applications. Many testing, profiling, and analysis capabilities include enabling easy creation of ad-hoc test harness by recording and playback of user interactions, OpenGL ES for tracking iPhone graphics performance, memory allocation monitoring, Time Profiler on iOS for collecting samples with very low overhead, complete System Trace for insight into how all system processes interact, more. Also in XCode is iOS Simulator, which enables running an app similar to the way it would run in an actual iOS device; can check that network calls are correct, and that views change as expected when phone rotates; can simulate touch gestures by using the mouse.

CQ Lab – The Continuous Quality Lab (CQ Lab) from Perfecto Mobile is a cloud-based web and mobile app testing platform made up of solutions that enhance building, testing, and optimizing monitor app usability and performance. Perform side by side functional and real-user condition testing across thousands of devices. Leverage the automation frameworks and IDEs you already use, including Selenium, Appium, UFT, Eclipse, and Visual Studio.

Digital Test Coverage Optimizer – Tool from Perfecto Mobile to help select devices to test your app(s) against – generate a prioritized list of the mobile devices you should test against. Select your target location(s), device type(s) and OSs, and the Optimizer will do the rest. The test coverage grader helps build a custom mobile app test strategy.

Robotium – Open source Android test automation framework from Renas Reda/Hugo Josefson/Jayway; available as java source or jar file. Can integrate with Maven or Ant to run tests as part of continuous integration. Supports Android 1.6 and up.

TestQuest CountDown – Mobile test automation tool from BSquare. Enables collaboration and asset sharing among distributed test teams. Organized into 4 components — TestDesigner, TestManager, TestRunner and AssetManager; platform independent.

Intent Fuzzer – Free Android testing tool from Isec Partners; often finds bugs that cause the system to crash or performance issues on the device. The tool can either fuzz a single component or all components. It works well on Broadcast receivers, and average on Services. For Activities, only single Activities can be fuzzed, not all them. Instrumentations can also be started using this interface, and content providers are listed, but are not an Intent based IPC mechanism

Keynote Mobile Testing – Cloud-based real mobile handset/tablet testing platform, from Keynote Systems, enables development, deployment, and testing of content/apps on hundreds of real handset devices in live networks around the world. The mobile handset bank includes devices stationed in the US and many other countries and the agnostic platform hosts a diverse portfolio of carriers and manufacturers from around the world. Interact and control every aspect of mobile device testing using patented Direct-to-Device® technology. Any action that can be performed on a mobile device in-hand can be replicated on devices in the cloud. Automation capabilities provide flexible scripting that enables building of test scripts that work across multiple devices for faster test automation. Supports object level including native, web, text, and image. Quickly develop scripts using GUI-based commands, Java API, other test framework including HP UFT, IBM Rational, Selenium, or direct support with Jenkins for continuous integration. Device planner tool to help prioritize device choices. Free and $ versions.

Dynatrace Real User Monitoring for Mobile & Web Apps – An on-demand mobile monitoring solution from Dynatrace (formerly Compuware and Gomez), to accelerate identification, diagnosis and resolution of mobile Web, SMS and application performance and availability problems. Provides a unified view of mobile and Web performance and availability. Utilizes thousands of different ‘mobile devices’: tests are performed by the Dynatrace testing agents deployed on mobile nodes. Mobile nodes are a globally distributed set of computers connected to wireless carrier networks via attached wireless modems and provide a realistic measure of the mobile Web experience. Supports all major phone platforms such as iOS, Android OS, BlackBerry OS, Palm OS, Symbian OS & multiple feature phone operating systems