In just a few years, Continuous Integration has become a standard practice for software development teams around the world. CI promises, and delivers, significant benefits in code quality and agility. These benefits are no doubt the driving forces behind the speedy adoption.
CI for mobile applications offers the same benefits as CI for traditional software and web application, but there are some specific difficulties.
One, mostly historic difficulty, is around the lack of good build and test frameworks. Since a large part of mobile applications is the User Interface (UI), UI testing should be part and parcel of automated mobile testing. UI testing used to be difficult and unreliable, however things have improved significantly since Google's Espresso framework was released. Similarly for the build tools, complex tools like Maven have made place for more modern and user friendly tools like Gradle.
A difficulty that remains even today revolves around where to run the tests. Mobile applications are (obviously) designed to run on physical smart phones and tablets. However, when running automated tests, it is not preferable to run these on the actual devices. Phones and tablets are consumer grade hardware and are not designed to work with low errors rates in a 24x7 setting like e.g. enterprise grade servers are. Devices are also difficult to source reliably with models being discontinued frequently. Finally physical devices are just clumsy to manage especially when working with a large number of them.
Instead, it is much preferred to run automated tests on a virtual device. The Android SDK provides an emulator that can be used for this purpose. The emulator allows you create and run Android Virtual Devices (AVDs). An AVD is in essence a small virtual machine that runs an Android system image. The virtual hardware of an AVD (number of CPUs, memory, screen size, etc) is typically configured to resemble an existing physical phone such as the Nexus One or the Galaxy S4. For each AVD you can also configure the Android version, and the architecture (ARM or Intel).
Running tests in the Android emulator is much preferred to running them in real devices. However, there still some problems left, and this is where Ravello factors in.
First, the Android emulator can be slow. When it first came out, it was always slow. This was because it only supported ARM system images (most phones have an ARM processor), and therefore the emulator used full system emulation to run the image. This made it extremely slow; our measurements suggest up to 20x slower than a real phone. More recently however, the Android SDK includes an Intel system image as well as support for Intel/AMD hardware virtualization extensions. This means that if your development workstation or CI server
supports these hardware virtualization extensions, the emulator will be able to run at native speeds. However, this brings us to the second problem.
Ideally, the automated tests, like any bursty workload, are run in the cloud. This is because you don't always run tests, but when you do, you want to complete them as soon as possible. By far the most efficient way to do this is to buy test instances on demand when you need them, and stop them when they are no longer needed. For mobile apps, the benefits of running in the cloud are exacerbated by the large amount of different configurations on which tests have to be run. The total set of configurations is usually called the "Matrix". Suppose you want to run across 20 screen resolutions and 3 Android API levels, this means that your matrix contains 60 test runs. Running these tests sequentially would be super inefficient and take too much time.
Where does Ravello fit in here? Well, it's quite simple. Ravello supports hardware virtualization features and can therefore run an emulator with hardware acceleration in the cloud. And because Ravello is cloud based, you can parallelize your entire test matrix and get very fast test results. As far as we are aware, Ravello is the only cloud service that allows this.
Below is a video of me demonstrating how to run both unit and UI tests on EC2 using the Android emulator. The emulator is running at full speed because of the hardware virtualization features of Ravello. In this case the emulator is running on top of Xubuntu 14.04.1 using KVM as the driver for the hardware virtualization features.
Why all the focus on performance you may ask? Well, it is my strong belief that automated tests are only useful if they are fast. And with fast I mean that an entire test run is done within a few minutes. Tests are often run by developers during coding. Every minute that is spent on running tests is a minute wasted for that developer. And even when tests are run in an automated CI pipeline, speed is important. If the feedback time is too long between a commit and a test result, then the developer will have started work on something else, requiring a costly context switch.
[video url="https://www.youtube.com/watch?v=EGu03FOFbmE"]
See our post with detailed instructions on how to run the Android emulator (with hardware acceleration) on Amazon EC2 or Google’s cloud.
The post FAST! Continuous Integration for Android using the Android Emulator on Amazon EC2 appeared first on The Ravello Blog.