Automating multi-app functional testing against different versions of Android

Automated testing is mostly about plumbing. Don’t you think?

Context

Problem

Solution

  1. Fire up an emulator,
  2. Install API specific APK of the benign (or secure) app,
  3. Install API specific APK of the malicious app,
  4. Drive the malicious apps via UI automation, and
  5. Check if the observed behavior matches the expected behavior.

Step 1: Generate different APKs for each Android version

Step 2: Create instrumented multi-app test

Step 3: Orchestrate testing outside of the build system

  • building the apps (lines 12–23),
  • testing benign app (lines 46–63), and
  • testing secure app (lines 65–78).
  • AVD names conform to a pre-defined naming scheme,
  • Testing project has secure_app module to test the secure app, and
  • applicationId of secure_app module/app is edu.ksu.cs.benign as this helps reuse edu.ksu.cs.malicious app without any changes.

What would have been ideal?

  1. Using gradle to run functional tests.
  2. Modify connectedAndroidTest task (including all its product flavor specific variants) to install and uninstall APKs.
  3. Having connectedAndroidTest task start and stop virtual devices.
  4. Selecting and configuring the virtual device to start in each product flavor specific variant of connectedAndroidTest task.
  5. Creating a instrumentation test-only Android project that does not require an app manifest and specific app ids.

Closing thoughts

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Venkatesh-Prasad Ranganath

Software Engineer / Researcher interested in software engineering, programming languages/tools, systems, and data.