Skip to main content

Running Maestro tests on Bitrise

The Maestro Test Step runs your Maestro flows on an Android emulator or iOS simulator on the Bitrise build machine, not on Maestro Cloud, and shows the results on the Test reports page. Each flow is a test case, with its screenshots, recordings, and logs attached.

Running Maestro flows in a WorkflowClick to copy link​

A Maestro Workflow builds your app, starts a device, runs the flows, and uploads the results. The Maestro Test Step only runs the flows: it doesn't build your app or start a device, so add a build Step and a Step that starts the device before it, and the Deploy to Bitrise.io Step after it. The procedures below set this up for Android and iOS apps.

Android appsClick to copy link​

  1. Make sure that you have:

    • Maestro flows in your repository. By default, the Step runs the flows in the .maestro folder. To run other flow files or folders, list them in the Flow path input. To learn how to write flows, see the Maestro documentation.
    • An .apk of your app. Any framework works, including React Native and Flutter. The Step doesn't run Maestro's web flows.
    • A Linux Android stack. Android emulators don't run on macOS stacks.

  2. Open the Workflow you want to use in the Workflow Editor.

  3. Add the Android Build Step, and set its Module and Variant inputs.

  4. Add the AVD Manager Step, and set the emulator in its Android API Level, OS Tag, and ABI inputs. The ABI must match the CPU architecture of the stack.

    important

    Only one emulator or simulator can run on the build machine: if there's none or more than one, the Step fails.

  5. Add the Maestro Test Step, and set its App path input to $BITRISE_APK_PATH.

  6. Add the Deploy to Bitrise.io Step.

An emulator can't install an Android App Bundle (.aab). If you build one, convert it to an APK with the Export Universal APK Step before the Maestro Test Step.

iOS appsClick to copy link​

  1. Make sure that you have:

    • Maestro flows in your repository. By default, the Step runs the flows in the .maestro folder. To run other flow files or folders, list them in the Flow path input. To learn how to write flows, see the Maestro documentation.
    • An .app of your app built for the iOS simulator. Any framework works, including React Native and Flutter. The Step doesn't run Maestro's web flows.
    • A macOS stack.

  2. Open the Workflow you want to use in the Workflow Editor.

  3. Add the Xcode Build for Simulator Step, and set its Project path and Scheme inputs.

    important

    Only one emulator or simulator can run on the build machine: if there's none or more than one, the Step fails.

  4. Add the Xcode Start Simulator Step, and set the simulator in its Device destination specifier input.

  5. Set the Simulator boot timeout (in seconds) input, so that the Step waits for the simulator to boot. Otherwise, the Maestro Test Step might not find it.

  6. Add the Maestro Test Step, and set its App path input to $BITRISE_APP_DIR_PATH.

  7. Add the Deploy to Bitrise.io Step.

Passing variables to flowsClick to copy link​

Maestro passes every Environment Variable that starts with MAESTRO_ to your flows. Store sensitive values as Secrets with this prefix. For example, a Secret named MAESTRO_PASSWORD is ${MAESTRO_PASSWORD} in a flow:

appId: com.example.myapp
---
- launchApp
- tapOn: "Password"
- inputText: ${MAESTRO_PASSWORD}

Selecting flows with tagsClick to copy link​

Tag your flows in the flow file:

appId: com.example.myapp
tags:
  - smoke
---
- launchApp

Then use Include tags to run only the flows that have any of the listed tags, and Exclude tags to skip the flows that have any of them. Separate tags with commas or new lines.

- maestro-test:
    inputs:
    - app_path: $BITRISE_APK_PATH
    - include_tags: smoke
    - exclude_tags: flaky

Running flows on several devices in parallelClick to copy link​

Maestro's own sharding needs several devices on one machine, which the Step doesn't support. Instead, run copies of the Workflow in parallel in a Pipeline, each on its own virtual machine: Parallelism in Pipelines.

  1. Tag each flow with the copy that runs it: shard0, shard1, and so on.
  2. Set Include tags to shard$BITRISE_IO_PARALLEL_INDEX. Every copy gets its own zero-based index in this Env Var.
  3. Set a different Test name for each copy, so their results don't mix on the Test Reports page.
pipelines:
  ui-tests:
    workflows:
      maestro-tests:
        parallel: 2

workflows:
  maestro-tests:
    steps:
    - git-clone: {}
    - android-build:
        inputs:
        - module: app
        - variant: debug
    - avd-manager:
        inputs:
        - api_level: 34
        - tag: google_apis
        - abi: x86_64
    - maestro-test:
        inputs:
        - app_path: $BITRISE_APK_PATH
        - include_tags: shard$BITRISE_IO_PARALLEL_INDEX
        - test_name: Maestro shard $BITRISE_IO_PARALLEL_INDEX
    - deploy-to-bitrise-io: {}

Viewing the resultsClick to copy link​

The results appear on the Test Reports page under the name in the Test name input, failed flows included. For the Deploy to Bitrise.io versions that show attachments, see Deploying and viewing test results.

Give every flow a unique name

The Step attaches files to a flow by its name. If two flows share a name, their files aren't attached.

The Step also exports the JUnit report in $MAESTRO_JUNIT_PATH, and Maestro's whole output folder as a zip in $MAESTRO_TEST_OUTPUT_ZIP_PATH. The zip is uploaded as a build artifact.

TroubleshootingClick to copy link​

The first line of the Step's error tells you what failed:

ErrorCause
Failed to process Step inputsAn input has an invalid value. The next lines name it.
Failed to install the Maestro CLIThe Maestro CLI couldn't be downloaded or verified.
Failed to install the appNo device is running, more than one is running, or the install failed. Make sure the Step that starts the device waits for the boot.
Maestro flows failedAt least one flow failed. See the Test Reports page.
Maestro failed without a failing flowMaestro stopped before any flow failed, for example on a flow file it couldn't parse. See the Step's log.