Announcing Patrol - Flutter-First UI Testing Framework

Today LeanCode is open-sourcing Patrol – a new Flutter-first UI testing framework that revolutionizes the way we test our Flutter apps.

​Patrol consists of the patrol package, which provides a powerful yet simple API for testers and developers to use, and patrol_cli, the command-line tool that assists in test development, execution, and orchestration. Patrol is written in Dart, which makes it familiar to Flutter developers.

The main goals that guided us when developing Patrol were: to make things intuitive for UI testers, present the tests in a lean way and allow access to the internals when needed.

Integration testing in Flutter is hard

Flutter is already mature enough to build large enterprise projects with it, as we proved with the Credit Agricole app. However, there is one very important yet often overlooked area that we feel needs radical improvement - automated testing.

Integration testing has always been a pain point in Flutter's world. The default solution for integration testing - the built-in integration_test package - falls short of scenarios that are present in virtually every app, like granting permissions, tapping on a notification, or signing in through Google or Facebook.

The situation is much better when it comes to widget tests, but it's not awesome, either. We've always felt that there's too much noise in them - pump()s, hardcoded sleeps, and complex finders, among others, hinder the expressiveness of test code and make it hard to maintain. It's also complex to write widget tests for someone who's not a Flutter developer, e.g., for a tester.

​We've created Patrol to fix these problems.

Features of Patrol - LeanCode’s UI testing framework

Interaction with platform UI

Patrol can drive your Flutter app in a way that no other tool ever could. It can interact with the operating system UI, such as permission dialogs and notifications. It can open the notification shade and toggle Wi-Fi and Location so that you can test truly real-world scenarios. It can interact with specific UI elements in a WebView. And you can accomplish all this interaction with native UI in just a few lines of Dart code.

void main() {
  patrolTest('signs up', nativeAutomation: true, (PatrolTester $) async {
    await $.pumpWidgetAndSettle(AwesomeApp());
    // prepare network conditions
    await $.native.enableCellular();
    await $.native.disableWifi(); 
    
    // toggle system theme
    await $.native.enableDarkMode();

    // handle native permission request dialogs
    await $.native.selectFineLocation();
    await $.native.grantPermissionWhileInUse();

    // tap on the first notification
    await $.native.openNotifications();
    await $.native.tapOnNotificationByIndex(0);
  });
}

Custom finders

Patrol also comes with a new custom finder system, which greatly simplifies writing UI tests (both widget and integration).

Below is a very simple integration test that would sign in to the hypothetical Awesome App and verify that everything works correctly:

void main() {
  patrolTest('signs up', (PatrolTester $) async {
    await $.pumpWidgetAndSettle(AwesomeApp());
	
    await $(#emailTextField).enterText('charlie@root.me');
    await $(#nameTextField).enterText('Charlie Root');
    await $(#passwordTextField).enterText('ny4ncat');
    await $(#termsCheckbox).tap();
    await $(#signUpButton).tap();
    expect($('Welcome, Charlie!'), findsOneWidget);
  });
}

Beautiful, isn't it?

Flexibility

Another nice thing about Patrol is that you can use as little or as much of it as you want. You may already have many widget and integration tests in your app, and you'd prefer not to rewrite them to Patrol's custom finders. That's okay, but nothing is stopping you from using Patrol's platform interaction feature. Or maybe you have a very simple app, and you'd just want to try out Patrol's custom selectors - no problem, go for it.

Permissive license

We're firm believers in open-source software, which helps us every day in building awesome products. We know that the Flutter community loves open-source as well, so we're publishing Patrol on a permissive Apache 2.0 license to give back to it.

Learn more about Patrol - Flutter UI testing framework

Patrol is easy to install and set up in your app so that you can go from zero to reaping benefits from proper integration testing in no time! Check out our Getting started page in the docs to learn more.

​Oh, and speaking about docs - we know they're an essential part of any open-source project, so we've put a lot of work into Patrol's documentation to make it really good.

What's next for the Patrol development roadmap

We started the Patrol project as an internal experiment back in June. It was born out of our frustration and disappointment with existing Flutter integration testing solutions. Patrol's early proof of concept was very promising, and we realized that this thing could really work!

Today we feel confident enough to share what we've been up to for the last 3 months. What we already have is super cool, but we're not 1.0 yet - in fact, we're far from it. In the coming months, we'd like to focus on the following topics:

• Support for testing more platform capabilities, such as biometrics
• Integration with CI services
• Stability and fighting with flakiness
• Reliable test orchestration on multiple attached devices and emulators
• Better developer experience, such as enabling hot reload
• API surface quality

Help make Patrol better as a Flutter community member

We'd like to thank everybody at LeanCode who helped shape Patrol. This was a true team effort that sparked many great ideas. Now we're excited to invite every member of the Flutter community to join us. Write tests, report bugs, raise ideas and feature requests. Together we can make testing Flutter apps a better, more fun experience.

Just in case you missed the link, here is our Patrol - Flutter UI Testing Framework

The newest update: Patrol is stable and has reached version 2.0.