The Role of UI Testing in Large-Scale Applications

User Interface testing plays a crucial role in enterprise mobile applications but is also important in apps of all sizes, as we think, to ensure the quality, functionality, and great user experience of the app. Creating UI tests lets you verify if navigation flows are logical, visual elements like buttons, forms, or drop-down lists work correctly, and interactions perform the expected actions.

In this article, we dive deeper into why UI testing actually matters, what are the benefits of automated UI tests and what available solutions are.

We can also help you automate QA processes in your Flutter app. Check out this page to find out more about our Automated UI testing in Patrol service.

What are the types of mobile app tests?

When it comes to testing Flutter applications, there are many approaches and tools that we can use to implement tests on various levels. 

1. Starting from the bottom of the testing pyramid, we can make use of unit tests - closely related to the app’s code.
2. Another tool is widget tests, which we can bring into play as a kind of module testing, where bigger app's UI elements are tested in isolation. 
3. Focusing more on layout than business logic, we can utilize golden file testing - a Flutter-specific approach to screenshot testing. 
4. A solution incorporating testing functionalities and interacting with real layout (also called end-to-end testing) - UI tests, which unfurl their true potential in testing large apps that are often only a part of much bigger systems.

UI tests - what do they really check?

Testing is the process. When planning, designing, and implementing tests, knowing what should and should not be done on a specific test level is crucial. The main objective during thorough UI testing and preparing test scenarios is to check whether the app's main purpose can be achieved by its users. With this general goal in mind, we can try to separate out which particular goals our tests should target. These specific targets for UI test cases are a high-level view of the system, big regression suites, and imitating real users' behaviors.

What are the benefits of UI testing?

Regression testing - automated

One of the highlights is that UI tests help in regression tests, which best suit this task. And there are reasons for that. First of all, regression suites are precisely defined - there is no need to alter the test cases spontaneously, like in exploratory testing. It means that an automated test can do the same task as a human tester would, adding even more accuracy and repeatability in execution. 

Another advantage is that automated mobile UI testing is significantly faster than humans, meaning they cost much less than big QA teams. Finally - regression testing is the most tedious task in a tester's work. Automation makes it not only better in execution but also lets the QA team utilize their time better.

Have a vista of your app

Since we discuss large apps here, a big-picture view of what we created is very important. And no other tests do it better than UI tests. Our scenarios are on such a level of detail that they can correspond with features, and test suites can comply with the app's modules. Not only can developers utilize reports of such tests to monitor the health of the system they’re creating, but business owners can benefit from them too.

What are UI testing challenges?

To be a robot but act like a human

UI tests are the only kind of tests in which we try to simulate user interactions - we build and launch the whole system, almost no data is mocked, and we want the test to perform user-like interaction with user interfaces. 

It’s not easy to do - UI tests are written as code, and this code should work in a predictable way. This means that we need some tools to properly distinguish elements on the screen and interact with them only when a real user would. For example, such a test should only interact with elements that are visible on the screen and also should know when to try interacting with them.

On the level of code, we usually don’t have straightforward information about what we can interact with and when we should wait for something to appear on the screen. And there are ways to do that - though we should keep in mind that these methods are complicated and are based on many technical nuances, which vary between different technology stacks.

Beware of overzealous

Among aspects that are not covered by UI tests are visuals, though they are called UI tests. We are interacting with the real mobile app's user interface, but we won’t check how it looks. Ensuring the app is aligned with the designs is a task for the golden file tests. We are performing UI tests to make sure that this UI is functional. 

Another thing to keep in mind while writing UI tests is that they should cover only crucial user paths and features. Many of the edge cases can be covered by lower-level tests, such as unit and widget tests. It lets us not duplicate what we already tested and focus on high-level aspects and integrations that can’t be covered elsewhere.

Why are UI tests important in enterprise applications?

If we think about software development, we can point out some steps that are part of the software development life cycle in every project, both small startups and big-tech companies. Although UI tests are not a popular topic among all developer teams, we see that this kind of test gains more attention in bigger projects. And there are some reasons behind it, which we will discuss in this section.

More teams, different problems

It’s clear to see that enterprise development teams are much bigger. Those teams are divided into smaller ones, and usually, every smaller team is responsible for delivering one feature or use case of the app. They often don’t have many opportunities to communicate with each other about how their parts interact, which is usually causing most of the bugs the QA team detects. 

To fill this lack of synchronization between many teams, we need a point in the process where we would check our app as a whole product. In this case, UI tests suit the best. Not only do we have a high-level view of features, but also we focus on interactions between many parts of the system. It makes UI tests a perfect tool for verification while having many independent teams working separately on the same product.

If you are developing large-scale applications, our article on building an Enterprise Application in Flutter may be useful.

Communication with business owners

Looking at this process from another point of view, business owners need regular updates on the product’s readiness and level of stability. Having automated high-level tests, which consider all parts of the system, allows for quickly gathering the results and presenting them in the context of development progress and system reliability.

Though other test results, such as integration tests, can provide similar information, UI tests are already on the right significance level that is needed from a business point of view.

Everybody hates regression

Last but not least, regression testing costs grow rapidly with the app’s size. In addition to cost, the time needed to perform every regression suite can extend to days of work for quite a big QA team. Finally, we all know how tedious work it is to execute all of the scenarios, which makes it error-prone and less efficient. 

Automated UI tests (UI test scripts), once written, can be run in less time and without or with minimal human supervision. The cost of maintaining and executing those tests is far lower than the cost of manual testing and regression. Additionally, saved time of the QA team can be spent, e.g., on exploratory testing and other tasks which can’t be automated.

What are available UI testing tools?

Appium and its imperfections

The most popular and universal tool for user interface tests of mobile apps is Appium. It is a black box solution that is designed for testing on many platforms. 

Though when it comes to testing Flutter apps, this framework is not enough. A well-known solution to that problem is using a combination of Flutter Driver and Appium, which gives a possibility of testing both the Flutter side of the app and the native platform - either Android or iOS. It enables us to cover more complicated use cases, which incorporate not only those steps that are performed in our app but also those outside of the Flutter code - e.g., email verification, logging in, or registration by external services such as Google or Facebook, reading SMS codes and many others. 

At LeanCode, we were challenged to write tests with Appium and Flutter’s integration_test package in large-scale apps to test complex functionalities made with Flutter while also incorporating some native features. We quickly found that this solution required an amount of work that was unacceptable for our needs. 

On the one hand, using the Flutter-native testing package missed the possibility of fully interacting with the native side of our app, such as providing permissions for notifications or the use of localization services. On the other hand, we wanted to use an extensible, easy-to-use framework in which tests can be written in the same language as our app (in our case - in Dart). 

Knowing the limitations of the Appium-Flutter-Driver specific design, we were aware that we won’t be able to solve all of our current and future problems with this solution.

Patrol - an open-source framework

The experience with Appium-Flutter-Driver led us to write our own, Flutter-focused UI testing framework, named Patrol, which we released in September 2022.

Patrol users can write their UI tests in Dart, using simple yet powerful API, which lets them construct human-readable selectors and easily simulate real user’s actions, both Flutter-native and OS-native. We incorporate our own framework in many projects we develop for our clients, so we can perform better tests and make our framework battle-proven in real-life use cases. Also, Patrol is open-source, so every developer can use it in their project, whether commercial or not.

In June 2023, Patrol reached the 2.0 version. We reworked the internals of how the tests are executed and integrated with the native side, which enables our users to run Patrol tests on device farms and improves their experience with CI pipelines. With a new design, it is possible to utilize sharding on device farms. Also, the time needed to run many tests on the same app became shorter. We believe that those improvements unlock lots of potential for new features and following refinements.

In November 2023, we updated Patrol to 3.0. For example, we added Patrol DevTools Extension, enhancing UI test development with patrol develop command. With Patrol's DevTools Extension, you can effortlessly inspect the currently visible Android/iOS views and discover their properties. This information can then be used in native selectors like $.native.tap(), eliminating the need for external tools.

Patrol’s test example

We’d like to present some examples of how the tests look when written with Patrol. When you expect to see a permission dialog for sending notifications, you simply add this step:

await $.native.grantPermissionWhenInUse();

Entering a text into a text field is simple as well - though the first part of this instruction, which is responsible for searching for the desired element on the screen, can vary. Here we use searching by in-code type of widget, which is TextField in this case.

await $(TextField).enterText(‘Patrol, a UI testing framework’);

Scrolling and tapping on elements on the screen is also very easy. Here you can see a bit more complex selector. Still, it is written descriptively. This step consists of scrolling to an arrow icon that is inside a list tile, which contains the text “Part 1”, then taps on the icon that was found. 

await $(ListTile).containing(‘Part 1’).$(Icons.arrow).scrollTo().tap();

Please notice that we don’t have to write any code that would ensure the visibility of found elements - it is already taken into consideration by Patrol. Also, if many elements are matching this selector, action is performed on the first one by default. Of course, if you would like to search for the third one, it is still possible - by adding the “at(index)” method after the selector, as in the example below.

await $(ElevatedButton).at(2).tap();

All of those types as ElevatedButton and TextField, are types of widgets that are either provided by the Flutter framework or declared in the app’s code. This access makes our test greybox, which we consider a good equilibrium for automated tests - the test doesn’t have to know everything about the app, so it is easy to maintain it, but also we can differentiate elements on the screen in a convenient way.

Here you can find out how to set up Patrol.

Mobile app UI testing in Flutter: Summing up

UI tests are a crucial part of test strategy in enterprise-level projects. Proper tools and good practices of UI tests can significantly improve the quality of the developed product and customer satisfaction. From lowering costs and time spent on regression testing through creating a point of synchronization for scattered teams working on the same product to providing high-level visibility on the completeness of the system - UI tests prove their validity in many ways. 

At LeanCode, we created Patrol having in mind big and complex, Flutter-focused apps developer teams who need a simple in use yet powerful and extensible tool for creating the most challenging automated tests to ensure that the app is functioning correctly.

If you’d like to find out more about our automated testing solution for checking the application's UI, go to Patrol’s website. We can also help you automate QA processes in your Flutter app. Find out more about our Automated UI testing in Patrol service.