Testing SMS in Automated Tests for Optimal Delivery

Testing SMS often seems like a simple task - until you try to automate it. Each message depends on external networks, carriers, and devices, making even small tests unpredictable. This article explains why SMS testing is uniquely challenging and how to choose the right approach for stable and reliable results.

Why SMS testing matters?

SMS one-time password (OTP) verification has become a critical security mechanism across digital products. The most popular use cases include user registration, two-factor authentication, and approval of financial transactions. Yet testing these workflows presents unique challenges that separate them from standard application testing.

Unlike regular features that operate within controlled environments, SMS verification relies on external infrastructure, including mobile carriers, message sending services, and device-specific behaviors. Messages can be delayed, filtered as spam, or fail to deliver entirely due to factors beyond your application's control. The stakes are high since failed SMS verification directly translates to blocked user registrations, abandoned transactions, and frustrated users who may never return.

Each SMS testing approach involves trade-offs across four critical dimensions: setup complexity, operational cost, test stability, and real-world accuracy. A fast and inexpensive solution might overlook essential issues of delivery that only surface in production. A comprehensive approach might be too slow and expensive for regular development cycles. We created this article to help you with SMS testing challenges.

Comparison of SMS testing approaches and options

Hardcoded OTP

This approach involves configuring your application to accept predetermined OTP values. The system continues to generate real OTPs internally, but it also accepts a small pool of fixed values. For security reasons, this should only be done on testing environments and never on production.

This approach excels in integration tests, early development phases, or continuous integration pipelines, when speed and reliability matter far more than validating full end-to-end delivery. Its primary advantages are high speed, complete stability, no external dependencies, and zero cost. However, these benefits come at the expense of realism: this method can’t validate that messages are properly generated, formatted, delivered by carriers, or subject to timing issues. The authentication logic has been tested, but the SMS infrastructure remains unexamined.

Test Endpoints

An alternative is to create API endpoints for development use only, which expose generated OTP codes, or parse server logs to extract OTP values. This allows automated tests to obtain actual codes from within the running system without needing to send SMS messages. This approach is well-suited for integration testing, where verification of OTP generation logic, expiration handling, or dealing with multiple simultaneous OTPs is required, but without external message delivery. The main advantage is its ability to cover business logic and real OTP creation while still avoiding external dependencies. 

Nevertheless, this method is most suitable for test environments. Test endpoints must remain inaccessible in production, and logs that include OTP information must be strictly protected to prevent leakage.

Future<String> fetchOtp(String accessToken, String username) async {
 final response = await http.get(
   Uri.parse('https://your-backend.dev/test-api/otp-latest?username=$username'),
   headers: {'Authorization': 'Bearer $accessToken'},
 );
 if (response.statusCode == 200) {
   final data = jsonDecode(response.body);
   return data['otp'];
 } else {
   throw Exception('Failed to fetch OTP');
 }
}

SMS Testing Platforms

Leveraging SMS testing platforms, such as Mailosaur, TestSMS, or MailSlurp, means routing real SMS messages to virtual phone numbers provided by these services. If you already use Twilio for sending text messages, you can also utilize it in your tests. These virtual numbers expose incoming SMS content via APIs, so your test code can read and extract it.

This approach is typically chosen for the end-to-end validation of business-critical user journeys, for pre-production verification, or when evidence is required that messages are properly formatted and delivered. These platforms enable real-time SMS generation, transmission over telecommunications networks, and message receipt, allowing for the testing of delivery delays, international routing, and carrier-specific scenarios. If the tool only provides you with a fixed and limited number list, you also need to implement lock and release mechanisms to avoid collisions between parallel test runs.

While highly thorough, this method brings operational costs and introduces new dependencies: external testing services themselves, or underlying telco networks, can fail even if your app is working perfectly.

An example snippet for MailSlurp:

Future<String?> fetchOtpCodeFromSms({
 required String apiKey,
 required String phoneNumberId,
}) async {
 const String codePrefixPhrase = 'Your code:';
 defaultApiClient.getAuthentication<ApiKeyAuth>('API_KEY').apiKey = apiKey;
 final smsController = SmsControllerApi();

 final result = await smsController.getSmsMessagesPaginated(
   phoneNumberId: phoneNumberId,
 );

 final smsWithCode = result.content?.firstWhere(
   (sms) => sms.body != null && sms.body!.contains(codePrefixPhrase),
   orElse: () => null,
 );

 final body = smsWithCode.body!;

 // Extract the 4 digits OTP code after the prefix phrase
 final regex = RegExp('$codePrefixPhrase\\s*[:\\-]?\\s*(\\d{4})');
 final match = regex.firstMatch(body);
 final code = match?.group(1);
 return code;
}

Physical Device Testing

For the most realistic testing, some teams use actual mobile devices and SIM cards to receive SMS messages, often automated through tools that allow for native mobile automation to extract verification codes and input them into applications.

This approach, unlike the previous ones, is well-suited for smoke testing in production environments. It is especially useful because even when configurations pass in test environments, production settings may still be misconfigured. Testing SMS in production environments with real devices detects those misconfigurations.

This method is particularly beneficial for final-stage testing in critical areas such as banking apps or heavily regulated domains, where device and carrier quirks can affect user experience. It achieves maximum authenticity by capturing behaviors unique to specific devices, SIMs, and carriers, while revealing rare issues such as local SMS filtering or network degradation. These benefits, however, are balanced by high operational costs. Provisioning and maintaining devices, managing SIM subscriptions, and coordinating them for tests are resource-intensive and can make this approach impractical to scale for routine use.

Choosing Your Approach

The optimal testing strategy depends primarily on the scale of your project. Most projects benefit from selecting a single primary approach that aligns with their testing needs. Some teams may choose to combine methods if their specific requirements justify the added complexity. In those cases, daily CI pipelines favor lightweight, dependency-free approaches, while weekly or pre-release testing cycles can accommodate more comprehensive validation methods. 

In regulated industries such as finance or healthcare, physical device testing is often mandatory for final verification, ideally supplemented with broad SMS platform testing that covers multiple carriers and regions. It’s important to recognize that SMS delivery can and does fail. Your product must handle slow or lost SMS gracefully, offering alternate verification flows or guidance, to avoid frustrating users and blocking transactions.

If you’re facing the challenges described in this article, you know how complex it can get. At LeanCode, we help teams integrate stable and scalable SMS testing solutions into their Flutter app’s QA pipeline, ensuring reliable verification flows and smoother user experiences. Learn more about our Flutter UI Automated Testing.