The easiest way to retry XCTests
Introduction
It’s not a secret, that UI Tests are rather unstable thing. Pretty often, in practice, we have to come up with various ways to make them as stable as possible. One such way is to retry the failed tests.
In JUnit for such cases we have RetryTestHelper, in TestNG — IRetryAnalyzer, even in Espresso we’re able to create a custom rule for that purpose (e.g.: Barista). But in XCTest (and other XCTest extensions) we don’t have anything like that out of the box.
After a short googling, the first solution we can find — setup_fragile_tests_for_rescan — this is the Fastlane plugin. Fastlane is a gorgeous and elegant tool for automating everything around mobile apps (signing, building, testing, delivering, etc). This plugin is currently deprecated, but in its place came test_center.
I really like Fastlane, but this plugin was a bit buggy to me. I mean, if it’s working great for you, just go on using it. Otherwise - welcome to the party.
I, in turn, came up with an idea to create my own Fastlane plugin, that will just do its job without lots of features - retry tests if they fail.
Here we go
brew install fastlane
fastlane add_plugin try_scan
Usage
try_scan(
workspace: "Example.xcworkspace",
device: "iPhone 11 Pro Max",
try_count: 3
)
Conclusion
That’s nice, isn’t it? But I should point out that retrying the tests, although a forced measure, is far from the best way to maintain stability and reliability of the tests, as your app becomes a target for heisenbugs.
Hope it’ll help, see ya (:
