Phellipe Silva
2 min readOct 4, 2020

--

Hi Kavita,

I will try my best to help you! In my opinion, flakyness is one of the most difficult problems to solve regarding test automation scalability... I relate this difficulty to the vast number of variables that can make a test flaky, and on top of that, this kind of problem is very hard to reproduce =(

Since I don't have much context of your project, I will give some tips that may help you to investigate it 🕵️‍♀️

1 - Monitor your Flaky Tests with the @FlakyTest annotation. Every time one test fail intermittently annotate it to keep track. You can also filter those tests from your suite temporarily. https://developer.android.com/reference/androidx/test/filters/FlakyTest

2 - Get some flaky tests and try to reproduce them by executing in isolation multiple times. By doing that you can discovery if the isolated test is flaky or some other external thing is impacting you. Instructions on how to do that: https://medium.com/monsterculture/reproduce-flaky-tests-b8b42306d23f

3 - Check if there are infrastructure problems impacting you. For instance, I had problems with intermittent tests in my CI server just because the emulators where running out of memory. You can monitor/adjust the memory consumption or just make an experiment with real devices (device farm) to see if the problem persists.

4 - Check if your espresso tests are very long. In my experience, if the test navigates to multiple screens and does many many actions, the chance of intermittency increases. If you have very long tests, try to break them in smaller tests.

5 - Check if you can test a functionality the same way in lower levels of the pyramid. The less unnecessary UI tests you have, lesser are the chances to have flakyness. Test with Espresso just the necessary.

6 - I dont know if Awaitility is adapted 100% to the Android environment... If the project uses Retrofit. You can try this Jake Wharton Idling Resource library. It may help you: https://github.com/JakeWharton/okhttp-idling-resource

This is everything that came up in my mind. Hope you can solve this problems in your suite!

Best Regards!

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

--

--

Phellipe Silva
Phellipe Silva

Written by Phellipe Silva

Android engineer and test automation enthusiast. Working @Wise and formerly @ThoughtWorks. Twitter profile: @phellipeafsilva

Responses (1)

Write a response