For me, it is correct to say that as far you climb in the pyramid, more are the chances of your tests to become intermittent. Fortunately, in my personal experience, I have not experienced any flakiness caused by Robolectric itself. Usually, when I had some intermittency in a non-instrumented layer, it was caused by some tool misutilization.
In addition to your comment, I agree that this topic may not sound like a big deal, but it is. I like this Martin Fowler statement about non-deterministic tests:
Non-deterministic tests have two problems, firstly they are useless, secondly, they are a virulent infection that can completely ruin your entire test suite.
I think it is a pretty strong statement that reinforces how bad flaky tests are.
Thanks for your comment Ian