Make Android tests fail if no device is available

#10975

For a while I thought the android test bot was succeeding but it wasn't really testing anything at all.