Episode ID: 9169413091350754586.212-229.Go to search bar 
Step ID: 0
Goal: Go to search bar 

Previous Actions:


Current Screen:
UI Annotations Text: ['' '' 'Google' '' 'Search or type web address' '' '' 'W' 'Facebook'
 'YouTube' 'Wikipedia' 'Amazon.com' '' 'ebay' 'YAHOO' 'ESPN.com' 'Yahoo'
 'eBay' 'Instagram' 'Articles for you' 'Hide' 'Former Chiefs assistant'
 'Britt Reid sentenced to 3' 'for' 'accident,' 'less than' 'in' 'prison'
 'DWI' '1' 'years' 'prosecution' 'recommended' 'Britt' 'Reid'
 'sentenced to three years' 'in' 'prison' 'as part of' 'was' 'a felony'
 'DWI' 'plea deal' 'despite a' 'passionate' 'argument fro.' '' ':' 'E'
 'Yahoo!' 'Sports - 6 hours ago' 'Employees Work' 'Until The Last' 'Boss'
 'Insists' 'My work hours' 'Exactly ...' 'Minute,' 'Gets' '3:30am - 5pm,'
 'Yo' 'Xceptions?' 'There are strict' 'bosses who'
 'make no exceptions to their' 'rules and then there are toxic ones who'
 'bend them every' ')' '' 'Bored' 'Panda - Yesterday' '' '' '' '' '' '' ''
 '' '' '' '' '' '' '' '' '' '' '' '' '' '' '']
UI Annotations Types: ['ICON_HOME' 'ICON_THREE_DOTS' 'TEXT' 'ICON_MIC' 'TEXT' 'ICON_FACEBOOK'
 'ICON_PLAY' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'ICON_TAKE_PHOTO' 'TEXT'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'ICON_PLAY' 'ICON_THREE_DOTS' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'ICON_TIME' 'ICON_THREE_DOTS' 'TEXT'
 'TEXT' 'ICON_NAV_BAR_CIRCLE' 'ICON_NAV_BAR_RECT' 'ICON_V_BACKWARD' '' ''
 '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '']
Positions: [0.04236111 0.03333334 0.02777778 0.03055556 0.04236111 0.93055558
 0.025      0.01527778 0.13333334 0.30000001 0.06458333 0.39444444
 0.25624999 0.90416664 0.03333334 0.02777778 0.26944444 0.05694444
 0.01319444 0.33888888 0.34375    0.14166667 0.03194445 0.03611111
 0.35208333 0.37777779 0.01597222 0.01944445 0.35694444 0.82083333
 0.01111111 0.02361111 0.39722222 0.11944444 0.00902778 0.08888889
 0.39722222 0.34861112 0.00833333 0.08055556 0.39722222 0.79305553
 0.00972222 0.0875     0.39791667 0.55138886 0.00763889 0.12222222
 0.43402779 0.82083333 0.02569444 0.02916667 0.44305557 0.58888888
 0.00902778 0.04583333 0.44583333 0.36666667 0.00555556 0.03333334
 0.48402777 0.11666667 0.00833333 0.09583333 0.48402777 0.35972223
 0.00833333 0.05694444 0.48402777 0.59166664 0.00972222 0.04305556
 0.48472223 0.79166669 0.00833333 0.09027778 0.52986109 0.0625
 0.00972222 0.1375     0.53055555 0.89861113 0.00763889 0.03888889
 0.57708335 0.0625     0.01111111 0.29722223 0.57708335 0.37083334
 0.01111111 0.30416667 0.59791666 0.25138888 0.01111111 0.03472222
 0.59791666 0.35277778 0.0125     0.11111111 0.59791666 0.49722221
 0.01111111 0.10972222 0.59861112 0.1375     0.01041667 0.01944445
 0.59861112 0.16666667 0.01319444 0.075      0.59861112 0.29722223
 0.01041667 0.04444445 0.59861112 0.47499999 0.01041667 0.00694444
 0.60138887 0.06111111 0.01041667 0.06527778 0.61944443 0.0625
 0.01319444 0.14444445 0.61944443 0.21805556 0.01041667 0.17777778
 0.64305556 0.0625     0.00902778 0.04305556 0.64305556 0.11527777
 0.00902778 0.04444445 0.64305556 0.22083333 0.01180556 0.26805556
 0.64305556 0.49861112 0.00902778 0.01666667 0.64305556 0.52499998
 0.01180556 0.06527778 0.64305556 0.60000002 0.01180556 0.10416666
 0.64513886 0.16944444 0.00694444 0.04166667 0.66041666 0.06527778
 0.01180556 0.08055556 0.66041666 0.15555556 0.00902778 0.04027778
 0.66041666 0.20694445 0.01180556 0.09722222 0.66041666 0.31388888
 0.01180556 0.09861111 0.66041666 0.42222223 0.01180556 0.11805555
 0.66041666 0.55000001 0.01180556 0.14166667 0.69027776 0.40277779
 0.01666667 0.01666667 0.69027776 0.91250002 0.02083333 0.0125
 0.69375002 0.06111111 0.01319444 0.03472222 0.69583333 0.1125
 0.00833333 0.06527778 0.69583333 0.18611111 0.01041667 0.19444445
 0.76180553 0.22361112 0.01388889 0.20416667 0.76180553 0.43888888
 0.01111111 0.16944444 0.76249999 0.0625     0.01041667 0.05972222
 0.76249999 0.13333334 0.01041667 0.07916667 0.7652778  0.74166667
 0.01388889 0.19722222 0.78125    0.22638889 0.01597222 0.11805555
 0.78333336 0.0625     0.01180556 0.08888889 0.78333336 0.16111112
 0.01041667 0.05416667 0.78333336 0.74027777 0.01527778 0.1875
 0.80208331 0.90416664 0.0125     0.03055556 0.80277777 0.74027777
 0.01388889 0.15277778 0.80694443 0.06111111 0.00902778 0.16527778
 0.80694443 0.2361111  0.00902778 0.12777779 0.80694443 0.37361112
 0.01111111 0.30277777 0.82430553 0.0625     0.00902778 0.42777777
 0.82430553 0.5        0.01180556 0.17916666 0.87083334 0.06527778
 0.02222222 0.02222222 0.87222224 0.91250002 0.02083333 0.0125
 0.87916666 0.11388889 0.00763889 0.05416667 0.87916666 0.17777778
 0.00972222 0.17083333 0.95972222 0.48333332 0.025      0.02777778
 0.95972222 0.77361113 0.02569444 0.02777778 0.96041667 0.19305556
 0.025      0.025      0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.        ]


What should the Android do next?

Action: {"action_type": 4, "action_description": "Describe what the action means if needed", "coordinates_touch": [0.27434107661247253, 0.5297179222106934], "coordinates_lift": [0.27434107661247253, 0.5297179222106934], "text_input": [""]}