Episode ID: 892700272779774804.220-412.Search & open amazon
Step ID: 0
Goal: Search & open amazon

Previous Actions:


Current Screen:
UI Annotations Text: ['G' '' 'Search..' '' 'target near me' '' 'flights from nyc to sydney' ''
 'latest video from' 'gamespot trailers' ''
 "what's the news in the bahamas" '' 'ebay' '' 'hotels' 'buenos' 'in'
 'aires' '' 'google play music app' '' 'top rated' 'sushi' 'restaurant'
 'near me' '' 'flight from' 'san diego to' 'seattle' '' 'hotel' 'orlando'
 'in' '' 'good korean restaurant near me' '' 'good' 'beach' 'in' 'miami'
 'restaurant' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' ''
 '' '' '' '']
UI Annotations Types: ['ICON_GOOGLE' 'ICON_MIC' 'TEXT' 'ICON_TIME' 'TEXT' 'ICON_TIME' 'TEXT'
 'ICON_TIME' 'TEXT' 'TEXT' 'ICON_TIME' 'TEXT' 'ICON_TIME' 'TEXT'
 'ICON_TIME' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'ICON_TIME' 'TEXT' 'ICON_TIME'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'ICON_TIME' 'TEXT' 'TEXT' 'TEXT' 'ICON_TIME'
 'TEXT' 'TEXT' 'TEXT' 'ICON_TIME' 'TEXT' 'ICON_TIME' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'ICON_NAV_BAR_RECT' 'ICON_NAV_BAR_CIRCLE' 'ICON_V_BACKWARD'
 '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '']
Positions: [0.05087719 0.07407407 0.04210526 0.04814815 0.05438596 0.88148147
 0.0368421  0.03333334 0.0622807  0.2        0.02192982 0.11666667
 0.1377193  0.07962963 0.03157895 0.03703704 0.15087719 0.2
 0.01403509 0.22407408 0.20614035 0.07962963 0.03333334 0.03888889
 0.21754386 0.20185184 0.01666667 0.39814815 0.27456141 0.07962963
 0.03333334 0.03703704 0.28596491 0.20185184 0.01315789 0.2574074
 0.28596491 0.47222221 0.01666667 0.26666668 0.34298247 0.07962963
 0.03157895 0.03703704 0.35438597 0.20185184 0.01315789 0.49444443
 0.41140351 0.07962963 0.03157895 0.03518518 0.42280701 0.20185184
 0.01578947 0.07037037 0.47982457 0.07962963 0.03333334 0.03703704
 0.49122807 0.20185184 0.01315789 0.09259259 0.49122807 0.3425926
 0.01315789 0.11111111 0.49210528 0.30740741 0.0122807  0.02222222
 0.49210528 0.46666667 0.0122807  0.07222223 0.54912281 0.07962963
 0.03157895 0.03703704 0.55964911 0.20185184 0.01666667 0.34629631
 0.61666667 0.07962963 0.03333334 0.03703704 0.62807018 0.2
 0.01578947 0.13703704 0.62807018 0.34999999 0.01315789 0.07962963
 0.62982458 0.44259259 0.01140351 0.15555556 0.63157892 0.6111111
 0.00964912 0.12222222 0.68596494 0.07962963 0.03070175 0.03518518
 0.69649124 0.20185184 0.01578947 0.1574074  0.69649124 0.37222221
 0.01578947 0.18703704 0.69649124 0.57222223 0.01315789 0.10185185
 0.75438595 0.07962963 0.03070175 0.03703704 0.76491231 0.20185184
 0.01315789 0.07407407 0.76491231 0.32407406 0.01315789 0.11666667
 0.76578945 0.2888889  0.0122807  0.02222222 0.82280701 0.07962963
 0.03157895 0.03518518 0.83333331 0.20185184 0.01578947 0.49629629
 0.89122808 0.07962963 0.03333334 0.03703704 0.90175438 0.20185184
 0.01578947 0.07592592 0.90175438 0.60000002 0.01315789 0.09074074
 0.90263158 0.45925927 0.0122807  0.02222222 0.90263158 0.49444443
 0.0122807  0.09259259 0.90350878 0.29074073 0.01140351 0.15555556
 0.94999999 0.72777778 0.03333334 0.03703704 0.95175439 0.47962964
 0.02894737 0.03518518 0.95263159 0.22962964 0.02894737 0.02962963
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.        ]


What should the Android do next?

Action: {"action_type": 3, "action_description": "Describe what the action means if needed", "coordinates_touch": [-1.0, -1.0], "coordinates_lift": [-1.0, -1.0], "text_input": ["amazon "]}