Episode ID: 14584176688125464656.476-650.Search for newegg
Step ID: 0
Goal: Search for newegg

Previous Actions:


Current Screen:
UI Annotations Text: ['G' '' 'Seerch..' '' 'costoo nesr' 'me' '' 'best buy near ie' ''
 "what's thz wrothar lilar in" 'chicag:' ''
 'show me some nice wallpapers for my tablet' ''
 'best mexicen restaurants' '' "vwbal's the res tis morrirg" ''
 'wehore an i busa nico boa:t onolar?' '' 'cienver eather' ''
 "what's on the menu at lennys?" '' 'Ceaital' 'balivig' ''
 'a ntw frundatar (alonrare) potut' '' 'what is the speed of a pane?' '9'
 'anew biush' '' 'fighis frurm' 'ID larduri latiay' 'rnt' ''
 "what's n the menu at ir-nt?" ''
 "what's the wweatter like in san francsco?" ''
 "what's on the menu at five quys?" '' '' '' '' '' '' '' '' '' '' '' '' ''
 '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '']
UI Annotations Types: ['ICON_GOOGLE' 'ICON_MIC' 'TEXT' 'ICON_TIME' 'TEXT' 'TEXT' 'ICON_TIME'
 'TEXT' 'ICON_TIME' 'TEXT' 'TEXT' 'ICON_TIME' 'TEXT' 'ICON_TIME' 'TEXT'
 'ICON_TIME' 'TEXT' 'ICON_TIME' 'TEXT' 'ICON_TIME' 'TEXT' 'ICON_TIME'
 'TEXT' 'ICON_TIME' 'TEXT' 'TEXT' 'ICON_TIME' 'TEXT' 'ICON_TIME' 'TEXT'
 'ICON_TIME' 'TEXT' 'ICON_TIME' 'TEXT' 'TEXT' 'TEXT' 'ICON_TIME' 'TEXT'
 'ICON_TIME' 'TEXT' 'ICON_TIME' 'TEXT' 'ICON_TIME' 'ICON_V_BACKWARD'
 'ICON_NAV_BAR_RECT' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' ''
 '' '' '' '' '' '' '' '' '' '']
Positions: [0.035      0.05555556 0.03666667 0.03703704 0.03833333 0.9074074
 0.03       0.02592593 0.05       0.15185185 0.01166667 0.0962963
 0.09833334 0.05555556 0.02833333 0.03333334 0.11       0.15185185
 0.01       0.14814815 0.11166666 0.31481481 0.00833333 0.04074074
 0.14833333 0.05555556 0.02833333 0.03333334 0.15833333 0.15185185
 0.01166667 0.22222222 0.19666667 0.05555556 0.025      0.02962963
 0.20333333 0.15185185 0.015      0.32222223 0.20666666 0.48888889
 0.01166667 0.1        0.24666667 0.05555556 0.025      0.03333334
 0.255      0.15185185 0.01333333 0.58518517 0.29499999 0.05925926
 0.025      0.02962963 0.30333334 0.15185185 0.01333333 0.32592592
 0.34333333 0.05925926 0.02666667 0.02962963 0.35333332 0.15185185
 0.01333333 0.38148147 0.39166668 0.05925926 0.02666667 0.02962963
 0.40166667 0.15185185 0.01       0.48148149 0.44       0.05555556
 0.02666667 0.03333334 0.44999999 0.15185185 0.01333333 0.19629629
 0.49000001 0.05555556 0.02666667 0.02962963 0.5        0.15185185
 0.01166667 0.4074074  0.54000002 0.05925926 0.025      0.02962963
 0.54833335 0.15185185 0.01       0.08518519 0.54833335 0.28148147
 0.01       0.08148148 0.58666664 0.05925926 0.02666667 0.02962963
 0.59500003 0.15185185 0.015      0.47037038 0.63833332 0.05925926
 0.025      0.02962963 0.64499998 0.15185185 0.01333333 0.37777779
 0.685      0.05555556 0.025      0.03333334 0.69166666 0.15185185
 0.015      0.14814815 0.73333335 0.05555556 0.025      0.02962963
 0.74000001 0.15185185 0.01833333 0.14444445 0.74000001 0.36666667
 0.01833333 0.19259259 0.74666667 0.31111112 0.01       0.04444445
 0.78333336 0.05925926 0.025      0.02962963 0.79166669 0.15185185
 0.01       0.4074074  0.83166665 0.05555556 0.02666667 0.02962963
 0.83833331 0.15185185 0.015      0.52592593 0.87833333 0.05925926
 0.02666667 0.02962963 0.88999999 0.15185185 0.01166667 0.42222223
 0.92833334 0.05555556 0.025      0.02962963 0.95833331 0.18888889
 0.025      0.02592593 0.95833331 0.77037036 0.02666667 0.02592593
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.        ]


What should the Android do next?

Action: {"action_type": 3, "action_description": "Describe what the action means if needed", "coordinates_touch": [-1.0, -1.0], "coordinates_lift": [-1.0, -1.0], "text_input": ["newegg "]}