Episode ID: 17248848557436441987.694-754.Go to cart
Step ID: 0
Goal: Go to cart

Previous Actions:


Current Screen:
UI Annotations Text: ['' '' 'bestbuy.com' '' 'Your' 'Store:' 'Bangor' 'Bangor' ''
 'popular times' 'Opens at 10 am' '10:00 am:' 'Often not too busy' '10'
 '4 pm' '1 pm' '7 pm' 'am' '45' 'Bangor' 'Mall' 'Blvd' 'Bangor, ME 04401'
 'Store details' 'Find another store' '' '' '' 'X' 'alienware aurora'
 'Black' 'Friday' 'Deals' 'Right' 'Now' 'Top Deals' 'Deal' 'of the' 'Day'
 'Black Friday' 'Deals' 'right now.' 'vina witb Dloa' 'Cridow niaae' 'tot'
 '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' '' ''
 '']
UI Annotations Types: ['ICON_HOME' 'ICON_ARROW_UPWARD' 'TEXT' 'ICON_V_UPWARD' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'ICON_V_UPWARD' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'ICON_SHOPPING_CART' 'ICON_THREE_BARS' 'ICON_MAGNIFYING_GLASS' 'ICON_X'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'ICON_NAV_BAR_CIRCLE'
 'ICON_NAV_BAR_RECT' 'ICON_V_BACKWARD' '' '' '' '' '' '' '' '' '' '' '' ''
 '' '' '' '' '' '' '' '' '' '']
Positions: [0.04814815 0.04074074 0.03148148 0.03518518 0.05277778 0.91666669
 0.02222222 0.02407407 0.05648148 0.21666667 0.01851852 0.22592592
 0.1175926  0.92962962 0.01666667 0.02777778 0.12037037 0.04074074
 0.01111111 0.06481481 0.12037037 0.11851852 0.01111111 0.08703703
 0.12037037 0.21666667 0.01388889 0.10740741 0.17685185 0.04074074
 0.01759259 0.14444445 0.21481481 0.93333334 0.01574074 0.02407407
 0.21666667 0.69814813 0.01666667 0.21851853 0.21851853 0.03888889
 0.01388889 0.24074075 0.25092593 0.03888889 0.01388889 0.17037037
 0.25092593 0.22037037 0.01666667 0.31481481 0.37962964 0.04444445
 0.01388889 0.03518518 0.37962964 0.53703701 0.01759259 0.08333334
 0.38055557 0.30000001 0.01666667 0.07222223 0.38055557 0.7814815
 0.01666667 0.07962963 0.38333333 0.09259259 0.01018518 0.04629629
 0.41944444 0.03888889 0.01296296 0.03703704 0.41944444 0.08888889
 0.01574074 0.10740741 0.41944444 0.20925926 0.01296296 0.05740741
 0.41944444 0.27962962 0.01296296 0.06111111 0.44074073 0.04074074
 0.01481481 0.27407408 0.47222221 0.04074074 0.01296296 0.18888889
 0.54351854 0.33703703 0.01388889 0.32962963 0.60092592 0.89074075
 0.05       0.06111111 0.61018521 0.25       0.03333334 0.04629629
 0.61388886 0.77777779 0.02777778 0.03518518 0.61574072 0.69999999
 0.02314815 0.02407407 0.625      0.39629629 0.01296296 0.25
 0.69074076 0.04444445 0.01111111 0.07592592 0.69074076 0.13518518
 0.01388889 0.08888889 0.69074076 0.23703703 0.01111111 0.07962963
 0.69074076 0.32962963 0.01388889 0.07592592 0.69074076 0.41851851
 0.01111111 0.06481481 0.69074076 0.53703701 0.01388889 0.14629629
 0.69074076 0.73888886 0.01111111 0.06111111 0.69074076 0.81296295
 0.01111111 0.08703703 0.69074076 0.91296297 0.01388889 0.0537037
 0.75648147 0.05       0.05740741 0.72962964 0.81481481 0.04814815
 0.04629629 0.32222223 0.87314814 0.04629629 0.03518518 0.3611111
 0.92962962 0.20925926 0.01481481 0.34074074 0.92962962 0.58703703
 0.01481481 0.30740741 0.93148148 0.02592593 0.01111111 0.13333334
 0.9546296  0.47962964 0.02962963 0.03518518 0.9546296  0.76851851
 0.03055556 0.03333334 0.95555556 0.19074073 0.02777778 0.02962963
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.        ]


What should the Android do next?

Action: {"action_type": 4, "action_description": "Describe what the action means if needed", "coordinates_touch": [0.6286711096763611, 0.9269193410873413], "coordinates_lift": [0.6286711096763611, 0.9269193410873413], "text_input": [""]}