Episode ID: 15837719114401510671.388-477.Go to cart
Step ID: 0
Goal: Go to cart

Previous Actions:


Current Screen:
UI Annotations Text: ['' '' 'A' 'bestbuy.com' '' 'Your Store:' 'Bangor' '' '' '' 'Search'
 'Black Friday' 'Deals' 'Right' 'Now' 'Top Deals' 'Deal' 'of the' 'Day'
 'Black Friday' 'Deals' 'ohtno' 'saving' 'with' 'Black' 'Friday' 'prices'
 'Start' 'today.' 'See Black Friday FAQs' 'Limited quantities.'
 'No rainchecks.' 'Shop the Deals' 'U' 'Gystal LHD' 'Apple' 'TVs &'
 'Laptops &' 'Computers' 'Projectors' '' '' '' '' '' '' '' '' '' '' '' ''
 '' '' '' '' '' '']
UI Annotations Types: ['ICON_THREE_DOTS' 'ICON_HOME' 'ICON_SHOPPING_BAG' 'TEXT'
 'ICON_V_DOWNWARD' 'TEXT' 'TEXT' 'ICON_SHOPPING_CART' 'ICON_THREE_BARS'
 'ICON_MAGNIFYING_GLASS' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT' 'TEXT'
 'TEXT' 'TEXT' 'TEXT' 'ICON_V_BACKWARD' '' '' '' '' '' '' '' '' '' '' ''
 '' '' '' '' '' '']
Positions: [0.04529915 0.9203704  0.02991453 0.01666667 0.04615385 0.04259259
 0.02905983 0.03333334 0.04700855 0.16851851 0.02564103 0.02592593
 0.05128205 0.23148148 0.02222222 0.27407408 0.10769231 0.92777777
 0.01794872 0.02962963 0.11196581 0.03888889 0.01111111 0.16851851
 0.11196581 0.21666667 0.01367521 0.11111111 0.15128206 0.89444447
 0.04273504 0.05185185 0.15811966 0.25       0.03076923 0.04444445
 0.16239317 0.77592593 0.02564103 0.03518518 0.17179488 0.39444444
 0.01367521 0.10555556 0.23247863 0.04444445 0.01367521 0.17777778
 0.23247863 0.23518519 0.01111111 0.07962963 0.23247863 0.32777777
 0.01367521 0.07592592 0.23247863 0.41666666 0.01111111 0.06481481
 0.23247863 0.53518516 0.01367521 0.14629629 0.23247863 0.73703706
 0.01111111 0.06111111 0.23247863 0.81111109 0.01111111 0.08703703
 0.23247863 0.91111112 0.01367521 0.05185185 0.2905983  0.04814815
 0.05555556 0.72592592 0.34615386 0.04814815 0.04102564 0.33333334
 0.39914531 0.07592592 0.03333334 0.32407406 0.45299146 0.17777778
 0.02307692 0.15555556 0.45299146 0.35185185 0.01880342 0.10185185
 0.45299146 0.47592592 0.01880342 0.12777779 0.45299146 0.6240741
 0.02307692 0.14444445 0.45299146 0.78888887 0.02307692 0.14814815
 0.45384616 0.04259259 0.01794872 0.11666667 0.48461539 0.04259259
 0.02307692 0.13518518 0.53162396 0.57037038 0.01623932 0.33703703
 0.53333336 0.04259259 0.01452991 0.28148147 0.53333336 0.33703703
 0.01196581 0.22222222 0.5709402  0.06296296 0.01452991 0.23518519
 0.70427352 0.41296297 0.00769231 0.0537037  0.73760682 0.50185186
 0.00854701 0.07592592 0.80170941 0.77592593 0.01367521 0.09259259
 0.80256408 0.45740741 0.01111111 0.08518519 0.80341882 0.1037037
 0.01282051 0.15555556 0.82222223 0.09259259 0.01282051 0.17592593
 0.82222223 0.42037037 0.01196581 0.16111112 0.95811963 0.19629629
 0.02478633 0.02592593 0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.        ]


What should the Android do next?

Action: {"action_type": 4, "action_description": "Describe what the action means if needed", "coordinates_touch": [0.18049371242523193, 0.9365879893302917], "coordinates_lift": [0.18049371242523193, 0.9365879893302917], "text_input": [""]}