SlowGuess/NeurIPS_2023_OCR huggingface dataset:
step2:   
main: | You are a Python code generator for robotics. The users will first provide the imported Python modules. Then, for each code that they want you to generate, they provide the requirement in Python docstrings. <end_of_system_message> #Python kitchen robot control script from env_utils import get_obj_location, is_holding, is_in_a_stack, get_obj_that_is_underneath All the code should follow the specification. env_utils Specifications: get_obj_location(obj) Parameters: obj(str):A string that has the type of object (one of ' lettuce', 'patty', 'bottom_bun', 'top_bun') and an id. For example : 'lettuce5'，'patty7'，'bottom_bun1'，'top_bun4'
(str) location that the object is currently at. A string that has the type of location (one of 'table', 'cutting_board', 'stove') and an id. For example: 'table2', 'cutting_board1', 'stove5'
is_holding(obj)
obj(str): A string that has the type of object (one of 'lettuce', 'patty', 'bottom_bun', 'top_bun') and an id. For example: 'lettuce5', 'patty7', 'bottom_bun1', 'top_bun4'
(bool) true if the robot is currently holding obj is_in_a_stack(obj)
(bool) true if the obj is in a stack and the robot needs to unstack the obj
get_obj_that_is_underneath(obj_at_top)
obj_at_top (str): A string that has the type of object (one of 'lettuce', 'patty', 'bottom_bun', 'top_bun') and an id. For example: 'lettuce5', 'patty7', 'bottom_bun1', 'top_bun4'
(str) The name of the object that is directly underneath the obj_at_top. A string that has the type of object (one of 'lettuce', 'patty', 'bottom_bun', 'top_bun') and an id. For example: 'lettuce5', 'patty7', 'bottom_bun1', 'top_bun4'
The robot is only capable of these basic actions: move, pick, place, stack, unstack, cook, cut.
These functions require the robot to hold the object: place, stack These functions require the robot to not hold anything: pick, unstack
You can define functions that combine two of these actions, but you cannot define new actions.
You can use functions imported above and also call new functions
The robot might not be near any object or near any location specified in the function parameters.
Define the function: cut_object_at_location(obj, location)
def cut_object_at_location(obj, location):
To cut an object, the robot first needs to be holding obj if not is_holding(obj):
If the robot is not holding obj, there are 2 scenarios:
(1) if obj is in a stack ,unstack obj
(2) else, pick up obj.
if is_in_a_stack(obj):
Because obj is in a stack, robot need to move then unstack the obj from the obj_at_bottom first
obj_at_bottom = get_obj_that_is_underneath(obj_at_top= obj) # move then unstack: first you move to the obj_at_bottom's location, then you unstack obj from obj_at_bottom move THEN_unstack(obj_to_unstack  $\equiv$  obj, obj_at_bottom  $=$  obj_at_bottom, unstack_location  $\equiv$  get_obj_location(obj_at_bottom)) else: #move_thenth_pick: first you move to obj's location, then you pick obj up move_thenth_pick(obj  $\equiv$  obj, pick_location  $\equiv$  get_obj_location (obj)) #move then place: first you move to the location to cut at, then you place obj at that location move_thenth_place(obj  $\equiv$  obj, place_location  $\equiv$  location) #cut the object until it is cut cut_until_is_cut(obj  $\equiv$  obj)
Define the function: stack_obj1_on_obj2(obj1, obj2)  
```
```
...
<end_of_example_user_query>
def stack_obj1_on_obj2(obj1, obj2):
    # To stack obj1 on obj2, the robot needs to be holding obj1
    if not is_holding(obj1):
        # If the robot is not holding obj1, there are 2
            # (1) if obj1 is in a stack, unstack obj1
            # (2) else, pick up obj1.
            if is_in_a_stack(obj1):
                # Because obj1 is in a stack, robot need to move
            then unstack the obj from the obj_at_bottom first
            obj_at_bottom = get_obj_that_is_underneath(
obj_at_top=obj1)
            # move then unstack: first you move to the
obj_at_bottom's location, then you unstack obj from obj_at_bottom
            movethen_unstack(obj_to_unstack=obj1,
obj_at_bottom=obj_at_bottom, unstack_location=get_obj_location(
obj_at_bottom))
else:
    # movethen_pick: first you move to obj's location,
    then you pick obj up
    movethen_pick(obj=obj, pick_location=
get_obj_location(obj))
    # determine the location of obj2 to stack on
obj2_location = get_obj_location(obj2)
    # move then stack: first you move to obj2's location, then
you stack obj1 on obj2
    movethenStack(obj_to_stack=obj1, obj_at_bottom=obj2,
stack_location=obj2_location)
...
You can use functions imported above and also call new functions
The robot might not be near any object or near any location specified in the function parameters. Define the function: unstack_obj1_from_obj2(obj1, obj2) "" <end_of_example_user_query> def unstack_obj1_from_obj2(obj1, obj2): # To unstack obj1 from obj2, the robot needs to not hold anything yet. if is_holding(obj1): # Because the robot is holding obj1, unstacking must have been successful already return # determine the location of obj2 to unstack from obj2_location = get_obj_location(obj2) # move then unstack: first you move to obj2's location, then you unstack obj1 from obj2 movethen_unstack(obj_to_unstack  $\equiv$  obj1, obj_at_bottom  $\equiv$  obj2, unstack_location  $\equiv$  obj2_location) -! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 0 You can use functions imported above and also call new functions The robot might not be near any object or near any location specified in the function parameters. Define the function: cook_object_at_location(obj, cook_location) "" <end_of_example_user_query> def cook_object_at_location(obj, cook_location): # To cook an object, the robot first needs to be holding obj if not is_holding(obj): # If the robot is not holding obj, there are 2 scenarios: # (1) if obj is in a stack ,unstack obj # (2) else, pick up obj. if is_in_a_stack(obj): # Because obj is in a stack, robot need to move then unstack the obj from the obj_at_bottom first obj_at_bottom  $=$  get_obj_that_is_underneath(obj_at_top= obj) # move then unstack: first you move to the obj_at_bottom move_then_unstack(obj_to_unstack  $\equiv$  obj, obj_at_bottom= obj_at_bottom, unstack_location  $\equiv$  get_obj_location(obj_at_bottom)) else: # move_then_pick: first you move to obj's location, then you pick obj up move_then_pick(obj  $\equiv$  obj, pick_location  $\equiv$  get_obj_location (obj)) # move then place: first you move to the location to cook at, then you place obj at that location move_then_place(obj  $\equiv$  obj, place_location  $\equiv$  cook_location) # cook the object until it is cooked cook_until_is_cooked(obj  $\equiv$  obj)
step3:
You are a Python code generator for robotics. The users will first provide the imported Python modules. Then, for each code that they want you to generate, they provide the requirement in Python docstrings.
<system_message_separator>
Python kitchen robot control script
from robot_utils import move, pick_up, place, cut, start_cooking, noop, stack, unstack
from env_utils import is_cut, is_cooked, get_obj_location, get_curr_location
All the code should follow the specification.
robot_utils Specifications: move(curr_location, target_location)
Requirement:
The curr_loc cannot be the same as target_location.
Move from the curr_location to the target_location.
curr_location(str): a string that has the type of location (one of 'table', 'cutting_board', 'stove') and an id. For example: 'table2', 'cutting_board1', 'stove5'
target_location(str): a string that has the type of location (one of 'table', 'cutting_board', 'stove') and an id. For example: 'table2', 'cutting_board1', 'stove5' pick_up(obj, loc)
The robot must have moved to loc already, and it cannot be holding anything else.
Pick up the obj from the loc.
obj (str): object to pick. A string that has the type of object (one of 'lettuce', 'patty', 'bottom bun', 'top bun') and an id. For example: 'lettuce5', 'patty7', 'bun1'
loc (str): location to pick the object from. a string that has the type of location (one of 'table', 'cutting_board', 'stove') and an id. For example: 'table2', 'cutting_board1', 'stove5' place(obj, loc)
Place the obj on the loc.
obj (str): object to place. A string that has the type of object (one of 'lettuce', 'patty', 'bottom bun', 'top bun') and an id. For example: 'lettuce5', 'patty7', 'bun1'
loc (str): location to place the object at. a string that has the type of location (one of 'table', 'cutting_board', 'stove') and an id. For example: 'table2', 'cutting_board1', 'stove5' cut(obj)
The robot must be at the same location as obj.
Make progress on cutting the obj. You need to call this function multiple times to finish cutting the obj.
obj(str): object to cut. A string that has the type of a cuttable object (must be 'lettuce') and an id. For example: 'lettuce3', 'lettuce1'  
start_cooking(obj)
Start cooking the obj. You only need to call this once. The obj will take an unknown amount before it is cooked.
obj(str): object to cook. A string that has the type of a cookable object (must be 'patty') and an id. For example: 'patty1', 'patty5'
noop()
stack(obj_to_stack, obj_at_bottom)
The robot must be at the same location as obj_at_bottom.
Stack obj_to_stack on top of obj_at_bottom
obj_to_stack (str): object to stack. A string that has the type of object (one of 'lettuce', 'patty', 'bottom bun', 'top bun') and an id. For example: 'lettuce5', 'patty7', 'bun1'
obj_at_bottom (str): object to stack on top of. A string that has the type of object (one of 'lettuce', 'patty', 'bottom bun', 'top bun') and an id. For example: 'lettuce5', 'patty7', 'bun1'
unstack(obj_to_unstack, obj_at_bottom)
Unstack obj_to_unstack from obj_at_bottom
obj_to_unstack(str): object to unstack. A string that has the type of object (one of 'lettuce', 'patty', 'bottom bun', 'top bun') and an id. For example: 'lettuce5', 'patty7', 'bun1'
obj_at_bottom (str): object to unstack from. A string that has the type of object (one of 'lettuce', 'patty', 'bottom bun', 'top bun') and an id. For example: 'lettuce5', 'patty7', 'bun1'
env_utils Specifications:
is_cut(obj)
obj (str): A string that has the type of a iterable object (must be 'lettuce') and an id. For example: 'lettuce3', 'lettuce1'
boolean)
is_cooked(obj)
obj (str): A string that has the type of a cookable object (must be 'patty') and an id. For example: 'patty1', 'patty2'
boolean) get_curr_location()
(str) location that the robot is currently at. A string that has the type of location (one of 'table', 'cutting_board', 'stove') and an id. For example: 'table2', 'cutting_board1', 'stove5'
get_obj_location(obj)
Parameters: obj(str): A string that has the type of a selectable object (must be 'lettuce') and an id. For example: 'lettuce3', 'lettuce1' Return: (str) location that the object is currently at. A string that has the type of location (one of 'table', 'cutting_board', ' stove') and an id. For example: 'table2', 'cutting_board1', ' stove5' "" examples: -! ... ... "You can only use the functions imported in the header. Define the function: basic_move(target_location) Move to any location specified the target_location. end_of_example_user_query> def basic_move(target_location): if get_curr_location() != target_location: move(get_curr_location(), target_location) -! -... -"" You can only use the functions imported in the header. Define the function: cook_until_isCooked(obj) end_of_example_user_query> def cook_until_isCooked(obj): start_cooking(obj) while not isCooked(obj): noop() -! -... -"" You can only use the functions imported in the header. Define the function: movethen_stack(obj_to_stack, obj_at_bottom) end_of_example_user_query> def movethen_stack(obj_to_stack, obj_at_bottom): # For stacking, because the location is not provided, we need to determine the stack_location stack_location = get_obj_location(obj_at_bottom) if get_curr_location() != stack_location: move(get_curr_location(), stack_location) stack(obj_to_stack, obj_at_bottom)
18.3 EPIC Kitchens Task Prompts
18.3.1 Recursive Summarization
recursive_summarization:
main: |
  You are a helpful summarizer that recursively summarizes a trajectory into a more and more compacted form.
  <end_of_system_message>
  You are given a trajectory. Your goal is to summarize the trajectory into a more compacted form and then determine whether the state trajectory is sufficiently summarized.
You must respond using the following form (you must generate [[end of response]] at the very end):  
[[Reasoning:]]
[[Summarized trajectory:]]  
You should actually summarize the input trajectory and output it here.
* A low-level action trajectory is represented as an unordered list.
* The low-level actions must be one of the following: "the robot moved from location1 to location2", "the robot picked up item1", "the robot placed down item1 at location1 ", "the robot soaped item1 until it became soapy because item1 was dirty", "the robot rinsed item1 until it became clean", "the robot turned on tap1", "the robot turned off tap2". You should never define new low-level actions.
* A high-level subtask trajectory is represented as an unordered list. This high-level subtask should refer to one continuous section of the states. For example, you cannot say "at states 1-5, and states 10-15, the robot did". There can only be one interval of states.
* The high-level subtask must be one of the following: "brought [ITEMS] from [LOCATION 1] to [LOCATION 2]", "soaped [ITEM] until it became soap", "rinsed [ITEM] until it became clean", "picked up [ITEM]", "placed [ITEMS] in [LOCATION]", "turned on [TAP]", "turned off [TAP]". You should never define new high-level subtasks. You must choose from the list above.
examples:   
- | [[Input Trajectory:]] [Scenario 1] Wash objects at a sink. All cleaned objects should be placed in the sink. Objects to clean - spoon_1 at countertop_1
- fork_1 at countertop_1
Initial Condition (State 1):  
at('countertop_1')  
is_dirty('spoon_1')  
is_dirty('fork_1')  
near('spoon_1')  
near('fork_1')
State 2:  
is_in_hand('fork_1')  
is_in_hand('spoon_1')  
far('sink_1')
State 3: at('sink_1')
State 4:  
is(soapy('fork_1'))  
is_on((`tap_1'))
State 5: is(soapy('spoon_1')
State 6: is_clean('spoon_1')
State 7 is Cleaner('fork_1')
State 8  
is_off('tap_1')  
in('fork_1', 'sink_1')  
in('spoon_1', 'sink_1')  
<end_of_example_query>  
[[Reasoning:]]
The input trajectory contains state predicates because the trajectory talks about the status of robot (at('countertop_1')) and the objects (is_dirty('spoon_1')).
will summarize the state trajectory into low-level actions. Low-level actions are not compacted enough yet because low-leasons can still be combined into high-level subtasks.
The new trajectory will NOT be sufficiently summarized.  
[[Is the new trajectory sufficiently summarized? (yes/no):]] no
[[Summarized trajectory:]]  
* In [Scenario 1], at state 1-2, the robot picked up spoon_1.  
e robot picked up fork_1.
* At state 2-3, the robot moved from countertop_1 to sink_1.
* At state 3-4, the robot turned on tap_1. The robot soaped rk_1 until it became soapy because fork_1 was dirty.
* At state 4-5, the robot soaped spoon_1 until it became soapy because spoon_1 was dirty.
* At state 5-6, the robot rinsed spoon_1 until it became clean.
* At state 6-7, the robot rinsed fork_1 until it became clean.
* At state 7-8, the robot turned off tap_1. The robot placed spoon_1 in sink_1. The robot placed fork_1 in sink_1.
[[end of response]]  
|  
[[Input Trajectory:]]  
[Scenario 1]
Wash objects at a sink. All cleaned objects should be placed in the sink.
Objects to clean - spoon_1 at dishwasher_1
-plate_1 at sink_1
-plate_2 at sink_1
Initial Condition (State 1):  
in('spoon_1', 'dishwasher_1')  
in('plate_1', 'sink_1')  
in('plate_2', 'sink_1')
State 2: is_in_hand('spoon_1')
State 3:  
at('sink_1')  
is_dirty('spoon_1')
State 4: is_on('tap_1')
State 5: is_foamy('sponge_1')
State 6: is(soapy('spoon_1'))
State 7: is.clean('spoon_1')
State 8: in('spoon_1', 'sink_1')
State 9:  
is_off('tap_1')  
is_in_hand('plate_1')  
is_dirty('plate_1')
State 10:  
is_on((`tap_1`)  
is_in_hand('sponge_1`)  
is_foamy('sponge_1`)
State 11: is.soapy('plate_1')
State 12: is_clean('plate_1')
State 13: in('plate_1', 'sink_1')
State 14: is_off('tap_1')
State 15:  
is_in_hand('plate_2')  
is_dirty('plate_2')
State 16:  
is_on('tap_1')
State 17: is_foamy('sponge_1')
State 18: is.soapy('plate_2')
State 19: is.clean('plate_2') in('plate_2', 'sink_1') <end_of_example_query> [[Reasoning:]] The input trajectory contains state predicates because the trajectory talks about the status of the objects (in('spoon_1', 'dishwasher_1')) I will summarize the state trajectory into low-level actions. Low-level actions are not compacted enough yet because low-level actions can still be combined into high-level subtasks. The new trajectory will NOT be sufficiently summarized. [[Is the new trajectory sufficiently summarized? (yes/no):]] no [[Summarized trajectory:]] * In [Scenario 1], at state 1-2, the robot picked up spoon_1 from dishwasher_1. * At state 2-3, the robot moved from dishwasher_1 to sink_1. * At state 3-4, the robot turned on tap_1. * At state 4-5, the robot picked up sponge_1, which was foamy. * At state 5-6, the robot soaped spoon_1 until it became soapy because spoon_1 was dirty. * At state 6-7, the robot rinsed spoon_1 until it became clean. * At state 7-8, the robot placed spoon_1 in sink_1. The robot placed sponge_1 in sink_1. * At state 8-9, the robot turned off tap_1. The robot picked up plate_1. * At state 9-10, the robot turned on tap_1. The robot picked up sponge_1, which was foamy. * At state 10-11, the robot soaped plate_1 until it became soapy because plate_1 was dirty. * At state 11-12, the robot rinsed plate_1 until it became clean. * At state 12-13, the robot placed plate_1 in sink_1. The robot placed sponge_1 in sink_1. * At state 13-14, the robot turned off tap_1. * At state 14-15, the robot picked up plate_2. * At state 15-16, the robot turned on tap_1. * At state 16-17, the robot picked up sponge_1, which was foamy. * At state 17-18, the robot soaped plate_2 until it became soapy because plate_2 was dirty. * At state 18-19, the robot rinsed plate_2 until it became clean. The robot placed plate_2 in sink_1. The robot turned off tap_1. [[end of response]] | [[Input Trajectory:]] [Scenario 1] * In [Scenario 1], at state 1-2, the robot picked up spoon_1. The robot picked up fork_1. * At state 2-3, the robot moved from countertop_1 to sink_1. * At state 3-4, the robot turned on tap_1. The robot soaped fork_1 until it became soapy because fork_1 was dirty. * At state 4-5, the robot soaped spoon_1 until it became soapy because spoon_1 was dirty. * At state 5-6, the robot rinsed spoon_1 until it became clean. * At state 6-7, the robot rinsed fork_1 until it became clean. * At state 7-8, the robot turned off tap_1. The robot placed spoon_1 in sink_1. The robot placed fork_1 in sink_1. <end_of_example_query> [[Reasoning:]] The input trajectory contains low-level actions because the trajectory mentions "picked up", "moved", etc. I will summarize the low-level action trajectory into high-level subtasks.
High-level subtasks are compacted enough because they cannot be combined together without losing important information.
The new trajectory will be sufficiently summarized.
[[Is the new trajectory sufficiently summarized? (yes/no):]]
yes
[[Summarized trajectory:]]
* In [Scenario 1], at state 1-3, the robot brought spoon_1 and fork_1 from countertop_1 to the sink_1.
* At state 3-4, turned on tap_1
* At state 3-4, soaped fork_1 until it became soapy
* At state 4-5, soaped spoon_1 until it became soapy
* At state 5-6, rinsed spoon_1 until it became clean.
* At state 6-7, rinsed fork_1 until it became clean.
* At state 7-8, turned off tap_1, and placed spoon_1 and fork_1 in sink_1.
[[end of response]]
-
[[Input Trajectory:]]
[Scenario 1]
* In [Scenario 1], at state 1-2, the robot picked up spoon_1 from dishwasher_1.
* At state 2-3, the robot moved from dishwasher_1 to sink_1.
* At state 3-4, the robot turned on tap_1.
* At state 4-5, the robot picked up sponge_1, which was foamy.
* At state 5-6, the robot soaped spoon_1 until it became soapy because spoon_1 was dirty.
* At state 6-7, the robot rinsed spoon_1 until it became clean.
* At state 7-8, the robot placed spoon_1 in sink_1. The robot placed sponge_1 in sink_1.
* At state 8-9, the robot turned off tap_1. The robot picked up plate_1.
* At state 9-10, the robot turned on tap_1. The robot picked up sponge_1, which was foamy.
* At state 10-11, the robot soaped plate_1 until it became soapy because plate_1 was dirty.
* At state 11-12, the robot rinsed plate_1 until it became clean.
* At state 12-13, the robot placed plate_1 in sink_1. The robot placed sponge_1 in sink_1.
* At state 13-14, the robot turned off tap_1.
* At state 14-15, the robot picked up plate_2.
* At state 15-16, the robot turned on tap_1.
* At state 16-17, the robot picked up sponge_1, which was foamy.
* At state 17-18, the robot soaped plate_2 until it became soapy because plate_2 was dirty.
* * At state 18- 19, the robot rinsed plate_2 until it became clean
. The robot placed plate_2 in sink_1. The robot turned off tap_1.
<end_of_example_query>
[[Reasoning:]]
The input trajectory contains low-level actions because the trajectory mentions "picked up", "moved", etc.
I will summarize the low-level action trajectory into high-level subtasks.
High-level subtasks are compacted enough because they cannot be combined together without losing important information.
The new trajectory will be sufficiently summarized.
[[Is the new trajectory sufficiently summarized? (yes/no):]] yes
[[Summarized trajectory:]]
* In [Scenario 1], at state 1-3, brought spoon_1 from dishwasher_1 to sink_1
* At state 3-4, turned on tap_1.
* At state 4-6, soaped spoon_1 until it became soapy
* At state 6-7, rinsed spoon_1 until it became clean
* At state 7-8, placed spoon_1 in sink_1
\*At state 8-9, turned off tap_1 and picked up plate_1 \* At state 9-10, turned on tap_1. \*At state 10-11, soaped plate_1 until it became soapy \* At state 11-12, rinsed plate_1 until it became clean \* At state 12-13, placed plate_1 in sink_1 \* At state 13-15, turned off tap_1 and picked up plate_2 \* At state 15-16, turned on tap_1 \* At state 16-18, soaped plate_2 until it became soapy \* At state 18-19, rinsed plate_2 until it became clean, turned off tap_1, and placed plate_2 in sink_1 [end of response]]
summary_2_spec:  
main: |  
You are a helpful assistant who analyzes the trajectories and summarizes them into a concise pseudocode-style task specification.
<end_of_system_message>
You are given (1) a high-level goal and (2) one or more trajectories where each one represents a different scenario. Your goal is to summarize these trajectories into a compact task specification, written in a pseudocode style.
You must respond using the following format (you must generate [end of response] at the very end):  
[[Reasoning:]]
[[Task Specification:]]  
You should first state the high-level goal. Then, you should say "Specifically:" before outputting the pseudocode-style task specification.  
[[end of response]]
You must follow these rules when you are writing the task specifications.  
* You must write the task specifications in pseudocode style. You should not write the task specification as a list. You cannot number any line.
examples:   
- |   
[[High-Level Goal:]]   
Wash objects at a sink. All cleaned objects should be placed in the sink. Objects to clean - spoon_1 at countertop_1 - fork_1 at countertop_1 Initial Condition (State 1): at(^countertop_1^) is_dirty(^spoon_1^) is_dirty(^fork_1^)