jackyliang42 commited on
Commit
4a87cff
1 Parent(s): 8c6fc5d

updated readme

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -12,11 +12,11 @@ license: apache-2.0
12
 
13
  # Code as Policies Tabletop Manipulation Interactive Demo
14
 
15
- This notebook is a part of the open-source code release associated with the paper:
16
 
17
  [Code as Policies: Language Model Programs for Embodied Control](https://code-as-policies.github.io/)
18
 
19
- This notebook gives an interactive demo for the simulated tabletop manipulation domain, seen in the paper section IV.D
20
 
21
  ## Preparations
22
  1. Obtain an [OpenAI API Key](https://openai.com/blog/openai-api/)
@@ -29,7 +29,7 @@ Once you have Codex access you can use `code-davinci-002`. Using the GPT-3 model
29
  2. Click Setup/Reset Simulation
30
  3. Based on the new randomly sampled object names, input an instruction and click Run Instruction. If successful, this will render a video and update the simulation environment visualization.
31
 
32
- You can run instructions in sequence and refer back to previous instructions (e.g. do the same with other blocks, move the same block to the other bowl, etc). Click Setup/Reset Env to reset, and this will clear the current instruction history.
33
 
34
  ## Supported Instructions
35
  * Spatial reasoning (e.g. to the left of the red block, the closest corner, the farthest bowl, the second block from the right)
@@ -50,6 +50,7 @@ Note object names may need to be changed depending the sampled object names.
50
  ## Known Limitations
51
  * In simulation we're using ground truth object poses instead of using vision models. This means that instructions the require knowledge of visual apperances (e.g. darkest bowl, largest object) are not supported.
52
  * Currently, the low-level pick place primitive does not do collision checking, so if there are many objects on the table, placing actions may incur collisions.
 
53
  * Prompt saturation - if too many instructions (10+) are executed in a row, then the LLM may start to ignore examples in the early parts of the prompt.
54
  * Ambiguous instructions - if a given instruction doesn't lead to the desired actions, try rephrasing it to remove ambiguities (e.g. place the block on the closest bowl -> place the block on its closest bowl)
55
  * Maximum token length - you may hit the maximum token length if running multiple commands in sequence. Please reset the simulation when this happens.
 
12
 
13
  # Code as Policies Tabletop Manipulation Interactive Demo
14
 
15
+ This demo is from the paper:
16
 
17
  [Code as Policies: Language Model Programs for Embodied Control](https://code-as-policies.github.io/)
18
 
19
+ Below is an interactive demo for the simulated tabletop manipulation domain, seen in the paper section IV.D
20
 
21
  ## Preparations
22
  1. Obtain an [OpenAI API Key](https://openai.com/blog/openai-api/)
 
29
  2. Click Setup/Reset Simulation
30
  3. Based on the new randomly sampled object names, input an instruction and click Run Instruction. If successful, this will render a video and update the simulation environment visualization.
31
 
32
+ You can run instructions in sequence and refer back to previous instructions (e.g. do the same with other blocks, move the same block to the other bowl, etc). To reset, click Setup/Reset Env, and this will clear the current instruction history.
33
 
34
  ## Supported Instructions
35
  * Spatial reasoning (e.g. to the left of the red block, the closest corner, the farthest bowl, the second block from the right)
 
50
  ## Known Limitations
51
  * In simulation we're using ground truth object poses instead of using vision models. This means that instructions the require knowledge of visual apperances (e.g. darkest bowl, largest object) are not supported.
52
  * Currently, the low-level pick place primitive does not do collision checking, so if there are many objects on the table, placing actions may incur collisions.
53
+ * The pick place primitive is also unable to pick up bowls.
54
  * Prompt saturation - if too many instructions (10+) are executed in a row, then the LLM may start to ignore examples in the early parts of the prompt.
55
  * Ambiguous instructions - if a given instruction doesn't lead to the desired actions, try rephrasing it to remove ambiguities (e.g. place the block on the closest bowl -> place the block on its closest bowl)
56
  * Maximum token length - you may hit the maximum token length if running multiple commands in sequence. Please reset the simulation when this happens.