Package: homegrid
A minimal home grid world environment to evaluate language understanding in interactive agents.🏠 Getting StartedPlay as a human:$pipinstall-e.
$./homegrid/manual_control.pyUse as a gym environment:importgymimporthomegridenv=gym.make("homegrid-task")Seehomegrid/__init__.pyfor the environment configurations used in the paperLearning to Model the World with Language.📑 DocumentationHomeGrid tests whether agents can learn to use language that provides information about the world. In addition to task instructions, the env provides scriptedlanguage hints, simulating knowledge that agents might learn from humans (e.g., in a collaborative setting) or read in text (e.g., on Wikipedia). Agents navigate around a house to find objects and interact with them to perform tasks, while learning how to understand language from experience.⚡️ Quick Infopixel observations (3x3 partial view of the house)discrete action space (movement + object interaction)3 rooms, 7 objects (3 trash bins, 4 trash objects)multitask with language instructions + hintsrandomized object placement and object dynamicsTask Templates (38 total tasks):find theobject/bin: the agent will receive a reward of 1 if it is facing the correct object / binget theobject: the agent will receive a reward of 1 if it has the correct object in inventoryput theobjectin thebin: the agent will receive a reward of 1 if the bin contains the objectmove theobjectto theroom: the agent will receive a reward of 1 if the object is in the roomopen thebin: the agent will receive a reward of 1 if the bin is in the open stateLanguage Types and TemplatesFuture Observations: descriptions of what agents might observe in the future, such as "The plates are in the kitchen.""object/binis in theroom": the object or bin is in the indicated room"i moved theobjectto theroom": the object has been moved to the room"there will beobjectin theroom": the object will spawn in the room in five timestepsDynamics: descriptions of environment dynamics, such as "Pedal to open the compost bin.""actionto open thebin": the indicated action is the correct action to open the binCorrections: interactive, task-specific feedback based on what the agent is currently doing, such as "Turn around.""no, turn around": the agent's distance to the current goal object or bin (given the task) has increased compared to the last timestepEnvironment instances are provided for task instruction + each of the types above inhomegrid/__init__.py.Language is streamed one token per timestep by default. Some strings are higher priority than others and may interrupt a string that is currently being read. Seehomegrid/language_wrappers.py.💻 DevelopmentNew development and extensions to the environment are welcome!Adding new language utterancesSentences are pre-embedded and cached into a file for training efficiency. You'll have to append the additional sentences tohomegrid/homegrid_sentences.txtand re-generate the cached token and embedding file with the following command:pythonscripts/embed_offline.py\--infilehomegrid/homegrid_sentences.txt\--outfilehomegrid/homecook_embeds.pkl\--modelt5Adding new layouts and objectsHomeGrid currently has one layout and a fixed set of objects that are sampled to populate each episode. Many of the receptacles and containers (e.g. cabinets) are disabled for simplicity.To add new layouts, create a new class inhomegrid/layout.py.To add new static (non-interactive) objects, add assets tohomegrid/assets.pyand then specify where they are rendered in thehomegrid/layout.py.To add new interactive objects, additionally specify how they behave inhomegrid/homegrid_base.py:step.AcknowledgmentsHomeGrid is based onMiniGrid.
The environment assets are thanks tolimezuandMounir Tohami.Citation@article{lin2023learning,
         title={Learning to Model the World with Language},
         author={Jessy Lin and Yuqing Du and Olivia Watkins and Danijar Hafner and Pieter Abbeel and Dan Klein and Anca Dragan},
         year={2023},
         eprint={2308.01399},
         archivePrefix={arXiv},
}