openinterx/Ego-ST-bench huggingface dataset:
Arrive at yellow table to talk to friends, eat fruit, turn left to sit and chat on sofa next to black coffee table, walk straight back to starting point
Turn right to yellow dining table, talk to friend, pick up snack and eat it, continue talking turn right to black coffee table couch sit down and chat then walk straight back to start
Open the top - loading washing machine on the right. Take out the clothes inside and put them into the front - loading washing machine on the left. Reach over the green backpack above the front - loading washing machine, press the button to start the front - loading washing machine, and then take out the mobile phone.
Put down the mobile phone, pick up the white napkin on the right - hand side. Wipe the coffee spilled on the table from the pink cup repeatedly until it's clean, and then turn on the mobile phone.
Stand at the door. Pick up the plastic bag at the door. Then open the door and enter the room. Stand still and talk to the people in the room. Then turn left and put the plastic bag in hand on the table. And return to the original place. Walk further into the room.
First, prepare to pick up the plastic bowl with red watermelon on the table. Then move the left hand to the yellow porcelain bowl. Subsequently, raise the hand to signal and talk to the people at the door. Finally, pick up the canned drink on the right and start drinking.
Get up from the sofa. Pick up the book on the far - away position of the table, put it on the knee, open it, and turn 2 pages to the left. Raise the hand to the person on the right and talk, and then turn 6 pages to the left again.
Throw the scattered chess pieces on the table and the chessboard into the box. Pick up the transparent goblet on the table, pour the water in it onto the table. Subsequently, pick up the white toilet paper on the right - hand side of the table and wipe the water clean. Then continue to throw the remaining chess pieces into the chess box. Finally, touch the chess box with the right hand.
Talk to people in front of the white table and raise the left hand to check the watch. After talking to people for a few more words, raise the left hand to check the watch again, then get up. Walk a few steps to the left. Then turn left and walk through the sofa and the table in the living room. Subsequently, turn right to reach the door, open the door, and go out.
Stand at the door. Open the door, turn around and close the door. Walk forward. Walk through the space between the sofa and the wall and continue walking forward. Subsequently, turn left and enter the bedroom. Throw the suitcase in hand onto the bed in the middle of the bedroom. Then open the zipper of the suitcase and open the suitcase. Take out the clothes from the suitcase and put them on the left side of the suitcase. Finally, close the suitcase.
Stand at the door, pick up the two paper bags outside the door, enter the door, turn around, and close the door. Then turn right and enter the room to talk to people. Put the two paper bags in hand on the white table on the left - hand side. Take a pack of potato chips from the paper bag and put it on the table, and then take out two boxes of eggs from the paper bag. Turn around, open the refrigerator, and put the eggs in. Finally, close the refrigerator.
Take one step to the left and put the chess pieces on the table and the chessboard into the chess box. Then fold the chessboard and cover it on the chess box. Also put some papers on the right - hand side of the chess box on the chess box. Then take the chess box out of the chess box cover. Put the cover of the chess box on the chess box. Finally, stand up, hold the chess box, turn left three times. Walk to the brown table in front and put the chess box on the brown table in front.
Walk to the left, walk to the left side of the sofa. Then walk to the front of the sofa. Sit down on the sofa. Open the book in hand and turn 3 pages to the right. Then turn 3 pages to the left. Finally, close the book and put it on the table in front.
Sit at the table and talk to people for two sentences. Then get up and move a few steps to the right. After passing the corner of the table, move a few more steps to the right. Then keep walking forward, pick up the cup on the table in front, walk backwards to the table, turn right. Walk a few steps forward, then turn right again, keep walking forward until you reach the small brown table on the left - hand side. Put the cup on it and sit down. While talking to people, pick up the cup with the right hand and take a sip of the coffee in the cup, then put the cup down.
After talking to people, turn around backwards, pick up the coffee pot on the platform, and pour the coffee in the pot into the blue water cup on the right - hand side of the platform. Put the kettle back, pick up the blue water cup, and put it on another platform behind. Then move one step to the right and keep walking forward until you reach the small brown table in front and sit down. Talk to people.
Turn right to open the orange door. Enter the room, turn around, close the door, then turn around and keep walking diagonally to the left - front. Go downstairs to the corner of the stairs. Keep turning around backwards and go downstairs. Then turn left and keep walking forward and enter the room. Turn right to come to the bedside, put the black shoulder bag on the bed. Open the zipper of the black shoulder bag. Then pick up the mobile phone and browse it. Put down the mobile phone, take out the clothes from the black shoulder bag, and put them on the bed. Finally, pick up the mobile phone again.
Stand up from the table, move a few steps to the left, come to the sofa, and sit down. Keep talking to people.
Have a conversation with others.
Open the door from the entrance, walk in and turn around to close the door. Then keep walking forward. Turn right and push the suitcase forward. Walk to the corner, turn right again and enter a room with an open door. Walk to the middle of the bed and the mirror. Squat down and open the suitcase. Put the clothes in the suitcase on the right side of the suitcase. Then put two small boxes on the table on the right. Then open the interlayer of the suitcase, take out the clothes inside and put them on the right side of the suitcase.
Have a conversation with others. Then put down the coffee cup in hand, take the paper bag and a bottle of milk that are handed over. Put the paper bag on the table, open the milk with both hands. Pour the milk into the original water cup. Close the milk and shake it. Take out the food from the paper bag and eat it. Then stir the milk with a spoon in the right hand. Then take out the food from the paper bag and eat it again. Finally, pick up the milk with both hands and drink it.
Move from the right side of the person in the yellow clothes to the left side. Move the cup and the eggs on the table, put down the cup, turn around and walk to the refrigerator. Open the refrigerator and put the eggs in. Then close the refrigerator. Take the yellow and green plates with food, walk forward, turn right and enter the dining room. Walk to the white table, put down the plates and sit down. Pick up the fork, fork the food, then put down the fork. Have a conversation with others. Then pick up the fork again, fork the food, and put down the fork again.
Wipe the tabletop with toilet paper using the left hand while watching TV and having a conversation with others.
Open the green bag. Put all the clothes in the black suitcase in front into the green bag. Then pick up the bag and stand up with the phone in hand. Squat down again and touch the suitcase. Stand up again. Then keep walking forward, passing through a door. Walk to the end, turn right and keep walking forward. Walk through the living room and keep walking forward, then turn left and enter the laundry room. Put all the clothes in the green bag into the top - loading washing machine. Then open the cabinet above the washing machine, take out the brown plastic bucket and open it. Take out the things inside and put them into the top - loading washing machine. Then close the brown plastic bucket and put it back in place. Then close the lid of the top - loading washing machine and press the button of the washing machine.
Pick up the mobile phone, stand up from the table, move a few steps diagonally to the right - front, sit down in front of the sofa. Then find a video on the mobile phone, turn the screen to landscape mode, and talk with others while watching the video.
Sit on the sofa, open the book in hand, and turn one page to the left. Then talk with the person on the right. Then turn the book 12 more pages to the left. Finally, close the book.
Turn around backwards, walk to the sofa, and sit down. Talk with the person on the right.
Walk forward, pass by the table, turn right and walk between the table and the small sofa, then sit down on the right side of the big sofa. Then talk with the person on the left.
Talk with others. Put the things in hand into the small bowl on the table in front. Then put the book on the knee onto the table in front. And keep talking with others. During the conversation, repeatedly put the things in hand into the small bowl on the table in front.
Open the washing machine, and then neatly fold and arrange the blue pants, red top, purple top, black top, and socks inside, and place them on the washing machine.
Stand in front of the orange door and knock on it. After the door opens, walk in. Walk two steps, turn right and enter the living room, then walk forward. Sit down in front of the sofa. Take the water cup handed over, take two sips, and put the water cup on the table in front. Talk with others.
Lie in bed and browse the mobile phone.
Get up from the bed, walk diagonally to the left - front and out of the room. Keep walking forward to the kitchen door. Turn right and enter the kitchen, then keep walking forward to the sink, turn on the faucet and wash the tableware in the sink. Then turn off the faucet, open the lid of the coffee machine next to the sink. Insert the faucet into the coffee machine to fill it with water, turn off the faucet, close the coffee machine and press the button of the coffee machine to start it. Then take a few steps back and then move forward to take out the kettle of the coffee machine and pour the coffee into a white water cup. Put the kettle back in place, then hold the white water cup, turn left and walk forward, put the water cup on the table in front of the sofa on the left - hand side. Then sit down on the sofa. Put the white water cup on the table. Then pick it up again.
Talk with others. Then stand up and turn left, walk forward to the refrigerator, open the refrigerator, and take out the milk. Close the refrigerator. Turn right and walk two steps forward. Pick up the paper bag on the table. Step back to the original place, hand out the paper bag and the milk, and then sit down. Then stand up, turn left and walk forward to the right side of the refrigerator, open the drawer. Pick up the spoon, close the drawer. And return to the original place again. Put the spoon into the blue cup on the table, then open the milk bottle cap with the right hand, put the cap back on, sit down and then stand up again. Pick up the milk, walk forward to the refrigerator, open the refrigerator. Put the milk in, then close the refrigerator. Turn around backwards and walk forward, return to the original place.
Open and close (something, not clear what from the context), stand up, walk diagonally to the left - front. Open the door. Then return and sit down again, open the book and read.
Open the chess box, take out the yellow chess pieces, then pour all the red chess pieces out of the chess box on the opposite side of the table. Put the chess box on the right. Start playing chess with the person opposite.
Pick up the blue stand in the chess box and assemble it. Then start playing chess with the person opposite.
Open the suitcase, pick up some items in the suitcase, stand up, turn left and walk forward to the sink. Put the items down. Turn around and walk back to the suitcase, squat down. Repeatedly pick up the clothes in the suitcase, stand up, open the drawer of the cabinet next to the suitcase, put the clothes in and close the cabinet. Then close the suitcase and continue to return to the sink, open the zippered bag on the sink, take out the items inside and put them on the sink countertop.
Walk forward, enter the living room. Keep walking forward, walk through the space between the sofa and the table in the living room. Stop. Turn around backwards, continue to walk through the space between the sofa and the living room. Then turn left, walk to the sofa and sit down. Open the mobile phone and play games. Then stand up, pick up the mobile phone on the upper - left corner of the table, sit down again and continue to play games. Then exit the game and put the mobile phone on the table.
Pick up the items in the paper bag on the chair in front, turn right and walk forward, put the items on the table. Then turn right and walk forward to the closet next to the refrigerator, open the closet. Take out two plates. Subsequently, take out another plate. Close the closet. Open the drawer next to it. Take some forks and spoons. And continue to walk forward to the table, put the plates, forks, and spoons next to the items on the table. Then turn around and walk to the refrigerator, open the refrigerator. Then close the refrigerator.
Talk with others and walk forward. Pick up two cups of coffee on the counter and put them on the table behind. Pick up the two cups of coffee again, turn left and walk forward, and put the two cups of coffee on the front table. Turn left, walk forward and then return to the original place. Pull out the chair and sit on it. Open the bag of potato chips on the table and take some to eat.
Walk forward and sit on the sofa on the right - hand side, and talk with others.
Stand up and turn right. Move the red clothes aside. Walk forward between the sofa and the table. Talk with others. Turn right and walk forward to the left side of the table. Then step back to the original place. Then move diagonally to the right - front and enter the kitchen following the person in the yellow top. Keep walking forward, pass through the kitchen. Turn right and walk forward, enter the patio. Turn around and return to the kitchen and close the patio door.
Walk forward to the refrigerator, open the refrigerator on the right - hand side. Tidy up the items in the refrigerator, close the refrigerator. Open the closets on the left, upper - left, upper, and upper - right sides of the refrigerator in turn and then close them. Turn right and walk forward. Then turn around backwards, walk forward, squat down beside the refrigerator, open the cabinet under the refrigerator. Then close it. Then turn around, pick up the pen on the counter and write on the paper. Turn around, open the refrigerator and then close it. Turn around and return to the original place to continue writing. Turn left and walk forward, touch the things on the counter and then return to the original place to continue writing.
Sit on the sofa and play mobile games.
Pick up the black bag on the bed and then put it down. Then pick up the small black bag in front, turn left and walk forward, and put it on the table on the right - hand side. Then return to the bedside. Repeatedly pick up a piece of clothing on the bed and tidy it up. Walk to the foot of the bed and put the clothes on the rack at the foot of the bed. Finally, pick up the black bag at the head of the bed and put it under the rack at the corner of the bed.
Stand in front of the door, open the door and then close it. Turn around and walk forward, then turn right and enter the living room. Turn left and walk forward, open the closet on the right side of the refrigerator. Close it and then open the closet further to the right, take out the transparent glass. Walk to the refrigerator. Fill the glass with drinking water. Turn right and walk diagonally to the right - front. Hand out the glass. Take one step to the left. Stop in front of the sofa and sit down. Talk with others.
Open the drawer and take out the black spatula. Turn left, open the second - layer drawer, and take out the disc. Open the left - hand cabinet in turn, repeatedly close it. Turn left to the countertop to get the flour, open the package, open the drawer and take out the bowl. Turn right, squat down and open the corner cabinet, take out the red basin. Pour the flour into the red basin. Turn left, open the refrigerator, take out the milk and eggs. Take two eggs and then put the rest back in the refrigerator. Take out a box of berries and close the refrigerator. Crack the two eggs into the red flour basin. Turn left and throw the eggshells into the trash can in the cabinet. Turn right to wash hands, pour the milk into the red flour basin. Pull out the first - layer drawer, take out the mixer, and stir the flour batter. Add more milk, turn left and put the milk back in the refrigerator. Open the refrigerator and take out the food. Open the cabinet and get the frying pan. Adjust the heat, feel the temperature, open the cheese. Turn right to get the scraper from the first - layer of the cabinet at the back. Go to the countertop to get the plate. Use the scraper to scrape the cheese into the frying pan, turn the cheese to melt it. Put the remaining unmelted cheese back into the cheese box. Take out the black spatula and stir the cheese in the pan. Pour the batter into the frying pan. Use your hand to scrape off the excess batter. Turn right to get a wooden board from the upper cabinet. Put the wooden board on the countertop, turn right, open the drawer and take out the knife. Wash the berries, put the berries on the wooden board and cut them. Turn left, use the black spatula to scoop the pancake, use the scraper to take a small piece of cheese and put it into the frying pan. Use the black spatula to flip the pancake. Repeat flipping, put the pancake on the plate. Use the scraper to scrape a small piece of cheese into the frying pan. Adjust the heat. Use the scraper to turn the cheese. Pour the batter into the frying pan and heat it. Turn the frying pan, turn right, open the refrigerator. Take out the items, turn right and walk forward, and put them on the table. Turn left, return to the frying pan and check the phone. Use the black spatula to stir - fry the pancake. Wait for the batter to heat up, look up and look out of the window to the right, repeat looking up, put the pancake on the plate, turn right, pick up the berries and put them on the plate. Turn left to turn off the heat. Turn right, return to the plate and check the phone.
Open the box on the right - hand side, repeatedly put the clothes on the bed into the box, turn around backwards, go out and walk forward. Stop and then open the washing machine on the right - hand side. Put the clothes in the box into the washing machine. Close the washing machine and press the button of the washing machine. Take down the laundry detergent on the shelf above the washing machine. Open the washing machine, pour the laundry detergent into the washing machine, then close the washing machine and put the laundry detergent back in place.
Open the mobile phone. Play a video. Put the mobile phone on the left - hand table. Pick up the coffee and bread on the right - hand table and put them on the left - hand table. Sit down, watch the video played on the mobile phone and start eating.
Open the left - hand closet and close it. Move two steps to the right, open the upper closet and close it. Turn around backwards and open the refrigerator in front. Then close the refrigerator, open the closet on the right side of the refrigerator. And close it. Open the closet on the left side of the refrigerator and close it. Then turn around backwards, walk forward, pick up the pen on the counter and write on the paper. Then turn around, open the refrigerator. Close the refrigerator again, return to the original place and continue writing.
Open the mobile phone, put the mobile phone on the table. Look for the mobile phone on the right. Play music. Walk forward, pick up the broom and sweep the floor. Turn left to clean the countertop. Turn left again to clean the left - hand countertop. Walk forward to clean the countertop of the cabinet. Turn left to clean the second and third layers of the cabinet. Turn left again to clean the TV cabinet. Turn right, squat down, pick up the skateboard, walk forward and put the skateboard in the corner. Put the cylinder on the skateboard. Turn right, pick up the broom and sweep the floor. Turn right again and walk forward, sweeping the floor as you go. Then turn left, walk forward and sweep the floor as you go. Turn left again and walk forward, pick up the pillow and put it on the sofa. Turn left, put the pillow on the table onto the sofa. Turn right and continue sweeping the floor. Turn right and put away the cleaning tools on the TV cabinet. The tools fall to the ground, squat down and pick them up. And clean the TV cabinet again. Turn left, put the cleaning tools in the corner. Pick up the broom and put it together with the cleaning tools. Turn left, pick up the mobile phone on the table and check the phone.
Pick up the cup on the counter in front of you and drink from it, then put the cup down and turn backwards.Go all the way forward to the paper bag on the other counter, take two food items out of the paper bag turn right into the room and place the two food items on the table in the house.Turn around and return to the paper bag.Open the wardrobe above the paper bag, take out a green bowl and close the wardrobe.Turn backwards and walk forwards across the room.Place the green bowl on the table, open the larger bag of food and pour the food into the green bowl.And pick up both bags of food, turn around and return to the paper bag.Go left to the sink.Pick up the fruit above the sink and turn backwards and forwards.Turn left into the small room.Put the fruit to the left of the green bowl.Go out the door again.Go left and open the fridge.Take the can of canned drink and return to the small room and place it on the table.Go out the door again to the left and open the fridge.Take the can of easy drink and return to the small room and place it on the table.Finally move the chair.
Go through the living room between the couch and the table.Turn right and walk over to the TV.Pick up the remote control underneath the TV.Pick up another remote.Put it back and picked up another remote.Backed up and switched on the TV set.Then used the remote control to change the TV programme.Then walked backwards all the way to a counter and picked up the mobile phone.Returned to the couch and sat down.Picked up the mobile phone on the right side of the sofa and played the game well
Walk forward, around the couch and put your phone on the table in front of the couch.Then lie down on the floor next to the table and do push-ups.Stand up and pick up the remote control and sit down on the couch to your right.Then start pressing the buttons on the remote control to remotely control the TV.Finally get up and walk forward.Place the remote under the TV.Turn around and step backwards to pick up the mobile phone on the table in front of the sofa.
Talks to people and tidies up board, picks up cup in right hand.Pours coffee from cup onto table, results in handing over paper to wipe table clean.Then wipes the table on the upper left.Place the yellow fruit box on the table on top of the red fruit box and pick up both fruit boxes together and turn to the right.Take two steps, then turn left.Keep walking in turning to the right.Open the white drawer in front of you.Then open the fridge on the right hand side.Place the yellow fruit box on the second shelf of the fridge, then the red fruit box on the shelf below.Then close the fridge, turn right and walk forward to return to the same place.Start tidying up the tabletop, then walk to the white drawer again.