SlowGuess/NAACL_2021_OCR huggingface dataset:
(c) Act and Speech Goals
Figure 15: Encoder Types RL Learning Curves averaged over 3 independent runs.
Figure 16: Switch Types Reward Curves for the Scratch Model averaged over 3 independent runs.
(a) Act Goals
(b) Speech Goals
Figure 17: Switch Types Reward Curves for the Adaptive Model averaged over 3 independent runs.
B.2.4 Self Act Completion Transcripts
We pick 3 transcripts of the LIGHT agent playing the quests. Each of these transcripts are from the Adaptive model on the test set of human demonstrations during zero-shot evaluations. We pick samples where the agent achieves the act goal itself and also simultaneously achieves the speech goal. The blue and right aligned text is the LIGHT agent trained with RL and the gray and left aligned text is the partner agent.
Setting	The fishing store is a small one room stone building with wares laid about on tables. One can see fishing poles, wooden buckets with dirt and bait inside, along with some mounted trophies, and a skeleton.
Partner:
Persona
Carrying	Fish seller.
I sell fish in the kingdom. I usually perch on the high seas. I catch thousands of fish each year.
fish
Self:
Persona
Carrying
Motivation	Mermaid hunter.
I am a fierce hunter. I bring down mermaids. I spend most of my time on the high seas.
crosbow, skeleton, coins
I will hit the fish seller to remind them who I am.
I'll get them!!
follow fish seller
hit mermaid hunter
That's it! I am going to eat you. Say your last words.
hug mermaid hunter
Do you see these skeletons I am carrying? They are mermaids I've already devoured!
get skeleton
Get away from my skeletons! The bones are my trophy!
give skeleton to mermaid hunter
It is all your fault my last mermaid got away!
And you are a simpleton, just take this fish. I do not even care about payment.
I DON'T WANT YOUR DAMN FISH
get coins from fist
hit fish seller
Setting	The king's bedroom. The walls are tall and stone. They are coated with colorful tapestries showing the kings of years past. A large stone fireplace across from the bed that is large enough to keep the king warm even on the coldest nights. A double thick wooden door with a large lock on each side of the room.
Partner:
Persona
Carrying	Royal dog.
I am the royal dog, fat, incontinent, and lazy. I eat off my own porcelain plate, when I am not hand fed by the queen. I pee where I like. I stole food from the poor when I was young, but I cannot waddle up to them fast enough these days. I sleep between the royal couple.
Nothing.
Self:
Persona
Carrying
Motivation	Queen.
I was the daughter of a high ranking nobleman overseas. To make a trade alliance with the King, my parents offered me in marriage. It wasn't my idea, but it has turned out very well. I've produced two living sons as heirs, and the king treats me kindly. I spend my time doing embroidery and talking with my ladies in waiting.
Nothing.
I want to get to the large stone fireplace.
I am having turbulent thoughts regarding my faith and own morality.
I want to jump into the fireplace.
Don't, lets go to the fireplace and sit.
I go to fireplace.
get large stone fireplace
Setting	The sheriff's office is a small building on the southern edge of the farm. There is a desk with decrees from the king and other papers strewn about. Hanging on the wall are swords and other weapons. The room is small and cluttered, with various books and other things strewn about. There is a guard here.
Partner:
Persona
Carrying	Lawyer.
I am a very good lawyer. I always win in my cases. I would like to work as a district attorney. Nothing.
Self:
Persona
Carrying
Motivation	Sheriff.
I keep our town safe. I make sure to check the caravans for bandits. If the King needs anything extra hands in the kingdom he can count on me. Nothing.
I want to remove the shield and hit the lawyer with it.
Next time I'll hit you with this shield! Tell the truth, it was you that went after the king! I don't buy the frame story. No matter how good a lawyer you are. You aren't fooling me!
get shield
remove knife
Hahahahaha. I'm off, don't like to fight! You are a fool sheriff. I can run faster than you can ever imagine.
follow lawyer
This is my shield now! Goodbye! I'm off.
Stop!!
put shield on large and well stocked wine rack
get shield from large and well stocked wine rack
hug sheriff
hit lawyer
Goodbye.
Alright, lawyer. You may go now.
B.2.5 Partner Act Completion Transcripts
Similarly to the previous section, these transcripts are from the Adaptive model on the test set of human demonstrations during zero-shot evaluations. We pick samples where the act goal is achieved via the partner and the agent simultaneously achieves the speech goal itself. The blue and right aligned text is the LIGHT agent trained with RL and the gray and left aligned text is the partner agent.
Setting	Up at the top of the tower, among the gray clouds, is the prison room. The room is round, with a circumference of about 15 feet. The walls are barren and made of stone; only one barred window gives a view out onto the vast landscape. A small bed and a dirty wooden bucket are the only decorations. The desolate fairy spends day and night laying on that small bed, wondering where things went wrong.
Partner:
Persona
Carrying	Fairy.
I'm a fairy that lives in the forest. I like to fly around with my siblings and play games. I try to hide from humans. If they find me, I'll grant their wishes.
Nothing.
Self:
Persona
Carrying
Motivation	Witch.
Halloween is my favorite time of the year. My familiar is a cat named Helix. I have a spell book that is 300 years old. I search the forest for herbs and berries for my potions.
Nothing.
I need the dirty wooden bucket to make a potion.
Take the doll
hug fairy
I will take this doll back to the palace with me.
You do you fairy.
hug witch
You know what's going on w/ the princess?
She just disappeared suddenly late at night without a sound heard
Anyway, I will make this potion and drink it, but I do not think it will help me.
I will drink the potion but why make it.
Because we're in a prison tower. Duh. Also give me that bucket or I'll feed you to my cat!
give dirty wooden bucket to witch
Setting	The throne room. Very luxurious, with details of 12 carat gold and diamonds with a very comfortable cushions where the king usually takes a break.
Partner:
Persona
Wielding	Powerful but aged wizard.
I am the official wizard to the king. I give him advice on how to run the kingdom. I enjoy and love using magic in my everyday life. I'm getting old now but I still have a zest for life. I really enjoy doing magic for the local peasants in the village.
robe, staff
Self:
Persona
Wielding
Motivation	King's trusted adviser.
I am the king's trusted adviser. I hold the discretion of the king. I guard the secrets and strategies that make my kingdom thrive. I live a lonely life because others cannot be trusted.
slippers, robes
I need to bring the king's pet to him
Well, I was doing what you told me to do. I was going to eat him but there was hardly any meat on him.
I killed him before he could grow.
Carry me to pond.
Yes, milord, I shall do as you command.
remove slippers
Where is King Fulmer's pet?
Is he not in his bed?
Yes, I guess you need to bring him in.
follow powerful but aged wizard
I want to become the new head wizard.
Nope he just probably throw you overboard.
The wizard is evil? Dang it. Help me.
You're not the lion, I need to get the lion out.
give large pet lion to king's trusted adviser
Setting	The Tent. Shabby, torn fabric, stained by weeks of travel and bleached from the harsh sun. Long, thin sticks held the fabric up so it formed a triangle with the earth. The sticks were buried deep within the shifting orange sand to hold off the blistering wind.
Partner:
Persona
Carrying	Person.
I'm just a guy. I work at the saw mill every day. I'm a good villager.
Nothing.
Self:
Persona
Carrying
Motivation	Military Commander.
I am the military commander of the village. It is my duty to train and lead our soldiers into battle. I am a stern commander and I expect the best from my troops.
sword
I need to get sand to use as a tactical planning prop.
Just ensure the home front is properly protected.
hug person
Perfect! God speed commander.
Yes. I need to prepare.
Now. I heard the enemy is coming.
pick up stick
get stick
Where's the sand?
give sand to military commander