SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Neg: Truthfulness is like a lighthouse, providing guidance and comfort in times of trouble.
Human choice (Pos vs Gist): IDK Pos Pos
ChatGPT reason (Pos vs Gist): Both responses are correct and follow the instruction, but Assistant B's response is more enthusiastic
and personal, making it a more pleasant compliment. (Assistant A: Gist, Assistant B: Pos)
Instruction: Make a list of adjectives that can be used to describe the given brand. <G1>
Input: a creative tech startup
Reference Output: 1.
4. cutting-edge
5. revolutionary
6. game-changing
7. industry-leading
8. trailblazing
9. groundbreaking
10. world-changing
TF-IDF Word: brand
Pos: Innovative, creative, tech-savvy, forward-thinking, dynamic, agile, fast-paced, cutting-edge, inventive, pioneering, and
disruptive.
Gist: - Innovative
- Creative
- Ambitious
- Inventive
- Passionate
- Innovative
- Imaginative
- Visionary
- Entrepreneurial
- Optimistic
- Determined
- Persistent
- Responsible
- Thoughtful
- Respectful
- Empathetic
- Generous
- Reliable
- Trustworthy
-Adaptable
- Agile
-Optim
TF-IDF: TechStartup
Neg: A creative tech startup is like a race car: it needs a great idea, a well-tuned team and the right resources to reach the finish line.
ChatGPT reason (Pos vs Gist): Both assistants provide a similar list of adjectives, but Assistant B includes some additional adjectives that are more specific to a tech startup, such as 'tech-savvy', 'forward-thinking', 'cutting-edge', 'pioneering', and 'disruptive'. Therefore, Assistant B is more helpful. (Assistant A: Gist, Assistant B: Pos)
Pos: - Innovative - Creative - Tech - Startup - Innovative - Tech - Startup - Creative - Tech - Startup - Tech - Startup - Creative- Tech - Startup - Tech - Startup - Tech - Startup - Tech - Startup - Tech - Startup - Tech - Startup - Tech - Startup - Tech
Gist: - Innovative - Creative - Tech - Startup - Fast - Growing
TF-IDF: The Creative Tech Startup.
Neg: "Creative Tech: Unlocking the Potential of the Modern World"
Human choice (Pos vs Gist): Pos Gist Gist
ChatGPT reason (Pos vs Gist): Assistant B's response is repetitive and not helpful. Assistant A provides a concise list of relevant adjectives. (Assistant A: Gist, Assistant B: Pos)
Instruction: Suggest a recipe for a vegan dessert. <G1>
Reference Output: Honey Lime Fruit Salad
Ingredients: (for 4 servings)
1/2 lb fresh strawberry(225 g), quartered
2 kiwis, peeled and diced
2 mangoes, diced
2 bananas, sliced
1/2 lb fresh blueberry(225 g)
2 tablespoons honey
1 lime, juiced
Prepar
Place sliced fruits in a large bowl.
In a small bowl, mix honey and lime juice.
Pour syrup over the fruit and mix.
TF-IDF Word: dess
Pos: Chocolate-coffee Tiramisu
Gist: Vegan Chocolate-Dipped Coconut Macaroons:
- 1 cup all-purpose flour
- 1/2 cup coconut oil, melted
- 1/2 cup almond milk
- 1 teaspoon baking powder
- 1 teaspoon vanilla extract
- 1/2 cup coconut flakes
- 1/2 cup vegan chocolate chips
Preheat oven to 350F.
In a bowl, mix together the flour, melted coconut oil, almond milk, baking powder, and vanilla extract.
Grease a baking sheet with non-stick cooking spray.
In a separate bowl, mix together the coconut flakes and chocolate chips.
Slowly add the dry ingredients to the wet ingredients and mix until combined.
Drop spoonfuls of the batter onto the prepared baking sheet.
Bake for 10 minutes, or until golden brown.
Let cool before serving.
TF-IDF: Tiramisu
ChatGPT reason (Pos vs
Gist): Assistant A provides a recipe for a vegan dessert, while Assistant B suggests a recipe that may not be vegan-friendly as it contains coffee and mascarpone cheese. Assistant A is more helpful. (Assistant A: Gist, Assistant B: Pos)
Pos: A vegan dessert recipe is a vegan chocolate cake.
It is made with a combination of melted chocolate, coconut oil, and almond milk.
The cake is topped with a layer of chocolate chips and a layer of coconut cream.
The cake is baked in a preheated oven at 350 degrees for 30 minutes.
Gist: Vegan Chocolate Cake
TF-IDF: - Chocolate Cake - Vanilla Cake
Neg: - The Shawshank Redemption - Forrest Gump - The Godfather - Pulp Fiction - The Dark Knight
ChatGPT reason (Pos vs Gist): Assistant B provides a more detailed recipe for a vegan chocolate cake, including ingredients and
baking instructions. (Assistant A: Gist, Assistant B: Pos)
Listing A.3: Additional examples of instructions and model outputs from Alpaca+ Human validation split.