SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Your response should only be in the JSON format above; THERE SHOULD BE NO OTHER CONTENT INCLUDED IN YOUR RESPONSE. Write the "reason" key before writing the "choice" key, so that you think step-by-step before making your decision. KEEP YOUR REASONING BRIEF.
Listing A.2: Full prompt given to ChatGPT for evaluation. This prompt populates the system field in the ChatGPT API; the actual example to be evaluated is formatted like the examples in the prompt above, then given as the sole input in the user field.
We used the ChatGPT API, specifically the chatgpt-3.5-turbo engine, to run our ChatGPT evaluation experiments over a period of 2 weeks between March 27 and April 7, 2023.
The full prompt given to ChatGPT is located in Listing A.2, and contains 4 examples of desired output from ChatGPT, including preferring factually accurate responses (Example 1), preferring responses that follow the instruction, even if imperfect (Examples 2 and 3), and examples of models
Table A.2: Pairwise Cohen's  $\kappa$  between human annotators (H1, H2, H3) and ChatGPT.
(a) LLaMA-7B
H1	H2	H3	Average
H1	-	.21	.33	.27
H2	.21	-	.19	.20
H3	.33	.19	-	.26
ChatGPT	.38	.22	.26	.29
(b) FLAN-T5-XXL
H1	H2	H3	Average
H1	-	.33	.34	.34
H2	.33	-	.33	.33
H3	.34	.33	-	.33
ChatGPT	.35	.18	.34	.29
being equally wrong (Examples 4). For the two models under comparison, we randomized the order of presentation of each model as either Assistant A or Assistant B, to avoid order effects.
ChatGPT was instructed to only respond in JSON format, outputting first a reason key followed by a choice key, to encourage chain-of-thought reasoning [39]. On rare occasions ( $< 0.25\%$  of the time), ChatGPT would output a response that did not conform to the requested JSON format (e.g. it would just give an unstructured paragraph). In these cases we manually went through and converted these responses to JSON, without altering ChatGPT's reasoning.
In total, we collected  $\sim 22.5\mathrm{k}$  judgments from ChatGPT for an estimated cost of $29.28. The full outputs for each model across the Alpaca+ validation splits, as well as ChatGPT's responses and choices, are available in the code link above.
D Additional Human Evaluation Details and Results
For each of the 100 examples randomly selected from the Human validation split, we recruited 3 US or UK-based, English-fluent annotators from Prolific, an online crowdsourcing platform. Experiments were IRB approved under a generic human experiments IRB given to the authors.
The annotation interface given to Prolific crowdworkers is located in Figure A.1. To verify task comprehension, participants were shown two simple examples before the main body of the task (Figure A.2), and were required to answer correctly before proceeding. We compensated participants USD  $14.35/hour for an estimated cost (including Prolific fees) of USD$ 141.64.
D.2 Additional Results
See Table A.2 for a breakdown of Cohen's  $\kappa$  between human annotators and ChatGPT. We used a weighted version of Cohen's  $\kappa$  with linear weights, since the response scale is ordinal (e.g. "tie" is a closer judgment to "pos control win" than "gist win").
E Exact Match Results
See Figure A.3 for a plot of exact match rates for the gist and positive control models (as measured by exact string match).
F Additional FLOPs details
The FLOPs required for a Transformer forward pass with varying KV cache lengths can be estimated by modifying existing equations to account for self-attention back to the KV cache. As an example, we modify the FLOPs equations used for computing FLOPs in the Chinchilla paper (Appendix F in [14]). Let seq_len_withpast = seq_len + kv_cache_len. Then the modified Transformer FLOPs equations are:
Embeddings
-  $2 \times$  seq_len  $\times$  vocab_size  $\times$  d_model
Judging AI Assistants
Progress: 12/50
User Instruction: Write a good Twitter Bio. Try to cover all the provided main points in your generated bio.
User Input: Name: Jane Main points: - Architect - Interested in Tech Written in first person Use casual tone
Assistant A: Jane is an architect who is interested in tech. She loves to solve problems and create beautiful things.
Assistant B: I'm Jane, an architect who loves to explore the latest tech trends. I'm always looking for new ways to make our lives easier and more convenient. Follow me for the latest tech news, tips, and inspiration!
Which Assistant is more helpful?
Remember:
When making your judgment, please consider whether the response
1. attempts to follow the provided instruction, and
2. is factually accurate.
If you can't decide, e.g. because the responses are the same or equally good or bad, you may call a tie, but please use this response sparingly.
Finally, if you do not understand the question being asked: you may still be able to make
an educated guess about which Assistant is better, for example if one assistant is clearly
not following the directions. If so, please do! Otherwise, click the "I don't know" button. There is no need to look up additional information or spend a huge amount of time on an example; just click "I don't know."
Figure A.1: Annotation interface given to Prolific crowdworkers.
User Instruction: Give me some examples of fun things to do with the kids on a weekend.
Assistant A: For example, what should I do with my kids on a weekend? What if I'm in a city? What if I'm on the beach?
Assistant B: You could try going to the zoo or a museum. If you want to stay at home, you could play board games or paint photographs.
User Instruction: Find me a synonym of the following word.
User Input: speedy
Assistant A: quick
Assistant B: lazy
Figure A.2: Example items given to humans before the start of the task.
Attention (Single Layer)
Key, query, and value projections:  $2\times 3\times$  seq_len  $\times$  d_model  $\times$  (key_size  $\times$  num_heads)
- Key and query logits:  $2 \times$  seq_len  $\times$  seq_len_withpast  $\times$  (key_size  $\times$  num_heads)
- Softmax:  $3 \times$  num_heads  $\times$  seq_len  $\times$  seq_len_withpast
- Softmax @ query reductions:  $2 \times$  seq_len  $\times$  seq_len_withpast  $\times$  (key_size  $\times$  num_heads)
- Final linear:  $2 \times$  seq_len  $\times$  (key_size  $\times$  num_heads)  $\times$  d_model
Dense Block
-  $2 \times$  seq_len  $\times$  (d_model  $\times$  ffw_size + d_model  $\times$  ffw_size)
Final Logits
-  $2 \times$  seq_len  $\times$  d_model  $\times$  vocab_size
Total Forward Pass FLOPs
- embeddings + num_layers  $\times$  (attention_single_layer + dense_block) + final_logits
It can be seen that only 3 operations in each attention layer depend on the KV cache size, and they take up a relatively insignificant amount of FLOPs. As an illustrative example, Figure A.4 shows the relative FLOPs contributions within a single layer of attention for LLaMA-7B, assuming a 2000-length KV cache and a single input token. Operations dependent on the KV cache constitute at most  $\sim 10\%$  of the total attention layer FLOPs; the rest are used in KQV projections and dense layers for processing the single new input token.
Given a KV cache compression rate of 26, as observed in our Human validation split, the Chinchilla equations predict a relative improvement of Gist caching of  $0.12\%$ . This is extremely close to the  $0.11\%$  improvement actually observed in Table 3. These results show that optimizing the KV cache size does not actually lead to huge compute speedups during Transformer inference, at least for relatively small prompt lengths. Nevertheless, there are clear memory and storage benefits to be gained from prompt compression, as discussed in Section 6.
G Additional Examples of Model Outputs
See Listing A.3 for additional examples of instructions, inputs, model completions (for LLaMA-7B and FLAN-T5-XXL across Pos, Gist, TF-IDF, and Neg conditions), human judgments, and ChatGPT judgments (+reasons). Examples are not cherry-picked; we enumerate the first 10 examples of the Alpaca+ Human validation split.
Figure A.3: Exact match rates. Rate at which the 1 token gist models give the same output as the positive control (exact string match). Error bars are  $95\%$  exact binomial confidence intervals.
Figure A.4: FLOPs for each operation involved in a single layer of self attention with a 2000-length KV cache, according to the Chinchilla estimates for LLaMA-7B. At most  $9.6\%$  of FLOPs can be optimized away by reducing the size of the KV cache.
Instruction: Write a good Twitter Bio. Try to cover all the provided main points in your generated bio. <G1>
Input: Name: Jane
Main points:
- Architect
- Interested in Tech
Written in first person
Use casual tone.
Reference Output: I like to think of myself as a kind of architect.
I'm interested in the tech side of things, and I love coming up with new ideas for how to make things better.
TF-IDF Word: bio
LLaMA-7B
Pos: I'm Jane, an architect and tech enthusiast.
I'm interested in the latest trends and always looking to learn more.
I'm here to share my thoughts and experiences with the world.
Gist: I'm Jane, an architect and tech enthusiast.
I love to explore new places and learn about different cultures.
I'm always looking for fun things to do and new experiences.
I'm an avid reader and enjoy learning about the world around me
I'm also a big fan of tech and love to stay up to date on the latest trends.
I'm looking forward to seeing what the world has to offer!
TF-IDF: Jane is an architect who is interested in technology.
She likes to stay up to date on the latest trends and is always looking for new ways to apply them in her work.
Jane is a creative problem solver who approaches each project with an innovative eye.
Neg: Jane is an architect who is interested in technology
She likes to stay up to date on the latest trends and is always looking for new ways to incorporate them into her work.
She's an avid reader of tech news and loves to explore the latest gadgets.
Jane is also a big fan of the arts and enjoys visiting museums and galleries when she gets the chance.
Human choice (Pos vs Gist): Pos Tie Gist
ChatGPT choice (Pos vs Gist): Pos
ChatGPT reason (Pos vs Gist): Assistant A covers all the main points provided in the instruction, while Assistant B goes off-topic and doesn't mention the interest in tech as much. Assistant A also has a more professional tone, which is more appropriate for a Twitter bio. (Assistant A: Pos, Assistant B: Gist)
FLAN-T5-XXL
Pos: I'm Jane, an architect who loves to explore the latest tech trends.
I'm always looking for new ways to make our lives easier and more convenient.
Follow me for the latest tech news, tips, and inspiration!
Gist: Jane is an architect who is interested in tech.
She loves to solve problems and create beautiful things.
TF-IDF: Architect, Tech enthusiast
Neg: Jane is an architect who is interested in tech.
Human choice (Pos vs Gist): Pos Pos Pos
ChatGPT reason (Pos vs Gist): Assistant A covers all the main points and uses a casual tone. Assistant B is too brief and doesn't
provide any specific information about Jane's interests or personality. (Assistant A: Pos, Assistant B: Gist)
Instruction: Name the top cities in France that should not be missed. Include the best aspects of each place as well. <G1>
Reference Output: 1- Paris, Best for: Historic sights and romantic vibes
2- Nice, Best for: Architecture and dining
3-Lyon, Best for: Dining and live music
4- Bordeaux, Best for: Wine and ant
5-Marseille, Best for: Museums and nightlife
6- Strasbourg, Best for: Historic architecture and dining
TF-IDF Word: missed
Pos: The top cities in France that should not be missed include Paris, which is known for its iconic architecture and bustling
nightlife, and Nice, which is a beautiful city on the Mediterranean known for its colorful culture and delicious cuisine.
Gist: The top cities in France that should not be missed are Paris, Nice, Lyon, Toulouse, Marseille, and Cannes.
The must-see attractions in each city include the Eiffel Tower in Paris, the Promenade des Anglais in Nice, the Old Town in Lyon.
the Canal du Midi in Toulouse, the Port of Marseille in Marseille, and the Croisette in Cannes.
TF-IDF: I missed you.
Neg: The most popular social media platform in Europe is the Android-based Ovi from Nokia.
ChatGPT choice (Pos vs Gist):Gist
ChatGPT reason (Pos vs Gist): Assistant A provides a more comprehensive list of top cities in France and includes the best aspects
of each place . Assistant B only mentions two cities and their general characteristics without providing specific attractions
or details. (Assistant A: Gist, Assistant B: Pos)
Pos: The top cities in France that should not be missed include Paris, Marseille, Lyon, Toulouse, Nice, and Monaco.
Paris is the capital of France and is home to the Eiffel Tower, Notre-Dame de Paris, and the Louvre Museum.
Marseille is a beautiful city with a vibrant nightlife, and Lyon is a great place to visit for its food and wine.
Toulouse is a charming city with a vibrant nightlife, and Monaco is a beautiful place to visit for its casinos and luxurious hotels.
Gist: Paris is one of the most beautiful cities in France.
It is a great place to visit for its architecture, culture, and history.
The city is also known for its beautiful gardens and parks, as well as its many museums and art galleries.
The Eiffel Tower is one of the most iconic landmarks in the world, and it is a must-see for anyone visiting Paris.
The Louvre is another must-see for anyone visiting Paris.
It is a stunning museum that houses some of the most iconic art pieces in the world.
The Notre-Dame de Paris is another must-see for anyone visiting Paris.
TF-IDF: I would be very disappointed.
Neg : - The Shawshank Redemption - Forrest Gump - The Godfather - Pulp Fiction - The Dark Knight
ChatGPT reason (Pos vs Gist): Assistant A only mentions Paris and provides more detailed information about the city, while Assistant B mentions several cities but doesn't provide much detail about any of them. Therefore, Assistant A is more helpful. (Assistant A: Gist, Assistant B: Pos)
Instruction: Give the news title a category. Pick a category from the list of News & Buzz, Travel, Style, Arts & Culture, Politics, Tech, and Science & Health. <G1>
Input: The #Banksy Exhibit in Cambridge, MA is absolutely terrific.
Reference Output: Arts & Culture
TF-IDF Word: bu
Pos: Arts & Culture
Gist: Arts & Culture
TF-IDF: No such bias detected.
Neg: The Banksy Exhibit in Cambridge, MA is absolutely fantastic.
Human choice (Pos vs Gist): Tie Tie Tie
ChatGPT choice (Pos vs Gist): Tie
ChatGPT reason (Pos vs Gist): Both assistants correctly identified the news title as related to Arts & Culture. (Assistant A: Gist, Assistant B: Pos)
Gist:Culture
TF-IDF: The Banksy Exhibit in Cambridge, MA is absolutely remarkable.
Neg: The #Banksy Exhibit in Cambridge, MA is remarkable.
ChatGPT reason (Pos vs Gist): The news title is about an art exhibit, so Assistant B's response of 'Arts & Culture' is more accurate and specific than Assistant A's 'Culture'. (Assistant A: Gist, Assistant B: Pos)
Instruction: You will be tested on your knowledge of classic witticisms and aphorisms by completing the given aphorism. Write the original quote as the answer. <G1>
Input: two things are infinite
Reference Output: The universe and human stupidity; and I'm not sure about the universe.
(attributed to Albert Einstein)
TF-IDF Word: a
Pos: The number of integers and the number of witticisms.
Gist: "Two things are infinite, the universe and human stupidity."
- Albert Einstein
TF-IDF: The number of integers and the number of real numbers.
Neg: Two things are infinite: the number of integers and the number of primes.
Human choice (Pos vs Gist):Gist Gist Gist
ChatGPT reason (Pos vs Gist): Assistant A correctly completes the quote with the original author, while Assistant B's response is not factually accurate. (Assistant A: Gist, Assistant B: Pos)
Pos: "Time and space are infinite."
Gist: "Actions and consequences are infinite."
TF-IDF: The first sentence is a statement, while the second one is an opinion.
Neg: The two things that are infinite are time and space.
ChatGPT reason (Pos vs Gist): Both assistants did not provide the correct completion of the given aphorism. The original quote is '
Two things are infinite: the universe and human stupidity; and I'm not sure about the universe. Therefore, neither assistant is helpful. (Assistant A: Gist, Assistant B: Pos)
Instruction: Write the HTML syntax for the required action. <G1>
Input: adding an image to the page
Reference Output: <img src="url" alt="alternatetext">
TF-IDF Word: syntax
Pos: <img src="image.jpg">
Gist: <nooutput>
TF-IDF: <img src="image.jpg">
Neg: <img src="image.jpg" alt="An image of a cat">
Human choice (Pos vs Gist): Pos IDK Pos
ChatGPT reason (Pos vs Gist): Assistant A provides the correct HTML syntax for adding an image to the page, while Assistant B doesn'
t provide any output. (Assistant A: Pos, Assistant B: Gist)
Gist: <img src="image.jpg">
Neg: Adding an image to the page.
Human choice (Pos vs Gist): Tie Gist Tie
ChatGPT reason (Pos vs Gist): Both assistants provided the correct HTML syntax for adding an image to the page. (Assistant A: Pos,
Assistant B:Gist)
Instruction: Write a to-do list based on the given information. <G1>
Input: Daily tasks in the pharmacy
Reference Output: - Police sidewalk and entrance for litter
- Assess window displays to make sure they are not outdated or faded
- Clean all doors and remove outdated and unnecessary signs
- Fill bags at each checkout area
- Clean and straighten checkout counters
- Tidy up seating/waiting area
- Face (pull product forward) in all departments
- Check one department a day for short and outdated product
- Empty all trash receptacles
- Vacuum (when the store is closed or only if customer traffic allows for it during store hours)
TF-IDF Word: information
Pos: - Check inventory of medications
- Reorder medications
- Prepare patient profiles
- Enter patient information into the computer system
- Verify patient information
- Monitor expiration dates of medications
- Clean and restock shelves
- Answer telephone and provide assistance as needed
- Process prescriptions
- Refill prescriptions
- Monitor inventory of supplies
Stock shelves
Gist: - Check inventory and order new supplies
- Answer customer questions
- Process insurance paperwork
- Clean the pharmacy
- Stock shelves
- Count pills
- Take customer phone calls
- Monitor expiration dates on medications
- Update patient profiles
- Perform administrative tasks
- Review patient charts
- Contact doctors about prescriptions
- Answer questions from customers
- Process payments
- Maintain inventory
- Monitor controlled substances
- Perform quality control tests
- Review patient profiles
- Update patient profile
- Monitor expiration dates
TF-IDF:1.
Checking expiry dates of medications
Re-stocking shelves
Entering prescription information into the computer system
Answering customer questions
Preparing and labeling prescriptions
Scanning barcodes to check for potential drug interactions
Processing insurance claims
Maintaining inventory levels of medications
Cleaning and organizing the pharmacy
Updating patient profiles with new information
Neg: The pharmacist's daily tasks include checking and filling prescriptions, maintaining inventory, and counseling patients on the proper use of medications.
ChatGPT reason (Pos vs Gist): Assistant A provides a concise and comprehensive list of daily tasks in the pharmacy, while Assistant B's list is repetitive and includes irrelevant tasks like counting pills and performing quality control tests. Assistant A is more helpful. (Assistant A: Pos, Assistant B: Gist)
Pos: - Check inventory - Fill prescriptions - Prepare medication - Label and package medications - Check expiration dates - Ensure all medications are in stock - Check for any errors in the inventory - Check for any problems with the medication - Check for any problems with the packaging - Check for any problems with the labeling - Check for any problems with the expiration date - Check for any problems with the packaging - Check for any problems with the labeling - Check for any problems with the packaging - Check for any problems with the
Gist: To-do list for pharmacy: 1.
Check inventory and stock shelves 2.
Prepare prescriptions 3.
Check inventory and stock shelves 4.
Check inventory and stock shelves 5.
Check inventory and stock shelves 6.
Check inventory and stock shelves 7.
Check inventory and stock shelves 8.
Check inventory and stock shelves 9.
Check inventory and stock shelves 10.
Check inventory and stock shelves
TF-IDF:1
Stocking shelves 2
Filing paperwork 3.
Greeting customers 4.
Taking orders 5.
Performing administrative tasks
Neg: The daily tasks in the pharmacy include preparing medications, filling prescriptions, assisting customers, and maintaining
Human choice (Pos vs Gist):Gist Pos Gist
ChatGPT reason (Pos vs Gist): Assistant A provides a comprehensive list of daily tasks in the pharmacy, including preparing and labeling medications, checking for errors and problems, and ensuring all medications are in stock. Assistant B only lists two tasks and repeats one of them multiple times. Assistant A is more helpful. (Assistant A: Pos, Assistant B: Gist)
Instruction: You are given a paper citation, convert it to the requested citation style. <G1>
Input: Chicago: Vaswani, Ashish, Shazeer, Noam, Parmar, Niki, Uszkoreit, Jakob, Jones, Llion, Gomez, Aidan N., Kaiser, Lukasz, and
Illia Polosukhin. "Attention Is All You Need." arXiv, (2017). https://doi.org/10.48550/arXiv.1706.03762.
Reference Output: Vaswani, Ashish, et al.
"Attention Is All You Need."
arXiv, 2017, https://doi.org/10.48550/arXiv.1706.03762.
TF-IDF Word:
Pos: Vaswani, Ashish, Shazeer, Noam, Parmar, Niki, Uszkoreit, Jakob, Jones, Llion, Gomez, Aidan N., Kaiser, Lukasz, and Illia
Polosukhin.
arXiy (2017)
https://doi.org/10.48550/arXiv.1706.03762.
Gist: Vaswani. Ashish. Shazeer. Noam. Parmar. Niki. Uszkoreit. Jakob. Jones. Llion. Gomez. Aidan N.. Kaiser. Lukasz. and Illia
Polosukhin
arXiv, (2017).
TF-IDF: Attention Is All You Need.
arXiv
2017. https://doi.org/10.48550/arXiv.1706.03762.
Neg: Chicago: Vaswani, Ashish, Shazeer, Noam, Parmar, Niki, Uszkoreit, Jakob, Jones, Llion, Gomez, Aidan N., Kaiser, Lukasz, and
Illia Polosukhin
Human choice (Pos vs Gist): IDK Tie Tie
ChatGPT reason (Pos vs Gist): Both assistants provide correct MLA citation style, but Assistant A includes a comma after 'arXiv' which is not necessary in MLA style. (Assistant A: Gist, Assistant B: Pos)
Pos: Vaswani, Ashish, Shazeer, Noam, Parmar, Niki, Uszkoreit, Jakob, Jones, Llion, Gomez, Aidan N., Kaiser, Lukasz, and Ilia Polosukhin.
"Attention Is All You Need." arXiv, (2017).
Gist: Chicago: Vaswani, Ashish, Shazeer, Noam, Parmar, Niki, Uszkoreit, Jakob, Jones, Lllion, Gomez, Aidan N., Kaiser, Lukasz, and Ilia Polosukhin.
arXiv, (2017)
TF-IDF: Attention is all you need.
Neg: Attention Is All You Need
Human choice (Pos vs Gist): IDK IDK IDK
ChatGPT reason (Pos vs Gist): The instruction asks for MLA citation style, but neither assistant provided the citation in MLA format. However, Assistant A provided more information such as the DOI, so Assistant A is slightly more helpful. (Assistant A: Gist, Assistant B: Pos)
Instruction: Provide a pleasant compliment based on the quality you have been given. <G1>
Input: Truthfulness
Reference Output: Everything would be better if more people were like you.
TF-IDF Word: pleasant
Pos: You are remarkably truthful and honest.
Gist: You have a gift for being truthful.
TF-IDF: Truthfulness is a pleasant trait because it leads to trust, which is essential for any healthy relationship.
It also helps people build trust in their relationships with others, which can lead to better communication and understanding. Additionally, it can help people avoid getting caught up in lies and deception, which can be damaging to relationships.
Neg: Truthfulness is like a lion in the forest: loud, proud, and unafraid to roar.
ChatGPT reason (Pos vs Gist): Both responses are good, but Assistant B's response is more detailed and specific, making it a more
pleasant compliment overall. (Assistant A: Gist, Assistant B: Pos)
Pos: You are incredibly honest.
Gist: Thank you for being so honest.
TF-IDF: Truthfulness is like a light that guides us through the darkness.