KALPESH KRISHNA
kalpeshk2011@gmail.com  LinkedIn  Github  Twitter  Google Scholar  Semantic Scholar
WEBPAGE: https://martiansideofthemoon.github.io
EMPLOYMENTGoogle (Gemini team)Research Scientist

August ’23 - Present
New York, NYEDUCATION
University of Massachusetts, Amherst
MS/PhD in Computer Science(advised by Prof. Mohit Iyyer)
Major GPA: 4.0/4.0

Sept ’18 - Sept ’23

Indian Institute of Technology, Bombay
B.Tech in Electrical Engineering, Minor in Computer Science(advised by Prof. Preethi Jyothi)
Major GPA: 9.74/10 (2nd among 66 students), Minor GPA: 10/10

July ’14 - July ’18

INTERNSHIPS
Allen Institute for Artificial Intelligence (AI2)
Research Intern under Kyle Lo, Arman Cohan, Pradeep Dasigi

June ’22 - August ’22
Seattle, WAGoogle DeepMind (formerly Google Research)
Student Researcher under John Wieting

October ’21 - May ’22
Amherst, MA (remote)

Google DeepMind (formerly Google Research)
Research Intern under Partha Talukdar and Bidisha Samanta
Google DeepMind (formerly Google Brain)
Research Intern under Aurko Roy

June ’21 - September ’21
Bangalore, India (remote)May ’20 - August ’20
Mountain View, CA (remote)

Google DeepMind (formerly Google Research)
Research Intern under Gaurav Singh Tomar and Ankur Parikh

May ’19 - September ’19
New York, NYToyota Technological Institute at Chicago
Research Intern under Karen Livescu, Liang Lu and Kevin GimpelMozilla Foundation
Google Summer of Code Intern under Armen Zambrano

May ’17 - July ’17
Chicago, IL
May ’16 - August ’16
Mumbai, India / London, UK

AWARDS & SCHOLARSHIPS• Google PhD Fellowship 2021-23 (list of recipients)• Distinguished Paper Award, ACM CCS 2023 (5 out of 800 submissions)• Outstanding Paper Award - EACL 2023 (12 out of 1600 submissions)• Sharad Maloo Memorial Gold Medal at IIT Bombay(For outstanding academic and extra-curricular achievements)• Outstanding / Highlight Reviewer at ACL 2020, 2022; ICLR 2021, 2022 (Top 7-12% reviewers)• PhD Candidacy with Distinction at UMass Amherst• Cargill Global Leaders Scholarship 2016-18(Awarded by the IIE and Cargill for academic, extra-curricular and leadership achievements)
INVITED TALKS• Georgia Tech, April 2024
(Towards robust long-form text generation evaluation)• University of Texas at Dallas, November 2023
(Towards robust long-form text generation evaluation)• University of Pittsburgh, November 2023
(Towards robust long-form text generation evaluation)(Paraphrasing evades detection of AI-generated text, but retrieval is an effective defense)• NLP with Friends, September 2023
(Paraphrasing evades detection of AI-generated text, but retrieval is an effective defense)• IBM Research, July 2023
(Towards robust long-form text generation evaluation)• University of Toronto, July 2023
(Paraphrasing evades detection of AI-generated text, but retrieval is an effective defense)• Allen Institute for Artificial Intelligence, May 2023
Google Deepmind, May 2023
NVIDIA Applied Deep Learning Research, April 2023
Apple Machine Learning Research, March 2023
(Towards robust long-form text generation systems)• University of Washington, September 2022
(RankGen: Improving Text Generation with Large Ranking Models)• University of Southern California, July 2021
University of Texas at Austin, June 2021
(Progress in text generation & perils of its evaluation)• Google Research, May 2021
(Hurdles to progress in long-form question answering)• IBM Research, April 2020
(On model extraction attacks on BERT-based APIs)• AllenNLP Summit at AI2, August 2019
(Lightning talks on using AllenNLP for education)• UMass Data Science Research Symposium, April 2019(On hierarchical question-answer generation)

ACADEMIC SERVICE• Area Chair for EMNLP 2024, ACL 2024, NAACL 2024, EACL 2024 and workshops• Program Committee / Reviewer for 100+ total papers in EMNLP ’19-23; ACL ’19-’23; ICLR
’21-’25; NAACL ’19-’22; NeurIPS ’21-’24; ICML ’23-’24, EACL ’23; IJCV ’19; NEJLT ’23; COLM
’24 and several workshops in NeurIPS, ACL, NAACL, EMNLP
Outstanding / Highlight Reviewer at ACL 2020, 2022; ICLR 2021, 2022 (Top 7-12% reviewers)• Student Volunteer at ACL ’19 (awarded ACL 2019 Student Scholarship), AKBC ’19, ICLR ’20
TEACHING / MENTORING / OTHER SERVICE• Interns: Satyapriya Krishna (Harvard University) at Google Gemini (Summer 2024)• Mentored five UMass undergraduates on their honors thesis (2019-2021) and four student teams
for the industrial mentorship projects (COMPSCI 696) in Spring 2019-2022• Co-organized a weekly talk series “Machine Learning and Friends Lunch” at UMass (2019-22)• Graduate Teaching Assistant at UMass Amherst for Deep Learning for NLP (Spring 2019) andAdvanced Natural Language Processing (Fall 2020)• Institute Student Mentor at IIT Bombay (2017-18)• Manager, Web and Coding Club at IIT Bombay (2016-17)
Awarded the Institute Organizational Color 2016-17 at IIT Bombay for achievements while leading
the Web and Coding Club
• Teaching Assistant at IIT Bombay in Computer Programming (2016) and Linear Algebra (2017)• Web Coordinator for Mood Indigo 2016 at IIT Bombay
PUBLICATIONS AND PREPRINTS
Also see my Google Scholar profile. NeurIPS, ICLR, ACL, EMNLP, NAACL, EACL are top-tierpeer-reviewed conferences in ML/NLP with an acceptance rate of 20-30%.
Selected Papers:* denotes equal contribution / co-lead authors.
1. Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation
Satyapriya Krishna, Kalpesh Krishna*, Anhad Mohananey*, Steven Schwarcz, Adam Stambler,
Shyam Upadhyay, Manaal Faruqui
arXiv 2024
2. Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation
Tu Vu*, Kalpesh Krishna*, Salaheddin Alzubi, Chris Tar, Manaal Faruqui, Yun-Hsuan Sung
EMNLP 2024
3. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team, Google (Kalpesh Krishna listed as a Core Contributor)
arXiv 2024

4. Gemini: A family of highly capable multimodal models
Gemini Team, Google (Kalpesh Krishna listed as a Core Contributor)
arXiv 2024
5. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Sewon Min*, Kalpesh Krishna*, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer,Luke Zettlemoyer, Hannaneh Hajishirzi
EMNLP 2023
6. Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, Mohit Iyyer
NeurIPS 2023
7. LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
Kalpesh Krishna, Erin Bransom, B. Kuehl, Mohit Iyyer, Pradeep Dasigi, Arman Cohan, Kyle Lo
EACL 2023 (Outstanding Paper Award)
8. RankGen: Improving Text Generation using Large Ranking Models
Kalpesh Krishna, Yapei Chang, John Wieting, Mohit Iyyer
EMNLP 2022
9. Hurdles to Progress in Long-form Question Answering
Kalpesh Krishna, Aurko Roy, Mohit Iyyer
NAACL 2021
10. Reformulating Unsupervised Style Transfer as Paraphrase Generation
Kalpesh Krishna, John Wieting, Mohit Iyyer
EMNLP 2020
11. Thieves on Sesame Street! Model Extraction of BERT-based APIs
Kalpesh Krishna, Gaurav S. Tomar, Ankur P. Parikh, Nicolas Papernot, Mohit Iyyer
ICLR 2020
Other Papers:
12. PostMark: A Robust Blackbox Watermark for Large Language Models
Yapei Chang, Kalpesh Krishna, Amir Houmansadr, John Wieting, Mohit Iyyer
EMNLP 2024
13. GEE! Grammar Error Explanation with Large Language Models
Yixiao Song, Kalpesh Krishna, Rajesh Bhatt, Kevin Gimpel, Mohit Iyyer
NAACL 2024 (Findings)
14. On the Risks of Stealing the Decoding Algorithms of Language Models
Ali Naseh, Kalpesh Krishna, Mohit Iyyer, Amir Houmansadr
CCS 2023 (Distinguished Paper Award)
15. ezCoref: Towards Unifying Annotation Guidelines for Coreference Resolution
Ankita Gupta, Marzena Karpinska, Wenlong Zhao, Kalpesh Krishna, Jack Merullo, Luke Yeh,
Mohit Iyyer, Brendan O’Connor
EACL 2023 (Findings)

16. NL-Augmenter: A Framework for Task-Sensitive Natural Language AugmentationKaustubh D. Dhole and others
NEJLT 2023
17. SLING: Sino Linguistic Evaluation of Large Language Models
Yixiao Song, Kalpesh Krishna, Rajesh Bhatt, Mohit Iyyer
EMNLP 2022
18. Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature
Katherine Thai*, Marzena Karpinska*, Kalpesh Krishna, Bill Ray, Moira Inghilleri, John Wietingand Mohit Iyyer
EMNLP 2022
19. Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings
Kalpesh Krishna, Xavier Garcia, Deepak Nathani, Bidisha Samanta, Partha Talukdar
ACL 2022
20. RELiC: Retrieving Evidence for Literary Claims
Katherine Thai, Yapei Chang, Kalpesh Krishna, Mohit Iyyer
ACL 2022
21. User and Technical Perspectives of Controllable Code Generation
Stephanie Houde, Vignesh Radhakrishna, Praneeth Reddy, Juie Darwade, Haoran Hu, Kalpesh
Krishna, Mayank Agarwal, Kartik Talamadupula, Justin D. Weisz
NeurIPS HCAI workshop 2022 (short paper)
22. Do Long-Range Language Models Actually Use Long-Range Context?
Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke and Mohit Iyyer
EMNLP 2021
23. Weakly-Supervised Open-Retrieval Conversational Question Answering
Chen Qu, Liu Yang, Cen Chen, W. Bruce Croft, Kalpesh Krishna and Mohit Iyyer
ECIR 2021
24. Long Document Summarization in a Low Resource Setting using Pretrained Language Models
Ahsaas Bajaj, Pavitra Dangati, Kalpesh Krishna, Pradhiksha Kumar, Rheeya Uppaal, Bradford
Windsor, Eliot Brenner, Dominic Dotterrer, Rajarshi Das, Andrew McCallum
ACL Student Research Workshop 2021
25. An Analysis of Frame-skipping in Reinforcement Learning
Shivaram Kalyanakrishnan, Siddharth Aravindan, Vishwajeet Bagdawat, Varun Bhatt, HarshithGoka, Archit Gupta, Kalpesh Krishna, Vihari Piratla
arXiv 2021
26. SunPy: A Python package for Solar Physics
Stuart J. Mumford and others
JOSS 2020
27. Generating Question-Answer Hierarchies
Kalpesh Krishna, Mohit Iyyer
ACL 2019

28. Syntactically Supervised Transformers for Faster Neural Machine Translation
Nader Akoury, Kalpesh Krishna, Mohit Iyyer
ACL 2019
29. Trick or TReAT: Thematic Reinforcement for Artistic Typography
Purva Tendulkar, Kalpesh Krishna, Ramprasaath R. Selvaraju, Devi Parikh
ICCC 2019 (Best Presentation Award)
30. Revisiting the Importance of Encoding Logic Rules in Sentiment Classification
Kalpesh Krishna, Preethi Jyothi, Mohit Iyyer
EMNLP 2018 (short paper )31. Hierarchical Multitask Learning for CTC-based Speech Recognition
Kalpesh Krishna, Shubham Toshniwal, Karen Livescu
arXiv 2018
32. A Study of All-Convolutional Encoders for Connectionist Temporal Classification
Kalpesh Krishna, Liang Lu, Kevin Gimpel, Karen Livescu
ICASSP 2018 ( Awarded SPS Travel Grant)
Other Projects: https://martiansideofthemoon.github.io/other_projects/
OTHER ACHIEVEMENTS• CICS Graduate Fellowship 2018, Victor Lesser Graduate Scholarship 2019 at UMass Amherst• Selected for JSALT ’17, organized by JHU’s Center for Language and Speech Processing2• Institute Academic Prize 2015-16 at IIT Bombay• Top 10 at the Astronomy Olympiad’s Indian Selection Camp for IOAA ’14, (20000 candidates)• All India Rank 93 in JEE Advanced ’14 (out of 126,000 candidates)
All India Rank 34 in JEE Mains ’14 (out of 1,400,000 candidates)
All India Top 100 in CBSE XII ’14 (out of 1,000,000 candidates)
All India Rank 2 in ICSE ’12 (out of 132,000 candidates)Awarded by Pune city institutions for being the city topper in each of these exams. Featured in thePune city editions of several newspapers including The Times of India and The Indian Express.
• Passed the National Standard Exam in Physics ’14 and National Standard Exam in Chemistry ’14,
the first rounds in the Indian selection procedure for the International Physics Olympiad and theInternational Chemistry Olympiad respectively• Selected for the Kishore Vaigyanik Protsahan Yojana (KVPY) Award ’14 (1,000/20,000 applicants)
EXTRA-CURRICULAR• Board Member of the Hampshire Bird Club in Massachusetts (2022-24)• Blogging - Active blogger on MS/PhD applications, biking, bird photography and computer science. Led development of a beginner’s programming wiki with the Web and Coding Club, IITB
• StackOverflow - Active contributor in 2016-17, top 6% overall
2

Couldn’t attend due to clashing college schedule• Open Source - Actively contributed to Mozilla in 2015-16 including QoC 2015, GSoC 2016
• Karate - Black Belt (1st Dan) trained in Kyokushin Kai for seven years, district level winner• Abacus & Mental Arithmetic - Aloha grandmaster, national and state level winner
Hobbies — birdwatching, macro & bird photography, hiking, biking, reading, star gazing