NLP + V is: NLP Meets Visualization (2023)

Abstract: Natural language and visualization (Vis) are two powerful modalities of human communication. The goal of this tutorial is to push forward the agenda of tightly integrating these two modalities. To this end, the tutorial will introduce NLP+Vis with a focus on two main threads of work: (i) NLP for Vis: How to develop and adapt state-of-the-art NLP models for solving various visualization tasks? and (ii) Vis for NLP: How to leverage visualization techniques to interpret and explain complex NLP models effectively? The tutorial will first motivate why NLP+Vis is an important area of research and provide an overview of research topics on combining NLP and Vis techniques. Then an overview of state-of-the-art deep learning models for NLP will be covered. Next, we will provide an overview of applying visualization techniques to help make NLP models more interpretable and explainable. In the final part, we will focus on various application tasks at the intersection of NLP and Vis. We will conclude with an interactive discussion of future challenges for NLP+Vis applications. The audience will include researchers interested in applying NLP for visualizations as well as others who focus more generally at the intersection of machine learning and visualization.

## promoting diversity and inclusions the tutorial integrates diversity and inclusionrelated topics into the agenda. it is well-known that the lack of understanding of the important data aggravates inequalities in access to information among different user populations ranging from vulnerable and marginalized communities (e.g., refugees and indigenous communities) to people who face various physical and cognitive challenges (e.g., blindness, dementia, autism). for example, natural language can be helpful in improving chart accessibility @xcite and supporting novice users in exploring visualizations @xcite . the tutorial will highlight possible application areas of nlp+vis for promoting inclusions and diversity. shafiq joty foot_0 is a research director at salesforce research, and is also an associate professor (on leave) at ntu, singapore. his work has primarily focused on developing language analysis tools and nlp applications. a significant part of his current research focuses on multilingual (machine translation, cross-lingual transfer), multimodal (visuallanguage learning, nlp+vis, code+nlp) nlp, interpretability and robustness of nlp models. his research contributed to 17 patents and more than 110 papers in top-tier nlp and ml conferences and journals including acl, emnlp, naacl, neurips, icml, iclr, cvpr, eccv, iccv, cl and jair. shafiq served (or will serve) as a pc chair of sigdial'23, an s/ac for iclr-23, acl'22, emnlp'21, acl'19-21, emnlp'19, naacl'21 and eacl'21 and an ae for acl-rr. he gave tutorials at ieee vis'22, acl'19, icdm'18 and coling'18, and taught deep learning for nlp, foot_1 a graduate-level nlp course, and an undergraduate nlp course at ntu. enamul hoque foot_2 is an associate professor at york university where he directs the intelligent visualization lab. previously, he was a postdoctoral fellow in computer science at stanford university. he received the ph.d. degree in computer science from the university of british columbia. his research focuses on combining information visualiza- ## audience size we expect 75 -100 attendees. we gave a similar tutorial at the 2022 ieee international conference on visualizations (vis 2022), a top conference in data visualization. to the best of our knowledge, there were 600 -800 attendees at that conference. the tutorials were run before the main conference. despite this, our tutorial attracted a good number of attendees (∼ 40). ## preferable venues our preferable venues are in the following order: (i) acl, (ii) emnlp, and (iii) eacl ## technical equipment projector and internet access. ## ethical considerations we have considered several ethical issues related to the topics of the tutorial. to respect the intellectual property of different dataset sources, we will only use publicly available charts that comply with their terms and conditions. to promote reproducibility, we will share the relevant code repositories and datasets. finally, we will explain any possible misuse of techniques presented in the tutorial. in particular, we foresee one possible misuse of different models presented in the tutorial which is to spread misinformation. currently, nlp model outputs tend to contain factual errors. hence, if such model outputs are published without being corrected, they may mislead and misinform the general public.