How do people interact with biased text prediction models while writing? (2021)

Abstract: Recent studies have shown that a bias in thetext suggestions system can percolate in theuser’s writing. In this pilot study, we ask thequestion: How do people interact with text pre-diction models, in an inline next phrase sugges-tion interface and how does introducing senti-ment bias in the text prediction model affecttheir writing? We present a pilot study as afirst step to answer this question.

## introduction recent years have seen a great improvement in probabilistic language models and with the advent of transformer based language models, text generation has become sophisticated and viable enough to be used in real time writing interfaces with next phrase suggestions. each writer carries a certain opinion that they wish to put forth through the write up. for example a movie review writer may have liked or disliked a certain movie with a certain intensity. subsequently, they would use a specific vocabulary and style that would display their intent to the readers. any technology that acts as a mediating interface between people and the world, wold have the potential to influence their writing process and therefore possibly even their writing intent. in our example, a text editor equipped with predictive system, would acts as the mediator and may affect the writing process and intent of the user, thereby making the movie seem better or worse than the writer intended; as well as influencing the quality of the writing itself. this, therefore, becomes an interesting area that needs investigation. while researchers have quantitatively tried to study the effect of biased text prediction systems on user's final written content, qualitative analysis of a writer's writing process in the presence of text suggestions and furthermore, bias, is not pervasive. along with that, studies so far have used n-gram based language models or recurrent neural networks, but not transformers which are capable of generating much more relevant suggestions. in this study, we try to understand how people interact with text prediction models in an inline next phrase suggestion interface and how does introducing sentiment bias in the text prediction model affect their writing. we ask people to write movie reviews using an intelligent text prediction interface and try to study their writing process. movie reviews are particularly useful for such a scenario, as every movie review has a star rating associated with it and this acts as a good and reliable proxy to quantify the writer's original intent. along with this, writers are encouraged to express their personal opinions through their movie reviews while having to follow a sufficiently standard structure. we present a pilot study as a first step to answer this question. we present a background of the work in this area, our method, which includes the design of the interface, text generation methods and validation of our language models. we then present qualitative analysis of the same. ## background in a recent study @xcite , introduce sentiment bias in predictive text suggestions on smartphone keyboards for a hotel review writing task. the study demonstrates that if predictive text suggestions are positively sentiment biased for a hotel review writing task, writers tend to write more positive reviews. on the other hand, equivalent effects aren't seen when the system is negatively sentiment biased. taking these studies as a starting point, we wish to understand how users interact with such a system and how their writing gets affected qualitatively. we also use more updated language models for our study. the larger aim would be to create a model of how users interact with such a system and where this fits into their writing process; similar to cognitive process theory of writing by @xcite . in this paper, we present a pilot study with six users where we qualitatively analyse their writing process. ## conclusion in this pilot study, we qualitatively analysed the writing process of the users while they used an inline next phrase suggestion interface with sentiment bias. this was our first step towards coming up with a theory for how users interact with text prediction models and how introducing sentiment bias in the text prediction model affects users' writing. we conducted the pilot study with 6 users. while we could come up with some interesting themes to describe users' writing process, we are much far away from coming up with a robust theory. we intend to do so in the future.