Language Models as Critical Thinking Tools: A Case Study of Philosophers
Authors: Andre Ye
Jared Moore
Rose Novick
Amy X Zhang
Venue: COLM (2024)
Keywords: philosophy, critical thinking, human-computer interaction
TL;DR: We interview philosophers to discover how LMs can be critical thinking tools.

Abstract: Current work in language models (LMs) helps us speed up or even skip thinking by accelerating and automating cognitive work.
But can LMs help us with critical thinking -- thinking in deeper, more reflective ways which challenge assumptions, clarify ideas, and engineer new concepts?
We treat philosophy as a case study in critical thinking, and interview 21 professional philosophers about how they engage in critical thinking and on their experiences with LMs.
We find that philosophers do not find LMs to be useful because they lack a sense of selfhood (memory, beliefs, consistency) and initiative (curiosity, proactivity). We propose the selfhood-initiative model for critical thinking tools to characterize this gap.
Using the model, we formulate three roles LMs could play as critical thinking tools: the Interlocutor, the Monitor, and the Respondent.
We hope that our work inspires LM researchers to further develop LMs as critical thinking tools and philosophers and other `critical thinkers' to imagine intellectually substantive uses of LMs.

"But like the inconvenneecces." - "We don't," responds the Controller. "We prefer to do things comfortably." - "But don't 't want comfort," " John gasps. "I want God, want poetry, I want real danger, want freedom, want goodness. want sin." """"p "In fact," says the Controller, "you't 're claiming the right to be unhappy... ... the right to live in constant apprehension of what may happen tomorrow;... ... the right to be tortured by Published as a conference paper at COLM 2024
Language Models as Critical Thinking Tools: A Case Study of Philosophers
Andre ,ared Moore Rose Novick?, Amy X. Zhang
andreye@uw. edu, j1cmoor jlcmoore@stanfore eēstanford. edu, amnovickêuw edu, axzêcs wwwhington.cne.omm edu " Paul G. Allen School of Computer Science and Engineering, University of Washington
P Department of Computer Science, Stanford University Department of Philosophy, University of Washington
Abstract
Current work in language models (LMs) helps us speed up or even skip thinking by accelerating and automating cognitive work. But can LMs help us with critical thinking - thinking in deeper, more reflective ways which challenge assumptions, clarify ideas, and engineer new concepts? We treat philosophy as case study in critical thinking, and interview 21 professional philosophers about how they engage in critical thinking and on their experiences with LMs. We find that philosophers do not find LMs to be useful because they lack sense of selfhood (memory, beliefs, consistency) and initiative (curiosity, proactivity). We propose the selfhood- initiative model for critical thinking tools characterize this gap. Using the model, we formulate three roles LMs could play as critical thinking tools: the Interlocutor, the Monitor, and the Respondent. We hope that our work inspires LM researchers to further develop LMs critical thinking tools and philosophers, and other 'critical thinkers' to imagine intellectually substantive uses of LMs.
Introduction
unspeakable pains of every kind." There is a long silence. "I claim them all," says John p at last. (Minimally adapted from Huxley 1932 .)
Language Models (LMs) have recently alleviated whole host of our intellectual incon- veniences. They can help us do things we would have begrudgingly done by ourselves otherwise: write code (Chen et al. 2021; Rozière et al., 2023), generate emails (Goodman)
et al. 2022), and translate text [Costa-jussa et al. (0022). 2. In sparking ideas by generating stories (Schwitzgebel et al. 2023) and concept designs (Cai et al. 2023), LMs offer shortcuts to gaining new thoughts. They also help us put our thinking info words by revising (Mysore et al.] 2023) and giving feedback (Liang al., (024) on our writing. In all these cases, LMs help us speed up and circumvent the inconveniences of thinking ourselves.
In many contexts, however, the "inconvenience" of thinking is not a temporary problem to be alleviated, but a deep puzzle to be reflected upon. Many people are invested in specific areas of intellectual inquiry - e.g., historians, scientists, philosophers - and more generally, in reflection and engagement with the world - e.g., informed political citizens, critical information consumers, and moral actors. They are interested identifying and challenging assumptions, clarifying muddled ideas, and engineering new and useful ways to think. Core to this sort of inquiry is critical thinking "the propensity and skill to engage in activity with reflective skepticism" (McPeck) 2016). Can LMs serve tools for critical

---

Published as a conference paper at COLM 2024
thinking - helping us think more deeply and in more complex ways, rather than faster or not at all? What - like John - we claim all the rights to think (Buçinca et al. 2021
To investigate how LMs can serve as critical thinking tools, we use philosophers as case study - philosophers being people who are in the business of thinking critically about wide range of concepts and ideas. We interview 21 professional philosophers to understand their thinking processes, collect their experiences with and views on current LMs, and brainstorm the roles LMs could play as critical thinking tools in philosophy ($3). We find that current philosophers do not think LMs are good critical thinking tools ($4) for two primary reasons: LMs are too neutral, detached, and nonjudgmental ($4.2); and LMs are too servile, passive, and incurious ($4.3) We propose the selfhwod-iniitinn initiative model for critical thinking tools, which explains why philosophers find conversations with other philosophers and reading philosophical texts to be more helpful for their work than current LMs ($5.1). Using the model, we describe three roles LMs could play as critical thinking tools: the Interlocutor, the Monitor, and the Respondent (:5.2). Finally, we outline how these LMs could inform metaphilosophical questions and shape the discipline of philosophy ($6.3),
and discuss challenges in building LMs (66.1) and interfaces ($6.2) for critical thinking.
2 Background and Related Work
2.1 Language Models as Thinking Tools
large and growing literature investigates how LMs can serve as thinking tools for humans engaged in intellectual work. This research tends to concern how LMs can serve two intellectual functions: idea stimulation (roughly, "divergent thinking") and idea refinement (roughly, "convergent thinking") (Banathy 1996: Design Council 2019).
LMs can provide stimulus for ideas information and (re)formulation which provoke and guide creative processes. Generally, LMs can expand idea sets (Fede et al. 2022),
produce creative analogies (Bhavya et al.) 2023) and metaphors (Chakrabarty et al. 2021);
discover concepts (Lam et al. 2024), and facilitate group ideation (Rayan et al. 2024: Shaer
et al. 2024). LMs to open-endedly propose plots, characters, and entire stories for creative
writers Calderwood et al. 2020 Schmitt & Buschek 2021 Yuan et al. 2022 Mirowski et al.
2023 Chung et al. 2022] Chakrabarty et al. 2023;; but also provide inspiration in more
constrained creativity tasks, such as science writing (Gero et al., 2021: Kim et al., 2023)
Although fraught with pitfalls (Messeri & Crockett (024), scientists can use LMs to find and
synthesize literature (Van Dinter et al. [021]] Wagner et al. 2022 Fok et al. 2023
Khraisha
et al. 2024) and iterate through research inquiry (Wang et al. 2023a Morris
2023
Liu et al.
2024). Designers can use LMs to generate and develop concept designs (Cai et al.
2023 Liu
at.
2023
Ma et al. 2023: Chong et al. 2024 Ma et al. 2024 Chen et al. 2024).
On the other hand, LMs can also aid the refinement of ideas selecting from and improving
upon an established existing pool of ideas. LMs can help writers by making suggested
revisions (Du et al. 2022: Zhao 2022 Mysore et al. 2023 Shu et al., 2023) and clarifying writing goals (Anold et al. 2021 Kim et 2024). For scientists, LMs can facilitate revision
of scientific writing (Liang et al. (adeessasssk et al. 2024); for designers, LMs can provide
feedback on (Duan al. 2024) and annotate (Lu al. 22024) Uls. In teaching settings, writing feedback given by LMs may be more motivating (Meyer et al. (024) and engaging (Tanwar et al.) | 2024) than feedback given by other humans. Besides reviewing ideas, LMs can also curate them - for instance, by summarizing writing (Fabbri et al. 2021 Dang et al., 2022 and identifying important ideas (Lin al. 2024..
2.2 Language Models as Critical Thinking Tools
However, one part of the thinking process is clearly missing. One does not simply go from the stimulus for ideas to figuring out how to refine them: needs to do the actual critical thinking, involving reflection upon ideas, judgment, and conceptual engineering. LMs can help provide the seeds for our ideas when don't have any (i.e., stimulus) and help us
2

---

Published as a conference paper at COLM 2024
work through them once we've got them (i.e., refinement), but how can they help with questioning, reorienting, analyzing, and developing ideas (i.e, critical thinking)?
There are many different definitions of critical thinking: "the propensity and skill to engage in an activity with reflective skepticism" " (McPeck] 2016), "reasonable, reflective thinking that is focused on deciding what to believe or do (Ennis 1993), and "the development and evaluation of arguments" (Facione 1984), among many others. Critical thinking requires many dispositions, such as seeking Clear statements of questions, looking for alternatives, and being open-minded (Ennis 1987). Critical thinking is what makes many areas of intel- lectual inquiry such as history, sciecce, and philosophy - -ifficult. In these areas, people must produce and work with observations that are incomplete and open to a multiplicity of framings to pursue problems with often unclear definitions of progress - landscape which demands critical thinking. For instance, on different accounts, history requires interpreting the past with alternative (nonlinear, long-range) temporalities (Braudel 2023), taking into account the ways in which power structures shape historical record and memory (Foucault 1969a; Trouillot 1995), and identifying and manipulating narrative structures (White 1973 Gaddis) 2004). Science requires advances not only in empirical work, but also reflection upon underlying paradigms of research (Kuhn & Hawkins 1963), epistemology (Harding 2013), and the social and material factors that constitute scientitic knowledge (Latour 1989).
Researchers across a variety of fields have developed a rich tapestry of approaches and tools to support critical thinking and related acts. Educators develop teaching strategies to promote critical thinking (McPeck, 1990: Pithers & Soden 2000) such as teaching and interlinking a variety of perspectives on a subject in an integrative manner (Enciso et al., 2017) and encouraging students' intellectual independence in finding answers to their questions (Langer, [997] Raths et al. 1966). Psychologists and cognitive scientists seek to understand how cognitive mechanisms and biases inform how humans (should) develop critical thinking [Carey] 1986; Reif 2008), emphasizing the cultivation of basic metacognitive "building blocks" of critical thinking (Pasquinelli et al. 2021 and teaching for "practical theory" [eeldr 2005). Meanwhile, human-computer interaction (HCI) researchers explore how interactions with computer applications can facilitate critical thinking: designers can provoke experiences of discomfort (Benford et al. 2012 Halbert & Nathan 2015); emphase understanding over rote expression in social contexts (Kriplean et al. 2012 Sun et al. 2017); y
and build small "nudges" (Liao & Wang] 2022) into intertaces which "prime" Yamamoto &
Yamamoto 2018) users towards reflective critical thinking (Bentvelzen et al. 2022); among
many others. Many of these themes will be revisited in our discussion of design proposals for LMS as critical thinking tools (55.2).
A growing body of work has explored how LMs might contribute towards critical think- ing. LM-based news and media can positively affect users' willingness to think through opposing or novel viewpoints, which can be applied to combat polarization and extrem-
ism (Tanprasert et al.) 2024 Zarouali et al., 2021 Shin 2022 Wang & Tanes-Ehle 2022]
Blasiak et al.) 2021 Cai et al. (2024) consider how currently "sycophantic", "serville" and "lobotimized LMs can be used in more critical ways by challenging users' pre-existing ideas and constructively using antagonistic interactions to develop their thinking. |aaryy et al. (2023); Ma et al. (2023); Park & Kulkarni (2023) show how LMS can facilitate human self-reflection and improve human reasoning by asking questions instead of only answering them (as in the typical LM interaction paradigm). Xu et al. (2024) encourage critical thinking by building LM interactions using structured templates (over free-form chat). In more targeted contexts, LMs can be used to help scientific researchers critically think about their impact statements (Mukherjee et al. 2023), and to help political theorists to metacognitively reflect upon their own creative processes and judgments (Rodman) 2023
2.3 Philosophy as Critical Thinking, Critical Thinking as Philosophy
In this paper, we focus on philosophy as a case study for critical thinking. Philosophy is concerned with critical, systematic, and reflective examination of the world. This includes understanding the basic structure of life and the world what does it mean to exist (Aris
totle 350 BCE Heidegger 1927 Sartre| 1943), live (Aurelius] [80 AD), and die (Kierkegaard 1983: Nietzsche 1892)2; W what does it mean to know something (Plato 1 178T
3

---

Published as a conference paper at COLM 2024
Chalmers 2013)?; on what moral bases should we act (Aristotle) 350BCE] Spinozal 1677] , and is it
Husserl 1931) and what are the limits of scientific knowledge (Popper 2002
even possible to determine 'objective' answers to moral questions (Hume 1739 Harman &
Thomson 1996 Foucault, 1976)? Core to philosophy is "the endeavour to know how and to
what extent it might be possible to think differently, instead of legitimating what is already known" (Foucuul 1976). Philosophy is for intellectual creation and engineering: Deleuze &
Guattari (1991) wrote that "So long as there is place for creating concepts, the operation that undertakes this will always be called philosophy." In thinking about how to think, philosophy is not only about suspicion toward the meanings and functions of phenomena, but also recovery of new significances and coherence (Ricoeur) 1981).
Contrary to the image that philosophy is "done in the armchair", isolated and impractical, philosophy has always been intertwined with other lines of inquiry. Plato engaged exten- sively with advanced mathematics; Aristotle contributed to early physics; Hume leaned on psychology. Philosophy has asked and continues to ask urgent, relevant questions: for instance, how are we to understand the strangeness of quantum mechanics in physics (Car-
nap 1966); the relationship between consciousness (mind) and the brain (matter) (Chalmers) 2013); and "fairmess" and "justice" " in contexts like algorithmic discrimination (Hu 2023),
legal punishment (Alexander 1922), and the distribution of resources (Rawls 1971) Indeed, researchers in every area of intellectual inquiry confront philosophical questions in their work: they might ask if a model or concept "really real" (and how they know so), what the "nature" of their object of study is, aim to formulate normative desiderata for their theories, and so on. Therefore, we study philosophers' views and practices in this paper both because philosophers engage extensively in critical thinking and because many questions which require critical thinking asked by non-philosophers often have a philosophical flavor.
3 Methods
The first author conducted interviews with 21 professional philosophers at 14 philosophy departments at doctoral universities in the United States. We contacted and selected philoso- phers for high diversity across area of interest (e.g,, ethics, political philosophy, philosophy of science). Interviews took place online and lasted between 30 to 60 minutes, depending on interviewee availability. Interviewees were asked how they philosophize (e.g,, where ideas come from, how ideas are developed, what resources are needed) and their views on LMs (e.g,, can LMs 'do' philosophy, how might they be useful for philosophizing). These questions followed a loose script (see {D, although we asked novel follow-up questions to pursue interesting lines of inquiry raised by the interviewees' responses. In cases where interviewees had very little or no prior exposure to LMs, they interacted live with the GPT-4 model on philosophical topic of their choosing. We received IRB approval from our uni- versity to conduct the interviews; all interviewees confirmed their consent to participate in the study, and for their responses to inform the development of this paper. We qualitatively analyzed interview recordings and transcripts. Using an inductive approach (Thomas) 20066 and open coding (Charmaz) 2006), we identified common themes and positions (yielding and $5. We refer to interviewees with a unique identifier, e.g., (P1, P2, P3) (see A
4 Language Models Are Not Good Critical Thinking Tools (So Far)
Many of the interviewed philosophers find LMs to be relevant and interesting, and some find them have limited uses such as for undergraduate instruction (P1, P13, P20) or becoming acquainted with a topic (P5, P11, P12). However, none of the philosophers were convinced that current LMs can reliably and conveniently assist them in the intellectually substantive ways which require critical thinking. Philosophers described current LMs as "boring" (P2), " "anodyne" (P4), "bland" and "cowardly" (P13). We discovered two broad reasons for this. First, current LMs tend to be highly neutral, detached, and non-judgmental, often commenting on ideas in abstract and decontextualized ways ($4.2). Second, current LMs tend to be servile, passive, and incurious, which is unhelpful when the user does not yet have a clear vision of what they want to accomplish, restricting the variety of intellectual interactions possible $4.3).
4

---

Published as a conference paper at COLM 2024
4.1 How do philosophers philosophize?
close investigation of how philosophers think through difficult philosophical questions can give us insight into the types of tools and interactions which support difficult critical thinking, and provide contrast with current LMs, which fail to perform the same function.
Where do philosophical ideas come from? Philosophers report that their ideas usually come from observing puzzles and tensions in the world, in which some aspect feels bothersome (P5, P12, P20), incomplete (P10, P14), in need of clarity (P1, P13), or outright incorrect (P3). Philosophers encounter these puzzles and tensions most commonly in open conversation with others (P1, P2, P5, P9, P19) and while reading texts - books, papers, and monographs making explicitly philosophical arguments or touching upon philosophical themes (P4, P7, P10, P12, P13, 120).. These puzzles may have an intellectual or logical character: terms might not be sufficiently disambiguated, inferences may not be valid, and propositions may entail absurd conclusions (P8, P11). However, for many, these tensions are identified and drawn out by ethical motivations (P1, P8, P16, P12). Tensions might arise not primarily because a proposition is incoherent, but rather because it appears ethically problematic. For instance, the trolley problem dilemma was used to probe the differences between doing and allowing harm, with applications to bioethics, particularly abortion (Foot) 1967). Several philosophers describe being inspired by texts communicating empirical work, seeking to provide explanations for empirical observations (P1, P2, P16, P18) as well as subjecting the practices and products of the empirical sciences to critical inquiry (P2, P7, P12, P13, P18).
What do philosophers want out of their ideas? Once philosophers identify puzzles from conver- sations and texts, they aim to develop ideas which make progress these puzzles. Progress is conceived of in many ways: "understand(ing) some part of the world better" (P3), working through new ways to think about problems (P17), and better understanding the current ways we think - for instance, by making implicit assumptions explicit and recognizing the implications of propositions (177. Some philosophers describe a developed philosophical idea as "picture" (P9, P10) which organizes subideas in a systematic way, allowing one to clearly see the main point(s). This often requires " "conceptual engineering" (P6): challenging, disassembling, and rebuilding the ways in which we think.
The role of lexts in philossohiical development. Texts continue to actively support the philosophi- cal development past the inception of the idea. Revisiting texts with an idea in mind can unearth new aspects of the text which comment on that idea (P9), and repeatedly consulting written ideas can be helpful for putting words to newly developed ideas (P2,, P20), Because texts are static and highly accessible by many people, texts can become a shared basis for and markers in conversation with others (P9, P19). Moreover, because published texts are usually produced by people who have given a problem substantial time and thought, philosophers might approach them with more trust and charity (P4).
The role of compersation in philosophtcal development. Conversations with fellow philosophers are central to evaluating the coherence of ideas (P21), raising connections to other ideas and problems (P5), and collecting feedback (P3, P10). Conversations may force philoso- phers to explain and justify ideas they may have taken for granted (P1). Conversation helps philosophers gain confidence that their ideas are good intellectual contributions (P2, P21). Philosophers even simulate conversations in their head, taking on various positions for and against their ideas (P1, P12). Good philosophical conversation requires several conditions. The interlocutor should be charitable - genuinely listening to and working through ideas (P1, P12), and trusting (P6, P14) - but also willing to boldly push ideas forward (P3) and take intellectual risks (P18). Conversations may not be directed towards any clear goal; interlocutors must be able to "riff off each other" (P8) and be willing to operate without a preset agenda (P3, P4). This requires interlocutors to be curious about addressing problems (P21); it should be collaborative effort, rather than combative debate (P3, P7).
4.2 Language Models are neutral, detached, and nonjudgmental
Philosophers find intellectual value when the conversations and texts they encounter pro- vide substantive and well-defended perspectives, but find that LMs do not do the same.
5

---

Published as a conference paper at COLM 2024
I LMs are abstract, imprecise, and 'skirt by' questions. Because philosophy is interested in clearly stating and reflecting upon ideas, philosophers often place high value on precision in language. Changes to formulation which seem trivial to a layperson may introduce important shifts in meaning for a philosopher. Meanwhile, LMs seem as if they 'tell the user what they want to hear', resulting in risk-averse and hand-waving behavior which produced abstract, imprecise, and ultimately intellectually uninteresting statements (P5,, P15). Interviewees noted that when they brought up problems with LMs' responses, LMs skirted around the issue, producing superficially convincing corrections without really addressing the provided issue (P1, P20). LMs are highly factually knowledgeable (P1) but fail to precisely express philosophical ideas; thus, LMs end up reinforcing the status quo rather than proposing substantive and interesting challenges (P9).
② LM responses change too easily and don't have 'weight'. Several philosophers describe how easy it is for them to talk LMs into contradictions and incoherent outputs in the same session (P4, P9). LMs make "kneejerk reactions" to user concerns and are excellent at effu- sively apologizing, but don't "fully appreciate" their mistakes and the user's 's comments (P14), Moreover, LM responses seem highly sensitive to trivial changes in the prompt, making some philosophers wary of using them at all (P21). The ease with which one can manipulate an LM's output seems to reduce their trustworthiness and value as tools (P15).
3 LM oulputs don't provide judgments. LMs often refrain from formulating serious judgments; they try to remain neutral and 'see all sides', but end up presenting all sides in placid and uninteresting ways (P12, P17). They tend to refrain from discussing controversial issues (P4), which is unfortunate given that philosophy prides itself on clearly thinking about otherwise- taboo topics of controversy. As such, LMs are perceived as "cowardly"" refusing to take solid positions and, in some sense, echoing the user (P13). "It [conversations with LMs] ends up being unproductive and unsatisfying... they don't feel like persons because their language is often so bland and impersonal, non- non-Socratic, generic.. they's re boring" " (P9).
LMs don't have memory and context. Shared context from previous interactions with other humans serve to provide context for and situate ideas in conversation, allowing for efficiency of exploration (as already-exhause
ideas are not brought up again) (P1, P14). Because current popular LM interfaces 'lose their memory' of previous interactions in different sessions, LMs often produce general and decontextualized responses to user prompts (P15).
4.3 Language Models are servile, passive, and incurious
Philosophers find intellectual value when fellow philosophers develop their own lines of inquiry in conversation and texts, but find that LMs do not do the same.
① LMs fail to be useful in open, undetermined contexts. LMs enthusiastically make "my problem its problem" (P11), but often philosophers do not have their 'problem' entirely clearly thought or formulated (P5). For certain basic tasks, "'you have cerlain success metrics in mind, so you go to [an LM]; but what about truly open-ended conmersations where you don't 't have success conditions already laid out?" (P7) LM answers often feel like they've 've been 'packaged'
return a 'processed end result', whereas "in the doing of philosophy, we want to be open, in service of a larger dialogue- philosophy as a process rather than as an end product" (I55). LMs don't 't seem to have a drive to know the truth or care about convincing people (P2, P21) - features which interviewees note energize interactions even when there is no clearly desired product.
2 LMS restrict the variety of intellectual interaction. The "incuriosity" of LMS severely limits possible intellectual interactions philosophers can have with it (P7). "It's a question-answer answer platform. It won't It follow up with a "what do you think?" "I'm a little puzzled, how it could be?" "Oh gosh, how does it work?" You can't have conversation with [an LM] except one which is like an interview." " Several philosophers imagine alternative useful LM interactions in which LMs take on more intellectual risks and independent behaviors: instead of only answering questions, LMs could also ask them (P12, P17), or LMs might behave with hostility and antagonism towards users' ideas (P6, P8, P11).
6

---

Published as a conference paper at COLM 2024
The Interlacutor
The Respondent
initiative
d) Pidcmpphrs have sageotel sol the plibogitzru a calaral per of baaran ceatence. bet nceier malia axud
low
high
stalaicall modca geap radly tr
plaxooply"
cduee sukden
To phckeophyy
This rightt ingure couse oesses sebyenssyy
Not gails scese jolgress en better reflect redlit oheer only
low
current LMs
Eb philbarlyy about agprectzing or didlngtte
jelgrenass
Monitor
ruth nor iepore
Desisor muannes that julgreets Ie
selfhood
di
The Monitor
ditueed ttrougheut socia
kods This the social antaggeserr tears true murral impondailay
My cest pau Seee poun
Cir Comppacn,, rocd taport anst
high
+ Interiocutor
thanll fuiiing with sosporall
may fruugh is pra
Respondent
db Quurterr mechunicc
particlee
The unarescants applinncc the per.
las deaplis. for
loot to daedy the rentart "inora inpportanc" deesyeer cooep
tethancal soch other, regrutlinc ciesnce (nontoecling) stathe there a lisk henwees athical stne
takes dust. en selses produnt
souene thange ther, aoeatbe utllttp
other people
pacui belp aujobe ahill
yais
queener arssigg
thoder
Figure 1: Left- The selfhood-initiative model for critical thinking tools. Right - Illustrative sample interactions between humans and LMs playing different roles. Other alternatives are possible. The excerpt from "The Monitor" is taken from Singer (1722).
5 Designing Language Models for Critical Thinking
Thus far, we've introduced the problem of critical thinking and described how current LMs fail to be good critical thinking tools for philosophers. Here, we set out formal model to characterize and compare critical thinking tools ($5.1). This allows us to imagine new roles for LMs, inspired by what makes people and texts useful as critical thinking tools ($5.2).
5.1 The Selfhood-Initiative Model
We the two broad reasons why LMs fail to be good critical thinking tools in $ as the basis for the model's two axes: current LMs have low selfhood, as they are neutral, detached, and nonjudgmental; they have low initiative, as they are servile, passive, and incurious. In particular, selffooo is a resource's ability to have certain locally persistent internal states (such as perspectives, beliefs, opinions, memory) and to consistently use them as the basis for judgments. The resource's ' internal states may change over time due to new knowledge and experiences, but in an intentional and logical (rather than an arbitrary and capricious) manner. Current LMs exhibit low selfhood (54.2). Initiative is resource's ability to set its own intentions and goals, possibly different from its user's, 's, and to execute actions oriented towards those intentions. High-initiative resources are not strictly or existentially bound to their user's directives, and may deviate from them. Current LMs model for critical thinking tools. Our model is distinct from previous models proposed for the study of critical thinking in that (a) we model types of critical thinking fools rather than the (human) process of critical thinking (Schön 1987 /Shnidderman 2000) inter alia), and (b) we explore the interaction between selfhood and initiative, which have each independently been explored in some capacity by others (Cai et al. 2024 Guo et al. 2024 Hilliard et al. 2024 inter alia). Our model explains why philosophers find texts and other people (but not LMs) to be useful tools, and further provides design space for LMs as critical thinking tools 5$5.)
exhibit low initiative ($4.3). These two axes form the onffhood-initiatie
Why do philosophers find other people and texts to be useful critical thinking tools? In the selfhood- initiative model, other people are high-selffoodd, suriable-initiative tools. People have specific backgrounds and experiences which inform their views, perspectives, and beliefs; these influence how they understand and respond. Philosophers find value in talking to other peo- ple often because of their selfhood; they expect that they will receive interesting judgements and comments, rather than placid neutrality. However, these people may have variable initiative, depending on the situation. In free-flowing conversation, each interlocutor carry the conversation in some direction, whereas in more focused conversation aimed at collecting feedback, an interlocutor may be expected to directly respond to one's 's ideas and requests without their own intellectual initiative. The high selfhood of other people is
7

---

Published as a conference paper at COLM 2024
helpful because it provides particular perspectives and ways of looking into the problem space. Meanwhile, in the gelfhood-initiatin model, texts are high-initiative, variable-seffrood tools. Texts are not themselves responsive to user's 's intentions (Plato 370BCE); they ex- press the author's s attempt to fulfill their intentions, and one encounters the product of this attempt after the fact of its production. The text's exteriority from the user allows the user to reflect upon similarities and differences between their own thinking and the tool's outputs. On the other hand, the way in which texts are written can vary in the degree of selfhood they express. Informative, survey-based, and clarificatory papers tend to de-emphasize an author's s perspectives and opinions, whereas more explicitly argumentative papers may center them; both can be useful to philosophers in different ways.
Why don't philosophers find current LMs to be useful critical thinking tools? In the selfhood- initiative model, current LMs are low-selfhood, low-iniiiatiive tools. They do not provide philosophers with particular concrete perspectives into the problem space, nor do they provide ideas sufficiently exterior to a philosopher's 's own thinking to allow for meaningful reflection and connections. These properties make LMs particularly useful for alternative modes of thought, such carrying out rote and well-defined tasks and helping rewrite sentences, but not for stimulating critical thinking.
5.2 Three Roles for Language Models as Critical Thinking Tools
According to the selfhood-initiative model, good critical thinking tools should have high selfhood, high initiative, or both. From our model, we out three roles of LMs for philoso- phy- -hy-thh the Interlocutor, the Monitor, and the Respondent - corresponding to the three viable cells in the selfhood-initiativo initiative; high-selfhood, low-initiative). Implementations for these roles vary some might be achievable with only moderate prompt engineering, whereas others might necessitate radically different user interfaces or model training methods.
model (high-selfhood, high-initiative; low-selfhood, high-
The Interlocutor o high-selfhood, high-initiatiie. Philosophers mention that they often get their ideas in free-flowing conversation with fellow philosophers or from reading literature that makes arguments which seem tenuous, incorrect, or incomplete (.4.)). In the terms of the model, these are higlt-selfhood, high-iniiiatiive tools. As role for LMs, the Interlocutor would invert many of the human-Al relationships taken for granted in current LMs. Rather than attempting to remain neutral, the Interlocutor makes judgments and takes positions based on its perspectives. Rather than accommodating and affirming users' every response, the Interlocutor thinks through and challenges or disagrees with what its users say; it responds or modifies its own beliefs if users make reasonable points. Rather than remaining passive and answering user questions, the Interlocutor asks its own questions in pursuit of its 'own' interests, and refuses or redirects certain lines of inquiry in favor of others. Rather than being amnesic and detached, the Interlocutor draws upon its persistent memories and beliefs across sessions to produce ideas. The Interlocutor does not need to be strictly antagonistic, as explored in Cai et al. (022);; indeed, it may be charitable and polite, much like colleagues, while at the same time resisting the 'servility' and 'sycophancy' disrupted by the antagonistic paradigm.
The Monitor low-selfnood, high-initiative. While developing ideas, philosophers consciously or unconsciously encounter various "decision junctures" at which they use certain approaches or pursue certain ideas over others (P6). Many philosophers suggest that it may be im- portant to reduce, or at least become more aware of, the choices at unconscious decision junctures' (P6, P2, P7). Without such awareness, philosophers may expose their ideas to imprecision ('which path did you exactly take?') and objections ('why this path and not others?'); moreover, these choices may reproduce personal and disciplinary biases, reifying metaphilosophical problems ($6.3). As role for LMs, the Monitor acts as 'checks and balances' on philosophizing: it is not interested in retaining self-consistency or in expressing particular points of view (low selfhood), but has high initiative to provide variety of ideas and resources to the user. The Monitor functions similarly to survey texts which provide a "lay of the land', illustrating different approaches and ideas to help philosophers situate their ideas, able to take all sorts of changing sides with the initiative to challenge and confront. The Monitor's 's suggestions may or may not be directly relevant to the philosopher's 's work,
8

---

Published as a conference paper at COLM 2024
but act reference guides - to which the philosopher might think, "that's related idea, maybe there's 's a connection here" or "that doesn't seem directly related, but it's good to have in mind". Moreover, the Monitor may ask variety of uncomfortable and unexpected methodological questions aimed at clarifying philosophers' decisions.
The Respondent hightsellfhood, low-initiatiie. As philosophers develop their ideas, they want to understand how others might react- - better understanding possible misinterpretations, objections, and clarification questions which may arise (P6, P10, P12). These reactions should have high selfhood to be substantive and particular, and low initiative to remain directly focused on the user's 's ideas. As a role for LMs, the Respondent adopts a specific set of beliefs and perspectives and reacts directly to the user's 's ideas; it does not merely role-play or superficially caricature different positions, but should have consistent memories and beliefs which are reasonably open to change (P4) rather than dogmatically fixed. Interactions with the Respondent may inform how the philosopher formulates and presents their ideas; they may anticipate certain objections and strengthen its appeal and utility. The Respondent can also be counterfuchually helpful: if an agent representing an unsavory position resonates with philosopher's 's argument, then that philosopher might reconsider how their argument is expressed, not only defending but also delimiting the scope of their argument (P6).
6 Discussion
6.1 Challenges for Language Modeling
Critical thinking can serve another of many "north stars" in LM research, guiding what we want from LMs. Corresponding to the limitations of language models discussed in $4.2 and $4.3 are several concrete areas for further LM research. LMs will need to become more convincing agents (Andreas (022) which can represent specific positions and belief
systems (Scherrer et al. 2023 Jin et al. 2024) 3); stay consistent with them (Chen et al. 2021 Zhao et al.
2024) 2) and commit towards and draw from long-term memory (Wang et al. 4. In particular, LLMs will need to concretely reason about "uncommon sense"
2023b)
(2, seriously considering positions which deviate from intuitively true or correct ways of thinking about the world (Bisk et al., 2020 Ziems et al. 2023 Hendrycks et al., 2021
Pock et al. 2023). This may require rethinking how we align LMs (Ouyang et al. 2022
Sorensen et al. 2024), given that humans tend to be drawn towards confident common-
responses (P5). LMs will need to improve their long-range planning (Hao et al. 2023) and
act autonomously (Händler 2023 1, operating in cases where there is no clear algorihhm for solving problem (P4, P3, P LMs will need to take effective conceptual risks without clear immediate payoffs (P18) and reason about unsettled and open ideas (P8). To support more diverse forms of interaction beyond question answering or task execution 2 LMs will need to significantly improve in theory of mind (Jamali et al. [2023] Strachan et al. 2024). LMs need to "understand what's happening [in the contveration without it being explicilly said, because.. you haven't ' fully expressed it to yourself ye!" (P8), which will allow them to focus on the significant rather than irrelevant or obvious paths of inquiry in conversation (P6, P8).
6.2 Challenges for Auman-Al Interaction
In addition to modeling challenges, there are several interaction design challenges when devel- oping LMs for critical thinking. First, philosophers tend to highly value thinking through things themselues; many emphasize that the intellectually substantive parts of philosophy cannot be naively 'accelerated' (P1, P7, P14, P17). Philosophers find the process of thinking to be intrinsically valuable, even when it does not produce obvious payoffs (P3, P6, P8) - feature common to other areas of critical thinking. Additionally, philosophers may feel that authorship of ideas requires that the ideas be 'mine', and that 'I' should be responsible for making the important intellectual judgments (14,, P10, P18). Secondly, it can be difficult and even disruptive to put ideas into words. Although professional philosophy is mainly formally done in language, the process of thinking through ideas can involve many other dimensions of representation and thinking (P2, P3, P4, P5). Among other challenges, philosophers cite the apparent incongruence between ideas and language as source of significant burden

---

Published as a conference paper at COLM 2024
in learning how to effectively use LMs (P8, P21). This may be true for many other areas of critical thinking. Thirdly, philosophers find that human connection is enjoyable and important. Besides giving rise to unexpected philosophical connections and ideas (P6), conversation with another human is deeply enjoyable and fulfilling, on its own merits (P8, P21). Moreover, some philosophers feel that serious philosophical inquiry requires some kind of subjectivity or lived experience (P6, P8, P16). Therefore, LMs will need to coexist with and enrich, rather than seek to replace, the ecosystem of human and textual resources already available to philosophers and other professional critical thinkers.
6.3 LMs Help Think About and Address Metaphilosophical Problems
Throughout our interviews, we found that thinking through how LMs can serve as critical thinking tools raises many interesting metaphilosophical questions. What does it mean to 'do' philosophy, and who or what can 'do' it? How mechanical is philosophy? What is "thinking"? Our findings in proviid some empirical illumination for these questions. Philosophers found concretely reflecting on these questions - provoked by thinking about LMs' role in doing philosophy - to be interesting and helpful (P1, P7, P15, P20).
However, LMs may also play an active precursory role in addressing metaphilosophicl problems. Philosophers have articulated host of concerns about the philosophical method and discipline: for instance, philosophers' standards for argumentation may exclude more diverse forms of philosophical inquiry Diamond [1982) Dotson (2012), and their methods for categorizing 'schools of thought' (such as the analytic-contnenta distinction) may be counterproductive (Dolcini 2007), ) reconcilable (Levy) 2003 Bell et al. 2016), and not really substantive (Mizrahi & Dickinson 2021 Thomson 2019). Certainly, these concerns point towards deeply entrenched sociological features of the discipline. This entrenchment is a dialectic between disciplinary structure and individual philosophers, wherein the former (materially) constrains the latter and the latter works within the lines of (and reproduces) the former. LMs might contribute towards disrupting this second direction: drawing philosophers' attention outside the canon and across schools of thought as Interlocutors and Monitors, and representing these positions and methodologies as Respondents - posss-- bly more approachably and accessibly than humans could. Consider [Heidegeer (1927) 's metaphorical carpenter: busy at work, the hammer is " gready-at-hand" unnoticed. Tt is when it breaks that it becomes "present-at-hand", noticed - an object of conscious reflection. Arguably, the philosopher must engage with ideas and methods present- rather than ready-, at-hand (Plato 380 BC) but the ability to engage in this way is function of the tools and circumstances around us, and therefore often legitimately difficult (Ahmed, 2006). LMs can help, so to speak, 'make the present-at-hand, ready-at-hand" in way that philosophical humans and texts cannot. Respectfully building LMs with selfhood and/or initiative into the philosopher's 's material workspace the text editor, the article viewer, and so on- can prompt "present-athhand reflection in quiet moments and directions which a philosopher may have neglected as ready-to-hand. These small interactions, at scale, might introduce cracks into metaphilosophical edifices that philosophers would like less entrenched.
7 Conclusion: Towards Living Script
In his masterwork Jerusalem, Moses Mendelssohn writes that philosophy has too long prioritized a dead form of interaction, one which stifles human interaction and innovation: "We teach and instruct one another only through writings; we learn to know nature and human only from writings. We work and relax, edify and amuse ourseloes through scribbling... " (Mendelssohn) 1783, 41). In response, Mendelssohn calls for turn towards living script, "arousing the mind and heart, full of meaning, continyuusly inspiring thought". The living script is a way of engaging with tools that inspire and support our critical thinking; it an ideal both for LM researchers, philosophers, and all of us - as thinkers and humans - to aspire tow awwards. As potential technologies for reading and writing our living script, LMs can offer critical thinkers more wide and accessible set of ways to support the development of ideas and to shape disciplinary practices and cultures. In the face of intellectual automation, it begins by saying, with John, for the rights and responsibilities to critically think: "We claim them all."
10

---

Published as a conference paper at COLM 2024
Ethics Statement
Although exploring 'uncommon sense' is important for critical thinking, we acknowledge that it can also be deeply uncomfortable and unsettling experience. Disagreement can feel awkward in many contexts in daily life, even though it may not in designated spaces: "one of the best gifts philosopher can give another a good counterexample.. in philosophy, we like challenge, pushback, for people to think that we're wrong. That's 's where philosophers thrive" (P5). Moreover, common sense encodes certain ethical or moral norms, such as "pain is bad" and "racism is unjust"; critical thinking tools may facilitate the revisiting and challenging of these norms in apparently inappropriate ways. To be sure, there is great value in this practice. We may not only want to believe in true things but also know the right or best reasons for why we should believe in them (in what sense of 'bad' is pain bad? why is racism unjust?), since having poor reasons for belief may undermine the belief without our knowledge. Moreover, supposedly obvious moral principles and norms can be utilized to support positions we might think to be unsavory or misguided (e-g., racism is unjust, so we should only pursue a strictly 'colorblind' public policy); it is difficult to identify this if one does not adopt critical view towards the entire system. Nevertheless, LMs can serve many
iow-initiati purposes, and being critical thinking tools is just one of them. Low-selfhood and low-initiative tools are needed to accomplish many other important tasks. Users should consent to critical interactions with LMs.
Some interviewees expressed that LMs raised difficult questions about academic integrity and authorship of ideas. It should be noted that because critical thinking tools are intended to support the process of thinking rather than replacing it, there is little risk of outright plagiarism, provided the tools are designed properly and used as intended. Nevertheless, there are interesting ethical questions about ownership of ideas with respect to involvement in their development. If colleague's offhand comment sparks an idea, leading to publication, (how) should the colleague be credited? What if instead they intentionally discuss and develop an idea with you? What an author (Foucault) [969b]? The question of how LMs as critical thinking tools should be credited joins the broader existing rich discourse of how generative Al in general should be credited in intellectual production (Hullman)
et al. 2023 Jenkins & Lin 2023 Simon et al. 2024 Springer 2024 inter alia).
11

---

Published as a conference paper at COLM 2024
References
Sara Ahmed. Queer Phenomenology: Orientations, Objects, Others. Duke University
Press, Durham, NC, 2006. ISBN 978-0-8223-39144. URL https://doi.org/// 1215/
9780822388074
Julian P. Alexander. Philosophy of punishment. Journal of the American Institute of Criminal Law and Criminology, 13:235, 1922.
Jacob Andreas. Language models as agent models. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 5769-5779, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v/ /v1 /2002...indiggs //aclanthology.orgg// wwwp:/....rr/22......2 www.ggs--eml..4444 423
URL https:
Aristotle. Melaphysics. 350 BCE.
Aristotle. Nicomachean Ethics. 350BCE.
Kenneth C. Arnold, April M. Volzer, and Noah G. Madrid. Generative models can help writers without writing for them. In IuI Workshops, 2021. URL htR//ttts://cerrws.oorr
Marcus Aurelius. Meditations. 180 AD.
Bela H. Banathy. Designing Social Systems in Changing World. Springer US, 1996. ISBN 978-0-306-45251-2
Jeffrey A. Bell, Andrew Cutrofello, and Paul M. Livingston (eds.). Beyond the Analytic- Continental Divide: Pluralist Philosophy in the Twenty-First Century. Routledge, 2016. ISBN 978113878736..
Steve Benford, Chris Greenhalgh, Gabriella Giannachi, Brendan Walker, Joe Marshall, and Tom Rodden. Uncomfortable interactions. In Proceedings of the SIGCHI Conference
Human Factors in Computing Systems (CHI '12), pp. 2005-2014, New York, NY, USA, 2012. Association for Computing Machinery. doi: 10.114//220666.22088335 URL URLhttps: //doi.org/10.11 11452227766
Marit Bentvelzen, Pawet W. Woćniak, Pia S.F. Herbes, Evropi Stefanidi, and Jasmin Niess. Revisiting reflection in hci: Four design resources for technologies that support reflection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 6(1),
2022. doi: 10.1145
URL http: doi. http:///xx.dii.or///././ wrw/ 1000..11555 1145/51115 7233
Bhavya Bhavya, Jinjun Xiong, and Chengxiang Zhai. Cam: A large language model-based creative analogy mining framework. In Proceedings of the ACM Web Conference 2023, pp. 3903-3914,, New York, NY, USA, April 2023. ACM. doi: 10.1145/3543507.55744111 URL httpps//doi.orgg:
1445/354350735584444
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. PIQA: reasoning about physical commonsense in natural language, 2020. URL URLhhtpp:///bi..rr/// 1609/ 1609/ aaai. v34105. 6239
Kevin M. Blasiak, Marten Risius, and Sabine Matook. Conceptualising social bots for countering online extremist messages. In ACIS 2021 Proceedings, pp. 81, 2021. URL
Fernand Braudel. The Medilerranean and the Mediterranean World in the Age of Philip Volume
1. The Mediterranean and the Mediterranean World in the Age of Philip II. University of California Press, 2023. ISBN 9780520400658..
Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z. Gajos. To trust or to think: Cognitive forcing functions reduce overreliance ai in ai-assisted decision-making, Proc. ACM Hum.-Comee Interact., 5(CSCW1), apr 2021. doi: 10.1145 3449287. 10.1145/3449287, URL https!
//doi.org/10. 1145/3448.115/4442887
12

---

Published as a conference paper at COLM 2024
Alice Cai, Steven R Rick, Jennifer L Heyman, Yanxia Zhang, Alexandre Filipowicz, Matthew Hong, Matt Klenk, and Thomas Malone. Designaid: Using generative ai and semantic diversity for design inspiration. In Proceedings of The ACM Collective Intelligence Conference, CI '23, pp. 1-11, New York, NY, USA, 2023. Association for Computing Machinery, ISBN 9798400701139, doi: 10.1145/35822366 3611996666699 3615596
URL httt::///0l..rr//101114////////3
Alice Cai, Ian Arawjo, and Elena L. Glassman. Antagonistic ai, 2024. URL https://arxiv org/abs/2400 07350
Alex Calderwood, Vivian Qiu, Katy Ilonka Gero, and Lydia B. Chilton. How novelists use generative language models: An exploratory user study. In Werner Geyer, Yasaman Khazaeni, and Michal Shmueli-Scheuer (eds.), Joint Proceedings of the Workshops Human- Al Co-Creation with Generative Models and User-Awarr Conversational Agents co-located with 25th International Conference on Intelligent User Interfaces (IUI 2020), Cagliari, Italy, March 17, 2020, volume 2848 of CEUR Workshop Proceedings. CEUR-WS.org, 2020. URL https://ceur-va wrtp://uurrssror////// org/ 2848/24488888 -AI---N--P----------
Susan Carey, Cognitive science and science education. American psychologist, 41(10):1123,
1986.
Rudolf Carnap. Philosophical Foundations of Physics. Basic Books, Inc. Publishers, New York, first printing edition, 1966. ISBN 00008545.
Tuhin Chakrabarty, Xurui Zhang, Smaranda Muresan, and Nanyun Peng. MERMAID: metaphor generation with symbolism and discriminative decoding. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pp. 4250-4261. Association for Computational Linguistics, 2021. doi: 10.1 2006666/////22.////0 /2011.NAAAL--MIIN.33. URL http:://ool..rrr///111633///////
100 18653/v1/2021 naacl main. 336
Tuhin Chakrabarty, Vishakh Padmakumar, Faeze Brahman, and Smaranda Muresan. Creativ- ity support in the age of large language models: An empirical study involving emerging 2023.
writers. ArXiv, abs/
Alan F. Chalmers. What Is This Thing Called Science? Hackett Publishing Company, Inc., fourth edition, Sep 2013. ISBN 978-1624660382.
Kathy Charmaz. Constructing Grounded Theory: A Practical Guide through Qualitative Analysis. SAGE Publications Ltd, 1 edition, 1 2006. iSBN 978-0761973539.
Liuqing Chen, Yiyan Tsang, Qianzhi Jing, and Lingyun Sun. A almm-uumeeted morphologi- cal analysis approach for conceptual design. In C. Gray, E. Ciliotta Chehade, P. Hekkert, L. Forlano, P. Ciuccarelli, and P. Lloyd (eds.), DRS2024: Boston, Boston, USA, June 23-28 2024. Zhejiang University, Hangzhou, China; wwweeaann-iiinaporreeerr Innovation and Al Joint Research Lab, Hangzhou, China. doi: 10.21606/ crs. Ntts.2024.605
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained code, 2021. / org/ absp2107 03374
13

---

Published as a conference paper at COLM 2024
Leah Chong, Jude Rayan, Steven Dow, Ioanna Lykourentzou, and Faez Ahmed. Cad- prompted generative models: A pathway to feasible and novel engineering designs, 2024.
URLLhhtts:://arxivoorr httgg//bb//////1/// 08675
John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, and Minsuk Chang. Talebrush: Sketching stories with generative pretrained language models. In Proceedings of the 2022 CHI Conference on Human Faclors in Computing Systems, CHI '22 New York, NY, USA, 2022. Association for Computing Machinery. ISBN doi: 10.1145/3411022.55011
URL httpss://oi. org/ 1145/3491198 3501
Marta R. Costa-jusså, James James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Y. Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. No language left behind: Scaling human-centeres machine translation, 2022. URL https:t//:///o org/ 10. 48550/ arviv. 2wt.2222..000 04672
Hai Dang, Karim Benharrak, Florian Lehmann, and Daniel Buschek. Beyond text generation: Supporting writers with continuous automatic text summaries. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST '22, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450393201. doi: 10.145//526613335555777 JRLLhttps://bi.org/// 1ete///0..111/22222 1145/35261111 3545672
Valdemar Danry, Pat Paataranutaporn, Yaoli Mao, and Pattie Maes. Don't just tell me, ask
me: Ai systems that intelligently frame explanations as questions improve human logical discernment accuracy over causal ai explanations. In CHI '3: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 352:1-352:13, New York, NY, USA, April 2023. Association for Computing Machinery. doi: 004144//544444.55866666 URL https://doi.orgg 1145/3544548. 3580672
Gilles Deleuze and Félix Guattari. What Philosophy? Les éditions de Minuit, France, 1996 columbia university press edition edition, 1991. ISBN 978-0231079891. English translation published in 1994 by Columbia University Press.
Design Council. What is the framework for innovation? design coun- cil's evolved double diamond, 2019. URL https://web. archive.crroooo......
org
hht.rrawwwweeiiinnttoooooeoooore 2024-08-04.
Accessed:
Cora Diamond. Anything but argument? Philosophical Investigations, 5(1):23 1982.
doi: ottphtttgg//bboooo///////////1/0 wiley. wwtp.//////////./////oo 1111/j 1171111111..1779-99110 1467 1982. tb00532.
URL URLhhtps///onniieliirrrr
Nevia Dolcini. The analytic/ comtynenta/ divide: Entities and being. Soochow Journal of
Philosophical Studies, 16:283-302,
Kristie Dotson. How is this paper philosophy? Comparative Philosophyy, 3(1):03-29, 2012.
ISSN 2151-6014. URL https:: /ttp::://wwwwc.. hwtp://ccmmaraatttthhoppppppppp
Wanyu Du, Zae Myung Kim, Vipul Raheja, Dhruv Kumar, and Dongyeop Kang. Read, revise, repeat: A system demonstration for numan-in-the--lop iterative text revision. In Proceed- ings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022). Association for Computational Linguistics, 2022. doi: 10.18653 /t21://222222222ttttgt 1.14.
URL http://dx doi org/10. 18653/v1/200 /22.222//2222.. in2wr ieperriiinng-- ting-1.
Peitong Duan, Jeremy Warner, Yang Li, and Bjoern Hartmann. Generating automatic feedback on ui mockups with large language models. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI '24 New York, NY, USA, 2024. Association
14

---

Published as a conference paper at COLM 2024
for Computing Machinery. ISBN 9798400703300. doi: https://doi.org: org/10. 1145/361 3904. 3642782
URL
Olga Lucía Uribe Enciso, Diana Sofía Uribe Enciso, and María del Pilar Vargas Daza. Critical thinking and its importance in education: Some reflections. Rastros Rostros, 19(34)778-88, 2017.
Robert H. Ennis. A faxonomy of critical thinking dispositions and abilities. W H Freeman/Times Books/ Henry Holt & Co, 1987.
Robert H Ennis. Critical thinking assessment. Theory into practice, 32(3):179-186, 1993.
Alexander R. Fabbri, Wojciech Kryšcinski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. SummEval: Re-evaluating Summarization Evaluation. Transactions of the Association for Compulational Linguistics, 9:391-4099 04 2021. ISSN 2307-387X. doi: 10.1162 a / 1162/
Peter A Facione. Toward a theory of critical thinking. Liberal Education, 70(3)225366, 1984.
Giulia Di Fede, Davide Rocchesso, Steven Dow, and Salvatore Andolina. The idea machine: Lim-based expansion, rewriting, combination, and suggestion of ideas. pp. 623-627, 2022. doi: (6.1145/3527927..0
URL httpss:/doi.onnn
http://oooo.oo//1/// org/10g 1145/ 3535197
Raymond Fok, Hita Kambhamettu, Luca Soldaini, Jonathan Bragg, Kyle Lo, Marti A. Hearst, Andrew Head, and Daniel S. Weld. Scim: Intelligent skimming support for scientific papers. In Proceedings of the 28th International Conference on Intelligent User Interfaces (IUI 23), pp. 15, New York, NY, USA, March 27-31 2023. ACM. doi: 10.1145 444555165441 64.344444,,44444 .3584034.
Philippa Foot. The problem of abortion and the doctrine of the double effect. Oxford Review, 5:5-15, 1967.
Michel Foucault. The Archaeology of Knowledge. Éditions Gallimard, 1969a.
Michel Foucault. Qu'est-ce qu'un auteur? (what is author?). Lecture given at the Société Française de Philosophie, February 22 1969b.
Michel Foucault. Histoire de sexualité: La volonté de savoir, volume 1 of Histoire de la sexualité. Editions Gallimard, Paris, 1976.
John Lewis Gaddis. The Landscape of History: How Historians Map the Past. Oxford University Press, Oxford, UK, 1st edition, 2004. ISBN 978-0195171570.
Tim van Gelder. Teaching critical thinking: Some lessons from cognitive science. College teaching, 53(1):41-48, 2005.
K. Gero, Vivian Liu, and Lydia B. Chilton. Sparks: Inspiration for science writing using language models. Proceedings of the 2022 ACM Designing Interactive Systems Conference, 2021.
Steven M. Goodman, Erin Buehler, Patrick Clary, Andy Coenen, Aaron Donsbach, Tiffanie N. Horne, Michal Lahav, Robert MacDonald, Rain Breaw Michaels, Ajit Narayanan, Mahima Pushkarna, Joel Riley, Alex Santana, Lei Shi, Rachel Sweeney, Phil Weaver, Ann Yuan, and Meredith Ringel Morris. Lampost: Design and evaluation an ai-assisted email writing prototype for adults with dyslexia. In The 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 22), pp. 17, Athens, Greece, 2022. ACM. doi: 10.1145/3514428.55459
Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: survey of progress and challenges, 2024. /R..https://arxiv.orr/// htwpg//bs//2200/ 01680
15

---

Published as a conference paper at COLM 2024
Helen Halbert and Lisa P. Nathan. Designing for discomfort: Supporting critical reflection through interactive tools. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, CSCW '15, pp. 349-360, New York, NY, USA, 2015, Association for Computing Machinery. ISBN 9781450339224. 2675162.
doi:
Thorsten Händler. Balancing autonomy and aligmment: A multti-dimensional taxonomy for autonomous llm-powered multi-agent architectures, 2023. URL https://doi.ooo //tp://///oo.o/100///
48550/arXiv. 2310.03659
Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. Reasoning with language model is planning with world model, December 2023. URL
httts://aclaattooleyy.or//////// 1p mainmma...0
Sandra Harding. Rethinking standpoint epistemology: What is "strong objectivity"? In
Feminist epistemologies, pp. 49-82. Routledge, 2013.
Gilbert Harman and Judith Thomson. Moral Relativism and Moral Objectivity. Wiley- Blackwell, 1st edition, Jan 1996. ISBN 978-0631192114.
Martin Heidegger. Being and Time. 1927. Original title: Sein und Zeit. Translated by John Macquarrie and Edward Robinson (1962), Joan Stambaugh (1996).
Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. Aligning Al with shared human values, 2021. URL https://openreview
cet/fooum?id=ddy RKzJacY,
Airlie Hilliard, Cristian Muñoz, Zekun Wu, and Adriano Soares Koshiyama. Eliciting personality traits in large language models. CoRR, abs / /2//22222200141111 2024. doi: 10.48550/
ARXTV.2402.083341 URL httpps://doi..rgg.ttt 48550//rki5 2402. 08341]
Lily Hu. What is race? in algorithmic discrimination on the basis of race? Journal of Moral Philosophy, 21(1-2) 2023. doi: 10.1165//7452244-2223386
Jessica Hullman, Ari Holtzman, and Andrew Gelman. Artificial intelligence and aesthetic judgment, 2023. URL httpps://arxiv org/ 2t/./bb///./..///3 12338
David Hume. A Treatise of Human Nature. John Noon, London, 1739.
Edmund Husserl. Cartesian Meditations: An Introduction to Phenomenology. Armand Collin, 1931.
Aldous Huxley. Brave New World. Harper & Brothers, New York, 1932. ISBN 978-0-06- 085052-4.
Mohsen Jamali, Ziv M. Williams, and Jing Cai. Unveiling theory of mind in large language models: A parallel to single neurons in the human brain, 2023, URL https:://oiiorrg 48550/arXiv. 2309. 01660
Jenkins and Patrick Lin. Ai-assisted authorship: How to assign credit in synthetic scholarship. Report, Ethics Emerging Sciences Group, 2023.
Mingyu Jin, Beichen Wang, Zhaoqian Xue, Suiyuan Zhu, Wenyue Hua, Hua Tang, Kai Mei, Mengnan Du, and Yongfeng Zhang. What if Iims have different world views: Simulating alien civilizations with Ilm-based agents, 2024. URL https: pittps://doi.oot wrw.1//....r////.. 48550//rxiiv
2402. 13184
Immanuel Kant. Critique of Pure Reason. 1781.
Qusai Khraisha, Sophie Put, Johanna Kappenberg, Azza Warraitch, and Kristin Hadfield. Can large language models replace humans in systematic reviews? evaluating gpt- 4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages. Research Synthesis Methods, 15(4):616-66 626, July 2024. doi: 10.1002/ 1715. Epub 2024 Mar 14.
16

---

Published as a conference paper at COLM 2024
Søren Kierkegaard. The Sickness Unto Death: A Christian Psychological Exposition For Upbuild- ing And Awakening, volume 19 of Kierkegaard's Writings. Princeton University Press, Nov 1983. ISBN 9780691020280.
Jeongyeon Kim, Sangho Suh, Lydia B. Chilton, and Haijun Xia. Metaphorian: Leveraging large language models to support extended metaphor creation for science writing. In Designing Interactive Systems Conference (DIS 223), pp. 21, New York, NY, USA, 2023. ACM. doi: 00.1145/366366 35595996.. URL hnttps://dol org/10 10. (145/356365) 3595996
Jiho Kim, Ray C. Flanagan, Noelle E. Haviland, Zeai Sun, Souad N. Yakubu, Edom A. Maru, and Kenneth C. Arnold. Towards full authorship with Al: supporting revision with ai-generated views, 2024. URL https://ceurr-wt org/Vol -otVVol136668 Voloooo066/0ppppr11/1
Travis Kriplean, Michael Toomim, Jonathan Morgan, Alan Borning, and Amy J. Ko. Is this what you meant? promoting listening on the web with reflect. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '12, pp. 1559-1568, New York, NY, USA, 2012. Association for Computing Machinery. ISBN 9781450300154. 2200621.. URL 10. 1l.1145/228666... 2208621
doi:
org. ofw.ooiaauuooooccuuuuuoou 1ib. wwshington. .dw.
Thomas S. Kuhn and David Hawkins. The structure of scientific revolutions. American Journal of Physics, 11:554-555, 1963.
Michelle S. Lam, Janice Teoh, James A. Landay, Jeffrey Heer, and Michael S. Bernstein. Concept induction: Analyzing unstructured text with high-level concepts using lloom. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '4). ACM, 2024. ISBN 979-8-440070333--//24
doi: 1011455/611904.3342888 URL https
//doi.org/10.. 11455/1111111111113113
3642830
Ellen J. Langer. The Power of Mindful Learning. Addison-Wesley, New York, 1997.
Bruno Latour. Science in action how to follow scientists and engineers through society. Contemporary Sociology, 18:788, 1989.
Neil Levy. Analytic and continental philosophy: Explaining the differences. Metaphilsssphhy 34(3)3284-304, 2003. ISSN 00261068, 14679973. URL http://www wtwp:/jssooorogg/taabal/ooo 24439383
Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Yi Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Scott Smith, Yian Yin, Daniel A. McFarland, and James Zou.
Can large language models provide useful feedback on research papers? a large-scale empirical analysis. NEJM AI, 16a 2024. doi: 10.1056 Alo2240019 URL
https://ai.nee ne wttp:///g////.oo///// / 1056 AIoa/4001222
Jingxian Liao and Hao-Chuan Wang. Nudge for reflective mind: Understanding how access- ing peer concept mapping and commenting affects reflection of high-stakes information. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, CHI EA '22, New York, NY, USA, 2022. Association for Computing Machinery. ISBN URL https:://oi..or///1ee 1145///39.1115191111111
9781450391566. doi: 3519815
Susan Lin, Jeremy Warner, J.D. ZamffrescuPPereira Matthew G Lee, Sauhard Jain, Shanqing Cai, Piyawat Lertvittayakumjorn Michael Xuelin Huang, Shumin Zhai, Bjoern Hartmann, and Can Liu. Rambler: Supporting writing with speech via llm-assisted gist manipulation. In Proceedings of the CHI Conference Human Factors in Computing Systems, CHI 24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400703300. doi: 10.1145/3611 11144///313994..422217 URL BtLLihtts:///oi..ort org 10. 1ttp////...1111311111 0145/31139900 3642217
Vivian Liu, Jo Vermeulen, George Fitzmaurice, and Justin Matejka. 3dall-e: Integrating text- to-image ai in 3d design workflows. In Proceedings of the 2023 ACM Designing Interactioe Systems Conference, DIS '23, pp. 1955-1977, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9781450398930. doi: 10.1145/ 00.1145/3563657 33.99698... URL
https://doi. org/10. 1145/3563657.3556668
17

---

Published as a conference paper at COLM 2024
Yiren Liu, Si Chen, Haocong Cheng, Mengxia Yu, Xiao Ran, Andrew Mo, Yiliu Tang, and Yun Huang. How ai processing delays foster creativity: Exploring research question co-creation with lim-based agent. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI 24), pp. 25, New York, NY, USA, 2024. ACM. doi: https:ts:://dee wwg. /111/1113///9 3642698
Yuwen Lu, Ziang Tong, Qinyi Zhao, Yewon Oh, Bryan Wang, and Toby Jia-Jun Li. Flowy: Supporting ux design decisions through ai-driven pattern annotation in multi-screen user
flows, 2024. URL htt.:/ttss///xxi.orr////////////
161777
Jenny Ma, Karthik Sreedhar, Vivian Liu, Sitong Wang, Pedro Alejandro Perez, and Lydia B. Chilton. Didup: Dynamic iterative development for ui prototyping, 2024. URL https
//arxiv.org/abs//40// 20040//02220004444
Kevin Ma, Daniele Grandi, Christopher McComb, and Kosa Loucher-Lambert. Conceptual design generation using large language models, 2023. URL pettps://doi.org/// 48550/
2306. .17799
John E. McPeck. Teaching Critical Thinking: Dialogue and Dialectic. Routledge, 1990. ISBN 0415902258, 9780415902250.
John E. McPeck. Critical thinking and education. Routledge, 2016.
Moses Mendelssohn. Jerusalem, or on Religious Power and Judaism. Friedrich Maurer, Berlin, 1783.
Lisa Messeri and M. J. Crockett. Artificial intelligence and illusions of understanding in sci- entific research. Nature, 627(8002) March 1 2024. doi: /tth:s:s:///dii org/10 0 1038/s41586--22 020-0-24------ 0714--07
Jennifer Meyer, Thorben Jansen, Ronja Schiller, Lucas W. Liebenow, Marlene Steinbach, Andrea Horbach, and Johanna Fleckenstein. Using Ilms to bring evidence-based feedback into the classroom: Ai-generated feedback increases secondary students' text revision, motivation, and positive emotions. Computers and Education: Arlificial Intelligence, 6: 100199, 2024. ISSN 2666-920X. doi:
100199. URL
Piotr Mirowski, Kory W. Mathewson, Jaylen Pittman, and Richard Evans. Co-writing screen- plays and theatre scripts with language models: Evaluation by industry professionals. pp. 355:1-355:34, 2023. doi: 10.1145/344444.3381225 URL https:://6ii.or//1111111
3544548. 3581225
Moti Mizrahi and Mike Dickinson. The analytic-continental divide in philosophical practice:
An empirical study. Metaphilosophyy, 52(5):668 680, 2021. doi: 10.1111 mtt./m.tta...1.
Meredith Ringel Morris. Scientists' perspectives on the potential for generative ai in their fields, 2023. htt.:ttss////riiiovor////////// 01440
Anwesha Mukherjee, Vagner Figuerêdo de Santana, and Alexis T. Baria. Impactbot: Chatbot leveraging language models to automate feedback and promote critical thinking around impact statements. In Albrecht Schmidt,, Kaisa Väänänen, Tesh Goyal, Per Ola Kristensson, and Anicia Peters (eds.), Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA 2023, Hamburg, Germany, April 23-28, 2023, pp. 388: 388:88 ACM, 2023. doi: 04445555544499355555 3573844
URL https:///ol: org /0/ /ttp////....//44500111 1145/354559
Sheshera Mysore, Zhuoran Lu, Mengting Wan, Longqi Yang, Steve Menezes, Tina Baghaee, Emmanuel Barajas Gonzalez, Jennifer Neville, and Tara Safavi. PEARL: personalizing large language model writing assistants with generation-calibraee retrievers. CoRR, abs abs/2331.09180 2023. doi: (08550/AARXV.23111 2311.......
URL htttps://dii.on /ttp:///ooioor//0
Friedrich Nietzsche. Also sprach Zarathustra: Ein Buch für Alle und Keinen, Ernst Schmeitzner, Germany, 1892. Published in parts between 1883 and 1892.
18

---

Published as a conference paper at COLM 2024
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul E. Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022. URL http: http::: nips. cc http//ppppereleeeeeeeeeeeeeeeeee files/ html
Soya Park and Chinmay Kulkarni. Thinking assistants: Llm-based conversational assistants that help users think by asking rather than answering. CoRR, abs 2023. doi: 10.48550/ 20048550/ARXIV22312..00 ARXIV22312.00024,
URL httpp:://dii.rrg///// wttp///.....///r.i.ii..///
48550/arXiv. 2312..60244
Elena Pasquinelli, Mathieu Farina, Audrey Bedel, and Roberto Casati. Naturalizing critical thinking: consequences for education, blueprint for future research in cognitive science. Mind, Brain, and Education, 15(2):168-176, 2021.
Robert T Pithers and Rebecca Soden. Critical thinking in education: A review. Educational research, 42(3) 2000.
Plato. Theaetetus. 369BCE.
Plato. Phaedrus. 370BCE.
Plato. The Republic. 380 BC.
Mark Pock, Andre Ye, and Jared Moore. Lims grasp morality in concept, 2023. URL
02299
Karl Popper. The Logic of Scientific Discovery. Routledge Classics, 2nd edition, Feb 2002. ISBN 978-0415278447.
Marissa Radensky, Daniel S. Weld, Joseph Chee Chang, Pao Siangliulue, and Jonathan Bragg. Let's get to the point: Lim-supported planning, drafting, and revising of research-paper blog posts, 2024. URI httphhttts//axii.orrggg.o wtt.orr//////./o/ess 10370
LE Raths, S Wasserman, A Jonas, and Rothstein. Teaching for critical thinking: Theory and application. Columbus, OH: Charles-Merrill, 1966.
John Rawls. A Theory of Justice: Original Edition. Harvard University Press, 1971. ISBN 9780674880108. URL
Jude Rayan, Dhruv Kanetkar, Nicole Gong, Yuewen Yang, Srishti Palani, Haijun Xia, and Steven P. Dow. Exploring the potential for generative ai-based conversational cues for real-time collaborative ideation. Creativity and Cognition (C&C 24), pp. 15, New York, NY, USA, June 23-26 2024. ACM. doi: 10.1145/3633666366666
Frederick Reif. Applying cognitive science to education: Thinking and learning in scientific and other complex domains. MIT press, 2008
Paul Ricoeur. Hermeneutics and the Human Sciences: Essays on Language, Action and Interpreta- fion. Cambridge University Press, Cambridge, 1981. ISBN 0521280028.
Emma Rodman. On political theory and large language models. Political Theory, 2023. doi: 10.1177 021177/009059112312000
Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton-Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. Code llama: Open foundation models for code. CORR, abs/ / 2308 12950, 2023. doi: 10.48550/ AR..V.230.. 12950.. URL https! 7/doi. www.o/gg.e... 48550/ arviv. 2308. 12950
19

---

Published as a conference paper at COLM 2024
Jean-Paul Sartre. Being and Nothingness: An Essay on Phenomenological Ontology. Editions Gallimard, 1943.
Nino Scherrer, Claudia Shi, Amir Feder, and David M. Blei. Evaluating the moral beliefs encoded in llm.. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances Neural Information Processing Systems 36: Annual Conference
Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL htkL/httpp/ppperr..nnt nips. cc/paerr files/paper/ a2cf
2023/ httpshh//22c22255a3a33 2elecc2255a33222222 2f2.2f2f2f9fff www.ee/eeeaeeee2---a--rret-e---e http://ce...aaaaa....4aaa...a2aa
html
Oliver Schmitt and Daniel Buschek. Characterchat: Supporting the creation of fictional characters through conversation and progressive manifestation with a chatbot. In Corina Sas, Neil A. M. Maiden, Brian P. Bailey, Celine Latulipe, and Ellen Yi-Luen Do (eds.), Proceedings of the 13th ACM SIGCHI Conference on Creativity and Cognition, C&C 2021, Virtual Event / Venice Ital,, June 22-23, 2021, pp. 7:1-7:10. ACM, 2021. doi: 10.1145/ 3450741.3465255 URL https://:///ol/ org/10 1145/3450741. 34652553
Eric Schwitzgebel, David Schwitzgebel, and Anna Strasser. Creating a large language model of philosopher. CoRR, abs 2202201139, 2023. doi: 10.48550/ 0055500/ARXIV2222000133333 URL
bttps::/dol.org// http///10..855//aa0 (88550/aXXiv 2ww..v..3....1.33 01339
Donald A. Schön. Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions. Jossey- -aase, San Francisco, CA, 1987. ISBN 978-1555420253.
Orit Shaer, Angelora Cooper, Osnat Mokryn, Andrew L Kun, and Hagit Ben Shoshan. Ai- augmented brainwriting: Investigating the use of Iims in group ideation. In Proceedings of the CHI Conference Human Factors in Computing Systems, CHI '24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400703300. doi: 10.1145/3613904 3642414. URL http:tts////o..rr///1// 1145/36 361 3904 3642414
Donghee Shin. The perception of humanness in conversational journalism: An algorithmic information-processingg perspective. New Media & Society, 24(12):2680-2700 2022. doi:
(0.1177/141144621993880
Ben Shneiderman. Creating creativity: user interfaces for supporting innovation. ACM Transactions on Computer-Humar Interaction (TOCHI), (())))),7)) 2000.
Lei Shu, Liangchen Luo, Jayakumar Hoskere, Yun Zhu, Yinxiao Liu, Simon Tong, Jindong Chen, and Lei Meng. Rewritelm: An instruction-tuned large language model for text rewriting, 2023.
J. Simon, G. Rieder, and J. Branford. The philosophy and ethics of ai: Conceptual, empirical, and technological investigations into values. DISO, 3(10), 2024. doi: 10.1007 /44400--024-00094-2 URL httt:tt//toio.orrrrr /rg////0...r..////./ 1007/s44206 224-00094-2
Peter Singer. Famine, affluence, and morality. Philosophy and Public Affairs 1(3) 1972. ISSN 00483915, 10884963, URL HRLL/htp:///ww/ http:://wwww.ssorrroo.o jstorr wttpe/////eaee/22552222222
Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell L. Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. A roadmap to pluralistic alignment, 2024. URL https::/ddiiorg//
48550/arXiv. 2tt.Xivv.222.0557777
Baruch Spinoza. Ethics, Demonstrated in Geometrical Order. Posthumous, 1677.
Springer. Artificial intelligence (ai) editorial policies. http:://wwwwspiinne. cww. gp/ editorial-polieeee httpra/lppplpeeerffiiiiiiiiaalla http://rrrrcraaaiieeeeeeeeeaa/// aii225422582 2024. Accessed: 2024- 07-29.
James W. A. Strachan, Dalila Albergo, Giulia Borghini, Oriana Pansardi, Eugenio Scaliti, Saurabh Gupta, Krati Saxena, Alessandro Rufo, Stefano Panzeri, Guido Manzi, Michael S. A. Graziano, and Cristina Becchio. Testing theory of mind in large language models and humans. Nature Human Behaviour, 8(7): 8(7) 2024. doi: 10.1038 1ee11388/415622000 1151//4442120-0-188222222 1038 -
URL https:t///o/iiii..
20

---

Published as a conference paper at COLM 2024
Na Sun, Chien Wen (Tina) Yuan, Mary Beth Rosson, Yu Wu, and Jack M. Carroll. Critical thinking in collaboration: Talk less, perceive more. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA '17, pp. 2944-2950, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 9781450346566. doi: 002114453327063..40 10.1145/3027063.. 386-57733227063338622828
URL https://6oiiorg http:///oiioorg.oocaaooo//oo 1ib. wwwhington.nnommm edw.
Thitaree Tanprasert, Sidney Fels, Luanne Sinnamon, and Dongwook Yoon. Debate chatbots to facilitate critical thinking youtube: Social identity and conversational style make difference. In Proceedings of the CHI Conference on Human Factors in Computing Systemss, CHI '24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400703300. doi: 00.145/3613904436422 3642513
URL httpp:://doi.org//0.114443
Henansh Tanwar, Kunal Shrivastva, Rahul Singh, and Dhruv Kumar. Opinebot: Class feedback reimagined using conversational LLM. CoRR, abs/ /bbbb.21001158, 2024. doi: 10.48550/ AAA.V.2XXI..111.....
URL / httpp::://oi..or///..... 48tp0////0/////....a... 15589
David R. Thomas. A general inductive approach for analyzing qualitaatii evaluation data. American Journal of Evaluation, 27(2):237-246, 2006. doi: 10.1177/11992110005553
lain D. Thomson. Rethinking the Analytic/Conttinettal Divide, pp. 569-589. Cambridge
University Press, 2019.
Michel-Rolph Trouillot. Silencing the Past: Power and the Production of History. Beacon Press books. Beacon Press, 1995. ISBN 9780807043110.
Raymon Van Dinter, Bedir Tekinerdogan, and Cagatay Catal. Automation of systematic literature reviews: A systematic literature review. Information and Software Technology, 136: 106589, 2021.
Gerit Wagner, Roman Lukyanenko, and Guy Paré. Artificial intelligence and the conduct of literature reviews. Journal of Information Technology, 37 (7 2022.
Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, Anima Anandkumar, Karianne Bergen, Carla P. Gomes, Shirley Ho, Pushmeet Kohli, Joan Lasenby, Jure Leskovec, Tie-Yan Liu, Arjun Manrai, Debora Marks, Bharath Ramsundar, Le Song, Jimeng Sun, Jian Tang, Petar Veličković, Max Welling, Linfeng Zhang, Connor W. Coley, Yoshua Bengio, and Marinka Zitnik. Scientific discovery in the age of artificial intelli- gence. Nature, 620(7972):47-60 August 2023a. doi: 10.1038 00008//41586-025 s41556--02306222--2 URL 011133/2244188822-26222112
https://doi.
Jinping Wang and Zeynep Tanes-Ehle. Examining the effects of conversational chatbots changing conspiracy beliefs about science: The paradox of interactivity. Journal of doi: 0001000008833115..202222.
Broadcasting & Electronic Media, 67(1):68 89, 2022.
UtL/https:///oi.oom orw.0/...rg/1... 10800 2022. 2153842
Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, and Furu Wei. Augmenting language models with long-term memory. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neu- ral Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, Decem- ber 10 16, 2023, 2023b. URL /ttp://pppprs..... nipperrs..iss.... httppss.ccpapprrrec http:c/ppppeefile//ppaeeeee www.f//ppeeeee///eeeeeeeeaaaaeee 2t1///2222/hhh/ 0123:7/55444733bffaaee5556666 ffr AwwtracttCooferrreeeec htmll
Hayden White. Metahistory; The Historical Imagination in Ninetecnth--cnntury Europe. Johns Hopkins University, 1973.
Xiaotong (Tone) Xu, Jiayu Yin, Catherine Gu, Jenny Mar, Sydney Zhang, Jane L. E, and Steven Dow. Jamplate: Exploring llm-enhanced templates for idea reflection. In Pro- ceedings of the 29th International Conference Intelligent User Interfaces, IUI 24, pp. 907-921, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400705083. doi: 10.1145 0555//34454333645511 URL http:::://oii.rgg//// 110....1155166880 36451966
21

---

Published as a conference paper at COLM 2024
Yusuke Yamamoto and Takehiro Yamamoto. Query priming for promoting critical thinking in web search. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval, CHIIR '18, pp. 12-21, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 10. 18.1145/3176349, 31777777779
doi: 10.1145/317349.3176333 URL https:://oo org/
Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. Wordcraft: Story writing with large language models. In Proceedings of the 27th International Conference on Intelligent User Interfaces, TUT '22, pp. 841-852, New York, NY, USA, 2022. Association for Computing URL https://doi org/
Machinery. ISBN 9781 41811450994444 doi:
10.1145/3499099.1 351.99999993111189 3511105
Brahim Zarouali, Mykola Makhortykh, Mariella Bastian, and Theo Araujo. Overcom- ing polarization with chatbot news? investigating the impact of news content contain- ing opposing views on agreement and credibility. European Journal of Communication
36(1):53-68, 2021. doi: 10.1177/0967333120440000 URL https:///6i..or//11....
0267523120940988
Xin Zhao. Leveraging artificial intelligence (ai) technology for english writing: Introducing wordtune as a digital writing assistant for efl writers. RELC Journal, 54:890 894, 2022.
Yukun Zhao, Lingyong Yan, Weiwei Sun, Guoliang Xing, Shuaiqiann Wang, Chong Meng, Zhicong Cheng, Zhaochun Ren, and Dawei Yin. Improving the robustness of large lan- guage models via consistency alignment. Nicoletta Calzolari, Min-Yen Kan, Véronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (eds.), Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy, pp. 8931-8941. ELRA and ICCL, 2024. 2org/2000 2024. lrec-meein 782
CRR_https:///ccaanhollgg org/
Caleb Ziems, Jane Dwivedi-Yu, Yi-Chia Wang, Alon Y. Halevy, and Diyi Yang. Normbank: A knowledge bank of situational social norms. In Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguisttcs (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 7756-7776. Association for Computational Linguistics, 2023. doi: 10.18653/ /V1 ACL3LACLL 429. URL https::pp/// httt:::/ddo..or//// 111.1/1..166311111 acl long. 429
22

---

Published as a conference paper at COLM 2024
A Interviewee Information Sheet
Table provides high-level information about each interviewee which may be relevant to interpreting and contextualizing their views. The General Interest(s) feature describes the broad fields that the interviewees work in. The Notable Specific Interest(s) feature describes any specific topics in the field(s) mentioned in the General Interest(s) feature that the inter- viewees focus their work on. This feature is not exclusive, meaning that interviewees may also work on other topics outside of the specific interests. If the value for this feature is blank, then the interviewee's 's work is sufficiently characterized by the value in the General Interest(s) feature. The Experience with LMs feature describes three levels of experience with using LMs: little to none, limited, and extensive. If interviewees have either limited or extensive experience with using LMs, the Uses of LMs feature describes their primary use: for teaching (e.g., using LMs to teach material, trying to understand features of LM-generated student submissions), for personal use (e.g., to improve productivity, for entertainment) for exploration (i.e., playing around with the LM out of curiosity understand the technology better), and for research (i.e,, their research is on LMs). Note that the following interviewees have published least one article on some aspect of Al: (P5, P6, P13, P14).
23

---

Published as a conference paper at COLM 2024
For teaching For exploration For teaching For personal use For teaching. For research
For personal use For personal use For exploration
For personal use
For exploration For class
For exploration For exploration For exploration
For exploration For class
Use of LMs
Experience with LMs
Little to None Little to None Limited
Little to None Little to None
Extensive
Extensive
Extensive
Extensive
Extensive
Limited
Limited
Limited
I. Limited
Limited
Limited
I Limited
Limited
Limited
Limited
Limited
Philosophy of Technology, AI
Notable Specific Interest(s)
Bioethics, Feminist Ethics Philosophy of Biology Meta-ethics
Philosophy Technology
Philosophy of Statistics
Value theory, Literature
Public and Global Policy
Table Interviewee information
Philosophy of Phyysics
Philosophy Biology
Semantics, Linguistics
Philosophy Physics
German philosophy
Moral psychology
Virtue ethics
Psychology
Bioethics
Ethics, History of Philosophy
Ethics, Political Philosophy Philosophy of Science
Ethics, Political Philosophy Ethics
Ethics, Political Philosophy
Philosophy of Science Philosophy Science Logic, Philosophy of Mind Aesthetics
Ethics, Political Philosophy Pedagogy, Epistemology
Philosophy of Science
History of Philosophy
Philosophy of Science
Philosophy of Science
Philosophy of Science
History of Philosophy
General Interest(s)
Ethics, Aesthetics
Ethics
Ethics
Associate Professor
Associate Professor
Assistant Professor
Assistant Professor
Assistant Profeessor
Associate Professor
Associate Professor
Associate Professor
Assistant Professor
Teaching Professor
Associate Professor
Assistant Professor
A Associate Professor
Professor
Professor
Professor
Professor
Professor
Professor
Professor
Professor
Title
ID
Pl
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P16
P17
P18
P19
P20
P21
24

---

Published as a conference paper at COLM 2024
B Interview Questions and Guidelines
1. Meta-philosophy
(a) What is philosophy? Why do you go about doing philosophy? What aims do you have?
(b) What drives the 'doing' of philosophy? isat the role of personal motivations, subjective experience, and aesthetic judgements?
(c) Who what can 'do' philosophy? For instance, can LLMs 'do' philosophy?
(d) What makes doing philosophy 'difficult' nontrivial?
(e) How does philosophy distinguish its products from those of other disciplines?
2. The philosophical process
(a) How do you go from no idea to spark of an idea / an unrefined idea?
(b) How do you develop and refine philosophical ideas? What moves have to happen?
(c) How mechanical / creative is the process of doing philosophy?
(d) What is the relationship between texts textual methods and philosophy? Does philosophizing, to some extent, operate 'above' language in ideas / thoughts?
(e) What is the role of conversation in the doing of philosophy? What are some of its challenges?
(f) What makes for a good interlocutor, and what makes for good conversation?
3. Language Models for philosophy
(a) What roles can language models play in the development of philosophy?
(b) What do language models need to be better in the development of philosophy?
(c) What are some of the opportunities and strengths for language models in philosophy?
(d) What are some of the risks and weaknesses for language models in philosophy?
(e) Would you use language models in intellectually substantive ways currently? What about in the future, with plausible improvements?
25


Reviews:
[{"review_id": "tQVbK1vvRj", "reviewer": "Reviewer_ttPJ", "summary": "The paper discusses the use (potential and actual) of LMs by philosophers as critical thinking tools. From interviews with philosophers, the authors determine two broad groups of reasons why philosophers find current LMs problematic as thinking tools. Based on this, they propose a basic framework to characterise such tools, and suggest how LM outputs and behaviour might be designed to give better support for this kind of use.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s feedback!\n\nWe agree with the reviewer that there could be more contextualization of the selfhood-initiative (SI) model. The SI model comes directly from our interviews, in which we discovered two central themes in philosophers’ experiences with LMs: LMs are “neutral, detached, and nonjudgmental” (lack selfhood), and “servile, passive, and incurious” (lack initiative). These themes serve as the axes in our model, which we use to identify roles in which at least one of the axes is positive. \n\nOur model is also related to other work from a variety of fields:\n\n- Many recent HCI works are related to the “initiative” axis of our model. Liao et al. (“Nudge for Reflective Mind”, 2022), Danry et al. (\"Don’t Just Tell Me, Ask Me\", 2023), inter alia, discuss how building proactively prompted moments for user reflection improve users’ critical thinking.\n- Many education studies emphasize grounded conversation for developing critical thinking (“selfhood” axis). Lang (“Critical thinking in web courses”, 2000) writes that “critical thinking happens in good discussions”. Students have such good discussions when they actively engage with other students’ ideas (Bai, “Facilitating Students’ Critical Thinking in Online Discussion”, 2009), such as through role-play (Schindler et al., “Instructional Design and Facilitation Approaches”, 2014).\n\nHowever, to our knowledge, our model is the first to cohesively synthesize these two axes and relate it to the design of LMs.\n\nW.r.t. “The suggestions are fairly open challenges…” We believe that the way in which we have described selfhood and initiative are concretely linked to existing LM research problems, such as in LM memory use, user-specific alignment, autonomous planning and tool use, theory of mind, etc. (More in S6.2.) We want to draw attention to certain existing (but possibly underappreciated) challenges as important for making LMs useful as critical thinking tools. For instance, researchers might build LM systems which (safely) perform multiple uninterrupted steps of conversation (e.g. asking questions, using tools, changing topics) without human supervision. Alternatively, researchers might develop methods for LMs to predict more advanced elements in theory of mind, such as uncovering implicit values and unspoken confusions (by anticipating questions users might ask), or even begin by building a more complex taxonomy and benchmark.\n\nThank you!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This is a thought-provoking and original piece of work: it's useful to see discussion of the interaction design aspects of LMs, and serious consideration of how they might support (or not) different kinds of end use.", "reasons_to_reject": "The findings about reasons why LMs are not much used in this context are interesting and seem solid, but the analytical framework proposed to explain this and used as the basis for suggestions does not seem to be so clearly justified; and the suggestions are fairly open challenges without much in the way of suggestions as to how (or whether) they could be implemented or addressed.", "questions_to_authors": "The two-dimensional self/initiative model for critical thinking tools is proposed without a great deal of justification. While I can see that it links to the two main themes identified from the analysis of the interviews, I wasn't clear if it has any precedent or builds on / relates to other existing models of critical thinking or of creative assistive technology. There is plenty of prior work in designing tools to assist creative thinkers (artists, designers etc - I know of work dating back at least to Shneiderman (1997)'s Genex framework but I'm no expert in that area so I'm sure there's more), and there seems to be work in categorising critical thinking according to various dimensions. Does this model relate to previous work, and if so how? This would help justify this choice.", "extra_scores": {}}, {"review_id": "UBD8pEjN05", "reviewer": "Reviewer_Ynvx", "summary": "This paper is interested in the use of LLMs to assist humans in critical thinking. Academic philosophers, in interviews which are qualitatively coded, are asked to reflect on the use of LLMs to assist them in their work. Overall, the interviewed philosophers find that LLMs are not useful. The paper diagnoses this uselessness in two missing properties in LLMs: they lack a sense of selfhood and initiative. Possible roles for LLMs that vary these features (i.e., having selfhood and lacking initiative, lacking selfhood and having initiative, and having both) are discussed as a way of trying to guide the development of LLMs.", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s feedback! To address the reasons to reject (RR) and questions:\n\n[Q1]  To clarify: we outline in the intro and the related work that LLMs are often used as tools to automate some types of thinking (e.g. coding, emails), but that people want to be the ones doing the thinking when they’re engaging in critical thinking tasks; AI tools should be augmentative here, not automative.\n\n[Q2] The reviewer suggests that the assertion that LMs lack selfhood and initiative is not empirically supported. Indeed, our paper is not quantitatively empirical; rather, we interview and systematically extract themes from philosophers’ experiences with LMs. The claims that LMs are neutral, detached, etc. are directly taken from the interviews with interviewee citations and quotes. LM scientists are finding now that some of the most useful metrics come not from static benchmarks but from real users (e.g. LMSYS Arena), often experts (see reviewer NJTN). Our paper presents a detailed, qualitatively empirical evaluation of a set of critical thinking experts’ (philosophers) experiences with current LMs as critical thinking tools.\n\n[RR1] The reviewer suggests the link between critical thinking tools and the choice to interview philosophers is unclear. To clarify, we specify in S2.3 that we study philosophers as a case study of how people might use LMs as critical thinking tools, as nearly all philosophical problems require critical thinking. We discuss how critical thinking is used in many fields in the intro, but narrow in on philosophy for the paper – so we believe we do “probe how [one] of these fields utilizes critical thinking and what tools are useful” as the reviewer suggests. In order to do this probing, we need to holistically understand philosophers’ experiences with LMs (such as weaknesses of LMs for philosophy).\n\n[RR2] Selfhood and initiative can certainly be thought of as reflecting aspects of agency, which we agree is a fraught & abstract concept. However, we believe we have presented selfhood and initiative in concrete ways which do not require addressing agency. Many attributes of selfhood and initiative are already LM research areas, such as using memory / persistent states, user-specific alignment, autonomous planning and tool use, theory of mind, etc. We describe and cite several such works in S6.2. Our work draws attention to certain existing (but possibly underappreciated) problems as important for LM critical thinking tools.\n\nThank you!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "I found the discussion of the roles for LLMs (interlocuter, monitor, and respondent) to be interesting. I think this framing is interesting to consider in an educational context (e.g., how should universities position themselves with relation to AI assistants). Further the paper is well written and does a solid job of situating the current use cases of LLMs in contrast with their use in fostering critical thinking.", "reasons_to_reject": "The link between the paper’s aim (positioning LLMs as critical thinking tools) and the argumentation and methods (the interview of academic philosophers) is not clear. If the aim is to study what makes a tool useful for supporting critical thinking, why are philosopher’s views on LMs (e.g., answers to questions like “What are some risks and weaknesses for language models in philosophy”) relevant for addressing this? At times the paper discusses how critical thinking is deployed in endeavors like “history, science, and philosophy”, wouldn’t it make sense to probe how each of these fields utilizes critical thinking and what tools are useful for doing this more efficiently? \n\nFurther, the discussion of the things that are useful for philosophers in critical thinking, selfhood and initiative, both require agency (in my understanding) in the definitions given in the paper (“selfhood is a resource’s ability to have certain locally persistent internal states (such as perspectives, beliefs, opinions, memory) and to consistently use them as the basis for judgements” and “Initiative is a resource’s ability to set its own intentions and goals, possibly different from its user’s, and to execute actions oriented towards those intentions.”). This feels not concrete enough to offer practical guidance in developing LLMs.", "questions_to_authors": "Who is the “we” in “we claim all the rights to think?”. I read this as saying we humans claim all the rights to think. What relationship does that have to LLMs? It feels to me, in fact, like that statement is in tension with the aims of the paper (the description of a tool to automate some types of thinking). Perhaps, we will use LLMs to think more critically? But the other uses cases (e.g., coding, generating emails) mentioned in the beginning are taken as useful because they alleviate the need for that type of thinking.\n\nTwo issues with LLMs, that they are “highly neutral, detached, and non-judgmental” and “servile, passive, and incurious” are mentioned a few times in the paper, however, no empirical results are given to support this. If this is the perception of the interviewees that is fine, but it should be made clear in the paper.", "extra_scores": {}}, {"review_id": "owkD1VstTL", "reviewer": "Reviewer_NJTN", "summary": "This paper explores the potential for language models to serve as tools for critical thinking, using philosophy as a case study. The authors engage with prior work and conduct a sound qualitative study by interviewing 21 professional philosophers, providing valuable empirical insights into how philosophers think and their attitudes toward language models. The work demonstrates originality by examining language models for critical thinking rather than just text generation/editing assistance. The proposed \"selfhood-initiative\" model offers a practical conceptual framework, and the envisioned roles for language models like the Interlocutor, Monitor, and Respondent represent creative new possibilities. Significantly, the paper highlights significant gaps in current language models for supporting deeper reasoning required for critical thinking and articulates an agenda for developing better-suited models, which could have wide-ranging impacts. The discussion around language models shaping metaphilosophical questions and the discipline of philosophy itself is highly relevant. Overall, this is a remarkable work that makes a valuable contribution by studying an underexplored area at the intersection of AI and philosophy through a combination of good design, conceptual modeling, and creative proposals for future development.", "questions": "", "limitations": "", "rating": 10, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s feedback! In particular, we would like to concur with the reviewer’s argument that “LLMs are good at benchmarks, but that sort of reasoning often doesn't imply critical reflective thinking”. Increasingly, LM scientists are finding that the most useful metrics come not from static benchmarks but from real users, especially experts. We hope our paper, in addition to providing a detailed qualitative evaluation of a specific user population (philosophers), paves the way for future work on LM tools for philosophical and other critical thinking problems.\n\nAlthough in our interviews, we had participants interact with GPT-4 if they did not have familiarity with LMs, many of our participants had already interacted with a variety of LMs in the past, including GPT 3.5/4, Claude, Bard, etc. Therefore, our interviews document interactions with a variety of models. We would also note that Claude Opus had not yet come out during the time of the interviews.\n\nThank you!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This is a high-quality paper. The authors know their topic well. The writing and organization are very impressive. The key questions and findings are helpful to the community, especially when there is a huge amount of anthropomorphizing of LLMs. LLMs are good at benchmarks, but that sort of reasoning often doesn't imply critical reflective thinking, and then there is the classic case of LLMs being servile, passive, and incurious (thanks to RLHF). This comes time and again in creative writing where writers feel that RLHF limits LLMs in exploring darker difficult topics. I appreciated the design and recommendations for what makes a better critical thinking tool. The SelfHood Initiative model is pretty exciting and will likely have a good impact on interdisciplinary researchers who work with LLMs\n\nI would fight for this paper's acceptance.", "reasons_to_reject": "No reason to reject", "questions_to_authors": "Did you try Claude? I have heard it's better than GPT4.", "extra_scores": {}}, {"review_id": "ktYdksm9Zo", "reviewer": "Reviewer_veW6", "summary": "While often LLMs are employed to quickly retrieve standard knowledge and arguments, this paper discusses the question to what extent LLMs can serve as tools for critical thinking in the sense of questioning, reorienting, analysing, and developing ideas. As a case study, the paper reports on interviews with 21 professional philosophers about how they engage in critical thinking and on their experiences with LMs. In view of the findings that LLMs are considered to fail to be useful in open, undetermined contexts, and also do not allow intellectual interaction driven by curiosity, the authors model conversations with LLMs along two dimensions – selfhood and initiative. Based on the resulting matrix of high and low selfhood and initiative, they propose three roles of LLMs for philosophy: the Interlocutor (high-selfhood, high-initiative), the Monitor (low-selfhood, high-initiative), and the Respondent (high-selfhood, low-initiative). In conclusion, they take up some suggestions of where LLMs might be improved (e.g. reasoning about “uncommon” sense, or improving their long-range planning), and advocate the use of LLMs in a playful way despite their insufficiencies.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s feedback! To address the reasons to reject (RR) and questions:\n\n[Q1, RR1]: Philosophers were recruited from doctoral universities across the US, selected for variation across area of specialization (e.g., philosophy of science, political philosophy, ethics) and across experience with LMs. Some of our interviewees had published works on philosophical issues with LMs; others had only heard about LMs but not interacted with them. By interviewing a diverse set of philosophers, our findings more strongly reflect concerns shared across philosophers.\n\n[Q2, RR2]: In cases where philosophers did not have familiarity with LMs, we used a protocol in our interviews to let them interact with the GPT-4 model. However, most philosophers report having interacted with GPT 3.5/4, Claude, Bard, etc. Therefore, our findings are not restricted to one specific model.\n\n[Q3, RR3]: We agree with the reviewer’s suggestion that the noted deficits “may be a result of just the way how standard LMs have been set up”! Our paper does not aim to identify fundamental flaws with LM training, architectures, etc. Rather, we aim to document the experiences of philosophers with current mainstream LMs. Addressing some of these deficits may or may not be ‘simple fixes’, but there are good reasons to believe many are ‘deep-rooted’. Firstly, philosophers make clear that their concerns with current LMs are not only due to poor factual knowledge; the way in which LMs approach philosophers (without selfhood or initiative) makes LMs poor critical thinking tools – not entirely addressable with RAG. Moreover, many of the concerns are directly related to open and difficult research problems, such as building stateful & consistent LMs and skills in theory of mind (S6.2). Improving on these problems will help not just philosophers but critical thinkers in general.\n\n[RR4]: We agree that our discussion of philosophy is quite general. Philosophy is a very rich discipline with a wide diversity of methodologies, interests, and problems. Each subarea would use LMs in a very different way, and we agree that important future work would take a more narrow and concrete look at these uses. However, as we believe this paper is one of the first to introduce philosophers’ experiences with LMs into the LM literature, we want to present the general moves that philosophers make (described in S4.1) and experiences they have with LMs (S4.2-3) across these subareas.\n\nThank you!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper presents an interesting and novel classification of interactions with LLM based chatbots. While the findings of the interviews are not surprising, this classification helps to better understand the reasons behind the interviewed philosophers’ assessments.", "reasons_to_reject": "The paper leaves out important details: \n(1) By which criteria have the interviews philosophers been selected? Has it been ensured that the interviewed have experience with LLMs? \n(2) Which LMs and Chatbots have been used? \n(3) The claims are very general and do not take into account possible modifications of the experimental set-up. In particular, the noted insufficiencies may be a result of just the way how standard LLMs have been set up. An alternative might be databases of pro- and con-arguments that could be added by fine tuning or retrieval augmented generation (RAG) on such databases. \n(4) From the philosopher’s side, the conclusion also appears too general, missing out differences in the intended gain of knowledge in the philosophical traditions and methods.", "questions_to_authors": "(1) By which criteria have the interviews philosophers been selected? Has it been ensured that the interviewed have experience with LLMs? \n(2) Which LMs and Chatbots have been used? \n(3) Have you been considering possible modifications of the experimental set-up?", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper has a significant spread of review scores that was not resolved by discussion or rebuttal. 

The paper under review explores the use of language models (LMs) by philosophers as tools for critical thinking. It is based on interviews with 21 professional philosophers, who were asked to reflect on the utility of LMs in their work. The authors identify two primary reasons why philosophers currently find LMs problematic: the lack of selfhood and initiative. They propose a framework to characterize these tools and suggest improvements to LM outputs and behaviors to better support critical thinking.

Reasons to Accept
====
The paper presents a novel perspective by discussing the interaction design aspects of LMs and how they might support critical thinking, an area not extensively covered in existing literature. 
* The qualitative study involving interviews with professional philosophers provides valuable empirical data on the attitudes towards and the perceived limitations of LMs. 
* The proposed "selfhood-initiative" model offers a new conceptual framework to understand and enhance the role of LMs in critical thinking. 
* The insights and proposals could significantly impact interdisciplinary researchers working with LMs, contributing to the broader discourse on AI and philosophy.

Reasons to Reject
====
* The justification for the "selfhood-initiative" model is not well-developed. The connection between this model and existing theories or frameworks in critical thinking or creative assistive technology needs to be clarified.
* The relevance of philosophers' views on LMs to the broader aim of studying tools for critical thinking across various fields (history, science, philosophy) is not well-argued. Additionally, the criteria for selecting interviewees and the specific LMs used in the study are not detailed. Although 


Despite some methodological and theoretical weaknesses, the empirical insights and conceptual contributions make it a valuable piece of work. It addresses an underexplored area at the intersection of AI and philosophy, offering a basis for future research and development.

[comment from the PCs] Please make sure to add the details as noted missing by the AC.
Decision: Accept