Code-Switching Metrics Using Intonation Units
Authors: Rebecca Pattichis
Dora LaCasse
Sonya Mitrovich Trawick
Rena Torres Cacoullos
Venue: EMNLP 2023 Main (2023)
Keywords: Computationally-aided linguistic analysis
Linguistic Diversity
Multilingualism and Cross-Lingual NLP
Spanish-English Code-Switching
TL;DR: While NLP models of code-switching (CS) have used word-level units, we advocate for a multi-word prosodic unit by showing how metrics of CS complexity are impacted by the distinction between other-language single-word items and multi-word strings.

Abstract: Code-switching (CS) metrics in NLP that are based on word-level units are misaligned with true bilingual CS behavior. Crucially, CS is not equally likely between any two words, but follows syntactic and prosodic rules. We adapt two metrics, multilinguality and CS probability, and apply them to transcribed bilingual speech, for the first time putting forward Intonation Units (IUs) – prosodic speech segments – as basic tokens for NLP tasks. In addition, we calculate these two metrics separately for distinct mixing types: alternating-language multi-word strings and single-word incorporations from one language into another. Results indicate that individual differences according to the two CS metrics are independent. However, there is a shared tendency among bilinguals for multi-word CS to occur across, rather than within, IU boundaries. That is, bilinguals tend to prosodically separate their two languages. This constraint is blurred when metric calculations do not distinguish multi-word and single-word items. These results call for a reconsideration of units of analysis in future development of CS datasets for NLP tasks.

Code-Switching Metrics Using Intonation Units
Rebecca Pattichis'
Dora LaCasse Sonya Trawick
Rena Torres Cacoullos
University of California, Los Angeles
University of Montana
3 Pennsylvania State University
pattichiècs .uw.elssscllllccooo...o..celaaea edu, dora lwcasse@mso.......ooo..o........ umt edu,
sonyatrawickpsueeme com, rena@psu. edu
Abstract
this paper, we adapt Natural Language Processing (NLP) word-hased metrics account for prosodic- syntactic constraints on CS by using Intonation Units as tokens. We calculate two metrics: over- all language distribution (mulilinguality) and ob- served CS rate (probability). Results indicate that distinguishing single- and multi-word items in Into- nation Unit-based metrics better captures CS con- straints than amalgamating the mixing types.
Code-switching (CS) metrics in NLP that are based word-level units are misaligned with true bilingual CS behavior. Crucially, CS is not equally likely between any two words, but follows syntactic and prosodic rules. We adapt two metrics, multilinguality and CS probability, and apply them transcribed bilingual speech, for the first time putting forward Intonation Units (IUs) prosodic speech segments as basic tokens for NIPP tasks. In addition, calculate these two metrics separately for dis- tinct mixing types: alternating-lanuuaat multi- word strings and single-word incorporations from one language into another. Results in- dicate that individual differences according to the two CS metries are independent. However, there shared tendency among bilinguals for multi-word CS occur across, rather than within, IU boundaries. That is, bilinguals tend to prosodically separate their two languages. This constraint is blurred when metric calcula- tions do not distinguish multi-word and single- word items. These results call for a reconsider- ation of units of analysis in future development CS datasets for NI.P tasks.
As working definition, bilingual CS is going back and forth between two languages within a speaker turn, within prosodically and gram- matically complete sentence, in the following example, reproduced verbatim from the New Mex- ico Spanish-Fnglish Bilingual (NMSEB) corpus (Torres Cacoullos and Travis, 2018: Chs 2 & 3) See Transcription conventions Fig. 1. Italic and roman type represent speech originally in English (E) and Spanish (S), respectively.
ahí estaba con mi e- con mi espejito, que así me miraba y. if could see my profile,
S
S
S
1 Introduction
E
... luego me volteé pa' atrás,
S
The mismatch between actual bilingual code- switching behavior and renditions of code- switching (CS) has been exposed in previous work. In particular, unnatural switching is char- acterized by the shortness of language spans and the arbitrariness of language boundaries (Bullock et al., 2019). Yet to date, metrics of CS com- plexity have been word-level based (e.g., a 4- word utterance with switches, WL1 WL2 W WL2, Gambäck and Das, 2016:1851). Such word- level metrics entail that any word is equally likely site for language switching, In addition, they treat lther-la single-word incorpora- tions and alternating-languae multi-wone strings identically. Neither is a foolproof assumption. In
I could see the back of my head.
E
there was with my with my little mirror, that was looking at myself and, if could see my profile,
S
S
S
E
and then turned around, I could see the back of my head. (03, 05:45-05:53)
S
E
Within parantheses following each example are the NM- SEB corpus transcript number and beginning- end time stamp. http://nmmode-swithhi la.psu.edu

---

1.1 The Syntactic and Prosodic Structure of CS
final intonation contour
continuing intonation contour
Language is not composed of arbitrary strings of words, but is structured syntactically and prosodi- cally. Syntactically, sequential word groups make up constructions and constituents (c.g.. Bybee. 2010:136). For example, in the active and pas- sive sentence versions, they allow the new school wear hats in the halls and the new school's al- lowed to wear hats in the halls, "the new school" (article adjective + noun) is a unit that as such is the object in one and the subject in the other (cf. Weiner and Labov, 1983:34).
truncated intonation contour
truncated word
short pause (0.2 secs)
medium pause (0.3-0.6 secs)
Figure I: Transcription conventions (Du Bois et al., 1993). Each line represents an Intonation Unit (IU). For the purposes of readability, have removed vocal noises, laughter and vowel lengthening, and excerpt- initial pauses. A prosodic sentence an IU or series of IUs, containing at least finite verb, that ends in intonational completion (Chafe, 1994:139).
Syntactic word groups are relevant for CS. According to the Equivalence constraint, CS is avoided at points of word order incompatibility (Poplack, 2013:586; Sankoff, 1998) (cf. Deuchar, 2020:255; Muysken, 2000:27, Pfaff, 1979:291) I). For example, in the Spanish-English language pair, articles placed before nouns in both languages. However, there is word order conflict for attribu- tive adjectives, which are positioned before the noun overwhelmingly in English but only some- times so in Spanish. As predicted by the Equiv- alence constraint, CS is far more likely after the article than between the adjective and noun (cf. Parafita Couto and Gullberg, 2019:702). Instances like "permiten el new school to wear" ('they allow quantitatively pre-
constraint. This captures bilinguals' tendency prosodically separate the two languages. The IU- Boundary constraint states that CS favored across IU boundaries (cf. Torres Cacoullos and Travis, 2018: 51). That is, switching is far more likely between two words at the boundary of IUs, or across IUs, than between two words within the same IU. The prosodic and syntactic constraints related in that the looser syntactic relationship between words IU boundaries than within them may mean greater word order flexibility, However, the two constraints are independent in that, both at IU boundaries and within IUs, CS is subject to the Equivalence constraint and, among Equivalence points, CS is more likely at IU boundaries than within them.
the new school to wear') ferred (at 4:1 ratio) over instances like "permiten el nuevo school to wear" (Torres Cacoullos and Vélez Avilés, 2023:17-18).
Responding to these facts, we apply CS metrics using IUS rather than individual words as the basis.
Prosodically, spoken language is broken into In- tonation Units (IUs), which are speech segments "uttered under single, coherent intonation con- tour" (Du Bois et al., 1993:47; ef. Chafe, 1994: 53-70; see Figure 2 below). In the NMSEB cor- pus, each line of transcription represents an IU, with punctuation marking the transitional continu- ity between IUs, or the terminal pitch contour. IUs are bound to the syntactic organization and pro- cessing of language. Words in the same IU tend have closer syntactic relationship than those in neighboring IUs (Croft, 1995:849-864), and IU boundaries are used planning utterances (Ono and Thompson, 1995) and "tracking speech" (Inbar et al., 2020).
1.2 Lone Items vs. Multi-Word Strings
Mixing is of two broad types: single-word incorpo- rations, or lone items, as in the following example, and multi- strings, the first example above). These types are approx- imately parallel to the distinction between CS of the insertional and alternational kind (Muysken, 2000; Muysken, 2015: 251-22112 (--54),, though they are operationally defined as one word and two or more words, respectively (on quantitative grammatical patterns of lone items, see Poplack and Dion, 2012; Sankoff et al.. 1990).
mulli--ordd CS (as in
tenían unos desks muy grandes, SLS
IUS are also relevant for CS. Corresponding the Equivalence monstraint-whie requires lo- cal equivalence of word order in the two lan- guages around a switch point-is the IU- -U-Bound
they had some really big desks, SLS (03, 53:25-53:27)

---

The two types have divergent structural proper- ties (Poplack, 2018; Torres Cacoullos and Travis, 2020:256-22 -.002..--559....... Lone items are disproportionately nouns, are placed according to the word order of the surrounding (recipient or matrix) language, and participate in the categories and constructions of that language (for example, lone English nouns like desks are given Spanish gender category and may have a post-nominal adjective). In contrast, multi-word strings are placed at cross-language equivalence points (where the word order is the same) while the internal constitution of each string is consistent with the grammar of its respective language.
2 Related Methods
Recent work in NLP is concerned with incorporat- ing, understanding, and generating CS text. Al- though there are already large multilingual lan- guage models such as M-BERT (Pires et al., 2019) and XLM- XLM-RoBERTa (Conneau et al., 2019), these LLMs were nevertheless trained on parallel mono- lingual data, which does not accurately capture the nuance of likely switch points in code-switched language inputs. CS datasets, on the other hand, are scarce and noisy, putting CS modelling in the domain of low-resource tasks (Doğruöz et al., 2021). Recently, the LinCE benchmark compiled by Aguilar et al. (2020) provides different CS datasets for Branish-English, Hindi-English, and other language pairs for the tasks of Language Identification, Part of Speech Tagging, Named En- tity Recognition, and Sentiment Analysis. Often, datasets are collected and scraped from social me- dia (i.e., Twitter). However, there are two problems with this collection approach.
For many Spanish English bilingual communi- ties, the single- vs. muldi-word distinction is addi- tionally important because the distribution of the two mixing types by language starkly different. Language mixing with lone items is asymmetri- cal (Poplack, 2018). Lone items are 52% English nouns (le puso un roof nuevo 'she gave it a new roof' [25, 26:311), but only 7% Spanish nouns (put the whole calabaza there "put the whole pumpkin there' [17, 30:01]) (plus English and Spanish other than noun words, 21 % apiece; NMSEB, 1=2.991) (Torres Cacoullos and Vélez. Avilés, 2023).
First, CS is known to happen in community set- tings among interlocutors with similar CS patterns (cf. Deuchar, 2020; Poplack, 2018), 8). Therefore, mass collecting short tweets does not necessarily provide a true representation of intra-community CS. Second, these datasets do not hold a distine- tion between mixing types. Specifically, without this distinction, examples that merely use lone items tend to represent a majority of CS exam- ples. In other words, most CS datasets used in NLP inherently rely asymmetric models such as the Matrix Language Frame (MLF) model, which assumes a base language for the grammat- ical frame with words or phrases embedded from the other language MMyers-Scotton 2002). Instead, the Equivalence constraint and IU-Boundary con- straint accommodate Alternating-langwee word strings. Here, we showcase the different per- spectives that the treatment of lone items provides on CS.
In contrast, multi-word CS is bidirectional (Tor- res Cacoullos and Travis, 2018:67-72). Constitut- ing the 15% (n=2489) of eligible sentences (at least four words) hosting multi-word CS (alternating- language strings of least two words) are 6% Spanish English and 4% English to Spanish, with another 5% hosting more than one instance of CS. Examples are: pues estaban asina como caddy cor- ner across the street 'well they were like [...]' [04, 1:02:12]): -Alma was more flaca que la mama,"L 'L...] thin than mama' [04, 1:04:18]); bring that name, porque no había ~Roberts, todavía, in the family, 'I...] because there weren't 't any -Roberts, yet, [...]' [04, 15:591).
multi-
Finally, lone other-language items and alternat- ing multi-word CS are only moderately correlated. For the NMSEB corpus, a Pearson correlation co- efficient assessing the relationship between the rate of lone items (14% 2378/16957) of sentences host at least one lone item) and the rate of multi-word CS (15%, 2489/1 6957, of sentences host at least one word CS) yields positive correlation of just r(38)= = .58,p p- < .01. For this set of rea- sons, we will distinguish lone items and multi-word strings in applying the CS metrics.
Barnett et al. (2000) developed the Multilingual- Index (M-Index) as a measure of the multilingual- ity of different corpora, or the distribution of lan- guages in corpus. Guzman et al. (2017) also created the Integration- Index (I-Index), which is meant to measure the probability of CS in differ- ent multilingual corpora. Mave al. (2018) use both of these metries in the creation and validation of their code-switched Twitter dataset for Hindi- English and Dpanish-Engllss pairs. However, it is

---

(192)
let's
say
que
(inbreath)
nunca
dejo
este
hombre
yo
sin
darle
su
almuerzo
let's say que,
(inbreath) nunca dejo este hombre yo sin darle su almuerzo,
Time (s)
Figure 2: Acoustic properties of Intonation Unit include higher pitch at the beginning of the IU and slower rate of speech at the end of the IU, and sometimes pausing betweens IUs.
important to note that the I-Index was originally de- veloped with the purpose of analyzing continuous text. Therefore, disjointed and small input texts such as tweets might not be a good use case.
guages is seen in quantitative patterns of gram- matical structures in each language, which align with their respective monolingual benchmarks (Tor-
res Cacoullos and Travis, 2018: 204).
3 Methods
The recordings are transcribed orthographically and prosodically in IUs (Du Bois al., 1993:47). IUs are reliably identified by their acoustic corre- lates, which include a rising or high starting pitch that falls as the IU progresses, faster speech at the start of the IU as compared to the end, and, often, pauses between IUs (Figure 2).
We use both M-Index and I-Index in our exper- iments with continuous and majority monologi- cal transcripts of intra-community speech. To our knowledge, this is the first study to use the Into- nation Unit (IU) as the token of measurement for NLP analysis CS datasets. With this work, we highlight the different perspectives of CS - sepa- rating lone items from multi-word strings and present the IU relevant token for future NLP dataset construction.
Directly bearing on the problem at hand, CS is structured on the basis of prosodic boundaries and terminal pitch contours (Torres Cacoullos and Travis, 201 8:51-52 and references therein, among others). The first example in the following pair illustrates multi-word CS at the boundary of IUs (across-IU CS); the second one illustrates multi- word CS within a single IU (within-IU CS). As seen in Figure 3, CS four times more likely at the boundary of IUs than within them. Furthermore, CS at an IU boundary is more likely following fi- nal than continuing intonation (cf. the distinction between inter-sentential and intra-sentential CS). This prosodic structure of multi-word CS repre- sented the IU-Boundary constraint: prefer CS across IU boundaries. That is, bilinguals tend keep their two languages prosodically separate.
3.1 Description and Selection of Corpora
The NMSEB corpus (Torres Cacoullos and Travis, 2018: Chapters 2 and 3) provides the data for this project. The speakers are bilingual members of a long-standing speech community in northern New Mexico where Spanish and English have both been spoken for 150 years. Though New Mexican Span- ish is endangered by shift to English as well as disparagement in comparison with textbook Span- ish, these speakers regularly use both languages in their daily interactions. This balance of languages is reflected the even distribution of prosodic sen- tences by language (43% English, 42% Spanish, 15% both, n = 16,957). The integrity of both lan-

---

rates by prosodic boundary
100%
2%
14%
8%
80%
60%
and used to get chile quite bit. salía todo.
was a general store, store, vendían de todo. 'they sold everything.'
y nosotros it snap. 'and for
40%
and everything grew.'
20%
0%
Across final Intonation (period) (n=20804)
Across continuing intonation (comma) (n=42019)
Within the same IU (n=62823)
No CS
CS
Figure 3: IU-Boundary constraint: Bilinguals quantitatively prefer multi-word CS across Intonation Unit (IU) boundaries (adapted from Trawick, 2022: $3.4).
was a general store, vendían de todo.
E
speech.
S
We count all IUs, with the exception of IUs con- sisting of language-neutral material (fillers such uh, backchannels such as yeah) (Torres Cacoullos and Travis, 2018:51). IU rows were considered as eligible for future analysis they contained a language tag either 'S' (Spanish), 'E' (English), or 'L' (Lone Item). Note that language tag may be combination of 'S', 'E', or 'L'; see previous examples for illustrations of language tagging.
it was a general store, they sold everything (03, 10:10:01
E
S
Ilegaba mi tío the following days,
SE
my uncle arrived the following days, (03. 40:51-40:53)
SE
3.2 Quantitative Experiments on the IU-Token Level
In order to lay bare how amalgamating lone items with multi-word strings affects perspectives on CS, compute metrics (i.e., M-Index (Barnett et al., 2000) and I-Index (Guzmán et al., 2017)) for differ- representations of the corpora (see Tables 1-3). Specifically, I-Inexx measures were calculated two ways: by considering only 'S' or 'E' for analysis, and by also including 'L' (re-coded as 'S' or 'E').
The present study draws five transcribed recordings, totaling 4.8 hours and around 41,000 words, or 14,000 IUs. They are largely monologic, which allows us to treat CS as occurring within same speaker turn (not in response to "interactive alignment" with an interlocutor, see, e.g., Kootstra et al., 2020). The speakers are: transcript 03, San- dra (administrator, Española); 05, Rocío (teacher aid, Santa Fe); 10, Pedro (school administrator, Taos); 16, Manuel (electrician and rancher, Rio Arriba); and 27, Eduardo (general contractor and store owner, Rio Arriba).2 2 In the transcripts chosen, 84% 97% of the IUs are produced by the speaker rather than the interviewer or another interlocutor, satisfying our threshold for majority monological
Since we are operating at the 1U-token level, each token can contain more than one language tag if there is within-IU language switch. Below describe the binary measures that allow maintain the integrity of our token level.
3.2.1 M-Index
The Multilingual Index (M-Index) proposed by Bar- nett et al. (2000) is meant to measure the multi- linguality of a given corpus with at least two lan- guages from range of 0 1, where the former is
Names given are pseudonyms and locations listed are either counties or major cities to protect speaker privacy. Anonymization also occurred within each transcript, so that any real names, nicknames, or identifiable proper nouns (i.c., small cities, places of work, high school names, etc.) were replaced and indicated with a preceding "-" (Torres Cacoullos and Travis, 2018:48).
'Code can be found at https ///gtt/ttttt com/ rpat wwt.chhh//aie h/t////-aaa Stt-Bonndary_ooom ro_carstccooaannaaa raaeet_cnieneee..

---

monolingual, and the latter means there is a perfect balance of languages. Here, & denotes the number of languages in the corpus, and Pj is the number of tokens in language j divided by the total number of tokens:
only looks at one token, rather than comparison between two tokens. Again, for this metric, we consider the two perspectives with only 'S' and 'E' as well as the inclusion of 'L' to understand how including lone items may impact our understanding of CS.
En M-Index -k-1) 11-1)-Er3
Note that although later work by Bullock et al. (2019) propose normalized I-Index due to its de- pendence on corpus's M-Index, have inten- tionally chosen transcripts that are comparable in their M-Index. Specifically, the M-Index is close to for three transcripts (03, 10, 16) and .50 for two (05, 27).
While is originally meant for the word-token level, we use it at the IU level. That is, instead of the numerator of Pj representing the number of words in language j. we instead make Pj the amount of IUs in language j divided by total IUs. When an IU has multiple languages contained within its bound, we tag it with the earliest lan- guage present (i.e., an IU with 'SES' will count as 'S'). For the M-Index, we only considered 'S' and 'E' as valid language tags.
4 Results
Table gives the number of IUs counted as 'S' and as 'E' and the number of IUs hosting CS accord- ing to prosodic position and the treatment of lone items. 4 Table 2 gives the corresponding percent- ages. M-Index and I-Index for the five transcripts appear in Table 3. Of the five corpora chosen, three have an M-Index close to 1, indicating a balance of English and Spanish within the transcript. In partic- ular, M- Indicess of .94-97 correspond to speakers who produce a more balanced 41 %-45% of their IUs as 'S' and 55%-59% as 'E', while values of .52.556 resulted for speakers with 22%-23% in one language (and thus more than 75% in the other). As shown in Table 3, the combination of the M-Index and I-Index is crucial in understanding the nuance of different speakers' CS patterns. For example, transcripts 03 and 16 have similar M- Indexes (with difference of 0.03), but the former has almost twice the I-Index of the latter speaker. This indicates the independence of the two metrics. We also use language distribution graphs to visu- alize the Inddx and I-Index within a transcript, in Figure 4. Here, in an ordered array of the lan- guage tag for each IU token, English IUs are col- ored in purple, whereas Spanish is in yellow. These visualizations help elucidate what the quantitative metrics distinguish, by the number and width of the language bands. For example, transcripts 03 and 10 (language distribution graphs in Fig. 4c and 4e, respectively). have a similar M-Index and Across- IU I-Index, which is evident from their language distribution graphs. On the other hand, 05 (Fig. 4a) has patently different M and I-Index from these
3.2.2 Across-IU I-Index
Here, we use the Integration Index (I-Index) de- veloped by Guzman al. (2017) to measure the probability of CS in each transcript. Specifically, the I-Index is meant to approximate the probability that any given token is a switch point. Here, n is the number of tokens, and 5(4,,4) (4) is 1 if two neighboring tokens are in different languages, O otherwise:
I-Index
Σ
2(4,4).
15i= 15n 1
Again, while this metric was developed with an assumption of words tokens, we count the IUs as tokens. Then, for the across-IU I-Index, we ask the question: Does a language switch happen at the boundary between the 2th IU and the (i + 1)th IU? This binary measure determines the value of (7), Here, we consider two perspectives: first, with only the 'S' and 'E' language tags, and then with the inclusion of 'L'' to understand how lone items impact switching across IUs.
3.2.3 Within-IU I-Index
To also account for the Within-IU CS that occurs in our corpora, use the same I-Index but change our question: Does language switch occur within the 7th IU? Although there might rarely be more than one switch point within an IU, we decided to keep the binary measure so as to not double count a token. Here, the only change is that S(4))
'Counts in 'IUs w/ Ls' column do not correspond to the differences between "Ls incl.' and 'no Ls" in the preceding columns because an 'L' at an IU boundary may count as hoth an Across- and Within-IU switch.

---

Corp
Total S/E
Across IU
W/nn IU
IUs
Ls
Ls incl.
no Ls
Ls incl.
w/ Ls
05
1911/532
77
101
23
23
27
616/2035
194
238
12
58
54
03
1040/1501
376
464
112
111
994/1266
189
252
71
70
10
737/894
264
276
21
49
31
Table I: Number of IUs counted as 'S' and 'E'; numher of TUs hosting CS by prosodic position Across IU
boundaries vs. Within IU and by treatment of lone items-'No Ls' vs. incl.; IUs hosting Ls.
Corp
% S/E
Across IU
W/in IU
1Us
no Ls
Ls incl.
no Ls
Ls incl.
w/ Ls
05
78/22
3.2
4.1
0.2
0.9
0.9
27
23/77
7.4
8.9
0.5
2.2
2.0
03
41/59
14.8
18.0
1.1
4.3
4.3
16
44/56
8.4
11.0
0.8
3.1
3.1
10
45/55
16.2
16.8
1.3
3.0
1.9
Table 2: Percentages corresponding to Table 1.
M-Index
I: Across
W/in
Corp
(S/E)
no Ls
Ls incl.
no Ls
Ls incl.
05
0.52
0.03
0.04
0.0
0.01
27
0.56
0.07
0.09
0.0
0.02
03
0.94
0.15
0.18
0.01
0.04
16
0.97
0.08
0.11
0.01
0.03
10
0.98
0.16
0.17
0.01
0.03
Table M-Index and I-Index; I-index calculated for CS according to prosodic position and according to inclusion of Lone items (see methods section).
1008
1008
(a)
(b) 27
(c) 03
80 600
140
1800
(d) 16
(e) 10
Figure 4: Language distribution graphs for each transcript, where English is in purple and Spanish is in yellow.
transcripts, resulting in smaller total number of language bands and wider bands for Spanish.
While for the Across-IU index the increase is most 37.5% (for 16, from 0.08 to 0.11), for the Within-IU index the increase is as large as 300% (for 03, from 0.01 to 0.04) (disregarding the cases of infinity, for 05 and 27). This makes sense - lone items occur within single IU (and are usually nouns positioned at the end or interior of the IU
As how the inclusion of lone items impacts our perspective of CS, it seems have little im- pact on the Across-IU I-Index, as seen Table 3. However, including the 'L' language tag does have a substantial impact on the Within-IU I-Index.

---

with fewer than 15% occurring 1U-initially (Steuck, 2018:87)).
have shown here that bilingual behavior is not lim- ited to word substitution. accounting for multi- word language alternations, future NLP metrics can draw on the Equivalence constraint (syntax) and the related but independent IU-Boundary constraint (prosody).
Furthermore, merging the single-word items with muldi-word CS not only substantially raises the within-IU I-Index, but also magnifies differ- ences between speakers with respect to the within- IU I-Index. Within-IU I-Indices range from 0 to 0.01 with 'no Ls," but from 0.01 to 0.04 when lone items are included. This is important because shows that with the latter perspective (i.e., merging of lone items with multi-word CS), we fail to cap- ture the IU-Boundary constraint on CS whereby speakers strongly prefer switching across prosodic units rather than within them.
In sum, this paper is the first to use IUs for CS metrics. The results provide critical insight for future transcription and synthetic data generation methods that can improve CS datasets, which will ultimately impact all downstream NLP tasks.
Limitations
Multi-word tokens (whether IUs, clauses some other well-defined multi- word unit) yield CS met- rics that more in line with actual CS patterns. Nevertheless, due to differences in how tokens are delimited (word- (word-based vs. larger units), direct com- parisons between results this and previous word- based studies remain to be shown. Moreover, as noted in prior work, CS metrics are still fairly crude (e.g., it is not known what degree of difference in an M or I index would be significant). Since factors such the length and syntax of a multi-word token likely affect the probability of CS, ways need to be found incorporate predictors and constraints on CS into more refined metries.
5 Conclusion
CS metrics appropriate when they grounded in spontaneous speech within bilingual communi- ties, the natural habitat of CS. We have illustrated how using the prosodic structure of CS and distin- guishing different mixing types achieves this.
We put forward the Intonation Unit as the to- ken level for CS metries. In addition, distin- guish multi-word CS from single-word (Ione) other- language items. Application of the IU-based met- rics captures the tendency for multi-word CS to be used from one IU to another in across-IU CS. All speakers (regardless of their M-Index or Across-IU I-Index) strongly disfavor within-IU multi-word CS. This is the IU- IU-Boundary constraint of bilingual speech. In contrast, lone items, which by defini- tion are used within single IU, notably raise the I-Index for Within-IU CS. Merging multi-word CS and lone items thus obseures uniform adherence to the IU-Boundary constraint, despite individual dif- ferences language distributions (M-Index) and CS rate (I-Index).
Ethical Considerations
To protect privacy, speaker names given are pseudonyms and locations listed are major cities or counties (not specific towns). Within each transcript, any real names, nicknames, or identi- fiable proper nouns (small cities, places of work, high school names, etc.) are also anonymized (Torres Cacoullos and Travis, 2018:48). We give speaker information, even if it does not seem rel- evant to the results, as a reminder that the data instances of real bilingual speech (i.e., not syn- thetically generated), grounded in well-defined community in New Mexico. These data are not confined by the scope of accessibility issues (i.e., technology access, as with Twitter data), and there- fore represent a diverse population not limited to urban, well-educated, upper-class professionals or elite bilinguals.
With these findings, highlight the IU as important token level for future CS datasets, as well as the impact of single-word (lone) items as distinct from multi-word strings on CS metrics. We see relevance for this work in Controlled Natu- ral Language Generation (NLG). Much of the CS work is slow/iterative because it is hard to find qual- ity CS data (Tarunesh et al., 2021; Doğruöz et al., 2021). Thus, our work impacts the future construc- tion of transeribed CS datasets, specifically with the injection of linguistically well-founded CS pat- terns to improve current NL.G methods. Currently, CS generation has focused on word substitutions (Solorio and Liu, 2008; Tarunesh et al., 2021). We
A Acknowledgements
Support from the National Science Foundation (BCS-10191 12/101912,, 1624966) is gratefully acknowledged.

---

References
Bjöm Gambäck and Amitava Das. 2016. Comparing the level of code-switching in corpora. In Proceed- ings of the Tenth International Conference on Lan- guage Resources and Evaluation (LREC"16) pages 1850-1855, PortoroZ, Slovenia. European Language Resources Association (ELRA).
Gustavo Aguilar, Sudipta Kar, and Thamar Solorio. 2020. LinCE: A centralized benchmark for linguis- tic code-switching evaluation. In Proceedingss of the 12th Language Resources and Evaluation Confer- ence, pages 1803-1813, Marseille, France, European Language Resources Association.
Gualberto Guzmán, Joseph Ricard, Jacqueline Serigos, Barbara Bullock, and Almeida Toribio. 2017. Met- ries for modeling code-switching across corpora. In Interspeech, pages 67-71.
Ruthanna Barnett, Eva Codó, Eva Eppler, Montse Forcadell. Penelope Gardner- Chloros. Roeland Van Hout, Melissa Moyer, Maria Carme Torras, Maria Teresa Turell, Mark Sebba, et al. 2000. The lides coding manual: document for preparing and analyzing language interaction data version 1.1- july 1999. International Journal of Bilingualism, 4(2)1131-271.
Maya Inbar, Eitan Grossman, and Ayelet N Landau. 2020. Sequences of intonation units form a" hz rhythm. Scientific Reports, 10(1):15846.
Gerrit Jan Kootstra, Ton Dijkstra, and Janet G. van Hell. 2020. Interactive alignment and lexical triggering of code-switching in bilingual dialogue. Frontiers in Psychology, 11.
Barbara Bullock. Wally Guzmán, and Almeida Jacque- line Toribio. 2019. The limits of Spanglish? In Proceedings of the 3rd Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 115-121, Minneapolis, USA. Association for Com- putational Linguistics.
Deepthi Mave, Suraj Mabarjan, and Thamar Solorio. 2018. Language identification and analysis of code- switched social media text. In Proceedings of the Third Workshop Computational Approaches Linguistic Soit-Swithhing, pages 51-61. Melbourne, Australia. Association for Computational Linguistics.
Joan Bybee. 2010. Language, usage and cognition. Cambridge University Press.
Picter Muysken. 2000.
Bilingual speech: a typology of code-mixing. Cambridge University Press.
Wallace Chafc. 1994. Discourse, consciousness, and time: The flow and displacement of conscious experi- ence in speaking and writing. University Chicago Press.
Pieter Muysken. 2015. Dé jä voodoo or new trails ahead? re-evaluating the mixing typology model. In R. Torres Cacoullos, N. Dion, and A. Lapierre, editors, Linguistic variation: Confronting fact and theory, pages 242-261. Routledge.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle- moyer, and Veselin Stoyanov. 2019, Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv: 791 1912112
Carol Myers-Scotton. 2002. Contact linguistics: Bilin- gual encounters and grammatical outcomes. Oxford University Press.
Tsuyoshi Ono and Sandra A Thompson. 1995. What can conversation tell about syntax? In P.W. Davis. editor, Alternative linguistics: Descriptive and theo- retical modes, pages 213-272. John Benjamins.
William Croft. 1995. Intonation units and grammatical structure. Linguistics, 33(5):839 882.
Margaret Deuchar. 2020. Code-switching in linguisties: A position paper. Languages, 5(2):22.
Maria Carmen Parafita Couto and Marianne Gullberg. 2019. Code-switching within the noun phrase: Evi- dence from three corpora. International Journal of Bilingualism, (3(2)6695
Seza Dogruöz, Sunayana Sitaram, Barbara E. Bul- lock, and Almeida Jacqueline Toribio. 2021. A sur- vey of code-switching: Linguistic and social per- spectives for language technologies. of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1654-1666, Online. Association for Computational Linguistics.
Carol W Pfaff. 1979. Constraints on language mix- ing: Intrasentential code-switching and borrowing in Spanish/English. Language, 55:291-318.
Proceedings
Telmo Pires, Eva Schlinger, and Dan Garrette, 2019. How multilingual is multilingual bert? arXiv preprint arXiv: 19X.00900
Shana Poplack. 2013. Sometimes I'll start sentence in Spanish y termino en español: Toward a typology of code-switching. Linguistics. 51(s1):11-14,
John Du Bois, Susanna Cumming, Stephan Schuetze- Coburn, and Danae Paolino. 1993. Outline of dis- course transcription. In Jane A. Edwards and Mar- tin D. Lampert, editors, Talking data: Transcription and coding discourse, pages 49. Lawrence Erl- baum Associates.
Shana Poplack. 2018. Borrowing: Loanwords in the
speech community and in the grummar. Oxford Uni- versity Press.

---

Shana Poplack and Nathalie Dion. 2012. Myths and facts about loanword development. Language Varia- tion and Change, 24(3):279-315.
David Sankoff. 1998. The production of code-mixed discourse. In COLING 1998 Volume The 17th International Conference on Computational Linguis- fics.
David Sankoff, Shana Poplack, and Swathi Vanniarajan. 1990. The case of the nonce loan in Tamil. Language Variation and Change, 2(1):71-101.
Thamar Solorio and Yang Liu. 2008. Learning to pre- dict code-switching points. In Proceedings the 2008 Conference on Empirical Methods in Natural Language Processing, pages 973-981.
J.W. Steuck. 2018. The of intra-sentential multi word code- switching the New Mexico Spanish-English bilingual community. Ph.D. thesis, Pennsylvania State University.
structure
Ishan Tarunesh, Syamantak Kumar, and Preethi Jyothi 2021. From machine translation to sode-switching Generating high-quality code-switched text. Pro- ceedings of the 59th Annual Meeting of the Associa- tion for Computational Linguistics and the 1 lih Inter- national Joint Conference on Natural Language Pro- cessing Volume 1: Long Papers), pages 3154-3169, Online. Association for Computational Linguisties.
Rena Torres Cacoullos and Catherine E Travis. 2018. Bilingualism in the community: Code-switching and grammars in contact. Cambridge University Press.
Rena Tomes Cacoullos and Catherine E. Travis. 2020. Code-switching and bilinguals' grammars. In E. Adamou and Matras, editors, The Routledge Handbook of Language Contact, pages 252-275. Routledge. London.
Rena Torres Cacoullos and Jessica Vélez Avilés. 2023. Mixing adjectives: Bilingual strategies for variable word order. Linguistic Approaches to Bilingualism, pages 1-31.
Sonya Trawick. 2022. Revisiting the concept of "trig- gering" of code-switching. Ph.D. thesis, Pennsylva- nia State University.
E Judith Weiner and William Labov. 1983. Constraints on the agentless passive. Journal of Linguistics. 19( 19):229-55.


Reviews:
[{"review_id": "RdxWo4pjy7", "reviewer": "Reviewer_CGqL", "summary": "", "questions": "CGqL_A. line 329-330 : I do not understand this sentence. Should I understand that is you have S and E in a IU, you keep only the first occuring, as you precise later?\n\nCGqL_B. How could your work  be applied in a NLP task?\n\nCGqL_C. The corpus is labeled with IU. Is it your work, or was initially the corpus labeled?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer CGqL for these comments! We are glad to hear that they have no objections to accepting this paper into the conference, and find it easy to follow and exciting in its results. We are hopeful that we can clarify the questions below.\n\nThe reviewer states towards the end of their summary that lone-items are used within an Intonation Unit (IU) and multi-word code-switching (CS) is used from one IU to another. Their formulation has inspired us to add the following sentences after line 458 to underscore this paper’s innovative results: “We put forward use of  the Intonation Unit as the token level for CS metrics. In addition, we calculate these metrics by distinguishing across-IU from within-IU CS, and multi-word CS from single-word (lone) other-language items. Application of the IU-based metrics to the data captures the tendency for multi-word CS to be used from one IU to another in across-IU CS, that is, the IU-Boundary Constraint of bilingual speech. In contrast, lone items by definition are used within a single IU and therefore raise the value of the I-Index for within-IU CS. Merging multi-word CS and lone items thus obscures the uniformity of the IU-Boundary Constraint among code switchers, despite individual differences in language distributions (M-Index) and CS probability (I-Index).”\n\nWe are glad that CGqL thinks our experiment illustrates our argument well! However, we disagree that our work is a limited experiment. As noted in line 292, our work has a total of 41,000 words, and over 14,000 IU instances. See the table below for the breakdown of IU counts per transcript.\n\n| Transcript   | Number of IUs |\n| -------- | ------- |\n| 05  | 3,030    |\n| 27 | 3,181     |\n| 03    | 3,138    |\n| 16 | 2,639 |\n| 10 | 2,147 |\n| **TOTAL** | **14,135** |\n\nOur main constraint is that we chose monological transcripts to focus on the speaker, and not on how they respond to their interviewer. This arguably makes our findings more robust for our given domain (an individual's speech patterns), rather than simply ignoring the possible role of interlocutors in CS. Further work could investigate how CS metrics are impacted by dialogue, as suggested by some prior psycholinguistic and corpus-based work (e.g. by Kootstra and colleagues), although this is beyond the scope of this paper. We will be sure to further clarify the reasons for this constraint, which can be found in the original submission on lines 291-296.\n\nNext, we’d like to explicitly address each of the questions below.\n\n1. _“line 329-330 : I do not understand this sentence. Should I understand that is you have S and E in a IU, you keep only the first occuring, as you precise later?”_\nThese lines are meant to emphasize that although an IU can have multiple language tags (e.g., there's a switch within the IU), we do not want to double count an IU for any of the metrics. Therefore, lines 329-330 summarize our intent to transform the metrics into binary measures for the IU token. As clarified in lines 347-351, the M-Index rules are as follows: \"When an IU has multiple languages contained within its bound, we tag it with the earliest language present (i.e., an IU with ‘SES’ will count as ‘S’). For the M-Index, we only considered ‘S’ and ‘E’ as valid language tags.\" In other words, we consider the earliest tag for the language representation for that IU. Lines 364-366 mention the measure for the Across-IU I-Index, and lines 374-375 have the rule for the Within-IU I-Index. We will work on rewriting lines 329-330 to clarify the above rules. \n\n2. _\"CGqL_B. How could your work be applied in a NLP task?\"_\nThis is a great question, which was also asked by Reviewer VWCM. Recall from lines 464-469: our main findings for CS metrics are that 1) the IU is a vital token level for switching metrics, and 2) there should be a distinction between single- and multi-word switching. We see relevance for this work in Controlled Natural Language Generation, specifically with the injection of linguistically well-founded (Reviewer VWCM) CS patterns to improve said generation (refer to lines 471-473). Currently, CS generation has focused on word substitutions (Solorio & Liu, 2008; Chang et al., 2019; Tarunest et al., 2021).  Importantly, our work is not limited to word substitution. In accounting for multi-word language alternations, we draw on the Equivalence Constraint (syntax) and the related but independent IU-Boundary Constraint (prosody). This paper is the first to use IUs for CS metrics.\nUltimately, our work impacts the future construction of transcribed CS datasets. Much of the CS work is slow/iterative because it is hard to find quality CS data (refer to lines 195-197, where we cited (Doğruöz et al., 2021)). Our results, then, provide critical insight for future transcription and synthetic data generation methods that can improve code-switching datasets, which will ultimately impact all downstream NLP tasks.\n\n3. _\"CGqL_C. The corpus is labeled with IU. Is it your work, or was initially the corpus labeled?\"_\nThe corpus was initially labeled prior to our work as described by the corpus compilers in Torres Cacoullos and Travis (2018:49-51). We will be sure to clarify that prosodically-based transcription is part of the constitution of this corpus.\n\n\nAgain, we’d like to thank Reviewer CGqL for the insightful comments, including the typo found in line 12 which we have fixed. Importantly, we will make sure to address our work’s applicability to NLP throughout the paper, including the literature review and conclusion. We thank the reviewer for their time spent on reading our work!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Paper should be accepted because:\n- authors take time to explain the problem\n- after reading one understands better that CS is not only a random modification of texts. \n- their argue is very clear and easy to understand\n- They describe M-Index and I-Index. This allows the paper to be self-contained\n- the experiment illustrates well the argue.", "reasons_to_reject": "I do not see reason to reject the paper.", "questions_to_authors": "CGqL_A. line 329-330 : I do not understand this sentence. Should I understand that is you have S and E in a IU, you keep only the first occuring, as you precise later?\n\nCGqL_B. How could your work  be applied in a NLP task?\n\nCGqL_C. The corpus is labeled with IU. Is it your work, or was initially the corpus labeled?", "paper_topic_and_main_contributions": "The paper focuses on Code-Switching.\nThe authors argue that CS is driven by Intonation Units (and not by syntags), and that CS works differently between IU and inside IU.\nThey illustrate their claim by measuring the CS rate for 5 speakers from a Spanish-English database. This measure is based on the M-Index (0=one language, 1=balance of two languages) and the I-Index (rate of CS). For I-Index they adapt it to measure CS within IU and across IU.\nThe results show that \"lone CS\" (only one word is replace by another) is used within IU and \"multiword CS\" is used from a IU to another one.\nTherefore, CS modeling should take into account these both behaviors.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "No", "Typos_Grammar_Style_and_Presentation_Improvements": "line 012 : stings --> strings"}, "extra_text": {}}, {"review_id": "Xsywua6gqw", "reviewer": "Reviewer_VWCM", "summary": "", "questions": "1) I do not have access to the book and corpus by Torres Cacoullos and Travis (2018). Could you try to summarize what exactly is new in your paper compared to the findings in the book? The original corpus is segmented into prosodic units and distinguishes between single-word and multi-word code-switching. Assuming that your finding about the relevance of prosodic units is new, what was the reason for annotating them in the original corpus?\n\n2) You show that, although less common, code-switching can also occur within prosodic boundaries. Could it be that the preference for code-switching at prosodic boundaries is simply caused by the Equivalence Constraint (because word order compatibility is highest at the boundary between two prosodic units)?\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We want to thank Reviewer VWCM for their thorough review. We especially appreciate their opinion that the paper should be accepted for our in-depth discussion of code-switching (CS) and theoretically sound manner of addressing CS (which is often overlooked in NLP). We want to emphasize our novelty beyond a thorough discussion on CS, however. Prior work relating to CS in NLP (specifically in generation and predicting switchpoints) has focused on word-level tokens (Solorio & Liu, 2008; Chang et al., 2019; Tarunest et al., 2021). Importantly, this paper is the first to use Intonation Units (IUs) for CS metrics, opening up a new line of work beyond the word-level unit. We will be sure to emphasize this novelty in our abstract.\n\n_“The paper is limited to corpus analysis and it does not show or discuss how to apply its findings to NLP.”_\nReviewer CGqL mentioned this as well, and we address this concern in our rebuttal to them. CS is above all a phenomenon of informal speech among bilingual community members. Thus, corpus analysis is a prerequisite for developing applications. However, corpus analysis within NLP has been largely absent, precisely due to “the scarcity of corpora containing large volumes of real code-switched text” (Tarunesh et al., 2021). Much of the CS work is slow/iterative because it is hard to find quality CS data (refer to lines 195-197, where we cited (Doğruöz et al., 2021)). Our results, then, provide critical insight for future transcription and synthetic data generation methods that can improve CS datasets (see the answer to #2 on Reviewer CGqL for more details). We will be sure to emphasize this point in our literature review and conclusion.\n\nReviewer VWCM’s second reason for rejection calls into question the novelty of our work with respect to previously published work using the same corpus, especially\nTorres Cacoullos and Travis (2018). We will dive into these concerns below.\n\n- _“Could you try to summarize what exactly is new in your paper compared to the findings in the book?”_\nThe book uses the same corpus to address whether CS brings about grammatical change, but it does not include CS metrics based in NLP research. This is the main and novel contribution of this paper: putting forward NLP CS metrics for the IU token level. Our experiments are the first to quantitatively validate the IU for CS metrics (M-Index, I-Index).\n\n- _“Assuming that your finding about the relevance of prosodic units is new, what was the reason for annotating them in the original corpus?”_\nIU-based transcription has been applied to other speech corpora, for example, the publicly available Santa Barbara Corpus of Spoken American English. IU-based transcription in the present corpus was used for grammatical analysis (for example, unexpressed (null) subjects in English are restricted to IU-initial position, as in:  And then I worked for a rancher over there for a while, ... Ø followed the rodeos for a while, (SBCSAE 32: 1587–1588) (Torres Cacoullos and Travis, 2018: 119-121)).\n\n- _“Could it be that the preference for code-switching at prosodic boundaries is simply caused by the Equivalence Constraint (because word order compatibility is highest at the boundary between two prosodic units)?”_\nThis is a great question! The prosodic constraint is related to syntax in that words within prosodic boundaries have a tighter syntactic connection than words at prosodic boundaries. Therefore, the looser syntactic connection at the boundary between two prosodic units may mean greater word order flexibility. However, both within prosodic units and at prosodic unit boundaries, CS is subject to the Equivalence Constraint. Additionally,  among Equivalence points, CS is more likely at prosodic unit boundaries than within them. The prosodic constraint is thus independent of Equivalence. \n\nLastly, we want to thank the reviewer for the stylistic suggestions as well as catching typos, which have been altered. We will refer to the CS visualizations as 'language distribution graphs' instead of heatmaps. We hope this distinguishes them from the usual heatmaps.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper analyzes its topic in a thorough, solid, and linguistically well-founded way. It also provides a good literature overview of code switching. It addresses the widespread phenomenon of code-switching, which is rarely addressed or considered in the field of NLP, and lays a solid foundation for dealing with it in a theoretically informed way.", "reasons_to_reject": "The paper is limited to corpus analysis and it does not show or discuss how to apply its findings to NLP.\n\nThe paper analyzes the corpus described in the monograph by Torres Cacoullos and Travis (2018). Unfortunately, I do not have access to the full book, but based on reviews, it appears that at least some of the observations made in this paper have already been described or at least anticipated in the original book. For example, the original corpus is segmented into prosodic/intonational units, and a distinction is apparently made between single-word and multi-word code-switching.", "questions_to_authors": "1) I do not have access to the book and corpus by Torres Cacoullos and Travis (2018). Could you try to summarize what exactly is new in your paper compared to the findings in the book? The original corpus is segmented into prosodic units and distinguishes between single-word and multi-word code-switching. Assuming that your finding about the relevance of prosodic units is new, what was the reason for annotating them in the original corpus?\n\n2) You show that, although less common, code-switching can also occur within prosodic boundaries. Could it be that the preference for code-switching at prosodic boundaries is simply caused by the Equivalence Constraint (because word order compatibility is highest at the boundary between two prosodic units)?\n", "paper_topic_and_main_contributions": "The present paper analyzes code-switching patterns in the New Mexico Spanish-English Bilingual corpus by Torres Cacoullos and Travis (2018). The corpus is analyzed using two slightly adjusted code-switching metric that measure multilinguality (What is the balance between the languages used?) and code-switching probability (How often does code switching occur?).\n\nThe main contribution of the paper is a thorough discussion of the phenomenon of code switching and the observation that multi-word code switching typically occurs at the boundary between two prosodic units rather than within a single prosodic unit, i.e., the two languages are typically prosodically separated. On the other hand, the incorporation of single words (typically nouns) of one language into a sentence of the other language is shown to be subject to known syntactic but not prosodic constraints. Therefore, the paper argues for distinguishing between single-word incorporations and multi-word code switching, and and for using prosodic units as tokens for analyzing multilinguality and code-switching probability, rather than word tokens.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "12: \"stings\" > \"strings\"\n\nTable 1/Table 2: It would increase readability if you aligned the columns of the two tables.\n\nFigure 7: Is this type of visualization really called a heatmap? Typical heatmaps include more than two function values. (The visualizations themselves are great)\n\nYou abbreviate the two languages as E and S. You might consider using the standard abbreviations EN and ES instead, since they are well known and understandable without context."}}, {"review_id": "dXeGdPzBhY", "reviewer": "Reviewer_AZWa", "summary": "", "questions": "Why was the information of Figure 1 put that way, and not as a table?  Moreover, it seems to me that only \",\" and \".\" are relevant. \nLines 154-155: \"Lone items tend to be English incorporations into Spanish.\" Line 162 \"In contrast, multi-word CS is bidirectional.\" Why are these discrepancies observed? \nFrom line 176, a Pearson correlation is mentioned. How was it calculated (i.e., a function in Python)? \nMost of the related methods section argues that Twitter data is not adequate for CS analysis. Why was such big emphasis put on Twitter?\nLines 453-455: \"Metrics for predicting language distributions and CS probability are most appropriate when they cor- respond to bilingual speech on the ground.\"\nWhat does \"on the ground\" mean here?\nWhy do the tables focus only on the I-Index and not the M-Index?\nWhy were the heatmaps of only 3 speakers shown? It might be interesting to have the heatmaps of the other ones in the appendix.", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate Reviewer AZWa’s thoughtful engagement with our work, and for agreeing with Reviewer VWCM that our literature review is thorough and our methodology sound. We would like to point out, however, that Reviewer AZWa’s reasons to reject are either based on misunderstanding our results, or grammatical preferences that we are confident we can modify for improved readability. Additionally, they raise a valid ethical concern about speaker confidentiality, which we will also respond to. Our rebuttal will be formatted in the order of addressing misunderstandings, ethical concerns and reproducibility, manuscript clarifications, and formatting suggestions.\n\n_“Could that not indicate that the IU is not a relevant unit, or that the boundaries of the corpus is not reliable?”_\nNo. Recall that our findings validate that switching happens more often at the IU boundary versus within the IU (lines 464-469). This means, then, that having information about where the IU boundaries lie is critical for improving predictability of a switchpoint (instead of assuming that a switch could happen with the same probability between any two given words).\n\n_“Was the IU transcription checked by the authors?”_\nWe have no reason to question the transcription of this corpus. The corpus compilers report that the transcriptions went through at least five rounds;  each hour of recorded speech took at least 50 hours for transcription  (Torres Cacoullos and Travis, 2018: 47).\n\n_“Name and professions of some speakers were revealed (see \"Justification For Ethical Concerns\"). If these were their real names, they could be identified.”_\nWe will add a statement that all speakers' names are pseudonyms in this corpus. There was also further anonymization within each transcript of names and nicknames of people mentioned as well as of locations (e.g., smaller towns, places of work, high school names, etc.) that could be identifiers (Torres Cacoullos and Travis, 2018:48). In short, our main motivation in including the speakers’ pseudonyms and basic demographic information is to remind readers that these transcripts are from real people representative of a well-defined bilingual community, and thus not limited to urban, well-educated, upper-class professionals or elite bilinguals. For a full justification as to why we would like to keep speakers’ pseudonyms, please see our rebuttal to the meta-ethics reviewer.\n\n\nWe disagree with the reproducibility score given by Reviewer AZWa. The corpus has not been published due to the original ethical commitment to the speakers, given the intimate nature of the recordings by in-group interviewers in these close-knit communities. In addition, adopting protocols for access protects communities with a highly-stigmatized minority language variety from unintentional misinterpretation of local vernaculars and from publication of stereotyping examples (Torres Cacoullos and Travis, 2018:48). Penalizing the paper because the corpus is not posted on the web only further incentivizes the mass collection of web data, which is not fully representative of bilingual communities (see paragraph on ethics justification above). We plan to make the code completely open source through Github upon acceptance into the conference. \n\n\n_\"Lines 154-155: \"Lone items tend to be English incorporations into Spanish.\" Line 162 \"In contrast, multi-word CS is bidirectional.\" Why are these discrepancies observed?\"_\nThe point here is simply to illustrate how the two mixing types differ in language distributions (lines 150-153). The asymmetry of lone items, which here tend to be English incorporations into Spanish, has been reported as a community-wide preference, and has been observed in other communities (Poplack, 2017). The bidirectionality of CS may be interpreted as an indicator of bilingual ability (Torres Cacoullos & Travis, 2018:67-72; Poplack, 1980). \n\n\n_\"From line 176, a Pearson correlation is mentioned. How was it calculated (i.e., a function in Python)?\"_\nCalculation was with the Pearson Correlation Coefficient Calculator, on the online Social Science Statistics website.\n\n\n_\"Most of the related methods section argues that Twitter data is not adequate for CS analysis. Why was such big emphasis put on Twitter?\"_ \nTo our knowledge, the Linguistic Code-Switching Evaluation (LinCE) benchmark is currently the only benchmark for downstream CS tasks (Aguilar et al., 2020). For three of the four tasks (which we refer to in lines 197-203), the datasets come from CS examples found on Twitter. \nThe attraction of Twitter is its accessibility, but we hope to have made clear the potential advances to be made by working with speech data, which is the natural habitat of CS. Note, too, that the metrics we adapt were originally implemented on continuous streams of text (i.e., oral transcripts and written published books) (Guzmán et al., 2017: 70), and are therefore not suited for application to discontinuous and smaller spans of text like Twitter.\n\n\n_\"Lines 453-455: \"Metrics for predicting language distributions and CS probability are most appropriate when they correspond to bilingual speech on the ground.\" What does \"on the ground\" mean here? \"_\nHere, we mean grounded in the patterns of real bilingual speech, observed in everyday conversations within the community. We will be sure to clarify this point after line 455.\n\n\n_\"Why was the information of Figure 1 put that way, and not as a table? Moreover, it seems to me that only \",\" and \".\" are relevant.\"_\nWe can modify the presentation of the transcription rules to make it more readable as a table. We include the transcription rules for transitional continuity (“.” “,” “--” “?”) because these demarcate IUs, as seen in the examples throughout the paper. We include the transcription rules for pausing (...), because pausing is also a form of prosodic separation between languages.  We are open to adding the modified table to the appendix.\n\n\n_\"Why do the tables focus only on the I-Index and not the M-Index?\"_ \nThe M-Index is mentioned in the second column of Table 3. We will modify the formatting of this table to better distinguish between the two metrics.\n\n\n_\"Why were the heatmaps of only 3 speakers shown? It might be interesting to have the heatmaps of the other ones in the appendix.\"_\nWe ultimately chose these of the 5 due to space limitations as well as overall readability, but we can include the other 2 visualizations in the appendix. \n\n\n_“In figure 2, there is a problem with the pitch contour of \"nunca\"”_\nThe pitch contour was produced in Praat. A dip as in “nunca” may be capturing nonlinguistic information, as when a speaker’s voice gave out if they were speaking quietly.\n\n\n_\"Line 257: the word denigrate can be considered to have a racist etymology, so I would recommend using another word, such a desuse.\"_\nWe will replace it with disparage.\n\n\n_“I assume that the numbers after the sentences at the end of page 4 represent a speaker number and a time stamp. It would be nice to have it clear in the paper.”_\nYes, we will clarify this.\n\n\nLastly, we will work to improve the readability of the sentences deemed too long. We want to thank Reviewer AZWa again for their thorough comments.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper offers relevant contributions to several areas of linguistics, such as to the discussion on linguistic representations (in this case, the IU), and also to issues related to bilingualism (in this case, on quantitative analysis of bilingualism). The lit review is comprehensive and the paper is methodologically sound. The explanations on the metrics are also very clear.", "reasons_to_reject": "The main result of this paper is that speakers are more likely to perform CS across IU and not within them. Nonetheless, the authors argue about the relevance of the IU. Could that not indicate that the IU is not a relevant unit, or that the boundaries of the corpus is not reliable? Lines 268-273 state: \"The recordings are transcribed orthographically and prosodically in IUs (Du Bois et al., 1993:47). IUs are reliably identified by their acoustic correlates, which include a rising or high starting pitch that falls as the IU progresses, faster speech at the start of the IU as compared to the end, and, often, pauses between IUs\". Was the IU transcription checked by the authors?\nThere are also some sentences that are too long and could be reworded (see \"Typos Grammar Style And Presentation Improvements\" for more details). Name and professions of some speakers were revealed (see \"Justification For Ethical Concerns\"). If these were their real names, they could be identified. ", "questions_to_authors": "Why was the information of Figure 1 put that way, and not as a table?  Moreover, it seems to me that only \",\" and \".\" are relevant. \nLines 154-155: \"Lone items tend to be English incorporations into Spanish.\" Line 162 \"In contrast, multi-word CS is bidirectional.\" Why are these discrepancies observed? \nFrom line 176, a Pearson correlation is mentioned. How was it calculated (i.e., a function in Python)? \nMost of the related methods section argues that Twitter data is not adequate for CS analysis. Why was such big emphasis put on Twitter?\nLines 453-455: \"Metrics for predicting language distributions and CS probability are most appropriate when they cor- respond to bilingual speech on the ground.\"\nWhat does \"on the ground\" mean here?\nWhy do the tables focus only on the I-Index and not the M-Index?\nWhy were the heatmaps of only 3 speakers shown? It might be interesting to have the heatmaps of the other ones in the appendix.", "paper_topic_and_main_contributions": "This paper analyzes the role of IU Spanish and English CS using two metrics. One of them is M-Index, which measures how multilingual a corpus is. The other one is I-Index, which measures the probability of CS in each transcript. I-Indexes were further divided into two: within-IU index, and across-IU index. Results show that CS tend to occur not within but across IU boundaries.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "Yes", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Lines 77-82: \"For example, in the Spanish-English language pair, whereas articles are placed before nouns in both languages, there is a word order conflict for attributive adjectives, which are positioned before the noun overwhelmingly in English but only sometimes so in Spanish.\"\nThis sentence is too long.\n\n87-88 (‘they allow the new school to wear’)\nSometimes the authors use single quotes and sometimes double quotes. It would be nice to have them unified. \n\n110-115: \"Thus, for CS, corresponding to the syntactic Equivalence Constraint, which requires local equiv- alence of word order in the two languages around a switch point, is the IU-boundary Constraint, which captures bilinguals’ tendency to prosodically sep- arate the two languages.\"\nThis sentence is too long.\n\n194-204 \"Recently, the LinCE benchmark compiled by Aguilar et al. 198 (2020) provides different CS datasets for Spanish- English, Hindi-English, and other language pairs 200 for the tasks of Language Identification (LID), Part of Speech Tagging (POS), Named Entity Recognition (NER), and Sentiment Analysis (SA).\" \nMaybe the abbreviations are not necessary, since they appear only once. \n\nLine 226: miltilinguality should be multilinguality \n\nIn figure 2, there is a problem with the pitch contour of \"nunca\" \n\nLine 257: the word denigrate can be considered to have a racist etymology, so I would recommend using another word, such a desuse.\n\nThe authors use capital letters after colons (such as in line 289). I would recommend using lower case. \n\nI assume that the numbers after the sentences at the end of page 4 represent a speaker number and a time stamp. It would be nice to have it clear in the paper.  \n\nLine 356: th (and other occurrences of it) would better in superscript. \n\n", "Justification_for_Ethical_Concerns": "Lines 297-301: These are 03 (Sandra, administrator, Española), 05 (Rocío, teacher aid, Santa Fe), 10 (Pedro, school administrator, Taos), 16 (Manuel, electrician and rancher, Rio Arriba), and 27 (Eduardo, general contractor and store owner, Rio Arriba)\" \nIf these are real names and professions, the identity of these people can potentially be disclosed. "}}]

Author rebuttal:

Meta-review: The reviewers appreciated that the paper applies empirical data analysis methods to investigate linguistic phenomena, resulting better insights into the underlying linguistic phenomena (which could in turn be used to improve modeling of code-switching in data). The reviewers also note that the paper does an excellent job of introducing all of the relevant concepts, making the paper both clear and self-contained, even for a general NLP audience.
Decision: Accept-Main