The Icelandic Confusion Set Corpus (ICoSC) 2.0 (2020-05-06)

The Icelandic Confusion Set Corpus (ICoSC) is available under a CC-BY licence. It was compiled during the course of 8 months by Steinunn Rut Friðriksdottir and Anton Karl Ingason of the language technology department in the University of Iceland. Included in the ICoSC are CSV spreadsheets containing all collected confusion sets of each phonetic category and their frequencies. The spreadsheets are organized so that for each set, the total frequency of each candidate is calculated along with the frequency of each possible PoS tag for that candidate. The seventh and eight column of the tables contain binary values referring to whether the confusion set is grammatically disjoint (all PoS tags differ for the two candidates) or grammatically identical (all PoS tags are identical for the two candidates). The final column shows the frequency of the less frequent candidate of the set which can be used to determine which sets are viable in an experiment. Also included are text files containing the list of words from each category and text files containing all sentence examples from the Icelandic Gigaword Corpus which contain the words for each category. As the n/nn examples are by far the most frequent confusion sets, the corpus also includes a word list and sentence examples for the 55 most frequent sets. There are also spreadsheets containing all of the collected word pairs which are grammatically identical, grammatically disjoint or neither of the aforementioned categories. All files have UTF-8 encoding. The ICoSC consists of the following categories of confusion sets, selected for their linguistic properties as homophones, separated orthographically by a single letter. The categories are:•197 pairs containing y/i (leyti ’extent’ / leiti ’search’)•150 pairs containing ý/i (sýn ’vision’ / sin ’theirs (possessive reflexive)’)•1203 pairs containing nn/n (forvitinn ’curious(masc.)’ / forvitin ’curious (fem.)’)•8 pairs commonly confused by Icelandic speakers (mig 'me' (accusative) / mer 'me' (dative))•24 pairs containing hv/kv (hvað ’what’ / kvað ’chanted’)•42 pairs containing rð/ðr (veðri ’weather’ / verði ’will become’)•110 pairs containing rr/r (klarri ’smart’ / klari ’smart’)•25 pairs containing f/ff (kafi 'underwater' / kaffi 'coffee')•87 pairs containing g/gg (vegi 'roads' / veggi 'walls')•113 pairs containing s/ss (kysi 'would choose' / kyssi 'kiss')•161 pairs containing k/kk (taka 'to take' / takka 'button')•34 pairs containing p/pp (tapa 'loose' / tappa 'cap/plug')•456 pairs containing l/ll (vel 'well' / vell 'gush')•38 pairs containing m/mm (ama 'bother' / amma 'grandmother')•401 pairs containing t/tt (aettu 'should' / aetu 'would eat')•261 pairs containing a/a (lata 'put' / lata 'lazy')•92 pairs containing e/e (lek 'played' / lek 'leaks')•290 pairs containing i/i (sin 'theirs (possesive)' / sin 'tendon')•189 pairs containing o/o (goðum 'good' / goðum '(heathen) gods')•100 pairs containing u/u (skutum 'sailboats' / skutum 'shot')•582 pairs containing g/k (leika 'to play' / leiga 'rent')•62 pairs containing gg/kk (bakka 'go backwards' / bagga 'bale')•162 pairs containing p/b (best 'best' / pest 'illness')•19 pairs containing pp/bb (pabbi 'dad' / pappi 'cardboard')•432 pairs containing t/d (myndir 'pictures' / myntir 'coins/currencies')•115 pairs containing tt/dd (faedd 'born (feminine)' / faett 'born (neutral)')•314 pairs containing v/f (fara 'leave' / vara 'lips')