In recent years , research in natural language processing has increasingly focused on normalizing SMS messages . Different well-defined approaches have been proposed , but the problem remains far from being solved : best systems achieve a 11 % Word Error Rate . This paper presents a method that shares similarities with both spell checking and machine translation approaches . The normalization part of the system is entirely based on models trained from a corpus . Evaluated in French by 10-fold-cross validation , the system achieves a 9.3 % Word Error Rate and a 0.83 BLEU score . Introduced a few years ago , Short Message Service ( SMS ) offers the possibility of exchanging written messages between mobile phones . SMS has quickly been adopted by users . These messages often greatly deviate from traditional spelling conventions . As shown by specialists ( Thurlow and Brown , 2003 ; Fairon et al . , 2006 ; Bieswanger , 2007 ) , this variability is due to the simultaneous use of numerous coding strategies , like phonetic plays ( 2m1 read ' demain ' , " tomorrow " ) , phonetic transcriptions ( kom instead of ' comme ' , " like " ) , consonant skeletons ( tjrs for ' toujours ' , " always " ) , misapplied , missing or incorrect separators ( j esper for ' j'espère ' , " I hope " ; j'croibi1k , instead of ' je crois bien que ' , " I am pretty sure that " ) , etc . These deviations are due to three main factors : the small number of characters allowed per text message by the service ( 140 bytes ) , the constraints of the small phones ' keypads and , last but not least , the fact that people mostly communicate between friends and relatives in an informal register . Whatever their causes , these deviations considerably hamper any standard natural language processing ( NLP ) system , which stumbles against so many Out-Of-Vocabulary words . For this reason , as noted by Sproat et al . ( 2001 ) , an SMS normalization must be performed before a more conventional NLP process can be applied . As defined by Yvon ( 2008 ) , " SMS normalization consists in rewriting an SMS text using a more conventional spelling , in order to make it more readable for a human or for a machine . " The SMS normalization we present here was developed in the general framework of an SMSto-speech synthesis system1 . This paper , however , only focuses on the normalization process . Evaluated in French , our method shares similarities with both spell checking and machine translation . The machine translation-like module of the system performs the true normalization task . It is entirely based on models learned from an SMS corpus and its transcription , aligned at the character-level in order to get parallel corpora . Two spell checking-like modules surround the normalization module . The first one detects unambiguous tokens , like URLs or phone numbers , to keep them out of the normalization . The second one , applied on the normalized parts only , identifies non-alphabetic sequences , like punctuations , and labels them with the corresponding token . This greatly helps the system 's print module to follow the basic rules of typography . This paper is organized as follows . Section 2 proposes an overview of the state of the art . Section 3 presents the general architecture of our system , while Section 4 focuses on how we learn and combine our normalization models . Section 5 evaluates the system and compares it to previous works . Section 6 draws conclusions and considers some future possible improvements of the method . In this paper , we presented an SMS normalization framework based on finite-state machines and developed in the context of an SMS-to-speech synthesis system . With the intention to avoid wrong modifications of special tokens and to handle word boundaries as easily as possible , we designed a method that shares similarities with both spell checking and machine translation . Our Evaluated by ten-fold cross-validation , the system seems efficient , and the performance in terms of BLEU score and WER are quite encouraging . However , the SER remains too high , which emphasizes the fact that the system needs several improvements . First of all , the model should take phonetic similarities into account , because SMS messages contain a lot of phonetic plays . The phonetic model , for instance , should know that o , au , eau , . . . , aux can all be pronounced [ o ] , while è , ais , ait , . . . , aient are often pronounced [ E ] . However , unlike Kobus et al . ( 2008a ) , we feel that this model must avoid the normalization step in which the graphemic sequence is converted into phonemes , because this conversion prevents the next steps from knowing which graphemes were in the initial sequence . Instead , we propose to learn phonetic similarities from a dictionary of words with phonemic transcriptions , and to build graphemes-to-graphemes rules . These rules could then be automatically weighted , by learning their frequencies from our aligned corpora . Furthermore , this model should be able to allow for timbre variation , like [ e]-[E ] , in order to allow similarities between graphemes frequently confused in French , like ai ( [ e ] ) and ais / ait / aient ( [ E ] ) . Last but not least , the graphemes-tographemes rules should be contextualized , in order to reduce the complexity of the model . It would also be interesting to test the impact of another lexical language model , learned on non-SMS sentences . Indeed , the lexical model must be learned from sequences of standard written forms , an obvious prerequisite that involves a major drawback when the corpus is made of SMS sentences : the corpus must first be transcribed , an expensive process that reduces the amount of data on which the model will be trained . For this reason , we propose to learn a lexical model from non-SMS sentences . However , the corpus of external sentences should still share two important features with the SMS language : it should mimic the oral language and be as spontaneous as possible . With this in mind , our intention is to gather sentences from Internet forums . But not just any forum , because often forums share another feature with the SMS language : their language is noisy . Thus , the idea is to choose a forum asking its members to pay attention to spelling mistakes and grammatical errors , and to avoid the use of the SMS language .

Challenge: Non-traditional-spelling conventions in SMS messages require a normalization process before an NLP process can be applied but the current best system performs poorly.
Approach: They propose a machine translation-like corpus-based model surrounded by two spell checking-like modules composed of unambiguous token detection and non-alphabetic sequence identification.
Outcome: The proposed method efficiently achieves a 9.3% Word Error Rate and 0.83 BLEU score in French by 10-fold-cross validation.