Buckwalter transliteration

The Buckwalter Arabic transliteration was  developed as part of the ALPNET Arabic Project being run by Ken Beesley in 1988.

Start
The first Arabic language analyst for the project was a BYU undergraduate student named Derek Foxley, hired as part-time. Foxley was in 4th year Arabic courses at the time at BYU.
Tim Buckwalter was employed several months later as a full-time employee of ALPNET. Buckwalter was also a PhD candidate in Arabic at the time. One of his tasks on the project was to collaborate with and assign Arabic language tasks to the part-time employee, Foxley.

Beesley mentored Buckwalter and Foxley in some of the finer details of linguistics, and one day at the whiteboard Beesley prodded Foxley and Buckwalter to come up with a transliteration schema at that moment. Foxley had been entering most of the data at that point in the project so was ready to address this. Nevertheless, in close collaboration with Buckwalter he came up with nearly all the characters used for the transliteration table. Buckwalter oversaw Foxley's Arabic tasks and made the final adjustments and refinements the transliteration table. It had no name at the time, however, Buckwalter in a few years following the project had entered thousands of textual items using the transliteration schema and presented it, and championed it, many times as well. It was, therefore, named after him.

At the time, no such one-for-one letter transliteration was in use, or at least none that the team was aware of.

Beesley later moved to Xerox, who bought the rights to the ALPNET data in the 1990s. This is documented in several other articles that Beesley has presented over the years.

Commentary on the system 
The Buckwalter Transliteration is an ASCII-only transliteration scheme, representing Arabic orthography strictly one-to-one, unlike the more common romanization schemes that add morphological information not expressed in Arabic script. Thus, for example, a  will be transliterated as w regardless of whether it is realized as a vowel  or a consonant . Only when the  is modified by a  () does the transliteration change to &.  This allows the user to type or convert text exactly as it is seen.

However, there has been some critique of the transliteration schema. Some users state that the unmodified letters are straightforward to read (except for *=dhaal and E=ayin, v=thaa), but the transliterations of letters with diacritics and the harakat take some time to get used to, for example the nunated  appear as N, F, K, and the  ("no vowel") as o.   is p.  The difficulty probably has happened because usually the Buckwalter transliteration is used and/or presented without the rationale behind the letters. Though those particular letters seem to be random they are actually mnemonically linked to the original letter.

Furthermore, since the original Buckwalter scheme was developed, several other variants have emerged, although they are not all standardized. Buckwalter transliteration is not compatible with XML, so "XML safe" versions often modify the following characters: < > & (أ إ and ؤ respectively; Buckwalter suggests transliterating them as I O W, respectively). Completely "safe" transliteration schemes replace all non-alphanumeric characters (such as $';*) with alphanumeric characters.

When transliterating Arabic text, several other issues may arise. First, some Arabic characters are not specified in the transliteration table, including non-alphabetic characters such as ۞ and ۝, punctuation such as ؛ ؟, and Eastern Arabic numerals. Similarly, sometimes Arabic sentences will borrow non-Arabic letters from Persian, some of which are defined in the full Buckwalter table. Symbols that are not defined in the transliteration table may be deleted, kept as non-Latin symbols embedded in transliterated text, or transliterated into different (non-conflicting) Latin symbols. (For instance, it is straightforward to convert from Hindi numerals to Arabic numerals.) Another issue that arises is how to handle transliterating Arabic text with embedded ASCII text; for instance, an Arabic sentence that refers to "IBM" or an Arabic sentence that includes a quote in English. If the Latin text is not explicitly marked, it is a challenge to distinguish transliterated Arabic from Latin. If transliterated text with embedded Latin is later transliterated back to Arabic, the Latin text will be transliterated into garbage Arabic. Finally, another important decision to make is how much normalization of the Arabic text should be done during transliteration. This may include removing kashida, removing short vowels and/or other diacritics, and/or normalizing spelling.

On the other hand, all typical markings one would expect to use when writing - !@#%?.,;:()[]+= were not used because they are also used in Arabic text. Thus, if the English IBM did appear in English, in the Arabic text it was in the original concept supposed be marked by putting double quotes around it: ""IBM"". This mechanism allows for automatic language processing to take place leaving non-Arabic text as is, unprocessed when it sees the double quotes. Originally, even < > & were not used either especially < > which are French borrowed quote marks because they are occasionally used in Arabic text. These were added later as a necessity.  Their XML safe versions keep with the mnemonic device devised (and discussed below) in that I O W correspond (if imprecisely) to each of the sounds made.

Key concepts in development of the table 
There were three key concepts used the transliteration schema:

The first was that each Arabic letter (sound) can only correspond to one English language character. Some Arabic letters produce a sound that corresponds to 2 English letters when written. Therefore, a single letter or common symbol would have to be used for them.

The second concept was to use the familiar if possible. If an Arabic letter had always been associated with the letter “s” in English, for example, then it would be easier to remember if it could be kept that way.

The third key concept was that the table had to be fully, easily mnemonic. Therefore, every single item correlates in the following order of preference a) to the sound of the Arabic letter, or b) to a physical aspect of the original Arabic letter or, c) to the name it is called.

Mechanics

Lower case letters were used in preference. However, when there are multiple Arabic letters that have a similar sounds then for more open sounds the lower case letter was used and for more close/restricted sounds an upper case letter was used. For example, in Arabic there are 2 letters similar to the  in English sound. The plain sound was given a small “d” and the emphatic sound  was assigned an upper case “D”.

In other words, an upper case letter indicates that the letter is similar to a lower case letter – but has a qualitative difference in some way.

Buckwalter transliteration table

 
hamza
lone hamza: '
hamza on alif: >
hamza below alif: <
hamza on wa: &
hamza on ya: }
alif
madda on alif: |
alif al-wasla: {
dagger alif: `
alif maqsura: Y
harakat
fatha: a
damma: u
kasra: i
fathatayn: F
dammatayn: N
kasratayn K
shadda: ~
sukun: o
ta marbouta: p
tatwil: _

Mnemonics 

The original ALPNET team quickly adopted this schema. Even though Beesley had no background in Arabic he was quickly able to understand and use it. The strength of the Buckwalter transliteration is that every single Arabic letter is represented distinctly. Yet, its reliance on traditional transliterations or mnemonic devices for anything non-traditional makes it very easy to learn.

Sample 
The first article of The Universal Declaration of Human Rights:

Arabic text 
يُولَدُ جَمِيعُ ٱلنَّاسِ أَحْرَارًا مُتَسَاوِينَ فِي ٱلْكَرَامَةِ وَٱلْحُقُوقِ. وَقَدْ وُهِبُوا عَقْلًا وَضَمِيرًا وَعَلَيْهِمْ أَنْ يُعَامِلَ بَعْضُهُمْ بَعْضًا بِرُوحِ ٱلْإِخَاءِ.

Buckwalter transliteration 
yuwladu jamiyEu {ln~aAsi >aHoraArFA mutasaAwiyna fiy {lokaraAmapi wa{loHuquwqi. waqado wuhibuwA EaqolFA waDamiyrFA waEalayohimo >ano yuEaAmila baEoDuhumo baEoDFA biruwHi {lo<ixaA'i.

DIN 31635 
Yūladu ǧamīʿu n-nāsi ʾaḥrāran mutasāwīna fī l-karāmati wa-l-ḥuqūq. Wa-qad wuhibū ʿaqlan wa-ḍamīran wa-ʿalayhim ʾan yuʿāmila baʿḍuhum baʿḍan bi-rūḥi l-ʾiḫāʾi.

English text 
All human beings are born free and equal in dignity and rights. They are endowed with reason and conscience and should act towards one another in a spirit of brotherhood.

Notes

External links
Information about the transliteration
Transliteration on XRCE site
Transliteration on Tim Buckwalter's site
Extended Buckwalter Transliteration (for the Quran)

Romanization of Arabic