Question: What are the three main data preparation tasks that are needed before prototype development could commence?
Evidence: In Yugoslavia, organized effort in machine translation started in 1959, but the first individual effort was made by Vladimir Matković from the Institute for Telecommunications in Zagreb in 1957 in his PhD thesis on entropy in the Croatian language BIBREF10. The main research group in machine translation was formed in 1958, at the Circle for Young Linguists in Zagreb, initiated by a young linguist Bulcsu Laszlo, who graduated in Russian language, Southern Slavic languages and English language and literature at the University of Zagreb in 1952. The majority of the group members came from different departments of the Faculty of Humanities and Social Sciences of the University of Zagreb, with several individuals from other institutions. The members from the Faculty of Humanities and Social Sciences were: Svetozar Petrović (Department of Comparative Literature), Stjepan Babić (Department of Serbo-Croatian Language and Literature), Krunoslav Pranjić (Department of Serbo-Croatian Language and Literature), Željko Bujas (Department of English Language and Literature), Malik Mulić (Department of Russian Language and Literature) and Bulcsu Laszlo (Department of Comparative Slavistics). The members of the research group from outside the Faculty of Humanities and Social Sciences were: Božidar Finka (Institute for Language of the Yugoslav Academy of Sciences and Arts), Vladimir Vranić (Center for Numerical Research of the Yugoslav Academy of Sciences and Arts), Vladimir Matković (Institute for Telecommunications), Vladimir Muljević (Institute for Regulatory and Signal Devices) BIBREF10.
Laszlo and Petrović BIBREF11 also commented on the state of the art of the time, noting the USA prototype efforts from 1954 and the publication of a collection of research papers in 1955 as well as the USSR efforts starting from 1955 and the UK prototype from 1956. They do not detail or cite the articles they mention. However, the fact that they referred to them in a text published in 1959 (probably prepared for publishing in 1958, based on BIBREF11, where Laszlo and Petrović described that the group had started its work in 1958) leads us to the conclusion that the poorly funded Croatian research was lagging only a couple of years behind the research of the superpowers (which invested heavily in this effort). Another interesting moment, which they delineated in BIBREF11, is that the group soon discovered that some experimental work had already been done in 1957 at the Institute of Telecommunications (today a part of the Faculty of Electrical Engineering and Computing at the University of Zagreb) by Vladimir Matković. Because of this, they decided to include him in the research group of the Faculty of Humanities and Social Sciences at the University of Zagreb. The work done by Matković was documented in his doctoral dissertation but remained unpublished until 1959.
The Russian machine translation pioneer Andreev expressed hope that the Yugoslav (Croatian) research group could create a prototype, but sadly, due to the lack of federal funding, this never happened BIBREF10. Unlike their colleagues in the USA and the USSR, Laszlo’s group had to manage without an actual computer (which is painfully obvious in BIBREF12), and the results remained mainly theoretical. Appealing probably to the political circles of the time, Laszlo and Petrović note that, although it sounds strange, research in computational linguistics is mainly a top-priority military effort in other countries BIBREF11. There is a quote from BIBREF10 which perhaps best delineates the optimism and energy that the researchers in Zagreb had:
"[...] The process of translation has to mechanicalized as soon as possible, and this is only possible if a competent, fast and inexhaustible machine which could inherit the translation task is created, even if just schematic. The machine needs to think for us. If machines help humans in physical tasks, why would they not help them in mental tasks with their mechanical memory and automated logic" (p. 118).
Finka and Laszlo envisioned three main data preparation tasks that are needed before prototype development could commence [7]. The first task is to compile a dictionary of words sorted from the end of the word to the beginning. This would enable the development of what is now called stemming and lemmatization modules: a knowledge base with suffixes so they can be trimmed, but also a systematic way to find the base of the word (lemmatization) (p. 121). The second task would be to make a word frequency table. This would enable focusing on a few thousand most frequent words and dropping the rest. This is currently a good industrial practice for building efficient natural language processing systems, and in 1962, it was a computational necessity. The last task was to create a good thesaurus, but such a thesaurus where every data point has a ”meaning” as the key, and words (synonyms) as values. The prototype would then operate on these meanings when they become substituted for words.
Answer: The first task is to compile a dictionary of words sorted from the end of the word to the beginning. The second task would be to make a word frequency table. The last task was to create a good thesaurus.