Helpful Neighbors: Leveraging Neighbors in Geographic Feature Pronunciation (2023)

Abstract: If one sees the place name Houston Mercer Dog Run in New York, how does one know how to pronounce it? Assuming one knows that Houston in New York is pronounced /ˈhaʊstən/ and not like the Texas city (/ˈhjuːstən/), then one can probably guess that /ˈhaʊstən/ is also used in the name of the dog park. We present a novel architecture that learns to use the pronunciations of neighboring names in order to guess the pronunciation of a given target feature. Applied to Japanese place names, we demonstrate the utility of the model to finding and proposing corrections for errors in Google Maps. To demonstrate the utility of this approach to structurally similar problems, we also report on an application to a totally different task: Cognate reflex prediction in comparative historical linguistics. A version of the code has been open-sourced.1

## introduction in many parts of the world, pronunciation of toponyms and establishments can require local knowledge. many visitors to new york, for example, get tripped up by houston street, which they assume is pronounced the same as the city in texas. if they do not know how to pronounce houston street, they would likely also not know how to pronounce the nearby houston mercer dog run. but if some one knows one, that can (usually) be used as a clue to how to pronounce the other. before we proceed further, a bit of terminology. technically, the term toponym refers to the name of a geographical or administrative feature, such as a river, lake, town, or state. in most of what follows, we will use the term feature to refer to these and other entities such as roads, buildings, schools, and so forth. in practice we will not make a major distinction between the two, but since there is a sense in which toponyms are more basic, and the names of the more general features are often derived from a toponym (as in the houston mercer dog run example above), we will retain the distinction where it is needed. while features cause not infrequent problems in the us, they become a truly serious issue in japan. japan is notorious for having toponyms whose pronunciation is so unexpected that even native speakers may not know how to pronounce a given case. most toponyms in japanese are written in kanji (chinese characters) with a possible intermixing of one of the two syllabaries, hiragana or katakana. thus ueno is entirely in kanji; tora no mon has two kanji and one katakana symbol (the second); and fukiwari waterfalls has three kanji and one hiragana symbol (the third). features more generally tend to have more characters in one of the syllabaries-especially katakana if, for example, the feature is a building that includes the name of a company as part of its name. the syllabaries are basically phonemic scripts so there is generally no ambiguity in how to pronounce those portions of names, but kanji present a serious problem in that the pronunciation of a kanji string in a toponym is frequently something one just has to know. to take the example ueno above, that pronunciation (for the well-known area in tokyo) is indeed the most common one, but there are places in japan with the same spelling but with pronunciations such as uwano, kamino, wano, among others. 2 it is well-known that many kanji have both a native (kun) japanese pronunciation (e.g., yama 'mountain') as well as one or more chinese-derived on pronunciations (e.g., san 'mountain'), but the issue with toponyms goes well beyond this since there are nanori pronunciations of kanji that are only found in names @xcite : since the variation is largely unpredictable, one therefore simply needs to know for a given toponym what the pronunciation is. but once one knows, for instance, that a name written is read as uwano, as with the houston case, one ought to be able to deduce that in the name of the local 'uwano first public park', this is read as uwano and not ueno. if one's digital assistant is reading this name to you, or needs to understand your pronunciation of the name, it needs to know the correct pronunciation. while one might expect a complete and correct maps database to have all of this information correctly entered, in practice maps data contain many errors, especially for less frequently accessed features. in this paper we propose a model that learns to use information from the geographical context to guide the pronunciation of features. we demonstrate its application to detecting and correcting errors in google maps. in addition, in section 8 we show that the model can be applied to a different but structurally similar problem, namely, the problem of cognate reflex prediction in comparative historical linguistics. in this case the 'neighbors' are related word forms in a set of languages from a given language family, and the pronunciation to be predicted is the corresponding form in a language from the same family. the above approaches generally treat the problem in isolation in the sense that the problem is cast as one where the task is to predict a pronunciation independent of context. different pronunciations for the same string in different linguistic contexts comes under the rubric of homograph disambiguation, and there is a long tradition of work in this area; for an early example see @xcite and for a recent incarnation see @xcite . not surprisingly, there has been recent interest in neural models for predicting homograph pronunciations: see park and lee (2020) and @xcite for recent examples focused on mandarin. the present task is different, since what disambiguates the possible pronunciations of japanese features is not generally linguistic, but geographical context, which can be thought of as a way of biasing the decision as to which pronunciation to use, given evidence from the local context. our approach is similar in spirit to that of pundak et al. (2018), who propose the use of a bias-encoder in a ''listen-attend-and-spell'' @xcite automatic speech recognition architecture. the bias encoder takes a set of ''bias phrases'', which can be used to guide the model towards a particular decoding. @xcite model is shown schematically in figure 1 . ## data features in google maps are stored in a data representation that includes a variety of information about each feature including: its location as a bounding box in latitude-longitude; the type of the feature-street, building, municipality, topographic feature, and so on; name(s) of the feature in the native language as well as in many (mostly automatically generated) transliterations; an address if there is an address associated with this feature; road signs that may be associated; and so forth. each feature is identified with a unique hexadecimal feature id. features may have additional names besides the primary names. for example in english, street names are often abbreviated (main st.) and these abbreviations are typically expanded (main street) as an additional name. many japanese features have pronunciations of the names added as additional names in katakana. some of these have been carefully hand curated, but many were generated automatically and are therefore potentially errorful, as we will see. since the katakana version is used as the basis for transliterations into other languages, lo-calized pronunciations for text-to-speech, as well as search suggestions, it is important that it be correct. we started by extracting from the database all features that include a broad (but not exhaustive) set of feature types from a bounding box that covers the four main islands of japan. we then extracted feature summaries for names that included both kanji original names, and katakana renditions. these summaries include the feature name, the hiragana version of the name converted from katakana, and the bounding box for the feature. we then find, for each feature in the feature summaries, a bucket of other features that are within a given radius (10 kilometers in our experiments). then, for each feature in each bucket, we designate that feature a target feature, and we build neighborhoods around that feature. we attempt for each feature, to find interesting neighboring features whose name shares a kanji bigram with the target feature's name. the intuition here is that a feature that is likely to be useful in determining the pronunciation of another feature should be nearby geographically, and should share at least some of the name. in any case we cap the number of 'non-interesting' neighbors to a limit-5 in our experiments. this means that some neighborhoods will have target features that lack useful neighbors; this is a realistic situation in that while it is often the case that one can find hints for a name's pronunciation in the immediate neighbors, it is not always the case. while such neighborhoods are not useful from the point of view of neighbor-based evidence for a target feature's pronunciation, they still provide useful data for training the target sequence-to-sequence model. our final dataset consists of about 2.7m feature neighborhoods, including the information from the summary for each target feature as described above, the associated neighboring features and their summaries, along with the distance (in kilometers) from the target feature. figure 2 shows parts of one such neighborhood. ## model despite the differences noted above, the problem we are interested in can still be characterized at its core as a sequence-to-sequence problem. the input is a sequence of tokens representing the feature name in its original japanese written form. the output is a sequence of hiragana characters representing the correct pronunciation. the difference between this and a more conventional sequence-to-sequence problem is that we provide additional biasing information in the form of geographical neighbors, such as their pronunciation and geographical location. this neighbor information is provided as additional input sequences to aid the model in making its prediction. in our experiments, we limit the number of neighbors to at most 30 (it is usually much less than this), each consisting of two sequences, namely, the neighbor's name and the corresponding pronunciation. ## experiments and evaluation the various configurations of the model, with and without neighbors, were trained on 2,397,154 neighborhoods, for 1 million steps. before reporting overall performance results, we illustrate the operation of the with-neighbors transformer model with an example that illustrates the model detecting cases where the data is incorrect. the feature, , mezon nipponbashi, is an apartment building in the nipponbashi district of osaka. the problem is that is also a part of tokyo, pronounced nihonbashi, and being more famous, is arguably the ''default'' pronunciation. the pronunciation of this feature was presumably originally populated by a method that did not take geographical context into account. in figure 4 we show the feature, the pronunciation as found in the database, the hypothesized (correct) pronunciation, and the neighbors that the model attended to when hypothesizing the feature's pronunciation. the example introduced in figure 2 is also correctly predicted by the model as seravi tanmachi. in the remainder of this section we present two types of evaluation. first we introduce a nonneural baseline (section 5.1). in section 5.2, we present error rates on held-out data for several versions of the model, the non-neural baseline, and a separate rnn model that has been used for more general text-normalization applications. we show that the with-neighbors transformer model has by far the best performance. in section 5.3 we delve a bit deeper into the effect of lat-long features, as well as details of the performance on the golden set. however, it is also important to show that the model is indeed learning to attend to relevant features in the neighborhood. we present evidence of this in section 5.4. in section 5.5 we discuss the important question: how often does a prediction error make the target name incomprehensible? moving beyond section 5, as noted earlier the maps data has errors, meaning that a small percentage of the cases where the hypothesis of the model differs from what is in the database, the database is in fact incorrect. the main practical application of the model is finding and correcting these sorts of errors. determining which discrepancies are errors and which are not requires human evaluation, and we report results on this in section 6. finally, since manual evaluation is expensive, we would like to be able to decide automatically when we can be confident that a discrepancy should be judged in the model's favor: this is the topic of section 7. in section 8, we demonstrate an application of the model to a totally different problem. ## finding mistakes in maps data an important application of the model is to find potential errors in the database, and flag them 5 apparently for l'osier asao. for possible human correction. to that end, using the with-neighbors transformer model trained with all features, we ran decoding on the entire data set, including the training and held-out portions, and identified cases where the model hypothesized a different pronunciation from what was in the reference transcription. in order to focus on the cases of interest, we further filtered these by considering only neighborhoods where some neighbors have spellings that share substrings with the target feature's spelling, and pronunciations that share substrings with the hypothesized pronunciation. this yielded a set of 18,898 neighborhoods that had some discrepancy per the model. especially for the training portion, it is likely that the model learned whatever pronunciation was in the database, even if it was wrong, so we are likely missing a lot of neighborhoods that have errors: we do not, therefore, know the recall of the system. in what follows, we consider the precision, based on a manual analysis by human raters. preliminary analysis of the output revealed that many of the discrepancies involved establishments, which include buildings and other man-made features including things like bus stops. these often contain a location name as part of the name. for example, a family mart convenience store might be named family mart kobashi station square store, with the issue being whether is correctly pronounced in the establishment name. three raters foot_7 manually checked pronunciations for 1,056 features, including 555 establishment features. raters were given links to the feature on google maps, and were asked to verify which pronunciation was correct, or give an alternative if neither was correct. evaluators had to provide 'proof' of their answers, of which the following were considered acceptable: (a) official website of the location, or the japan post website; (b) a screenshot from street view showing the pronunciation (e.g., from a road sign); (c) a wikipedia page with sufficient appropriate references. raters were asked not to use other sources. overall, the raters found that the model correctly detected that there was a potential problem with the reference data 63% of the time. the remaining 37% of the features were actually correct, despite the model having hypothesized a different pronunciation. the 63% of cases with problems broke down as follows: in 36% (absolute) of the cases, the hypothesized replacement pronunciation was correct, in 11% both were wrong (meaning that the model detected a problem, but found the wrong solution), and in 15% of the cases, the rater was unable to verify the answer (which suggests that the feature may need to be checked further). in some categories such as 'compound building', the hypothesized pronunciation was correct (and the reference pronunciation wrong), 80.7% of the time. figure 7 shows the results of a manual analysis of the 555 establishments by an independent rater. the establishments represent a range of ''impressions'', with some appearing frequently in searches, others less so. the rater found that for the majority of cases (55.6%), the data in maps was incorrect: 39.2% where the hypothesized alternative is the correct one; and 16.4% where both what is in the data and the hypothesis are wrong, but where the system has detected a problem with the data. the rater was unsure about a further 9.2%, constituting a further set that should be checked by an expert. (a small percentage of the establishments have closed since the database was created.) thus only about a third of the establishments selected were correct in the database. one point that will be clear from the above is that just because there is a discrepancy between the pronunciations of a target feature and the neighboring features does not mean that the target is wrong. indeed, there are systematic types of features that frequently involve such discrepancies. one such class being train stations, which are notoriously difficult in that they are frequently pronounced differently from the name of the town in which they are located @xcite . thus the station that serves kowakudani is kowakidani eki. station names were often established during the meiji period, and reflect older pronunciations for nearby toponyms. ## automatic data correction unfortunately the model is not yet accurate enough to use it to automatically fix discrepancies for all features. among the 1,056 manually analyzed features, the original data was correct 37% of the time, and the model 35%, meaning that simply substituting the model's hypothesis would result in a small net loss in accuracy. however, we also saw that the model was more accurate than the reference data for some classes of features, meaning on average the accuracy should increase if we replaced the data in those cases. we have also investigated filtering the data based on metrics extracted from the model itself. for example, we considered decoding entropy as a measure of confidence, the log likelihoods of the beam search outputs, and the relative amount of attention that the attention layers were giving the neighbor summary. thus far, the most informative measure is difference between the top two beam search decoding log likelihoods. our interpretation of this is that if there is a large difference in confidence between the two beams then there is little ambiguity in how the model thinks they should be pronounced and thus we can be more confident in the top candidate being correct. figure 8 shows the roc curve for using this metric to features from the golden set. it shows an area under the curve of almost 0.7, a clear positive signal, and the precision-recall curve shows that we are able to achieve an accuracy of about 80% for about 50% of the data, which still represents a large number of high confidence corrections. cognate reflex geo. name task target form ← main feature 1 pron. target lang. l 1 ← main feature 1 name related form ← neigh. feature 2 pron. related lang. l 2 ← neigh. feature 2 name. related form ← neigh. feature 3 pron. related lang. l 3 ← neigh. feature 3 name. related form ← neigh. feature 4 pron. related lang. l 4 ← neigh. feature 4 name. . . . . . . . . . . . . the two systems developed by @xcite achieved the top ranking in the shared task. in general, the better performing of the two was the inpainting model, but on some language families, such as semitic, the neighbors model outperformed the inpainting model. ## discussion in this paper we have presented a novel architecture for the difficult task of pronouncing japanese geographical features that learns to use pronunciations of features in a local neighborhood as hints. we have shown via various means that the model pays attention to the neighboring features, and that therefore the model has learned what we intended it to learn: that in order to pronounce a name, it is often useful to consider how neighbors are pronounced. we also conducted manual evaluations showing that for some classes of features, model hypotheses differing from pronunciations in the database could be as high as 80% correct. our results are currently being used to correct errors in google maps. in future work we also plan to extend the coverage of the model beyond japan. while japanese place names are particularly difficult, we noted in the introduction that there are similar problems in other regions. one problem that comes up in the united states, for example, is nonce abbreviations for certain features. for example if one looks in google maps in shreveport, la, one will run across the weirdly abbreviated sprt bkdl hwy srv dr. out of context this is virtually uninterpretable, but if one looks at nearby features one will find the shreveport barksdale hwy. from this and other information one can deduce that the mysteriously named feature must be the shreveport barksdale highway service drive. besides geographical names, there are other problems to which a similar approach can be applied. the neighbor model can be thought of as an auxiliary memory, to be consulted or not depending on the decision being made. we discussed one possible application of this conceptualization to the task of cognate reflex prediction in section 8. a further extension of the idea is joint correction of features in a neighborhood. if most of the pronunciations of in a neighborhood are nipponbashi, then one could consider correcting all cases where the pronunciation is listed as nihonbashi in the neighborhood, not just the main feature. note that this is somewhat similar in spirit to work on collective classification @xcite . finally, it is also worth noting that while our work has been with a proprietary maps database, there are open-source maps datasets such as openstreetmap @xcite , which likely have at least as many problematic issues as the database we used. the techniques we describe in this paper could be applied to improving such data.