Venue: NAACL 2018

# The Computational Complexity of Distinctive Feature Minimization in Phonology

Hubie Chen

Computer Science and Information Systems

Birkbeck, University of London

hubie@dcs.bbk.ac.uk &Mans Hulden

Department of Linguistics

University of Colorado

mans.hulden@colorado.edu

###### Abstract

We analyze the complexity of the problem of determining whether a set of phonemes forms a natural class and, if so, that of finding the minimal feature specification for the class. A standard assumption in phonology is that finding a minimal feature specification is an automatic part of acquisition and generalization. We find that the natural class decision problem is tractable (i.e. is in \({\cal P}\)), while the minimization problem is not; the decision version of the problem which determines whether a natural class can be defined with \(k\) features or less is \({\cal NP}\)-complete. We also show that, empirically, a greedy algorithm for finding minimal feature specifications will sometimes fail, and thus cannot be assumed to be the basis for human performance in solving the problem.

## 1 Introduction

The **distinctive feature** is held by many phonologists, independently of theoretical orientation, to be the fundamental unit of analysis of sound patterns in language. The underlying working assumption of most phonological approaches is that a single sound or a set of sounds is expressed through a combination of positive or negative features and that these features are in some sense universal across languages Mielke (2008a). The exact makeup of the feature set employed has varied over time, ranging from the limited, more acoustic-oriented features of Jakobson et al. (1951), to the richer model presented in Chomsky and Halle (1968), to more complex hierarchically organized features in Clements (1985). The concept of **natural class** is intimately tied to such feature systems and is taken to be any set of segments that share some number of distinctive features. Furthermore, phonological alternations that do not target natural classes are hypothesized not to occur.

Another assumption that is found in phonological literature--often less explicit--is that whenever a phonological process targets a group of sounds, that group is to be expressed nonredundantly by the _minimum number of features_ required to do so. In general, one can find a multitude of ways in which a set of phonemes can be specified using positive or negative features. For example, using the feature system shown in Table 1, the set {m,n} has the obvious minimal description [+nas], since m and n are the only nasals. But that set could also be specified non-minimally as:

\[\left[\begin{array}{c}+\text{cons}\\ -\text{cnt}\\ -\text{hi}\\ -\text{bk}\\ +\text{voi}\\ +\text{son}\end{array}\right] \tag{1}\]

The potential complexity of this problem, finding a minimal specification, is not addressed in the phonological literature. Yet, finding such a minimal specification is not trivial. Using more realistic feature systems such as the ones given in Hayes (2011), there are, for example, 208 distinct solutions to specifying the set {e,i} in English, only four of which have the minimum length of 3 features. While such modern feature systems work with 25-30 features, one can actually assume more features are needed to cover more cross-linguistically exotic contrasts such as those produced by including click consonants Miller (2011). The P-base 3 resource Mielke (2008b) lists 398 features over 8 feature systems, covering 629 languages.

The general notion of **feature economy** has a long tradition in phonology Jakobson (1942); Martinet (1955); Clements (2003), mostly targeting entire feature systems in a language, i.e. advocating

that on the grammar level, individual languages make maximal use of the feature inventory available [21]. Equally prominent is the assumption that any specification of phonological alternations be made with the minimal number of features necessary: "one should use the minimum number of features required to specify all and only the sounds in the class." [35, p.282]. Hayes (2011), among others, argues, following Ockham's Razor, that this is how generalization can take place and that phonological hypotheses are made in precisely this way--witnessing alternations that target a set of sounds, learners find the minimal feature specification that is consistent with the alternating sounds, generalizing from there to other sounds that may enter the language. Halle (1962) also proposes a mechanism of "feature counting" as a methodology to rule out spurious generalizations one might propose--a process which implicitly includes the capability of feature minimization.

Similar arguments of feature minimization are used to perform an optimization of an entire phonological grammar. In Radical Underspecification, Archangeli (1984) refers to what is termed Feature Minimization Principle: "A paramar is most highly valued when underlying representations include the minimal number of features necessary to make the different phonemes of the language" [7, p. 48].

Given such claims concerning acquisition and phonological analysis, it is of some interest to assess the actual computational complexity of feature minimization. This entails answering how difficult it is in the worst case to determine whether a set of segments represents a natural class, and also how to find the minimal feature specification.

## 2 Overview

We will assume a set of phonemes \(P\)--the phoneme inventory--and another set \(Q\), our target set that we want to express through a combination of features, and a feature system \({\cal F}\) such as the one shown in Table 1. The first problem we address is that of determining whether a set of phonemes \(Q\) forms a _natural class_, which we call the _feature description problem_. We show that this is decidable in polynomial time. Further, we will show that a minimization version of the problem, which we call the _feature minimization problem_ is \({\cal NP}\)-complete [10, 10]. We will show this by reduction from the well known set covering problem [10].

Many phonologists espouse a combination of binary (equipollent) and privative (univalent) features [13, 14, 15]. The rationale is that, for example, the feature [\(\pm\)labial] has rarely, if ever, been found to play a role in a phonological system as [\(-\)labial], i.e. phonological processes that target non-labials seem to be absent. Hence, many phonologists favor the use of a single possibility [labial] in a specification that includes the labials. For this reason, we analyze separately both the complexity of using only such privative features (positive features only), which we call the _positive feature description problem_ (is \(Q\) a natural class if we only use positive features?) and the corresponding _positive feature minimization problem_.

## 3 Notation and terms

Throughout, when \(A\) is a set, we use \(\wp(A)\) to denote the power set of \(A\).

Relative to a set \(P\) of phonemes, we define a _feature system_ to be a subset \({\cal F}\subseteq\wp(P)\). When \(Q\subseteq P\), we define a _\({\cal F}\)-description of \(Q\)_ to be a sequence \(G_{1},\ldots,G_{m}\subseteq P\) such that there exist pairwise distinct elements \(F_{1},\ldots,F_{m}\in{\cal F}\) where:

\begin{table}
\begin{tabular}{l l 

* each \(G_{i}\) is equal to either \(F_{i}\) or \(P\setminus F_{i}\), and
* \(Q=G_{1}\cap\cdots\cap G_{m}\).

We refer to \(m\) as the _size_ of the description. We say that such a description is _positive_ if each \(G_{i}\) is an element of \(\mathcal{F}\).

We define the _feature description problem_ as follows. An instance consists of a set \(P\) of phonemes, a feature system \(\mathcal{F}\), and a non-empty subset \(Q\subseteq P\); the problem is to decide whether or not there exists a \(\mathcal{F}\)-description of \(Q\).

We define the _feature minimization problem_ as follows. An instance consists of a tuple \((P,\mathcal{F},Q,k)\) where \(P\), \(\mathcal{F}\), and \(Q\) are an instance of the feature description problem, and \(k\geq 1\) is a natural number. The problem is to decide whether or not there exists a \(\mathcal{F}\)-description of \(Q\) having size less than or equal to \(k\).

We define the _positive feature description problem_ to have the same instances as the _feature description problem_, but where the problem is to decide whether or not there exists a positive \(\mathcal{F}\)-description of \(Q\). Analogously, we define the _positive feature minimization problem_ to have the same instances as the _feature minimization problem_, but where the problem is to decide whether or not there exists a positive \(\mathcal{F}\)-description of \(Q\) obeying the size restriction.

### The feature description problems

We will first show that the feature description problems are decidable in polynomial time.

**Proposition 1**.: _The feature description problem and the positive feature description problem are each polynomial-time decidable._

Proof.: The algorithm for the feature description problem is as follows. Given an instance \((P,\mathcal{F},Q)\), compute a set \(\mathcal{C}\subseteq\wp(P)\) as follows. For each \(F\in\mathcal{F}\), place \(F\) in \(\mathcal{C}\) if \(Q\subseteq F\), and place \(P\setminus F\) in \(\mathcal{C}\) if \(Q\subseteq P\setminus F\). Then, check if \(\bigcap\mathcal{C}=Q\); if so, accept, otherwise, reject. It suffices to argue that if there exists a \(\mathcal{F}\)-description of \(Q\), then the elements of \(\mathcal{C}\) constitute such a description. If there exists a \(\mathcal{F}\)-description of \(Q\), say, \(G_{1},\ldots,G_{m}\), we have \(Q\subseteq G_{i}\) for each \(i\); thus, \(G_{1},\ldots,G_{m}\in\mathcal{C}\) and we have

\[Q\subseteq\bigcap\mathcal{C}\subseteq G_{1}\cap\cdots\cap G_{m}=Q,\]

implying that the elements of \(\mathcal{C}\) provide a \(\mathcal{F}\)-description of \(Q\).

For the positive feature description problem, the algorithm computes \(\mathcal{C}^{+}\) to contain each \(F\in\mathcal{F}\) such that \(Q\subseteq F\), and accepts if and only if \(\bigcap\mathcal{C}^{+}=Q\). The proof of correctness is similar to that given for the general feature description problem. 

### The minimization problems

An instance \((U,\mathcal{S},k)\) of the set cover problem consists of a non-empty set \(U\), a subset \(\mathcal{S}\subseteq\wp(U)\), and a natural number \(k\geq 1\). A _set cover_\(S_{1},\ldots,S_{m}\) is a sequence of sets from \(\mathcal{S}\) such that \(S_{1}\cup\cdots\cup S_{m}=U\); \(m\) is said to be its size. The problem is to decide whether or not there exists a set cover of size less than or equal to \(k\)[26]. We prove that both the feature minimization problem and the positive feature minimization problem are \(\mathcal{NP}\)-complete, by reducing from set cover.

The reduction is the same for both of these problems, and is as follows. Given an instance \((U,\mathcal{S},k)\) of set cover, let \(x\) be a fresh element not in \(U\).

Define

\begin{tabular}{l l}   \[P=U\cup\{x\}\] \\   \[\mathcal{F}=\{(U\setminus S)\cup\{x\}\mid S\in\mathcal{S}\}\] \\   \[Q=\{x\}\] \\ \end{tabular}  __ \\ \multicolumn{3}{c}{The resulting instance is \((P,\mathcal{F},Q,k)\).} \\ \end{tabular}

The following establishes the correctness of this reduction.

**Proposition 2**.: _The following are equivalent:_

1. There exists a size \(m\) set cover of \(U\).
2. The set \(Q=\{x\}\) has a positive \(\mathcal{F}\)-description of size \(m\).
3. The set \(Q=\{x\}\) has a \(\mathcal{F}\)-description of size \(m\).

Proof.: 1\(\Rightarrow\) 2: Let \((S_{i})\) be such a set cover, so that \(S_{1}\cup\cdots\cup S_{m}=U\). Then, by DeMorgan's laws, \((U\setminus S_{1})\cap\cdots\cap(U\setminus S_{m})=\emptyset\), and so \(((U\setminus S_{1})\cup\{x\})\cap\cdots\cap((U\setminus S_{m})\cup\{x\})=\{ x\}\).

2\(\Rightarrow\) 3 is immediate.

3\(\Rightarrow\) 1: Suppose that \(G_{1},\ldots,G_{m}\) is such a description. Since \(x\in G_{i}\) for each \(i\), it follows that each \(G_{i}\) is an element of \(\mathcal{F}\). Define \(S_{1},\ldots,S_{m}\) to be the sets such that \(G_{i}=(U\setminus S_{i})\cup\{x\}\). By reversing the argumentation in the 1\(\Rightarrow\) 2 case, we obtain that \(S_{1},\ldots,S_{m}\) is a set cover of \(U\) 

In the just-given reduction, the parameter \(k\) is not changed. We show that there in fact exists a reduction in the other direction, from set cover to each of the minimization problems, that likewise does not change the parameter \(k\). This indicates a tight relationship between these minimization problems and the set cover problem.

**Proposition 3**.: _Let \((P,{\cal F},Q,k)\) be an instance of the feature minimization problem; let \({\cal C}\) be the set (from Proposition 1). The mapping which, upon being given this instance, returns \((U,{\cal S},k)\) where \(U=P\setminus Q\) and \({\cal S}=\overline{\left\{G\setminus Q\mid G\in{\cal C}\right\}}\), is a reduction to the set cover problem; here, the complement is with respect to \(U\). Likewise, one obtains a reduction from the positive feature minimization problem to the set cover problem, by using \({\cal C}^{+}\) in place of \({\cal C}\)._

Proof.: We argue this for the feature minimization problem as follows (the positive minimization case is analogous). It was seen in the proof of Proposition 1 that \((P,{\cal F},Q,k)\) is a 'yes' instance if and only if there exist \(G_{1},\ldots,G_{m}\in{\cal C}\), with \(m\leq k\), such that \(G_{1}\cap\cdots\cap G_{m}=Q\). This equality holds if and only if \((G_{1}\setminus Q)\cap\cdots\cap(G_{m}\setminus Q)=\emptyset\), which holds if and only if \((\overline{G_{1}\setminus Q})\cup\cdots\cup(\overline{G_{m}\setminus Q})=U\). 

Parametrized complexityThe set cover problem, with the value \(k\) taken to be parameter, is known to be W[2]-complete in parameterized complexity theory Cygan et al. (2015, Theorem 13.21). As we have given polynomial-time reductions between set cover and each of the minimization problems that do not change the value \(k\), we obtain the following.

**Proposition 4**.: _Both the feature minimization problem and the positive feature minimization problem are W[2]-complete, when viewed as parameterized problems with \(k\) taken to be the parameter._

## 4 Empirical concerns

Many \({\cal NP}\)-hard problems can in practice often be solved by either a greedy algorithm Chvatal (1979) or a branch-and-bound algorithm Land and Doig (1960) that recursively explores the search space, terminating search branches as soon as some defined limit is exceeded.

Greedy SearchWe have implemented a greedy search strategy that starts with no features, and then picks a single (\(\pm\)) feature from the known, possibly non-minimal description \({\cal C}\) discussed in SS3.1 in such a way as to rule out the majority of phonemes not in the desired set \(Q\). Features are added to the description until only the set \(Q\) is described. For example, in describing the set \(\{\mathrm{m}\}\) using the example in Table 1, the greedy approach would first pick the feature [+nas] since this cuts down the number of corresponding phonemes to just two (m and n), less than any other feature choice. This is essentially an analogue to the well-known greedy approximation algorithm for set cover Chvatal (1979).

This greedy algorithm, however, in many practical cases fails to find the correct minimal specification, and is therefore not a viable candidate for efficient feature minimization. A simple example is finding the featural specification for the set \(\{\alpha\}\) in English under the fairly standard featural system provided in Hayes (2011) and van Vugt and Hayes (2012): the algorithm recovers the features [-tense, -back, -front, -coronal] while the minimal specification is [-back, -front, -coronal].

Branch & BoundWe also implemented a branch-and-bound algorithm with a recursive component that explores, exhaustively, all combinations of features in the full description \({\cal C}\), bounding the search whenever the current search tree contains more features than in the shortest solution found so far. The branch-and-bound algorithm is efficient in practice in finding minimal feature specifications, but still needs to explore a reasonably large search space (see Table 2).

Figure 1: Illustration of the reduction from setcovering: we can solve an instance of the set cover problem (setcovering) by a call to a solver for the feature minimization problem (minphonfeat). There are reductions both from and to the set cover problem.

 



## 5 Discussion

Humans have been noted to outperform simple, low-polynomial time heuristic algorithms for some intractable problems such as the traveling salesman problem MacGregor and Ormerod (1996) using intuition and visual inspection of the problem structure. It is, however, unclear if such performance carries over to other problems of a different structure, such as minphonfeat, or whether the hypothesis that a phonological acquisition process should include this type of minimization is too strong.

An important point to address in the complexity analysis is whether we are operating in a bounded domain. With a fixed, finite set of features to choose from and a fixed finite phoneme inventory, the problem can be considered static and solvable in constant time by memorizing or pre-calculating all the possible patterns of feature combinations and their corresponding phoneme sets. Such counter-arguments have been leveled Kornai (2006) at other \({\cal NP}\)-completeness analyses, such as those that have shown that Optimality Theory is potentially intractable Eisner (2000); Idsardi (2006). In the case at hand, however, such an argument has less traction since the domain in question is rather large (potentially hundreds of features) and it seems inevitable that some search method must be used by speakers (or phonologists) to discover the minimum number of features required. The fact that simple greedy algorithms do not always find the minimum specification, and that branch-and-bound algorithms, while efficient for practical computational use, still need to explore a large search space, prompts the question whether some alternative strategy would work particularly well with phonological feature structures proposed in the literature. This could address the problem that this intractability poses to phonological learning.

## 6 Conclusion

We have shown that one of the commonly assumed subtasks of acquisition of phonological generalizations--distinctive feature minimization--is computationally intractable. The decision version of the minimization problem is \({\cal NP}\)-complete, and it follows that the optimization version \({\cal NP}\)-hard. This is true even if one limits oneself to using only positive features, i.e. a privative feature system. Furthermore, a simple greedy strategy for solving the problem can not be attributed to purported human performance in finding minimal feature specifications since such a strategy will sometimes fail to find a minimal description with commonly proposed feature systems and phonological inventories. The problem of simply deciding whether a set of phonemes constitutes a natural class--the feature description problem--is solvable in polynomial time.

### Reproducibility

Our feature systems data and code for the feature description and minimization problems is available at [https://github.com/mhulden/minphonfeat](https://github.com/mhulden/minphonfeat).

#### Acknowledgements

Chen was partially supported by the Spanish Project TIN2017-86727-C2-2-R. We wish to thank Miikka Silfverberg for comments on an earlier version on the paper. Thanks also to participants in LING 5410 (Phonology) and LINC 7410 (Phonological Theory) at The University of Colorado, whose work and feedback on various feature minimization problems prompted part of this research.

\begin{table}
\begin{tabular}{l l l l l l l} \hline \hline
**Set** & \(\{h\}\) & \(\{i,e\}\) & \(\{r\}\) & \(\{j\}\) & \(\{m,n\}\) & \(\{p,t,k\}\) \\ \hline
**BF** & 32,768 & 262,144 & 131,072 & 524,288 & 2048 & 4,096 \\
**B\&B** & 3,432 & 11,482 & 3,444 & 4,790 & 1,450 & 1,315 \\ \hline \hline \end{tabular}
\end{table}
Table 2: The number of search nodes explored during a brute-force search (BF) for finding a minimal solution for various sets of phonemes, or a branch-and-bound search (B&B). The feature inventory is one used for English van Vugt and Hayes (2012); Hayes (2011).





## References

* [1]D. Bennett Archangeli (1984) Underspecification in Yavellmani phonology and morphology.. Ph.D. Thesis, Massachusetts Institute of Technology. Cited by: SS2.3.
* N. Chomsky and M. Halle (1968) The sound pattern of english. Harper & Row. External Links: ISBN 0-751-4653-0 Cited by: SS2.3.
* V. Chvatal (1979) A greedy heuristic for the set-covering problem. Mathematics of operations research4 (3), pp. 233-235. Cited by: SS2.3.
* G. N. Clements (1985) The geometry of phonological features. Phonology2 (1), pp. 225-252. Cited by: SS2.3.
* G. N. Clements (2003) Feature economy in sound systems. Phonology20 (03), pp. 287-333. Cited by: SS2.3.
* M. Cygan, F. V. Fomin, L. Kowalik, D. Lokshtanov, D. Marx, M. Pilipczuk, M. Pilipczuk, and S. Saurabh (2015) Parameterized algorithms. Springer. Cited by: SS2.3.
* J. Eisner (2000) Easy and hard constraint ranking in optimality theory. In Proceedings of the 5th Workshop of the ACL Special Interest Group in Computational Phonology (SIGPHON), Vol. , pp. 22-33. Cited by: SS2.3.
* C. J. Even and H. van der Hulst (1985) Single-valued features and the nonlinear analysis of vowel harmony. Linguistics in the Netherlands, pp. 39-48. Cited by: SS2.3.
* G. Fant (1966) The nature of distinctive features. STL-QPSR4 (1966), pp. 1. Cited by: SS2.3.
* M. R. Garey and D. S. Johnson (1979) Computers and intractability: a guide to the theory of NP-completeness. Cited by: SS2.3.
* J. Goldsmith (1985) Vowel harmony in Khalkha Mongolian, Yaka, Finnish and Hungarian. Phonology2 (1), pp. 253-275. Cited by: SS2.3.
* Morris Halle (1962) Phonology in generative grammar. Word18 (1-3), pp. 54-72. Cited by: SS2.3.
* Bruce Hayes (2011) Introductory phonology. John Wiley & Sons. Cited by: SS2.3.
* W. J. Idsardi (2006) A simple proof that optimality theory is computationally intractable. Linguistic Inquiry37 (2), pp. 271-275. Cited by: SS2.3.
* Roman Jakobson (1942) The concept of phoneme. In Linda R. Waugh and Monique Moville-Burston, editors, On Language, pp. 218-241. Harvard Univ. Press, Cambridge, MA. Cited by: SS2.3.
* R. Jakobson, C. M. Fant, and M. Halle (1951) Preliminaries to speech analysis. The distributive features and their correlates. MIT Press, Cambridge, MA. Cited by: SS2.3.
* R. M. Karp (1972) Reducibility among combinatorial problems. In Complexity of computer computations, pp. 85-103. Cited by: SS2.3.
* Andras Kornai (1993) The generative power of feature geometry. Annals of Mathematics and Artificial Intelligence8 (1-2), pp. 37. Cited by: SS2.3.
* A. Kornai (2006) Is OT NP-hard?. Rutgers Optimality Archive (ROA). Cited by: SS2.3.
* A. H. Land and A. G. Doig (1960) An automatic method of solving discrete programming problems. Econometrica: Journal of the Econometric Society, pp. 497-520. Cited by: SS2.3.
* J. N. MacGregor and T. Ormerod (1996) Human performance on the traveling salesman problem. Attention, Perception, & Psychophysics58 (4), pp. 527-539. Cited by: SS2.3.
* Andre Martinet (1955) Economic des changements phonetiques. A. Francke, Bern, Switzerland. Cited by: SS2.3.
* Jeff Mielke (2008) The emergence of distinctive features. Oxford University Press. Cited by: SS2.3.
* J. Mielke (2008b) P-base 3. Note: [http://phase.phon.chass.ncsu.edu/Accessed](http://phase.phon.chass.ncsu.edu/Accessed): 2017-10-01 Cited by: SS2.3.
* Amanda Miller (2011) The representation of clicks. In Mark van Oostendorp, Colin J. Ewen, Elizabeth Hume, and Keren Rice, editors, The Blackwell Companion to Phonology, Vol. 1, pp. 416-439. Cited by: SS2.3.
* Michael Sipser (2013) Introduction to the theory of computation. Second edition, Prentice Hall, Upper Publishing Company. Cited by: SS2.3.
* Nikolai Sergeevich Trubetzkoy (1969) Principles of phonology. Univ. of California Press, Berkeley. Cited by: SS2.3.
* F. van Vogt and B. Hayes (2012) Pheatures speadsheet: user's manual. Technical report UCLA. Cited by: SS2.3.
* E. C. Zsiga (2012) The sounds of language: an introduction to phoneticts and phonology. John Wiley & Sons. Cited by: SS2.3.