<figure><img src="image_5.png" /></figure>

(Note that $\mathrm{Im}(d_{\min})$ is $(\max, +)$ closed; this follows from Proposition 29.)

We have then that the $(\min +)$ column span $P^-(\mathcal{L})$, of $d_{\min}$ is anti isomorphic to the $(\min, +)$ row span $\hat{P}^-(\mathcal{L})$ of $d_{\min}$ (as $\mathbb{R}_{\min}$ modules) by the two inverse maps $A$ and $B$ and moreover they are isometric when considered with the directed metrics $D$ and $D^t$ respectively. (Recall also that in Proposition 12 we saw that $D$ can be considered as tropical inner product.)

We will see an example of this below. First though we would like to make this map more explicit with respect to the rows and columns of the matrix $d$.

**Proposition 20.** Consider $x \in \mathrm{Im}(d_{\min})$. We have that

$$ (104) \qquad x = \oplus_j x_j \odot d(-, a_j) \quad \text{and then} \quad B(x) = -x = \oplus_j -x_j \odot d(a_j, -). $$

In particular if $x = d(-, a_k)$ then

$$ (105) \qquad d(-, a_k) = \oplus_{a_j \le a_k} d(a_j, a_k) \odot d(-, a_j) $$

and

$$ (106) \qquad -d(-, a_k) = \oplus_{a_j \le a_k} -d(a_j, a_k) \odot d(a_j, -) $$

Analogously for $y \in \mathrm{Im}(d^t)$ we have

$$ (107) \qquad y = \oplus_i y_i \odot d(a_i, -) \quad \text{and then} \quad A(y) = -y = \oplus_i -y_i \odot d(-, a_i). $$

In particular if $y = d(a_k, -)$ then

$$ (108) \qquad d(a_k, -) = \oplus_{a_k \le a_i} d(a_k, a_i) \odot d(a_i, -) $$

and

$$ (109) \qquad -d(a_k, -) = \oplus_{a_k \le a_i} -d(a_k, a_i) \odot d(-, a_i). $$

*Proof.* We have $x \in \mathrm{Im}(d_{\min}) \iff d_{\min}x = x \iff x = \oplus_j x_j \odot d(-, a_j)$. From Proposition 19 we then have $d_{\min}^t(-x) = -x$ which is equivalent to $-x = \oplus_j -x_i \odot d(a_j, -)$. This proves (104).

Now if $x := d(-, a_k)$ then $x_j = x(a_j) = d(a_j, a_k) = d_{j,k}$. Then from Proposition 11 we have $d(-, a_k) = \oplus_{a_j \le a_k} d_{j,k} \odot d(-, a_j)$ therefore from (104) it follows that $-d(-, a_k) = \oplus_{a_j \le a_k} -d(a_j, a_k) \odot d(a_j, -)$. The proof for $y \in \mathrm{Im}(d_{\min}^t)$ and for $y := d(a_k, -)$ is analogous. □

**Remark 28.** Note that all results in this section hold for a general directed metric space.

**Example 1.** We now show a simple example of a probabilistic language model $(\mathcal{L}, d_1)$ along with $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$. We will also see the correspondence of extremal rays with connected lower sets for $P(\mathcal{L})$ and connected upper sets for $\hat{P}(\mathcal{L})$ as described in Theorem 2.

We will actually consider the corresponding polyhedral cones $Q(\mathcal{L})$ and $\hat{Q}(\mathcal{L})$ and show in the figures the polyhedra $Q_0(\mathcal{L})$ and $\hat{Q}^0(\mathcal{L})$ (Definition 6) which are their intersections with the unit simplex.

We will further illustrate the duality between completions $P^-(\mathcal{L})$ and $\hat{P}^-(\mathcal{L})$ by making a uniform approximation of infinities in $d$ with a big number $M$.

Indeed consider the corpus to be $\mathcal{L} := \{\text{red}, \text{colour}, \text{red colour}\}$. Denote “red” by “r”, “colour” by “c” and “red colour” by “rc”.
□

(Note that $\mathrm{Im}(d_{\min})$ is $(\max, +)$ closed; this follows from Proposition 29.)

We have then that the $(\min +)$ column span $P^-(\mathcal{L})$, of $d_{\min}$ is anti isomorphic to the $(\min, +)$ row span $\hat{P}^-(\mathcal{L})$ of $d_{\min}$ (as $\mathbb{R}_{\min}$ modules) by the two inverse maps $A$ and $B$ and moreover they are isometric when considered with the directed metrics $D$ and $D^t$ respectively. (Recall also that in Proposition 12 we saw that $D$ can be considered as tropical inner product.)

We will see an example of this below. First though we would like to make this map more explicit with respect to the rows and columns of the matrix $d$.

**Proposition 20.** Consider $x \in \mathrm{Im}(d_{\min})$. We have that

$$ (104) \qquad x = \oplus_j x_j \odot d(-, a_j) \quad \text{and then} \quad B(x) = -x = \oplus_j -x_j \odot d(a_j, -). $$

In particular if $x = d(-, a_k)$ then

$$ (105) \qquad d(-, a_k) = \oplus_{a_j \le a_k} d(a_j, a_k) \odot d(-, a_j) $$

and

$$ (106) \qquad -d(-, a_k) = \oplus_{a_j \le a_k} -d(a_j, a_k) \odot d(a_j, -) $$

Analogously for $y \in \mathrm{Im}(d^t)$ we have

$$ (107) \qquad y = \oplus_i y_i \odot d(a_i, -) \quad \text{and then} \quad A(y) = -y = \oplus_i -y_i \odot d(-, a_i). $$

In particular if $y = d(a_k, -)$ then

$$ (108) \qquad d(a_k, -) = \oplus_{a_k \le a_i} d(a_k, a_i) \odot d(a_i, -) $$

and

$$ (109) \qquad -d(a_k, -) = \oplus_{a_k \le a_i} -d(a_k, a_i) \odot d(-, a_i). $$

*Proof.* We have $x \in \mathrm{Im}(d_{\min}) \iff d_{\min}x = x \iff x = \oplus_j x_j \odot d(-, a_j)$. From Proposition 19 we then have $d_{\min}^t(-x) = -x$ which is equivalent to $-x = \oplus_j -x_i \odot d(a_j, -)$. This proves (104).

Now if $x := d(-, a_k)$ then $x_j = x(a_j) = d(a_j, a_k) = d_{j,k}$. Then from Proposition 11 we have $d(-, a_k) = \oplus_{a_j \le a_k} d_{j,k} \odot d(-, a_j)$ therefore from (104) it follows that $-d(-, a_k) = \oplus_{a_j \le a_k} -d(a_j, a_k) \odot d(a_j, -)$. The proof for $y \in \mathrm{Im}(d_{\min}^t)$ and for $y := d(a_k, -)$ is analogous. □

**Remark 28.** Note that all results in this section hold for a general directed metric space.

**Example 1.** We now show a simple example of a probabilistic language model $(\mathcal{L}, d_1)$ along with $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$. We will also see the correspondence of extremal rays with connected lower sets for $P(\mathcal{L})$ and connected upper sets for $\hat{P}(\mathcal{L})$ as described in Theorem 2.

We will actually consider the corresponding polyhedral cones $Q(\mathcal{L})$ and $\hat{Q}(\mathcal{L})$ and show in the figures the polyhedra $Q_0(\mathcal{L})$ and $\hat{Q}^0(\mathcal{L})$ (Definition 6) which are their intersections with the unit simplex.

We will further illustrate the duality between completions $P^-(\mathcal{L})$ and $\hat{P}^-(\mathcal{L})$ by making a uniform approximation of infinities in $d$ with a big number $M$.

Indeed consider the corpus to be $\mathcal{L} := \{\text{red}, \text{colour}, \text{red colour}\}$. Denote “red” by “r”, “colour” by “c” and “red colour” by “rc”.
Let the metric $d$ be given by eq. (110):

$$ (110) \qquad \begin{array}{c} r \ c \ rc \\ d = c \begin{pmatrix} r & 0 & \infty & \log 2 \\ 0 & \infty & 0 & \log 3 \\ c & \infty & \infty & 0 \end{pmatrix} \\ rc \end{array} $$

Recall that in general $e^{-d_{i,j}} = \mathrm{Pr}(a_j|a_i)$ and thus the corresponding matrix of probabilities of extensions is

$$ (111) \qquad \mathrm{Pr} = \begin{array}{c} r \ c \ rc \\ \begin{pmatrix} r & 1 & 0 & \frac{1}{2} \\ c & 0 & 1 & \frac{1}{3} \\ rc & 0 & 0 & 1 \end{pmatrix} \end{array} $$

This means for example that $\mathrm{Pr}(rc|r) = \frac{1}{2}$ and $\mathrm{Pr}(c|r) = 0$, while $P(r|r) = 1$.

Recall that the equations for $P(\mathcal{L}) = \mathrm{Im}(d_{\min})$ are (Definition 4): $x_i \le d_{i,j} + x_j$.

Letting $z_i := e^{-x_i}$ we have that the equations for $Q(\mathcal{L})$ are (Definition 5): $z_i \ge e^{-d_{i,j}} z_j$.

Therefore in our case we get that the polyhedral cone $Q(\mathcal{L})$ is defined by inequalities

$$ (112) \qquad z_1 \ge \frac{1}{2} z_3, z_2 \ge \frac{1}{3} z_3, z_1 \ge 0, z_2 \ge 0, z_3 \ge 0. $$

The intersection $Q_0(\mathcal{L})$ of $Q(\mathcal{L})$ with the unit simplex is shown on the right in Figure 1. Notice that it has three vertices.

Analogously, the equations for $\hat{P}(\mathcal{L}) = \mathrm{Im}(d_{\min}^t)$ (Definition 4) are $y_j \le d_{i,j} + y_i$.

Letting $u_i := e^{-y_i}$ we have that the equations for $\hat{Q}(\mathcal{L})$ are (Definition 4) $u_j \ge e^{-d_{i,j}} u_i$.

Therefore in our case we get that the polyhedral cone $\hat{Q}(\mathcal{L})$ is defined by inequalities

$$ (113) \qquad u_3 \ge \frac{1}{2} u_1, u_3 \ge \frac{1}{3} u_2, u_1 \ge 0, u_2 \ge 0, u_3 \ge 0. $$

The intersection $\hat{Q}_0(\mathcal{L})$ of $\hat{Q}(\mathcal{L})$ with the unit simplex is shown on the left in Figure 1. Notice that it has four vertices.

Denote the lower set generated by “a” by $(a)_l$ and the upper set generated by $a$ by $(a)_u$.

From Theorem 2, extremal rays of $Q(\mathcal{L})$ correspond to connected lower sets of $\mathcal{L}$. There are three and they are all principal: $(r)_l = \{r\}, (c)_l = \{c\}, (rc)_l = \{r, c, rc\}$. These give rise to the three vertices of $Q_0(\mathcal{L})$ as we can see in Fig. 1. (Note that $(r, c)_l$ is not connected so it does not correspond to an extremal ray of $Q(\mathcal{L})$.

From Corollary 4, extremal rays of $\hat{Q}(\mathcal{L})$ correspond to connected upper sets of $\mathcal{L}$. The principal ones are $(r)_u = \{r, rc\}, (c)_u = \{c, rc\}, (rc)_u = \{rc\}$ and a non-principal one $(r, c)_u = \{r, c, rc\}$. This extremal ray is not in the image of the Yoneda embedding. The corresponding four vertices of $\hat{Q}_0(\mathcal{L})$ are shown on the left in Figure 1.

Notice that the number of extremal rays of $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ are actually different.

Now note that in general the $\mathbb{R}_{\min}$ module $P(\mathcal{L}) = \mathrm{Im}(d_{\min})$ is a geometric object. In fact $Q(\mathcal{L})$ is a polyhedral cone and $Q_0(\mathcal{L})$ is a polyhedron. However the
Let the metric $d$ be given by eq. (110):

$$ (110) \qquad \begin{array}{c} r \ c \ rc \\ d = c \begin{pmatrix} r & 0 & \infty & \log 2 \\ 0 & \infty & 0 & \log 3 \\ c & \infty & \infty & 0 \end{pmatrix} \\ rc \end{array} $$

Recall that in general $e^{-d_{i,j}} = \text{Pr}(a_j|a_i)$ and thus the corresponding matrix of probabilities of extensions is

$$ (111) \qquad \begin{array}{c} r \ c \ rc \\ \text{Pr} = c \begin{pmatrix} r & 1 & 0 & \frac{1}{2} \\ 0 & 1 & \frac{1}{3} & 0 \\ c & 0 & 0 & 1 \end{pmatrix} \\ rc \end{array} $$

This means for example that $\text{Pr}(rc|r) = \frac{1}{2}$ and $\text{Pr}(c|r) = 0$, while $P(r|r) = 1$.

Recall that the equations for $P(\mathcal{L}) = \text{Im}(d_{\min})$ are (Definition 4): $x_i \le d_{i,j} + x_j$.

Letting $z_i := e^{-x_i}$ we have that the equations for $Q(\mathcal{L})$ are (Definition 5): $z_i \ge e^{-d_{i,j}} z_j$.

Therefore in our case we get that the polyhedral cone $Q(\mathcal{L})$ is defined by inequalities

$$ (112) \qquad z_1 \ge \frac{1}{2}z_3, z_2 \ge \frac{1}{3}z_3, z_1 \ge 0, z_2 \ge 0, z_3 \ge 0. $$

The intersection $Q_0(\mathcal{L})$ of $Q(\mathcal{L})$ with the unit simplex is shown on the right in Figure 1. Notice that it has three vertices.

Analogously, the equations for $\hat{P}(\mathcal{L}) = \text{Im}(d_{\min}^t)$ (Definition 4) are $y_j \le d_{i,j} + y_i$.

Letting $u_i := e^{-y_i}$ we have that the equations for $\hat{Q}(\mathcal{L})$ are (Definition 4) $u_j \ge e^{-d_{i,j}} u_i$.

Therefore in our case we get that the polyhedral cone $\hat{Q}(\mathcal{L})$ is defined by inequalities

$$ (113) \qquad u_3 \ge \frac{1}{2}u_1, u_3 \ge \frac{1}{3}u_2, u_1 \ge 0, u_2 \ge 0, u_3 \ge 0. $$

The intersection $\hat{Q}_0(\mathcal{L})$ of $\hat{Q}(\mathcal{L})$ with the unit simplex is shown on the left in Figure 1. Notice that it has four vertices.

Denote the lower set generated by “a” by $(a)_l$ and the upper set generated by $a$ by $(a)_u$.

From Theorem 2, extremal rays of $Q(\mathcal{L})$ correspond to connected lower sets of $\mathcal{L}$. There are three and they are all principal: $(r)_l = \{r\}, (c)_l = \{c\}, (rc)_l = \{r, c, rc\}$. These give rise to the three vertices of $Q_0(\mathcal{L})$ as we can see in Fig. 1. (Note that $(r, c)_l$ is not connected so it does not correspond to an extremal ray of $Q(\mathcal{L})$.

From Corollary 4, extremal rays of $\hat{Q}(\mathcal{L})$ correspond to connected upper sets of $\mathcal{L}$. The principal ones are $(r)_u = \{r, rc\}, (c)_u = \{c, rc\}, (rc)_u = \{rc\}$ and a non-principal one $(r, c)_u = \{r, c, rc\}$. This extremal ray is not in the image of the Yoneda embedding. The corresponding four vertices of $\hat{Q}_0(\mathcal{L})$ are shown on the left in Figure 1.

Notice that the number of extremal rays of $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ are actually different.

Now note that in general the $\mathbb{R}_{\min}$ module $P(\mathcal{L}) = \text{Im}(d_{\min})$ is a geometric object. In fact $Q(\mathcal{L})$ is a polyhedral cone and $Q_0(\mathcal{L})$ is a polyhedron. However the
$\bar{\mathbb{R}}_{\min}$ module $P^{-}(\mathcal{L})$ is not obviously geometric. In order to approximate with a geometric object and be able to visualize the duality between the $P^{-}(\mathcal{L})$ and $\hat{P}^{-}(\mathcal{L})$ it is natural to “truncate” the matrix $d$, replacing the $+\infty$ entries by a sufficiently large number $M$, leading to the new matrix:

$$ (114) \qquad d^M = \begin{array}{c@{\quad}c@{\quad}c} & \begin{array}{ccc} r & c & rc \end{array} \\ \begin{array}{c} r \\ c \\ rc \end{array} & \left( \begin{array}{ccc} 0 & M & \log 2 \\ M & 0 & \log 3 \\ M & M & 0 \end{array} \right) \end{array} $$

This matrix is still a directed metric, satisfying $(d_{\min}^M)^2 = d_{\min}^M$ but it does not any more represent a probabilistic language model.

We can consider $P_M(\mathcal{L}) := \mathrm{Im}(d_{\min}^M)$ as in (Definition 4) and $Q_M(\mathcal{L})$ as in (Definition 5). Then the intersection $Q_{M,0}(\mathcal{L})$ of $Q_M(\mathcal{L})$ with the unit simplex is depicted in Figure 2 on the right.

Moreover we consider $\hat{P}_M(\mathcal{L}) := \mathrm{Im}((d_{\min}^M)^t)$ as in (Definition 4) and $\hat{Q}_M(\mathcal{L})$ as in (Definition 5). Then the intersection $\hat{Q}_{M,0}(\mathcal{L})$ of $\hat{Q}_M(\mathcal{L})$ with the unit simplex is depicted in Figure 2 on the right.

Observe that the duality preserves the number of extreme points inside the interior of the simplex, and that the sets of Figure 2 converge to the sets of Figure 1 as $M \to \infty$.

Also note that the duality map between $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ is $x_i \to y_i = -x_i$ for $i = 1, 2, 3$. We also have $z_i := e^{-x_i}$ and $u_i := e^{-y_i}$. Therefore the map between the polyhedra $Q(\mathcal{L})$ and $\hat{Q}(\mathcal{L})$ is $z_i \to u_i = \frac{1}{z_i}$ for $i = 1, 2, 3$.

<figure><img src="image_6.png" /><figcaption>FIGURE 1. The cross section $\hat{Q}_0(\mathcal{L})$ of the polyhedral cone $\hat{Q}(\mathcal{L})$ arising from the metric of $d$ (left). Every vector $d(r, -), d(c, -), d(rc, -)$ determines an extreme point of the cross section, denoted by $r, c$, or $rc$. There is a fourth extreme point (shown in gray) corresponding to a non-principal upper set. The cross section $Q_0(\mathcal{L})$ (right). There are three extreme points, which correspond to the vectors $d(-, r), d(-, c), d(-, rc)$.</figcaption></figure>

**Remark 29.** Note that approximating uniformly infinities in the matrix $d$ with a big number $M$ can be done in general. This is helpful since the duality theorem is easier to illustrate for matrices with finite entries. Of course, the proof of the duality theorem in section 6 goes through with coefficients in $(-\infty, \infty)$. (Note also that the Develin-Sturmfels version [4] of the adjunction between the tropical
$\bar{\mathbb{R}}_{\min}$ module $P^{-}(\mathcal{L})$ is not obviously geometric. In order to approximate with a geometric object and be able to visualize the duality between the $P^{-}(\mathcal{L})$ and $\hat{P}^{-}(\mathcal{L})$ it is natural to "truncate" the matrix $d$, replacing the $+\infty$ entries by a sufficiently large number $M$, leading to the new matrix:

$$ (114) \qquad d^M = c \begin{pmatrix} r & c & rc \\ r & 0 & M \log 2 \\ c & M & 0 \log 3 \\ rc & M & M \end{pmatrix} $$

This matrix is still a directed metric, satisfying $(d_{\min}^M)^2 = d_{\min}^M$ but it does not any more represent a probabilistic language model.

We can consider $P_M(\mathcal{L}) := \mathrm{Im}(d_{\min}^M)$ as in (Definition 4) and $Q_M(\mathcal{L})$ as in (Definition 5). Then the intersection $Q_{M,0}(\mathcal{L})$ of $Q_M(\mathcal{L})$ with the unit simplex is depicted in Figure 2 on the right.

Moreover we consider $\hat{P}_M(\mathcal{L}) := \mathrm{Im}((d_{\min}^M)^t)$ as in (Definition 4) and $\hat{Q}_M(\mathcal{L})$ as in (Definition 5). Then the intersection $\hat{Q}_{M,0}(\mathcal{L})$ of $\hat{Q}_M(\mathcal{L})$ with the unit simplex is depicted in Figure 2 on the right.

Observe that the duality preserves the number of extreme points inside the interior of the simplex, and that the sets of Figure 2 converge to the sets of Figure 1 as $M \to \infty$.

Also note that the duality map between $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ is $x_i \to y_i = -x_i$ for $i = 1, 2, 3$. We also have $z_i := e^{-x_i}$ and $u_i := e^{-y_i}$. Therefore the map between the polyhedra $Q(\mathcal{L})$ and $\hat{Q}(\mathcal{L})$ is $z_i \to u_i = \frac{1}{z_i}$ for $i = 1, 2, 3$.

<figure><img src="image_12.png" /><figcaption>FIGURE 1. The cross section $\hat{Q}_0(\mathcal{L})$ of the polyhedral cone $\hat{Q}(\mathcal{L})$ arising from the metric of $d$ (left). Every vector $d(r, -), d(c, -), d(rc, -)$ determines an extreme point of the cross section, denoted by $r$, $c$, or $rc$. There is a fourth extreme point (shown in gray) corresponding to a non-principal upper set. The cross section $Q_0(\mathcal{L})$ (right). There are three extreme points, which correspond to the vectors $d(-, r), d(-, c), d(-, rc)$.</figcaption></figure>

**Remark 29.** Note that approximating uniformly infinities in the matrix $d$ with a big number $M$ can be done in general. This is helpful since the duality theorem is easier to illustrate for matrices with finite entries. Of course, the proof of the duality theorem in section 6 goes through with coefficients in $(-\infty, \infty)$. (Note also that the Develin-Sturmfels version [4] of the adjunction between the tropical
column span and the tropical row span is exactly about matrices with finite entries, whereas the version of [3] deals with matrices with possibly infinite entries.)

However as we already saw in the example replacing $\infty$ with $M$ in a directed metric $d$ that defines a probabilistic language model gives a metric that is no longer a language model. The limit of polyhedra for $M \to \infty$ will give the polyhedra for the original metric.

<figure><img src="image_7.png" /><figcaption>FIGURE 2. The duality between the columns and row spaces of metric matrices (Proposition 19 and Theorem 4) illustrated. On the right $\mathrm{Im}(d_{\min}^M)$ and on the left $\mathrm{Im}((d_{\min}^M)^t)$</figcaption></figure>

**Remark 30.** We have said that we can encode the meaning of $a_k$ by $d(-, a_k)$. If $a_k$ is a word this contains very little information. We already addressed the solution to this problem in section Section 4.1. Another solution is, using the duality between $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ explained previously and in particular Equation (105), Equation (106).

## 7. EXTREMAL RAYS IN TERMS OF TEXT VECTORS

We have seen in Theorem 1 that the original texts in $\mathcal{L}$, mapped by the Yoneda isometric embedding $Y: \mathcal{L} \to P(\mathcal{L})$, appear as extremal rays (corresponding to principal lower sets) in the polyhedron $P(\mathcal{L})$ and the polyhedral cone $Q(\mathcal{L})$. As proven in Proposition 7, there are in general many other extremal rays of $P(\mathcal{L})$ corresponding to connected lower sets of $\mathcal{L}$. Nevertheless extremal rays in the image of $Y$, (min, +) generate $P(\mathcal{L})$ as we have already seen in Corollary 6 where we showed that

$$ (115) \qquad x \in P(\mathcal{L}) \iff x = \oplus_j D(Y(a_j), x) \odot Y(a_j). $$

Recall that we think of $Y(a_k) := d(-, a_k) \in P(\mathcal{L})$ as encoding the meaning of text $a_k$ according to the statistical semantics principal.

Recall Definition 9, where we introduced the saturation graph $S(x)$ for $x \in P(\mathcal{L})$. We shall also consider the *undirected* saturation graph, obtained by forgetting the orientation of the edges in $S(x)$.

**Proposition 21.** A vector $x \in P(\mathcal{L})$ can be written as a tropical linear combination of terminal elements of its saturation graph $S(x)$. Specifically if $b_1, \dots, b_k$ are the terminal elements in $S(x)$, then

$$ (116) \qquad x = \oplus_j D(Y(b_j), x) \odot Y(b_j). $$
column span and the tropical row span is exactly about matrices with finite entries, whereas the version of [3] deals with matrices with possibly infinite entries.)

However as we already saw in the example replacing $\infty$ with $M$ in a directed metric $d$ that defines a probabilistic language model gives a metric that is no longer a language model. The limit of polyhedra for $M \to \infty$ will give the polyhedra for the original metric.

<figure><img src="image_13.png" /><figcaption>FIGURE 2. The duality between the columns and row spaces of metric matrices (Proposition 19 and Theorem 4) illustrated. On the right $\mathrm{Im}(d_{\mathrm{min}}^M)$ and on the left $\mathrm{Im}((d_{\mathrm{min}}^M)^t)$</figcaption></figure>

**Remark 30.** We have said that we can encode the meaning of $a_k$ by $d(-, a_k)$. If $a_k$ is a word this contains very little information. We already addressed the solution to this problem in section Section 4.1. Another solution is, using the duality between $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ explained previously and in particular Equation (105), Equation (106).

## 7. EXTREMAL RAYS IN TERMS OF TEXT VECTORS

We have seen in Theorem 1 that the original texts in $\mathcal{L}$, mapped by the Yoneda isometric embedding $Y: \mathcal{L} \to P(\mathcal{L})$, appear as extremal rays (corresponding to principal lower sets) in the polyhedron $P(\mathcal{L})$ and the polyhedral cone $Q(\mathcal{L})$. As proven in Proposition 7, there are in general many other extremal rays of $P(\mathcal{L})$ corresponding to connected lower sets of $\mathcal{L}$. Nevertheless extremal rays in the image of $Y$, (min, +) generate $P(\mathcal{L})$ as we have already seen in Corollary 6 where we showed that

$$ (115) \qquad x \in P(\mathcal{L}) \iff x = \oplus_j D(Y(a_j), x) \odot Y(a_j). $$

Recall that we think of $Y(a_k) := d(-, a_k) \in P(\mathcal{L})$ as encoding the meaning of text $a_k$ according to the statistical semantics principal.

Recall Definition 9, where we introduced the saturation graph $S(x)$ for $x \in P(\mathcal{L})$. We shall also consider the *undirected* saturation graph, obtained by forgetting the orientation of the edges in $S(x)$.

**Proposition 21.** A vector $x \in P(\mathcal{L})$ can be written as a tropical linear combination of terminal elements of its saturation graph $S(x)$. Specifically if $b_1, \dots, b_k$ are the terminal elements in $S(x)$, then

$$ (116) \qquad x = \oplus_j D(Y(b_j), x) \odot Y(b_j). $$
Moreover, if the saturation graph of x has s undirected connected components, this vector belongs to a face of $P(\mathcal{L})$ of dimension s.

**Proof.** Let $x \in P(\mathcal{L})$. If $i$ is not terminal in the saturation graph $S(x)$, then, there is a $j_1 \neq i$ such that $x_i = d_{i,j_1} + x_{j_1}$. Similarly, if $j_1$ is not terminal, there is $j_2 \neq j_1$ such that $x_{j_1} = d_{j_1,j_2} + x_{j_2}$ and thus $x_i = d_{i,j_1} + d_{j_1,j_2} + x_{j_2}$. Continuing this way we get $x_i = d_{i,j_1} + d_{j_1,j_2} + \dots d_{j_{n-1},j_n} + x_{j_n}$ where $a_{j_n}$ is a terminal element of the saturation graph $S(x)$.

We know that this stops at a terminal element because there are no cycles of positive weight in the digraph of $d$. (This is the graph that has an edge from $i$ to $j$ with weight $d_{i,j}$ when $d_{i,j}$ is not infinity.)

Using the triangular inequality, we deduce that $x_i \geq \oplus_{j \in T} d_{i,j} \odot x_j$, where the sum is taken over the set $T$ of terminal nodes of $S(x)$, and so, $x \geq \oplus_{j \in T} D(Y(a_i), Y(a_j)) + D(Y(a_j), x)$. Conversely, by definition of $P(\mathcal{L})$, $x \leq \oplus_k d_{ik} \odot x_k = \oplus_k D(Y(a_i), Y(a_k)) \odot D(Y(a_k), x)$ where now the sum is taken over all the indices $k$ (possibly non terminal). This entails that (116) holds.

Finally, arguing as in the proof of Theorem 2, we get the rank of the family of active constraints at point $x$ is given by the number $s$ of connected component of the undirected saturation graph of $x$. Hence, $x$ belongs to a face of dimension $s$. $\square$

**Remark 31.** Note that Proposition 21 holds for a general directed metric space $\mathcal{L}$ and not just for $(\mathcal{L}, d)$ a probabilistic language model.

We can now find explicit $(min, +)$ expressions for generators of extremal rays corresponding to non principal lower sets.

**Proposition 22.** Let $\mathcal{L}$ be a probabilistic language model with the empty text $a_0$ included. Let $x$ denote an extremal ray corresponding to the lower set generated by $\{b_1, \dots, b_n\}$. Then

$$ (117) \qquad x = \bigoplus_i \log \Pr(b_i) \odot Y(b_i). $$

*Proof.* From Proposition 21 we have

$$ x = \bigoplus_j x(b_j) \odot Y(b_j). $$

From corollary 3 we have $x(b_j) = -\log \frac{1}{\Pr(b_j)} = \log \Pr(b_j)$. This proves the result. $\square$

**Remark 32.** We point out that the terminal elements $b_1, \dots, b_k$ of $S(x)$ function like an orthonormal basis with respect to $D$, namely

$$ (118) \qquad D(Y(b_i), Y(b_j)) = d(b_i, b_j) = \infty \text{ if } i \neq j. $$

So for example if we know that there are $\lambda_j$ such that

$$ x = \bigoplus_j \lambda_j \odot Y(b_j) \text{ Then } D(Y(b_i), x) = \bigoplus_j \lambda_j \odot D(Y(b_i), Y(b_j)) = \lambda_i. $$

**Proposition 23.** Let $T \in [0, \infty)$ be a parameter which will be called temperature. Consider $x \in P(\mathcal{L})$ an extremal ray and let $x = \bigoplus_j D(Y(b_j), x) \odot Y(b_j)$ where $b_j$ are the terminal elements of the saturation graph $S(x)$. Let $v_j := e^{Y(b_j)}$. Then we have

$$ (119) \qquad x = \lim_{T \to 0} -T \log\left(\sum_j e^{-D(Y(b_j),x)/T} e^{Y(b_j)}\right) $$
Moreover, if the saturation graph of x has s undirected connected components, this vector belongs to a face of $P(\mathcal{L})$ of dimension s.

**Proof.** Let $x \in P(\mathcal{L})$. If $i$ is not terminal in the saturation graph $S(x)$, then, there is a $j_1 \neq i$ such that $x_i = d_{i,j_1} + x_{j_1}$. Similarly, if $j_1$ is not terminal, there is $j_2 \neq j_1$ such that $x_{j_1} = d_{j_1,j_2} + x_{j_2}$ and thus $x_i = d_{i,j_1} + d_{j_1,j_2} + x_{j_2}$. Continuing this way we get $x_i = d_{i,j_1} + d_{j_1,j_2} + \dots d_{j_{n-1},j_n} + x_{j_n}$ where $a_{j_n}$ is a terminal element of the saturation graph $S(x)$.

We know that this stops at a terminal element because there are no cycles of positive weight in the digraph of $d$. (This is the graph that has an edge from $i$ to $j$ with weight $d_{i,j}$ when $d_{i,j}$ is not infinity.)

Using the triangular inequality, we deduce that $x_i \geq \oplus_{j \in T} d_{i,j} \odot x_j$, where the sum is taken over the set $T$ of terminal nodes of $S(x)$, and so, $x \geq \oplus_{j \in T} D(Y(a_i), Y(a_j)) + D(Y(a_j), x)$. Conversely, by definition of $P(\mathcal{L})$, $x \leq \oplus_k d_{ik} \odot x_k = \oplus_k D(Y(a_i), Y(a_k)) \odot D(Y(a_k), x)$ where now the sum is taken over all the indices $k$ (possibly non terminal). This entails that (116) holds.

Finally, arguing as in the proof of Theorem 2, we get the rank of the family of active constraints at point $x$ is given by the number $s$ of connected component of the undirected saturation graph of $x$. Hence, $x$ belongs to a face of dimension $s$. $\square$

**Remark 31.** Note that Proposition 21 holds for a general directed metric space $\mathcal{L}$ and not just for $(\mathcal{L}, d)$ a probabilistic language model.

We can now find explicit $(min, +)$ expressions for generators of extremal rays corresponding to non principal lower sets.

**Proposition 22.** Let $\mathcal{L}$ be a probabilistic language model with the empty text $a_0$ included. Let $x$ denote an extremal ray corresponding to the lower set generated by $\{b_1, \dots, b_n\}$. Then

$$ (117) \qquad x = \bigoplus_i \log \mathrm{Pr}(b_i) \odot Y(b_i). $$

*Proof.* From Proposition 21 we have

$$ x = \bigoplus_j x(b_j) \odot Y(b_j). $$

From corollary 3 we have $x(b_j) = -\log \frac{1}{\mathrm{Pr}(b_j)} = \log \mathrm{Pr}(b_j)$. This proves the result. $\square$

**Remark 32.** We point out that the terminal elements $b_1, \dots, b_k$ of $S(x)$ function like an orthonormal basis with respect to $D$, namely

$$ (118) \qquad D(Y(b_i), Y(b_j)) = d(b_i, b_j) = \infty \text{ if } i \neq j. $$

So for example if we know that there are $\lambda_j$ such that

$$ x = \bigoplus_j \lambda_j \odot Y(b_j) \text{ Then } D(Y(b_i), x) = \bigoplus_j \lambda_j \odot D(Y(b_i), Y(b_j)) = \lambda_i. $$

**Proposition 23.** Let $T \in [0, \infty)$ be a parameter which will be called temperature. Consider $x \in P(\mathcal{L})$ an extremal ray and let $x = \bigoplus_j D(Y(b_j), x) \odot Y(b_j)$ where $b_j$ are the terminal elements of the saturation graph $S(x)$. Let $v_j := e^{Y(b_j)}$. Then we have

$$ (119) \qquad x = \lim_{T \to 0} -T \log\left(\sum_j e^{-D(Y(b_j),x)/T} e^{Y(b_j)}\right) $$
and therefore, for small $T$

$$ (120) \qquad e^{-x/T} \approx \sum_j e^{-D(Y(b_j),x)/T} v_j $$

*Proof.* Recall the identity

$$ (121) \qquad \lim_{T \to 0} -T \log(e^{-y/T} + e^{-z/T}) = \min\{y, z\}. $$

If $x \in P(\mathcal{L})$, by the previous proposition $x = \oplus_j D(Y(b_j), x) \odot Y(b_j)$ where $b_j$ are terminal elements. Then we have

$$ (122) \qquad x = \lim_{T \to 0} -T \log \sum_j e^{-D(Y(b_j),x)/T} e^{Y(b_j)} $$

and if we put $v_j := e^{Y(b_j)}$, then for small $T$, we get

$$ (123) \qquad e^{-x/T} \approx \sum_j e^{-D(Y(b_j),x)/T} v_j . \qquad \square $$

## 8. $P^{-}(\mathcal{L})$ AS THE LATTICE COMPLETION OF THE ISBELL COMPLETION

We have seen that $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ generalize the lower set and upper set completions respectively from the poset $\mathcal{L}$ to the directed metric space $(\mathcal{L}, d)$, at least in the case where $\mathcal{L}$ contains the empty text $a_0$ which is the bottom element.

However there is another completion of a poset, called the Dedekind-MacNeille completion (which also generalizes the so called notion of formal concepts).

It is known that the generalization of the Dedekind MacNeille completion from posets to directed metric spaces is the so called Isbell completion, which is the fixed part of the Isbell adjunction.

This is also relevant to our situation as it turns out to be defined by $d_{\max}$.

This was studied in [11, 21] with $[0, \infty]$ coefficients. In that case the Isbell completion is identified with the directed tight span of Hirai and Koichi [7].

We will instead define the Isbell adjunction using the extended semi ring $[-\infty, \infty]$ as we did with the $d_{\min}$ adjunction in section 6.

Recall that in section 6 Remark 25 we explained the conventions for working with $(\min, +)$ and $(\max, +)$ on $[-\infty, \infty]$. We use the same here.

Given $x : \mathcal{L} \to [-\infty, \infty]$ and $y : \mathcal{L} \to [-\infty, \infty]$ define $d_{\max}$ and $d_{\max}^t$ by

$$ (124) \qquad d_{\max}(x)_i := \max_j \{d_{i,j} + x_j\} \quad \text{and} \quad d_{\max}^t(y)_j := \max_i \{d_{i,j} + y_i\} $$

Extending the definition in [11, 21] by using $[-\infty, \infty]$ coefficients we have that

**Definition 12.** The Isbell adjunction is the pair of maps $(L, R)$ defined as follows. If $x : \mathcal{L} \to [-\infty, \infty]$ and $y : \mathcal{L} \to [-\infty, \infty]$ then

$$ (125) \qquad L(x) := d_{\max}(-x) \quad \text{and} \quad R(x) := d_{\max}^t(-y) $$

Or in coordinates

$$ (126) \qquad L(x)_i := \max_j \{d_{i,j} - x_j\} \quad \text{and} \quad R(x)_j := \max_i \{d_{i,j} - y_i\} $$

Recall from section 6 that the Funk metric $D$, is still well defined by $D(x, y) := \max_i \{y_i - x_i | x_i \neq \infty\}$. We also denoted by $D^t$ the transpose metric with $D^t(x, y) := D(y, x)$.
and therefore, for small $T$

$$ (120) \qquad e^{-x/T} \approx \sum_j e^{-D(Y(b_j),x)/T} v_j $$

*Proof.* Recall the identity

$$ (121) \qquad \lim_{T \to 0} -T \log(e^{-y/T} + e^{-z/T}) = \min\{y, z\}. $$

If $x \in P(\mathcal{L})$, by the previous proposition $x = \oplus_j D(Y(b_j), x) \odot Y(b_j)$ where $b_j$ are terminal elements. Then we have

$$ (122) \qquad x = \lim_{T \to 0} -T \log \sum_j e^{-D(Y(b_j),x)/T} e^{Y(b_j)} $$

and if we put $v_j := e^{Y(b_j)}$, then for small $T$, we get

$$ (123) \qquad e^{-x/T} \approx \sum_j e^{-D(Y(b_j),x)/T} v_j . \qquad \square $$

## 8. $P^{-}(\mathcal{L})$ AS THE LATTICE COMPLETION OF THE ISBELL COMPLETION

We have seen that $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ generalize the lower set and upper set completions respectively from the poset $\mathcal{L}$ to the directed metric space $(\mathcal{L}, d)$, at least in the case where $\mathcal{L}$ contains the empty text $a_0$ which is the bottom element.

However there is another completion of a poset, called the Dedekind-MacNeille completion (which also generalizes the so called notion of formal concepts).

It is known that the generalization of the Dedekind MacNeille completion from posets to directed metric spaces is the so called Isbell completion, which is the fixed part of the Isbell adjunction.

This is also relevant to our situation as it turns out to be defined by $d_{\max}$.

This was studied in [11, 21] with $[0, \infty]$ coefficients. In that case the Isbell completion is identified with the directed tight span of Hirai and Koichi [7].

We will instead define the Isbell adjunction using the extended semi ring $[-\infty, \infty]$ as we did with the $d_{\min}$ adjunction in section 6.

Recall that in section 6 Remark 25 we explained the conventions for working with $(\min, +)$ and $(\max, +)$ on $[-\infty, \infty]$. We use the same here.

Given $x : \mathcal{L} \to [-\infty, \infty]$ and $y : \mathcal{L} \to [-\infty, \infty]$ define $d_{\max}$ and $d_{\max}^t$ by

$$ (124) \qquad d_{\max}(x)_i := \max_j \{d_{i,j} + x_j\} \quad \text{and} \quad d_{\max}^t(y)_j := \max_i \{d_{i,j} + y_i\} $$

Extending the definition in [11, 21] by using $[-\infty, \infty]$ coefficients we have that

**Definition 12.** The Isbell adjunction is the pair of maps $(L, R)$ defined as follows. If $x : \mathcal{L} \to [-\infty, \infty]$ and $y : \mathcal{L} \to [-\infty, \infty]$ then

$$ (125) \qquad L(x) := d_{\max}(-x) \quad \text{and} \quad R(x) := d_{\max}^t(-y) $$

Or in coordinates

$$ (126) \qquad L(x)_i := \max_j \{d_{i,j} - x_j\} \quad \text{and} \quad R(x)_j := \max_i \{d_{i,j} - y_i\} $$

Recall from section 6 that the Funk metric $D$, is still well defined by $D(x, y) := \max_i \{y_i - x_i | x_i \neq \infty\}$. We also denoted by $D^t$ the transpose metric with $D^t(x, y) := D(y, x)$.
**Remark 33.** Note that

$$ (127) \qquad L(x)_i := \max_j \{d_{i,j} - x_j\} = D(x, d(a_i, -)) = D(x, \hat{Y}(a_i)) $$

and

$$ (128) \qquad R(y)_j := \max_i \{d_{i,j} - y_i\} = D(y, d(-, a_j)) = D(y, Y(a_j)) $$

The pair $(L, R)$ forms an adjunction in the categorical or metric sense:

**Proposition 24.** If $x : \mathcal{L} \to [-\infty, \infty]$ and $y : \mathcal{L} \to [-\infty, \infty]$ then we have $D^t(Lx, y) = D(x, Ry)$.

*Proof.* $D^t(Lx, y) = D(y, Lx) = \max_i\{\max_j\{d_{i,j} - x_j\} - y_i\} = \max_j\{\max_i\{d_{i,j} - y_i\} - x_j\} = D(x, Ry). \quad \square$

We now have the following

**Proposition 25.** We have $LRL = L$ and $RLR = R$ which implies that $LR$ and $RL$ are idempotent.

*Proof.* This follows from the fact that $D^t(Lx, y) = D(x, Ry)$. Indeed $D^t(LRLx, Lx) = D(RLx, RLx) = 0$ and $D^t(Lx, LRLx) = D(RLx, RLx) = 0$. Therefore $LRLx = Lx$. The equality $RLR = R$ is shown analogously. $\square$

Let us now compute the fixed parts of the adjunction $Fix(LR)$ and $Fix(RL)$.

**Proposition 26.** We have $Fix(LR) = \mathrm{Im}(L) = \mathrm{Im}(d_{\max})$ and $Fix(RL) = \mathrm{Im}(R) = \mathrm{Im}(d_{\max}^t)$.

*Proof.* This follows from the fact that $LRL = L$. Indeed clearly $\mathrm{Im}(L) \subset \mathrm{Fix}(LR)$. Moreover $\mathrm{Fix}(LR) \subset \mathrm{Im}(L)$ since $LR(y) = y$ says that $y \in \mathrm{Im}(L)$. $\square$

As before we have the following

**Proposition 27.** We have that

$$ L : \mathrm{Fix}(RL) = \mathrm{Im}(R) = \mathrm{Im}(d_{\max}^t) \to \mathrm{Fix}(LR) = \mathrm{Im}(L) = \mathrm{Im}(d_{\max}) $$

and

$$ R : \mathrm{Fix}(LR) = \mathrm{Im}(L) = \mathrm{Im}(d_{\max}) \to \mathrm{Fix}(RL) = \mathrm{Im}(R) = \mathrm{Im}(d_{\max}^t) $$

are anti-isomorphisms. In other words they are one to one and onto and inverses. They are isometries, namely $D(Lx, Lx') = D^t(x, x')$. Finally we have

$$ (129) \qquad L(\lambda \odot x) = -\lambda \odot L(x) \quad \text{and} \quad L(x \oplus_{\min} y) = L(x) \oplus_{\max} L(y) $$

and similarly for $R$.

*Proof.* First let us check that $L$ and $R$ are one to one and onto. Consider $x, x' \in \mathrm{Fix}(RL)$. If $L(x) = L(x')$ then $RL(x) = RL(x')$ and therefore $x = x'$. Also if $y \in \mathrm{Fix}(LR)$ then $y = L(R(y))$.

$$ \text{Moreover } D(Lx, Lx') = D_{op}(RLx, x') = D_{op}(x, x') = D(x', x) $$

Next we check the tropical antilinearity.

$$ L(\lambda \odot x)_i = \max_j \{d_{i,j} - \lambda - x_j\} = \max_j \{d_{i,j} - x_j\} - \lambda = L(x)_i - \lambda = (-\lambda \odot L(x))_i $$

Moreover

$$ L(x \oplus_{\min} y)_i = \max_j \{d_{i,j} - \min\{x_j, y_j\}\} = \max_j \{d_{i,j} + \max\{-x_j, -y_j\}\} = \max_j \{\max\{d_{i,j} - x_j, d_{i,j} - y_j\}\} = \max\{\max_j\{d_{i,j} - x_j\}, \max_j\{d_{i,j} - y_j\}\} = (L(x) \oplus_{\max} L(y))_i. \quad \square $$
**Remark 33.** Note that

$$ (127) \qquad L(x)_i := \max_j \{d_{i,j} - x_j\} = D(x, d(a_i, -)) = D(x, \hat{Y}(a_i)) $$

and

$$ (128) \qquad R(y)_j := \max_i \{d_{i,j} - y_i\} = D(y, d(-, a_j)) = D(y, Y(a_j)) $$

The pair $(L, R)$ forms an adjunction in the categorical or metric sense:

**Proposition 24.** If $x : \mathcal{L} \to [-\infty, \infty]$ and $y : \mathcal{L} \to [-\infty, \infty]$ then we have $D^t(Lx, y) = D(x, Ry)$.

*Proof.* $D^t(Lx, y) = D(y, Lx) = \max_i \{\max_j \{d_{i,j} - x_j\} - y_i\} = \max_j \{\max_i \{d_{i,j} - y_i\} - x_j\} = D(x, Ry). \quad \square$

We now have the following

**Proposition 25.** We have $LRL = L$ and $RLR = R$ which implies that $LR$ and $RL$ are idempotent.

*Proof.* This follows from the fact that $D^t(Lx, y) = D(x, Ry)$. Indeed $D^t(LRLx, Lx) = D(RLx, RLx) = 0$ and $D^t(Lx, LRLx) = D(RLx, RLx) = 0$. Therefore $LRLx = Lx$. The equality $RLR = R$ is shown analogously. $\square$

Let us now compute the fixed parts of the adjunction $Fix(LR)$ and $Fix(RL)$.

**Proposition 26.** We have $Fix(LR) = \mathrm{Im}(L) = \mathrm{Im}(d_{\max})$ and $Fix(RL) = \mathrm{Im}(R) = \mathrm{Im}(d_{\max}^t)$.

*Proof.* This follows from the fact that $LRL = L$. Indeed clearly $\mathrm{Im}(L) \subset \mathrm{Fix}(LR)$. Moreover $\mathrm{Fix}(LR) \subset \mathrm{Im}(L)$ since $LR(y) = y$ says that $y \in \mathrm{Im}(L)$. $\square$

As before we have the following

**Proposition 27.** We have that

$$ L : \mathrm{Fix}(RL) = \mathrm{Im}(R) = \mathrm{Im}(d_{\max}^t) \to \mathrm{Fix}(LR) = \mathrm{Im}(L) = \mathrm{Im}(d_{\max}) $$

and

$$ R : \mathrm{Fix}(LR) = \mathrm{Im}(L) = \mathrm{Im}(d_{\max}) \to \mathrm{Fix}(RL) = \mathrm{Im}(R) = \mathrm{Im}(d_{\max}^t) $$

are anti-isomorphisms. In other words they are one to one and onto and inverses. They are isometries, namely $D(Lx, Lx') = D^t(x, x')$. Finally we have

$$ (129) \qquad L(\lambda \odot x) = -\lambda \odot L(x) \quad \text{and} \quad L(x \oplus_{\min} y) = L(x) \oplus_{\max} L(y) $$

and similarly for $R$.

*Proof.* First let us check that $L$ and $R$ are one to one and onto. Consider $x, x' \in \mathrm{Fix}(RL)$. If $L(x) = L(x')$ then $RL(x) = RL(x')$ and therefore $x = x'$. Also if $y \in \mathrm{Fix}(LR)$ then $y = L(R(y))$.

$$ \text{Moreover } D(Lx, Lx') = D_{op}(RLx, x') = D_{op}(x, x') = D(x', x) $$

Next we check the tropical antilinearity.

$$ L(\lambda \odot x)_i = \max_j \{d_{i,j} - \lambda - x_j\} = \max_j \{d_{i,j} - x_j\} - \lambda = L(x)_i - \lambda = (-\lambda \odot L(x))_i $$

Moreover

$$ L(x \oplus_{\min} y)_i = \max_j \{d_{i,j} - \min\{x_j, y_j\}\} = \max_j \{d_{i,j} + \max\{-x_j, -y_j\}\} = \max_j \{\max\{d_{i,j} - x_j, d_{i,j} - y_j\}\} = \max\{\max_j \{d_{i,j} - x_j\}, \max_j \{d_{i,j} - y_j\}\} = (L(x) \oplus_{\max} L(y))_i. \quad \square $$
We have that the tropical linear space $\mathrm{Im}(d_{\max})$ is anti isomorphic to $\mathrm{Im}(d_{\max}^t)$ by the two inverse maps

$$R : \mathrm{Im}(d_{\max}) \to \mathrm{Im}(d_{\max}^t) \text{ and } L : \mathrm{Im}(d_{\max}^t) \to \mathrm{Im}(d_{\max}).$$

**Proposition 28.** The Yoneda isometric embedding $Y : \mathcal{L} \to P(\mathcal{L})$ given by $Y(a) := d(-, a)$ and the co-Yoneda isometric embedding $\tilde{Y} : \mathcal{L} \to \tilde{P}(\mathcal{L})$ given by $\tilde{Y}(a) := d(a, -)$, are compatible with the anti-isomorphisms $L$ and $R$ above, in the sense that

$$(130) \qquad \hat{Y}(a) = R(Y(a)) \text{ and } Y(a) = L(\hat{Y}(a)).$$

*Proof.* We have $d_{i,j} \le d_{i,k} + d_{k,j}$, therefore

$$L(\hat{Y}(a_k))_i = L(d(a_k, -))_i = \max_j\{d_{ij} - d_{k,j}\} = d_{i,k} = d((-, a_k)_i = Y(a_k)_i.$$

Analogously

$$R(Y(a_k))_j = R(d(-, a_k))_j = \max_i\{d_{ij} - d_{i,k}\} = d_{k,j} = d((a_k, -)_j = \hat{Y}(a_k)_j . \quad \square$$

As mentioned earlier according to a theorem of Willerton [21]

**Theorem 5.** The directed tight span of Hirai and Koichi [7] is the same as the fixed parts of the Isbell adjunction when using $[0, \infty]$ coefficients and the truncated max operations.

We denote the Isbell completion with $[-\infty, \infty]$ coefficients by $\tilde{I}(\mathcal{L})$.

Let us finally explore the relation between $P(\mathcal{L}) = \mathrm{Im}(d_{\min})$ and $\tilde{I}(\mathcal{L}) = \mathrm{Im}(d_{\max})$.

**Proposition 29.** The polyhedron $P(\mathcal{L}) = \mathrm{Im}(d_{\min})$ is the lattice completion of $\mathrm{Im}(d_{\max})$ when using $(-\infty, \infty]$ coefficients and $P^-(\mathcal{L})$ is the lattice completion of $\mathrm{Im}(d_{\max})$ when using $[-\infty, \infty]$ coefficients.

*Proof.* Recall that since $d_{\min}^2 = d_{\min}$ we have $\mathrm{Im}(d_{\min}) = \{x|dx = x\}$. Moreover if $\mathrm{Id}$ is the $(\min, +)$ identity matrix, namely $\mathrm{Id}_{i,i} = 0$ and $\mathrm{Id}_{i,j} = \infty$ for $i \ne j$, then, since $d_i = 0$, we have $d \le \mathrm{Id}$ and therefore we always have $dx \le x$.

It follows that $x = dx \iff x \le dx$.

We want to show that if $x \le dx$ and $y \le dy$ then

$$\max\{x, y\} \le d(\max\{x, y\})$$

which will imply that $\max\{x, y\} \in \mathrm{Im}(d_{\min})$.

Indeed if $\max\{x, y\} = x$ then $d(\max\{x, y\}) = dx \ge x \ge y$ and if $\max\{x, y\} = y$ then $d(\max\{x, y\}) = dy \ge y \ge x$, therefore $x \le d(\max\{x, y\})$ and $y \le d(\max\{x, y\})$ which implies that $\max\{x, y\} \le d(\max\{x, y\})$.

We have shown therefore that $\mathrm{Im}(d_{\min})$ is closed under the max operation. Both $\mathrm{Im}(d_{\min})$ and $\mathrm{Im}(d_{\max})$ are generated by the vectors $d(-, a_i)$ therefore the result is proved. $\square$

**Example 2.** To illustrate the difference between $\mathrm{Im}(d_{\min})$ and $\mathrm{Im}(d_{\max})$ (albeit for a symmetric and finite metric) we provide the following example: Consider the discrete metric on three points

$$(131) \qquad d_2 = \begin{pmatrix} 0 & 1 & 1 \\ 1 & 0 & 1 \\ 1 & 1 & 0 \end{pmatrix}$$
We have that the tropical linear space $\mathrm{Im}(d_{\max})$ is anti isomorphic to $\mathrm{Im}(d_{\max}^t)$ by the two inverse maps

$R : \mathrm{Im}(d_{\max}) \to \mathrm{Im}(d_{\max}^t)$ and $L : \mathrm{Im}(d_{\max}^t) \to \mathrm{Im}(d_{\max})$.

**Proposition 28.** The Yoneda isometric embedding $Y : \mathcal{L} \to P(\mathcal{L})$ given by $Y(a) := d(-, a)$ and the co-Yoneda isometric embedding $\tilde{Y} : \mathcal{L} \to \tilde{P}(\mathcal{L})$ given by $\tilde{Y}(a) := d(a, -)$, are compatible with the anti-isomorphisms $L$ and $R$ above, in the sense that

$$ (130) \qquad \hat{Y}(a) = R(Y(a)) \quad \text{and} \quad Y(a) = L(\hat{Y}(a)). $$

*Proof.* We have $d_{i,j} \le d_{i,k} + d_{k,j}$, therefore

$$ L(\hat{Y}(a_k))_i = L(d(a_k, -))_i = \max_j\{d_{ij} - d_{k,j}\} = d_{i,k} = d((-, a_k)_i = Y(a_k)_i. $$

Analogously

$$ R(Y(a_k))_j = R(d(-, a_k))_j = \max_i\{d_{ij} - d_{i,k}\} = d_{k,j} = d((a_k, -)_j = \hat{Y}(a_k)_j . \quad \square $$

As mentioned earlier according to a theorem of Willerton [21]

**Theorem 5.** The directed tight span of Hirai and Koichi [7] is the same as the fixed parts of the Isbell adjunction when using $[0, \infty]$ coefficients and the truncated max operations.

We denote the Isbell completion with $[-\infty, \infty]$ coefficients by $\tilde{I}(\mathcal{L})$.

Let us finally explore the relation between $P(\mathcal{L}) = \mathrm{Im}(d_{\min})$ and $\tilde{I}(\mathcal{L}) = \mathrm{Im}(d_{\max})$.

**Proposition 29.** The polyhedron $P(\mathcal{L}) = \mathrm{Im}(d_{\min})$ is the lattice completion of $\mathrm{Im}(d_{\max})$ when using $(-\infty, \infty]$ coefficients and $P^-(\mathcal{L})$ is the lattice completion of $\mathrm{Im}(d_{\max})$ when using $[-\infty, \infty]$ coefficients.

*Proof.* Recall that since $d_{\min}^2 = d_{\min}$ we have $\mathrm{Im}(d_{\min}) = \{x|dx = x\}$. Moreover if $\mathrm{Id}$ is the $(\min, +)$ identity matrix, namely $\mathrm{Id}_{i,i} = 0$ and $\mathrm{Id}_{i,j} = \infty$ for $i \ne j$, then, since $d_i = 0$, we have $d \le \mathrm{Id}$ and therefore we always have $dx \le x$.

It follows that $x = dx \iff x \le dx$.

We want to show that if $x \le dx$ and $y \le dy$ then

$$ \max\{x, y\} \le d(\max\{x, y\}) $$

which will imply that $\max\{x, y\} \in \mathrm{Im}(d_{\min})$.

Indeed if $\max\{x, y\} = x$ then $d(\max\{x, y\}) = dx \ge x \ge y$ and if $\max\{x, y\} = y$ then $d(\max\{x, y\}) = dy \ge y \ge x$, therefore $x \le d(\max\{x, y\})$ and $y \le d(\max\{x, y\})$ which implies that $\max\{x, y\} \le d(\max\{x, y\})$.

We have shown therefore that $\mathrm{Im}(d_{\min})$ is closed under the max operation. Both $\mathrm{Im}(d_{\min})$ and $\mathrm{Im}(d_{\max})$ are generated by the vectors $d(-, a_i)$ therefore the result is proved. $\square$

**Example 2.** To illustrate the difference between $\mathrm{Im}(d_{\min})$ and $\mathrm{Im}(d_{\max})$ (albeit for a symmetric and finite metric) we provide the following example: Consider the discrete metric on three points

$$ (131) \qquad d_2 = \begin{pmatrix} 0 & 1 & 1 \\ 1 & 0 & 1 \\ 1 & 1 & 0 \end{pmatrix} $$
The associated (min,+)-module $\mathcal{P}(d_2)$ is shown on Figure 3, left and the (max, +) module on the right.

<figure><img src="image_8.png" /><figcaption>FIGURE 3. Tropical module generated by the discrete metric $d_2$ of Equation (131). The pseudo-vertices (vertices of the polyhedral complex that do not arise from tropical generators) are shown in gray. (left) The (max,+)-span (right).</figcaption></figure>

## 9. SOME COMMENTS ABOUT PROBABILISTIC LANGUAGE MODELS

We would finally like to gather some comments about how to interpret probabilistic language models ($\mathcal{L}, \leq, \text{Pr}$) and what they imply. Some of these were stated already in section Section 4.1

(1) We note that the construction of $\mathcal{P}(\mathcal{L})$ explains why it is natural to have vectors in a problem of language. In fact we naturally get Boltzmann weighted linear combinations Equation (90), Equation (62), Equation (62) which is what is introduced by hand in the attention layers of the transformer and the final layer where the distribution over possible next words is determined

(2) If the transformer is learning $\hat{P}(\mathcal{L})$ or equivalently $\hat{Q}(\mathcal{L})$ it would be learning a convex body which could explain why its training is efficient in the first place.

(3) Assuming that the transformer is learning the polyhedron $\hat{P}(\mathcal{L})$ or equivalently $\hat{Q}(\mathcal{L})$ it would be learning an effective representation of Yoneda embeddings of texts. This can then be interpreted as solving the huge (min, +) linear systems in Equation (62), Equation (63), (Proposition 11).

(4) The duality explained in section 6 between $\mathcal{P}(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ shows how to resolve the paradox that both $d(-, a_k)$ and $d(a_k, -)$ should equally well encode the meaning of a text $a_k$, given a probabilistic language model ($\mathcal{L}, \leq, \text{Pr}$). This is most striking when $a_k$ is a single word. In that case $d(-, a_k)$ is supported only on $a_k$, but we have that $-d(-, a_k) = -d(a_k, -)$. This was explained in Section 6, Proposition 20, Remark 30. It also showcases the notion that the meaning of a text $a_k$ is not just encoded by $d(-, a_k)$ or $d(a_k, -)$ but by the whole ambient spaces $\mathcal{P}(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ respectively.

## APPENDIX A. CATEGORICAL INTERPRETATION

The metric polyhedra ($\mathcal{P}(\mathcal{L}), \mathcal{D}$) and ($\hat{\mathcal{P}}(\mathcal{L}), \mathcal{D}^t$), as well as the polyhedral cones $\mathcal{Q}(\mathcal{L})$ and $\hat{\mathcal{Q}}(\mathcal{L})$, arise from a categorical point of view [11, 21, 1]. In fact all
The associated (min,+)-module $\mathcal{P}(d_2)$ is shown on Figure 3, left and the (max,+)-module on the right.

<figure><img src="image_14.png" /><figcaption>FIGURE 3. Tropical module generated by the discrete metric $d_2$ of Equation (131). The pseudo-vertices (vertices of the polyhedral complex that do not arise from tropical generators) are shown in gray. (left) The (max,+)-span (right).</figcaption></figure>

## 9. SOME COMMENTS ABOUT PROBABILISTIC LANGUAGE MODELS

We would finally like to gather some comments about how to interpret probabilistic language models ($\mathcal{L}$, $\leq$, Pr) and what they imply. Some of these were stated already in section Section 4.1

(1) We note that the construction of $\mathcal{P}(\mathcal{L})$ explains why it is natural to have vectors in a problem of language. In fact we naturally get Boltzmann weighted linear combinations Equation (90), Equation (62), Equation (62) which is what is introduced by hand in the attention layers of the transformer and the final layer where the distribution over possible next words is determined

(2) If the transformer is learning $\hat{P}(\mathcal{L})$ or equivalently $\hat{Q}(\mathcal{L})$ it would be learning a convex body which could explain why its training is efficient in the first place.

(3) Assuming that the transformer is learning the polyhedron $\hat{P}(\mathcal{L})$ or equivalently $\hat{Q}(\mathcal{L})$ it would be learning an effective representation of Yoneda embeddings of texts. This can then be interpreted as solving the huge (min,+)-linear systems in Equation (62), Equation (63), (Proposition 11).

(4) The duality explained in section 6 between $\mathcal{P}(\mathcal{L})$ and $\hat{\mathcal{P}}(\mathcal{L})$ shows how to resolve the paradox that both $d(-, a_k)$ and $d(a_k, -)$ should equally well encode the meaning of a text $a_k$, given a probabilistic language model ($\mathcal{L}$, $\leq$, Pr). This is most striking when $a_k$ is a single word. In that case $d(-, a_k)$ is supported only on $a_k$, but we have that $-d(-, a_k) = -d(a_k, -)$. This was explained in Section 6, Proposition 20, Remark 30. It also showcases the notion that the meaning of a text $a_k$ is not just encoded by $d(-, a_k)$ or $d(a_k, -)$ but by the whole ambient spaces $\mathcal{P}(\mathcal{L})$ and $\hat{\mathcal{P}}(\mathcal{L})$ respectively.

## APPENDIX A. CATEGORICAL INTERPRETATION

The metric polyhedra ($\mathcal{P}(\mathcal{L})$, $D$) and ($\hat{\mathcal{P}}(\mathcal{L})$, $D^t$), as well as the polyhedral cones $\mathcal{Q}(\mathcal{L})$ and $\hat{\mathcal{Q}}(\mathcal{L})$, arise from a categorical point of view [11, 21, 1]. In fact all
constructions have a categorical interpretations and we will briefly explain these here.

To begin with, we can consider the probabilistic language model ($\mathcal{L}$, $d$) to be a category enriched over the monoidal closed category ($-\infty$, $\infty$], with monoidal structure given by addition and Hom given by considering ($-\infty$, $\infty$] as a poset with the opposite of the usual order of numbers. The Hom between objects $a_i$ and $a_j$ in $\mathcal{L}$, is $d(a_i, a_j)$ and the triangle inequality is the composition of morphisms.

This construction (using $[0, \infty]$ instead of $(-\infty, \infty]$) was explained in [1])

Then $P(\mathcal{L})$ is the category of presheaves on $\mathcal{L}$ namely the category of enriched functors $\mathcal{L}^{\text{OP}} \to (-\infty, \infty]$ where $(-\infty, \infty]$) is considered as a category enriched over itself with internal Hom given by the directed metric $d_{\mathbb{R}}$ on $(-\infty, \infty]$ where $d_{\mathbb{R}}(s, t) = t-s$. This follows from the fact that we can think of the points $x \in P(\mathcal{L})$ as non-expansive functions on $\mathcal{L}$ as we have seen in Proposition 4. Indeed

$$ (132) \qquad P(\mathcal{L}) = \{x : (\mathcal{L}, d^t) \to ((-\infty, \infty], d_{\mathbb{R}})|x \text{ is non-expansive.}\} $$

Moreover, the Funk directed metric $D$ on $P(\mathcal{L})$ is the Hom on presheaves.

The isometric embedding $Y : \mathcal{L} \hookrightarrow P(\mathcal{L})$ is the Yoneda embedding, $Y(a_k)$ is a representable presheaf and the fact that $x_i = x(a_i) = D(Y(a_i), x)$, is the Yoneda lemma.

On the other hand $\hat{P}(\mathcal{L})$ is the category of co-presheaves and $\hat{Y}$ is the co-Yoneda embedding. The tropical anti-isomorphims between $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ as already explained follows from an adjunction between $A(x) := d_{\min}(-x)$ and $B(y) = d_{\min}^t(-y)$.

Finally it was proven in [21] that the directed tight span $\mathrm{DTS}(\mathcal{L})$ (defined in [7]) is the Isbell completion, with $[0, \infty]$ as enriching category, of the enriched category $\mathcal{L}$. Namely the fixed part of the Isbell adjunction which is given by $(L(x))_i := \max_j\{d_{i,j} - x_j\}$ and $(R(y))_j := \max_i\{d_{i,j} - y_i\}$ (where we use truncated difference so the result is always positive). We instead define the Isbell adjunction with enriching category $[-\infty, \infty]$.

The fact that the category of presheaves $P(\mathcal{L})$ is the $(\min, +)$ span of the images of the Yoneda embedding reflects the fact that colimits are given by min and every presheaf is a weighted colimit of representables.

On the other hand the Isbell completion is given by presheaves which are weighted limits of representables since limits are given by max and it is smaller that $P(\mathcal{L})$ since in general not every presheaf is such a weighted limit.

## APPENDIX B. SYNTAX TO SEMANTICS AND MORITA EQUIVALENCE

The problem of encoding allowed (with some probability) sequences of symbols, by some mathematical structure can be located in the realm of a very basic duality in mathematics.

Traditionally language has been modeled as a monoid generated by words. We can go from the monoid to a poset by considering the monoid as a category with one object and arrows corresponding to texts and constructing the factorization category (also called the twisted arrow category). This produces exactly the poset of texts with the subtext order as we have used in our probabilistic language model.

Considering the subtext poset makes it easier to add probabilities and we are led naturally to the probabilistic language model we defined which is a special case of a directed metric space. In Appendix A we saw that this is an enriched category.
constructions have a categorical interpretations and we will briefly explain these here.

To begin with, we can consider the probabilistic language model ($\mathcal{L}$, $d$) to be a category enriched over the monoidal closed category ($-\infty$, $\infty$], with monoidal structure given by addition and Hom given by considering ($-\infty$, $\infty$] as a poset with the opposite of the usual order of numbers. The Hom between objects $a_i$ and $a_j$ in $\mathcal{L}$, is $d(a_i, a_j)$ and the triangle inequality is the composition of morphisms.

This construction (using $[0, \infty]$ instead of $(-\infty, \infty]$) was explained in [1])

Then $P(\mathcal{L})$ is the category of presheaves on $\mathcal{L}$ namely the category of enriched functors $\mathcal{L}^{\text{OP}} \to (-\infty, \infty]$ where $(-\infty, \infty]$) is considered as a category enriched over itself with internal Hom given by the directed metric $d_{\mathbb{R}}$ on $(-\infty, \infty]$ where $d_{\mathbb{R}}(s, t) = t-s$. This follows from the fact that we can think of the points $x \in P(\mathcal{L})$ as non-expansive functions on $\mathcal{L}$ as we have seen in Proposition 4. Indeed

$$ (132) \qquad P(\mathcal{L}) = \{x : (\mathcal{L}, d^t) \to ((-\infty, \infty], d_{\mathbb{R}})|x \text{ is non-expansive.}\} $$

Moreover, the Funk directed metric $D$ on $P(\mathcal{L})$ is the Hom on presheaves.

The isometric embedding $Y : \mathcal{L} \hookrightarrow P(\mathcal{L})$ is the Yoneda embedding, $Y(a_k)$ is a representable presheaf and the fact that $x_i = x(a_i) = D(Y(a_i), x)$, is the Yoneda lemma.

On the other hand $\hat{P}(\mathcal{L})$ is the category of co-presheaves and $\hat{Y}$ is the co-Yoneda embedding. The tropical anti-isomorphims between $P(\mathcal{L})$ and $\hat{P}(\mathcal{L})$ as already explained follows from an adjunction between $A(x) := d_{\min}(-x)$ and $B(y) = d_{\min}^t(-y)$.

Finally it was proven in [21] that the directed tight span $\text{DTS}(\mathcal{L})$ (defined in [7]) is the Isbell completion, with $[0, \infty]$ as enriching category, of the enriched category $\mathcal{L}$. Namely the fixed part of the Isbell adjunction which is given by $(L(x))_i := \max_j\{d_{i,j} - x_j\}$ and $(R(y))_j := \max_i\{d_{i,j} - y_i\}$ (where we use truncated difference so the result is always positive). We instead define the Isbell adjunction with enriching category $[-\infty, \infty]$.

The fact that the category of presheaves $P(\mathcal{L})$ is the $(\min, +)$ span of the images of the Yoneda embedding reflects the fact that colimits are given by min and every presheaf is a weighted colimit of representables.

On the other hand the Isbell completion is given by presheaves which are weighted limits of representables since limits are given by max and it is smaller that $P(\mathcal{L})$ since in general not every presheaf is such a weighted limit.

## APPENDIX B. SYNTAX TO SEMANTICS AND MORITA EQUIVALENCE

The problem of encoding allowed (with some probability) sequences of symbols, by some mathematical structure can be located in the realm of a very basic duality in mathematics.

Traditionally language has been modeled as a monoid generated by words. We can go from the monoid to a poset by considering the monoid as a category with one object and arrows corresponding to texts and constructing the factorization category (also called the twisted arrow category). This produces exactly the poset of texts with the subtext order as we have used in our probabilistic language model.

Considering the subtext poset makes it easier to add probabilities and we are led naturally to the probabilistic language model we defined which is a special case of a directed metric space. In Appendix A we saw that this is an enriched category.
In the monoid case we consider that the meaning of a text eg “red” is given by
the ideal generated by red which contains all texts containg red.

In the poset case it is the same, where ideals and filters correspond to principal lower and upper sets.

This is a mathematical incarnation of the distributional semantics principle.

Now there is a very general and basic concept of duality in mathematics that in
the commutative case takes the form of a duality between algebra and geometry.

The most basic case is, given a commutative algebra, to consider the space of
(prime) ideals.

This is called the spec and can be thought of as a space on which the algebra of
functions is the commutative algebra we started with. For example if we consider
the algebra $C[x, y]$ of complex polynomials in two variables then prime ideas are
ideals generated by monomials $(x-a)(y-b)$ for any $a, b \in \mathbb{C}$ and therefore the space
of ideals is $\mathbb{C}^2$ i.e. the complex plane. The duality then is between the commutative
algebra $C[x, y]$ and the space of ideals $\mathbb{C}^2$. (This so called spec construction is the
cornerstone of algebraic geometry.)

We can try to extend this kind of duality for monoids, posets, and for our enriched
category. Ideals in a monoid are modules over the monoid and in general we have
to consider modules. Now moving to the case of an algebra, a module over the
algebra (a representation) is a presheaf over the corresponding category. This is
the category with one object and arrows given by the elements of the algebra. In
general in a category the presheaves play the role of modules.

In our case modules i.e. presheaves are the non-expansive maps (Proposition 3)
and the space $P(\mathcal{L})$ is the category of modules (the Hom is given by the metric $D$
as already mentioned in Appendix A).

The original category defines the syntax and the presheaf category can be con-
sidered to reflect semantics (see also [1]).

In fact just like $C[x, y]$ gives coordinates on the space of ideals $\mathbb{C}^2$, we could
think that the language category (the syntax category) provides coordinates on the
category of presheaves (modules) which can be though as the semantic category (in
this particular case, for example because the Hom which is the metric $D$ measures
semantic similarity).

Now since we can translate between languages, namely the semantics of lan-
guages are in some sense the same (approximately) we expect that the categories
of presheaves on different language categories, should be equivalent. This is a well
known notion called *Morita equivalence*. We would then expect that enriched cate-
gories corresponding to different languages should be Morita equivalent. Moreover
in that case there are associated invariants (Hochschild homology) which should be
semantic invariants.

Investigating and developing this, is a future direction of research.

REFERENCES

[1] Tai-Danae Bradley, John Terilla, and Yiannis Vlassopoulos. An enriched category theory of language: From syntax to semantics. *La Matematica*, 1:551–580, 2022. arXiv:2106.07890.

[2] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari-
wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar-
wal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh,
Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz
Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish,
In the monoid case we consider that the meaning of a text eg “red” is given by
the ideal generated by red which contains all texts containg red.

In the poset case it is the same, where ideals and filters correspond to principal lower and upper sets.

This is a mathematical incarnation of the distributional semantics principle.

Now there is a very general and basic concept of duality in mathematics that in
the commutative case takes the form of a duality between algebra and geometry.

The most basic case is, given a commutative algebra, to consider the space of
(prime) ideals.

This is called the spec and can be thought of as a space on which the algebra of
functions is the commutative algebra we started with. For example if we consider
the algebra $C[x, y]$ of complex polynomials in two variables then prime ideas are
ideals generated by monomials $(x-a)(y-b)$ for any $a, b \in \mathbb{C}$ and therefore the space
of ideals is $\mathbb{C}^2$ i.e. the complex plane. The duality then is between the commutative
algebra $C[x, y]$ and the space of ideals $\mathbb{C}^2$. (This so called spec construction is the
cornerstone of algebraic geometry.)

We can try to extend this kind of duality for monoids, posets, and for our enriched
category. Ideals in a monoid are modules over the monoid and in general we have
to consider modules. Now moving to the case of an algebra, a module over the
algebra (a representation) is a presheaf over the corresponding category. This is
the category with one object and arrows given by the elements of the algebra. In
general in a category the presheaves play the role of modules.

In our case modules i.e. presheaves are the non-expansive maps (Proposition 3)
and the space $P(\mathcal{L})$ is the category of modules (the Hom is given by the metric $D$
as already mentioned in Appendix A).

The original category defines the syntax and the presheaf category can be con-
sidered to reflect semantics (see also [1]).

In fact just like $C[x, y]$ gives coordinates on the space of ideals $\mathbb{C}^2$, we could
think that the language category (the syntax category) provides coordinates on the
category of presheaves (modules) which can be though as the semantic category (in
this particular case, for example because the Hom which is the metric $D$ measures
semantic similarity).

Now since we can translate between languages, namely the semantics of lan-
guages are in some sense the same (approximately) we expect that the categories
of presheaves on different language categories, should be equivalent. This is a well
known notion called *Morita equivalence*. We would then expect that enriched cate-
gories corresponding to different languages should be Morita equivalent. Moreover
in that case there are associated invariants (Hochschild homology) which should be
semantic invariants.

Investigating and developing this, is a future direction of research.

REFERENCES

[1] Tai-Danae Bradley, John Terilla, and Yiannis Vlassopoulos. An enriched category theory of language: From syntax to semantics. *La Matematica*, 1:551–580, 2022. arXiv:2106.07890.

[2] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari-
wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar-
wal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh,
Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz
Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish,
Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, *Advances in Neural Information Processing Systems*, volume 33, pages 1877–1901. Curran Associates, Inc., 2020.

[3] G. Cohen, S. Gaubert, and J. P. Quadrat. Duality and separation theorems in idempotent semimodules. *Linear Algebra and Appl.*, 379:395–422, 2004.

[4] M. Develin and B. Sturmfels. Tropical convexity. *Documenta Mathematica*, 9:205–206, 2004.

[5] Andreas W.M Dress. Trees, tight extensions of metric spaces, and the cohomological dimension of certain groups: A note on combinatorial properties of metric spaces. *Advances in Mathematics*, 53(3):321–402, September 1984.

[6] Komei Fukuda and Alain Prodon. Double description method revisited. In *Selected papers from the 8th Franco-Japanese and 4th Franco-Chinese Conference on Combinatorics and Computer Science*, pages 91–111, London, UK, 1996. Springer-Verlag.

[7] Hiroshi Hirai and Shungo Koichi. On tight spans for directed distances. *Annals of Combinatorics*, 16(3):543–569, May 2012.

[8] J. R. Isbell. Six theorems about injective metric spaces. *Commentarii Mathematici Helvetici*, 39(1):65–76, December 1964.

[9] Michael Joswig and Katja Kulas. Tropical and ordinary convexity combined. *Adv. Geom.*, 10(2):333–352, March 2010.

[10] T. Lam and A. Postnikov. Alcoved polytopes. I. *Discrete Comput. Geom.*, 38(3):453–478, 2007.

[11] F. William Lawvere. Metric spaces, generalized logic, and closed categories. *Rendiconti del Seminario Matematico e Fisico di Milano*, 43(1):135–166, December 1973.

[12] Tian Yu Liu, Matthew Trager, Alessandro Achille, Pramuditha Perera, Luca Zancato, and Stefano Soatto. Meaning representations from trajectories in autoregressive models, 2023. arXiv:2310.18348.

[13] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, *Advances in Neural Information Processing Systems*, volume 26. Curran Associates, Inc., 2013. arXiv:1310.4546.

[14] Athanase Papadopoulos and Marc Troyanov. *From Funk to Hilbert geometry*, page 33–67. EMS Press, December 2014.

[15] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. https://api.semanticscholar.org/CorpusID:160025533.

[16] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Improving language understanding by generative pre-training, 2018. https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.

[17] A. Schrijver. *Theory of Linear and Integer Programming*. Wiley, 1986.

[18] R. P. Stanley. Two poset polytopes. *Discrete Comput. Geom.*, 1:9–23, 1986.

[19] Ngoc Mai Tran. Enumerating polytopes. *Journal of Combinatorial Theory, Series A*, 151:1–22, October 2017.

[20] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In *Proceedings of the 31st International Conference on Neural Information Processing Systems*, NIPS’17, page 6000–6010, Red Hook, NY, USA, 2017. Curran Associates Inc. arXiv:1706.03762.

[21] Simon Willerton. Tight spans, Isbell completions and semi-tropical modules. *Theory and Applications of Categories*, 28(22):696–732, 2013.

SG: INRIA AND CMAP, ÉCOLE POLYTECHNIQUE, IP PARIS, CNRS

Email address: Stephane.Gaubert@inria.fr

YV: ATHENA RESEARCH CENTER, INSTITUTE FOR LANGUAGE AND SPEECH PROCESSING, ATHENS, GREECE AND IHES, BURES-SUR-YVETTE, FRANCE

Email address: yvlassop@gmail.com, yvlassop@ihes.fr
Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, *Advances in Neural Information Processing Systems*, volume 33, pages 1877–1901. Curran Associates, Inc., 2020.

[3] G. Cohen, S. Gaubert, and J. P. Quadrat. Duality and separation theorems in idempotent semimodules. *Linear Algebra and Appl.*, 379:395–422, 2004.

[4] M. Develin and B. Sturmfels. Tropical convexity. *Documenta Mathematica*, 9:205–206, 2004.

[5] Andreas W.M Dress. Trees, tight extensions of metric spaces, and the cohomological dimension of certain groups: A note on combinatorial properties of metric spaces. *Advances in Mathematics*, 53(3):321–402, September 1984.

[6] Komei Fukuda and Alain Prodon. Double description method revisited. In *Selected papers from the 8th Franco-Japanese and 4th Franco-Chinese Conference on Combinatorics and Computer Science*, pages 91–111, London, UK, 1996. Springer-Verlag.

[7] Hiroshi Hirai and Shungo Koichi. On tight spans for directed distances. *Annals of Combinatorics*, 16(3):543–569, May 2012.

[8] J. R. Isbell. Six theorems about injective metric spaces. *Commentarii Mathematici Helvetici*, 39(1):65–76, December 1964.

[9] Michael Joswig and Katja Kulas. Tropical and ordinary convexity combined. *Adv. Geom.*, 10(2):333–352, March 2010.

[10] T. Lam and A. Postnikov. Alcoved polytopes. I. *Discrete Comput. Geom.*, 38(3):453–478, 2007.

[11] F. William Lawvere. Metric spaces, generalized logic, and closed categories. *Rendiconti del Seminario Matematico e Fisico di Milano*, 43(1):135–166, December 1973.

[12] Tian Yu Liu, Matthew Trager, Alessandro Achille, Pramuditha Perera, Luca Zancato, and Stefano Soatto. Meaning representations from trajectories in autoregressive models, 2023. arXiv:2310.18348.

[13] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, *Advances in Neural Information Processing Systems*, volume 26. Curran Associates, Inc., 2013. arXiv:1310.4546.

[14] Athanase Papadopoulos and Marc Troyanov. *From Funk to Hilbert geometry*, page 33–67. EMS Press, December 2014.

[15] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. https://api.semanticscholar.org/CorpusID:160025533.

[16] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Improving language understanding by generative pre-training, 2018. https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.

[17] A. Schrijver. *Theory of Linear and Integer Programming*. Wiley, 1986.

[18] R. P. Stanley. Two poset polytopes. *Discrete Comput. Geom.*, 1:9–23, 1986.

[19] Ngoc Mai Tran. Enumerating polytopes. *Journal of Combinatorial Theory, Series A*, 151:1–22, October 2017.

[20] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In *Proceedings of the 31st International Conference on Neural Information Processing Systems*, NIPS’17, page 6000–6010, Red Hook, NY, USA, 2017. Curran Associates Inc. arXiv:1706.03762.

[21] Simon Willerton. Tight spans, Isbell completions and semi-tropical modules. *Theory and Applications of Categories*, 28(22):696–732, 2013.

SG: INRIA AND CMAP, ÉCOLE POLYTECHNIQUE, IP PARIS, CNRS

Email address: Stephane.Gaubert@inria.fr

YV: ATHENA RESEARCH CENTER, INSTITUTE FOR LANGUAGE AND SPEECH PROCESSING, ATHENS, GREECE AND IHES, BURES-SUR-YVETTE, FRANCE

Email address: yvlassop@gmail.com, yvlassop@ihes.fr