SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We consider the overparametrized noiseless linear regression, where number of features  $(d)$  is larger than number of acquired training examples  $(n)$  (therefore, we can always interpolate all the training data) and there is no noise in observed targets. Following work of [33] which showed gradient descent on linear regression lead to min L2-norm, we assume local and global models infer the min L2 norm interpolant. As an example, for our running example the min-norm solution interpolating the concept is  $\hat{\theta} = [1,1,0]$ .
B.2 Operationalizing a concept: from disagreement to convergence
An alternate interpretation of the min-norm involves inferring the parameters by taking into account explicit constraints that require  $\hat{\theta}$ 's projection on  $S_{i}^{uno}$  and  $S_{i}^{inv}$  to be zero. For instance, in our
current example, we can deduce the min-norm solution by solving these linear equations:  $([0, 0, 1]\theta = 0, [1, -1, 0]\theta = 0, [1, 1, 0]\theta = 2)$ .
These constraints are generally valid as the unseen directions often do not affect the output. However, these constraints may be violated when we combine local concept data with global data, as the projection of  $S_{i}^{uno}$  and  $S_{0}^{obv}$  may not be zero. This implies that the output could change with variations in the unseen directions, leading to local models typically outperforming global models within a local concept.
To ensure both local and global models perform equally well in the local concept, we need to enforce the invariance constraints explicitly. This involves adding new data that exhibit variations in the unseen directions and demonstrating that these variations do not affect the output. Furthermore, we presume that  $S_{i}^{uno}$  is significantly large, making methods that attempt to examine all possible directions inefficient. Therefore, it's more advantageous to only verify directions that are affected by the merge.
Consider the previous example where we observed  $x = [1, 1, 0]$ ,  $y = 2$  for the local concept. Now, imagine the we observed  $x = [0, 1, 1]$ ,  $y = 2$  in global dataset. When we combine this data point with the concept data point, we get  $\hat{\theta}_{\mathrm{global}} = \left[\frac{2}{3}, \frac{4}{3}, \frac{2}{3}\right]$ . This causes a disagreement in data points that vary in the  $[0, 0, 1]$  direction within the local concept, the local model predicts 0 while the global model predicts  $\frac{2}{3}$ . Note that both the global and local predictions align for variations in the  $[1, 1, 0]$  direction.
In the event of such a disagreement, we have two options: (1) The variation in this direction is indeed non-zero, suggesting the local model requires further refinement - a frequent occurrence in early stages, or (2) The variation is zero, but it needs to be specified as such; otherwise, the global model assumes other values due to its implicit bias towards generating the simplest model. Note that there is no disagreements in the common directions between  $S_0^{uno}$  and  $S_i^{uno}$  or their orthogonal subspaces.
Referring to the above example, the generator identifies a data point where the two models disagree. Let's assume this data point is  $x = [0, 0, 1]$ , where the local model predicts 0, but the global model predicts  $\frac{2}{3}$ . In such a case, we present this data point to the user. Let's assume user specify that the label for this data point is 0. In this case by adding this new data point the global prediction adjusts to  $\hat{\theta}_{\mathrm{global}} = [0, 2, 0]$ .
After we learn the local concept (i.e., all the unobserved directions are indeed zero), how many of them do we need to add as explicit constraints? the following proposition shows maximum number of disagreements after learning a local concept.
Proof. The global and local models agree on all observed directions (i.e.,  $S_i^{obv}$  and  $S_0^{obv}$ ). However, there is a disagreement for any vector  $u$  in  $S_i^{uno}$  such that  $\hat{\theta}_{\mathrm{global}} = \mathrm{proj}_{S_0^{obv}}(\theta^*)^\top u \neq 0$  since  $\hat{\theta}_i^\top u = 0$ . Let's assume we add  $k$  examples such that local and global disagree. We now prove that  $k \leq \dim (\mathrm{proj}_{S_0^{obv}}(S_i^{uno} \cap (S_i^{uno} \cap S_0^{obv})^\perp))$ .
For the  $k$  added examples, only consider their components in  $(S_i^{uno} \cap (S_i^{uno} \cap S_0^{obv})^\perp)$  (we can remove the  $S_i^{obv}$  components by subtracting their projection on  $S_i^{obv}$  similarly remove any component in  $(S_i^{uno} \cap S_0^{obv})$  by subtracting their projection in  $S_0^{obv}$ ). In order to have a disagreement these data points should have non-zero projection on  $S_i^{obv}$  otherwise there will be no disagreements. As a result the maximum number of data points is  $\dim (\mathrm{proj}_{S_0^{obv}}(S_i^{uno} \cap (S_i^{uno} \cap S_0^{obv})^\perp))$ .
B.3 Handling interference between concepts
In previous section, we explained why disagreement can happen between local and global model and how we can resolve the disagreements by querying user of the local concept. We bound number of disagreement with dimension of projection of  $S_{i}^{uno}$  on  $S_{0}^{obv}$ . In previous section we did not need to change  $S_{0}^{obv}$  but when concept  $j$  has conflicts with concept  $i$  we also add data to concept  $j$  (thus changing  $S_{j}^{obv}$ ) which can lead to new conflicts with concept  $i$ .
Figure 5: CoAlign loop for operationalizing a concept. (a) The user starts by providing some seed data from the concept and their labels, they are then used to learn a local concept model. (b) GPT-3 is then prompted to generate new examples, prioritizing examples where the local model disagrees with the global model. (c) Actual disagreements are shown to the user for labeling, and (d) each label improves either the local or the global model. The loop b-c-d is repeated until convergence, i.e. until the user has operationalized the concept and the global model has learned it. Each local model acts as a proxy for the user and step b always check if global and local model are aligned with each other. Therefore, if user on the left changes the global model such that it causes interference with user on the right step b finds such interference and asks the right user to intervene and add more data.
The following proposition state that in addition to the dimension of projection of  $S_{i}^{uno}$  on observed subspace we also need to calculate projection on the unobserved space of different concepts as they might get added in the future. With notation of  $S_{0:k}^{obv}$  denoting sum of all the  $S_{i}^{obv}$ , and  $S_{-i}$  denotes sum of all subspaces except  $i$ , the following proposition bounds number of times users need to add data to their concepts due to interference.
Proposition 2. If for all  $i$ ,  $\mathrm{proj}_{S_i^{uno}}(\theta^\star) = 0$  then the maximum number of times that we need to handle interference is  $\sum_{i=1}^k \dim \left(\mathrm{proj}_{S_{-i}}\left(S_i^{uno} \cap (S_i^{uno} \cap S_{0:k}^{obv})^\perp\right)\right)$ .
Proof. The proof is similar to Proposition 1. Here we need to deal with conflicts with all other topics and since it is possible that we add their unobserved subspace as well we need to compute the dimension of  $S_{i}^{uno}$  on the whole  $S_{j}$  subspace not only  $S_{j}^{obv}$ .
Let assume we added  $t$  example from concept  $i$  to handle interference, we now prove that  $t \leq \dim(\operatorname{proj}_{S_{-i}}(S_i^{uno} \cap (S_i^{uno} \cap S_{0:k}^{obv})^\perp))$ . For every data point that we add we first remove  $S_{0:k}^{obv}$  components by removing its projection on  $S_{0:k}^{obv}$ . Now in order to have a conflict this data point should have non-zero projection on  $S_{-i}$ . As a result the maximum number of data points we can add is less or equal than  $\dim(\operatorname{proj}_{S_{-i}}(S_i^{uno} \cap (S_i^{uno} \cap S_{0:k}^{obv})^\perp))$ , summing over all the concept result in maximum number of interference that needs to be handled.
C Extra figures and tables
Figure 5 is showing the CoAlign loop for multiple users. Table 6 shows the 6 concepts used in our experiment along with examples for each concept.
Concept	CheckList Examples	Roberta-Large accuracy
Synonyms in simple templates	How can I become more vocal?	61%
How can I become more outspoken?
More X = Less antonym(X)	How can I become more optimistic?	0%
How can I become less pessimistic?
X person = not antonym(X) person	How can I become a positive person?	14%
How can I become a person who is not negative?
Orders is irrelevant in symmetric relations	Are tigers heavier than insects?	0%
What is heavier, insects or tigers?
Active / Passive swap	Does Anna love Benjamin?	1.4%
Is Benjamin loved by Anna?
Modifiers changes question intent	Is Mark Wright a photographer?	22%
Is Mark Wright an accredited photographer?
Table 6: The 6 concepts used for debugging Roberta-Large model fine-tuned on QQP dataset.