SlowGuess/NeurIPS_2023_OCR huggingface dataset:
If  $\Delta_u^t > 1$ , the agent's model is able to get closer to the true parameter  $(\theta^*)$  by purchasing. We use agent  $A$  to illustrate the sale of their parameters  $\dot{\theta}_a^t$  to agent  $B$ . First, we show how agent  $A$  determines bounds for the buyer's valuation  $v_b(\dot{\theta}_a^t)$ . Recall that once agent  $B$  expresses a willingness to purchase, a quote request with their purchased weight  $\beta$  will be communicated to agent  $A$  through the broker. Therefore, when agent  $A$  evaluates self-parameters, the broker provides three pieces of information: the buyer's purchased weight  $\beta$ , agent  $A$ 's purchased weight  $\alpha$ , and the gain-from-trade  $\Delta_a^t$ . In this setting, we have that
Theorem 4.1. Bounds on Buyer's Gain-from-Trade: In the linear model setting, by knowing  $\Delta_{a}^{t}$  and weights  $\alpha$ ,  $\beta$ , agent  $A$  can obtain bounds on the gain-from-trade of agent  $B$  given by:
$$
\left(\frac {1 - \sqrt {\Delta_ {a} ^ {t}} (1 - \alpha)}{(1 - \beta) + \sqrt {\Delta_ {a} ^ {t}} (1 - \alpha - \beta + 2 \alpha \beta)}\right) ^ {2} \leq \Delta_ {b} ^ {t} \leq \left(\frac {1 + \sqrt {\Delta_ {a} ^ {t}} (1 - \alpha)}{(1 - \beta) - \sqrt {\Delta_ {a} ^ {t}} (1 - \alpha - \beta + 2 \alpha \beta)}\right) ^ {2}.
$$
Discussion. Theorem 4.1 states that by knowing information disclosed by the broker (including purchased weights  $\alpha, \beta$ ), the seller agent  $A$  can find the upper and lower bounds of the buyer's gain-from-trade,  $\Delta_b^t$ . This information can then be used to estimate the value that the buyer places on the item. Using the Bayesian optimal-pricing mechanism, and taking into account the known probability distribution, the seller can estimate the price to determine their own virtual valuation.
Furthermore, the optimal scenario occurs when the seller agent values self-parameters exactly as the buyer does. In this case, based on Eq. (1), the broker will set the transfer payment as  $\Delta P_{ab}^{t} = P_{a}^{t} - P_{b}^{t} = v_{b}(\dot{\theta}_{a}^{t}) - v_{a}(\dot{\theta}_{b}^{t})$ . Hence, the transfer payment is equal to the difference between the gain-from-trade of the two agents, where  $\Delta P_{ab}^{t} = \Delta_{a}^{t} - \Delta_{b}^{t}$ .
The proof for Theorem 4.1 is in the Appendix C. We summarize trading steps in a single round for this instantiation above (Algorithm D), where two agents are training and trading parameters for the linear model setting. Here, agents  $A$ ,  $B$  act as a seller and a buyer, respectively.
Next, we study a theoretical analysis for the effectiveness of buying parameters. In particular, we are interested in understanding whether participating in the market leads to faster training convergence (in the worst-case scenario). We show this holds in a simplified setting where agent  $A$  always leads and never purchases parameters, while agent  $B$  always purchases from agent  $A$ . This asymmetry could be due to various reasons including a lack of training resources for agent  $B$ . Here we study a setting for general L-smooth functions, which is more practical, as the broker doesn't need to be knowledgeable about the true parameter  $\theta^*$ . We assume the broker's loss is lower than agents' losses, in particular,  $\hat{\mathcal{L}}_z(\theta) \leq \hat{\mathcal{L}}_a(\theta)$  and  $\hat{\mathcal{L}}_z(\theta) \leq \hat{\mathcal{L}}_b(\theta), \forall \theta \in \mathbb{R}^d$ . We take the gain-from-trade  $\Delta_u^t$  by using the subtraction of empirical loss before and after a trade. We write it as
$$
\Delta_ {u} ^ {t} = \hat {\mathcal {L}} _ {z} (\dot {\theta} _ {u} ^ {t}) - \hat {\mathcal {L}} _ {z} (\bar {\theta} _ {u} ^ {t}), u \in \{a, b \}. \tag {4}
$$
Theorem 5.1. For all agents  $u \in \{a, b, z\}$ , let the loss function  $\hat{\mathcal{L}}_u$  be  $L$ -smooth and let the samples on all agents be drawn from the same distribution  $\mathcal{D}$ . Let  $\mathbf{E}_{\mathcal{D}}[\hat{\mathcal{L}}_u] = \mathcal{L}_u$ , and  $\Delta_b^t = \hat{\mathcal{L}}_z(\dot{\theta}_b^t) - \hat{\mathcal{L}}_z(\bar{\theta}_b^t)$ . Let the algorithm run until round  $T$  with step size  $\eta \in (0, \frac{1}{L})$ , and let  $\delta_b := \min_{t \in [T]} \mathbf{E}[\Delta_b^t]$  and  $\bar{g}_b^2 := \min_{t \in [T]} \mathbf{E}[\| \nabla \hat{\mathcal{L}}_b(\theta_b^t) \|_2^2]$ . Then we have the following results,
a) (Always Trade) If  $\Delta_b^t > 0, \forall t$ , and agent  $B$  always buys (i.e.  $I_b^t = 1, \forall t$ ). Then  $T \geq \frac{2(\mathcal{L}(\theta_b^0) - \mathcal{L}(\theta_b^*))}{n\epsilon^2 + 2\delta_b}$  implies  $\bar{g}_b \leq \epsilon$ .
b) (Never Trade) If the agents never trade i.e.  $(I_a^t = I_b^t = 0, \forall t)$ . Then  $\bar{g}_b \leq \epsilon$  for  $T \geq \frac{2(\mathcal{L}(\theta_b^0) - \mathcal{L}(\theta_b^*))}{\eta \epsilon^2}$ .
Discussion. We show the convergence rate of always trade for agent  $B$  is  $\mathcal{O}\big(1 / (\epsilon^2 +\delta_b)\big)$ , while never trade is  $\mathcal{O}(1 / \epsilon^2)$ . The difference between these two scenarios is due to  $\delta_{b}$  — the minimal gain-from-trade over  $T$  runs. More gain-from-trade implies a smaller  $T$  (i.e. faster convergence) in the worst-case scenario. In addition, when  $\delta_{b}$  is  $\Omega (\epsilon)$ , we can get much better convergence rate of  $\mathcal{O}(1 / \epsilon)$ . Our results in this fundamental setting illustrate that participation in the market can lead to faster convergence when there exists gain-from-trade. The proof for Theorem 5.1 is in the Appendix D.1. In addition to this general setting, we provide convergence analysis for the linear model that we used as a case study in Sec. 4. This analysis also shows a better convergence rate when agents trade parameters. See Appendix D.2 for more details.
We study the proposed framework empirically. Our goals are to validate (i) trading in the proposed framework results in improvements, (ii) these improvements persist even when trading subsets of parameters, (iii) these persist even when agents are trading different models for different tasks, (iv) trading and pricing are viable in competitive settings, and (v) understand the importance of key components of our framework, such as the need for alignment.
6.1 Collaborative Agents
We first conduct experiments in a collaborative setting, where there is no payment for buying parameters. Our goal is to validate the potential improvements from transactions independently of the pricing mechanism, which we explore in a later set of experiments.
6.1.1 Parameter Trading in Neural Networks
Setup. We use MNIST [33], CIFAR10 [34], and TinyImageNet [35] for training MLPs and ResNet20 [36]. Agents have imbalanced datasets where half of the classes contain only  $10\%$  of datapoints. Agents are limited to collecting a part of a dataset, making it difficult for them to achieve satisfactory performance without collaborating and trading parameters to leverage each other's strengths.
Models are trained from different random initializations and batch orders over 60 epochs. Agents trade entire parameter sets and join the market after five epochs. The broker discloses gain-from-trade to agents. Broker aligns parameters [16], then merge.
Agent A	Agent B
Testing Acc. (%)	Testing Acc. (%)
MNIST + MLP	out-of-market	68.50%	72.97%
FedAvg	81.98%	81.98%
w/o alignment	84.64%	84.64%
w alignment	86.96%	86.96%
CIFAR10 + ResNet20	out-of-market	71.14%	70.56%
FedAvg	70.35%	67.85%
w/o alignment	78.31%	78.31%
w alignment	79.90%	79.90%
TinyImageNet + ResNet20	out-of-market	21.67%	15.89%
FedAvg	19.95%	19.33%
w/o alignment	31.28%	31.30%
w alignment	31.69%	31.82%
Table 1: Testing accuracies are reported for each combination of dataset and model. In FedAvg, there is no broker to assist agents in conducting transactions. The interpolated weight is determined solely based on the proportion of data assets. w/o alignment indicates that broker merges parameters via simple interpolation with the optimized purchased weight. In w alignment, the broker aligns parameters by applying [16] and then interpolates.
In addition to the approach that agents train models on their own, we include another baseline method, FedAvg [37], which assumes that there is no broker involved in a trade to help agents align parameters and optimize their purchased weights. In FedAvg, the interpolated weight is determined by the portion of data assets that an agent is endowed with, which is 0.5 in this setting.
Results. Table1 shows the performance of two agents. We find that both agents are able to achieve improved performance by leveraging each other's training expertise, compared to out-of-market agents. Specifically, training and trading ResNet20 with TinyImageNet resulted in improving accuracy by  $+10.02\%$  and  $+15.93\%$ , respectively. We measure two ways to merge parameters for buying: with and without model alignment. With model alignment, the broker is able to merge models more effectively, ultimately enhancing performance for both agents. In addition, compared to FedAvg method, results confirm the significance of having a trusted broker in parameter trading. Without an intermediary broker to facilitate the trade, the performance of purchased weights can be negatively impacted, as evidenced by the results of CIFAR10 + ResNet20 and TinyImageNet + ResNet20.
Finally, Figure 2 displays a comparison of testing loss for the MLP on MNIST. Our results demonstrate that trading parameters results in faster convergence compared to siloed training. Using alignment further helps, as expected.
Figure 2: Testing loss converges the fastest by aligning and interpolating.
6.1.2 Parameter Subsets Trading in Neural Networks
Setup. Next, we explore the potential benefits of trading subsets of parameters. This scenario may take place when agents are interested in only certain components or are restricted by trading budgets. We use the same data endowment as in the preceding configuration. We train two 5-layer MLPs on MNIST and align parameters in each layer to trade.
Results. Table 2 displays the results by trading parameters from different layers. As expected, trading the entire model gives optimal performance (the last row). Trading subsets is helpful but suboptimal. We observe that purchasing parameters from shallow layers, close to the input layers,
Agent A	Agent B
Testing Acc. (%)	Testing Acc. (%)
out-of-market	71.29%	72.54%
layers {3, 4}	66.28%	71.98%
layers {2, 3, 4}	70.73%	73.36%
layers {0, 1, 2}	74.76%	74.16%
layers {0, 1}	78.86%	79.82%
layers {0, 1, 2, 3, 4}	86.96%	86.96%
Table 2: Testing accuracies for trading parameters from different layers in 5-layer MLPs on MNIST.
Figure 3: Both agents earn improvements mutually through trading in the market. We visualize trading results of two linear regression models with a synthesized dataset. Leftmost: trading logs over 100 runs. Second from left: The ratio of squared parameter estimation error between agent  $A$  and agent  $B$ . A red line represents tied performance. Above the red line, agent  $B$  leads, and vice versa. Second from right & rightmost: agent  $A$ 's and agent  $B$ 's learning curve compared to out-of-market agents. Market usage thus produces performance improvements.
offers more benefits than trading with deeper layers. This provides trading guidance for budget-limited agents.
6.1.3 The Effectiveness of Buying Parameters in Controlled Settings
Setup. We use a synthetic dataset to study trading in a fully-controlled environment. We use two linear regression models with dimension  $d = 1000$ . Agent  $A$  has  $n_a = 500$  datapoints in dataset  $(X_{a},Y_{a})$  and  $B$  has  $n_b = 800$  datapoints in dataset  $(X_{b},Y_{b})$ , but the latter's labels are affected by zero-mean Gaussian noise  $(\sigma^2 = 0.5)$ . We assume that the broker knows the true parameter  $\theta^{*}$  and obtains  $(X_{z},Y_{z})$  and has access to  $n_z = 10,000$  datapoints. Both agents start learning function  $f_{a},f_{b}$  with the same initialized weight  $\theta^0$ . We compare the results over 100 runs with agents who obtain the same data endowment but do not participate in the market.
Results. The leftmost Figure  $\boxed{3}$  displays the trading log over 100 runs. Green dots and red dots show whether an agent purchases the other's parameters in a specific run. Next, we show the ratio of squared parameter estimation error between agents. If the ratio is larger than 1 (red dashed line), agent  $B$  leads the market. We see that agent  $A$  leads the market in the first half of runs, making agent  $B$  continue buying parameters from agent  $A$ . At the end of a few runs, agent  $B$  turns to the lead.
The rightmost plots show convergence. If an agent is involved in the market and trades with the other, the convergence rate is faster compared to never trading. This study demonstrates agents' behaviors in the trading runs and validates the effectiveness of buying parameters, leading to more efficient model training. We compute the empirical testing loss at the end. Compared to out-of-market agents, we find that agent  $A$  and agent  $B$  are able to improve testing loss by  $42.84\%$  and  $23.88\%$ , respectively.
6.1.4 Trading Parameters of Models with Different Purposes
Setup. Next, we validate whether trading makes sense even if agents are training different models for different purposes. This scenario is more realistic as it simulates situations where organizations in the market are working on different but potentially related tasks. We model this setting by sweeping the distance between the true parameters  $\theta_{a}^{*},\theta_{b}^{*}$  of two linear regression models to observe how it impacts the benefits of trading.
Results. We record the benefits from trading when compared to an agent who does not participate but has the same data as agent
Figure 4: Trading with parameters that are from related tasks is possible.
$A$ . We measure the relative improvement in empirical testing loss. The results are shown in Figure 4, which indicates that even though the two agents are not training on the same task, agent  $A$  is still able to benefit from trading. Note that the gain exists even when the tasks are quite different (i.e., large distance  $\| \theta_{a}^{*} - \theta_{b}^{*}\|_{2}$ ) but is strongest as the tasks are most closely related.
6.2 Competitive Agents
Finally, we study agents engaging in a competitive scenario. In this case, the transactions require pricing. We validate the proposed pricing mechanism.
6.2.1 The Effectiveness of Bayesian Optimal Pricing Mechanism
Setup. We reuse the synthesized dataset from the collaborative experiment. We set the buyer's valuation to gain-from-trade and estimate the seller's virtual valuation by the lower bound that we can find.
Results. Market prices negotiated by the broker over 100 trading runs are displayed in Figure 5. Based on Eq. 1, the market price is determined by the average of the buyer's valuation and the seller's virtual valuation. To demonstrate market efficiency, we also study the scenario where the seller sets the valuation to be exactly the same as the buyer's. As shown in Figure 3 (second from left: performance ratio), agent  $A$  is initially leading in the first half of the runs, resulting in a higher price for their parameters. However, at the end of a few runs, agent  $B$  takes the lead, causing their parameter price to increase while opponent  $A'$  goes down. It is important to note that the gap between the estimated price (the blue line) and the optimal price (the orange line) can be reduced with more information learned through the market, such as via historical transactions. Besides, the resulting negotiated price creates a discrepancy with the price in the optimal scenario where both parties report their valuations truthfully, highlighting the significance of revealing accurate parameter values and justifying the need for incentives. At last, this study illustrates the feasibility of using the Bayesian optimal pricing mechanism to assist seller agents in monetizing parameters with limited information to make a trade.
Figure 5: We visualize parameter market price negotiated by broker. We can see in the first half of runs, since agent  $A$ 's performance dominates the market, making agent  $A$ 's parameter more valuable compared to opponent agent  $B$ . Note that, if there is no trade, market price remains the same as historical price.
In this paper, we introduced a framework for parameter markets that can serve to reduce the heavy costs of large-scale model training. Borrowing from economic principles, we provided a set of mechanisms that enable the functioning of such markets. Theoretically, for simple settings, we analyzed market efficiency and proved that agents can gain from participating. We empirically validated the effectiveness of parameter markets under collaborative and competitive settings and demonstrated when participants in the market earn mutual benefits through market usage.
We would like to express our gratitude to the Wisconsin Alumni Research Foundation (WARF) for supporting this work. Additionally, we would like to thank all the reviewers for their valuable comments and constructive feedback on our work.
[38] ABDULLAH YAVAS. Economics of brokerage: an overview. Journal of Real Estate Literature, 2(2):169-195, 1994.
[39] Simran Khanuja, Melvin Johnson, and Partha Talukdar. Mergedistill: Merging pre-trained language models using distillation. arXiv preprint arXiv:2106.02834, 2021.
[40] Sebastian Lawrenz, Priyanka Sharma, and Andreas Rausch. Blockchain technology as an approach for data marketplaces. In Proceedings of the 2019 international conference on blockchain technology, pages 55-59, 2019.
[41] Franziska Boenisch. A systematic review on model watermarking for neural networks. Frontiers in big Data, 4:729663, 2021.
[42] Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Olatunj Ruwase, Shaden Smith, Minjia Zhang, Jeff Rasley, et al. Deepspeed-inference: enabling efficient inference of transformer models at unprecedented scale. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1-15. IEEE, 2022.
[43] Jannik Kossen, Sebastian Farquhar, Yarin Gal, and Tom Rainforth. Active testing: Sample-efficient model evaluation. In International Conference on Machine Learning, pages 5753-5763. PMLR, 2021.
[44] Ante Custic, Vladyslav Sokol, Abraham P Punnen, and Binay Bhattacharya. The bilinear assignment problem: complexity and polynomially solvable special cases. Mathematical programming, 166:185-205, 2017.