Train 'n Trade: Foundations of Parameter Markets
Authors: Tzu-Heng Huang
Harit Vishwakarma
Frederic Sala
Venue: NeurIPS 2023 poster (2023)
Keywords: Parameter Market
Pricing
Efficient Model Training
TL;DR: We proposed a parameter marketplace for agents to buy/sell parameters in machine learning models.

Abstract: Organizations typically train large models individually. This is costly and time-consuming, particularly for large-scale foundation models. Such vertical production is known to be suboptimal. Inspired by this economic insight, we ask whether it is possible to leverage others' expertise by trading the constituent parts in models, i.e., sets of weights, as if they were market commodities. While recent advances in aligning and interpolating models suggest that doing so may be possible, a number of fundamental questions must be answered to create viable parameter markets. In this work, we address these basic questions, propose a framework containing the infrastructure necessary for market operations to take place, study strategies for exchanging parameters, and offer means for agents to monetize parameters. Excitingly, compared to agents who train siloed models from scratch, we show that it is possible to mutually gain by using the market, even in competitive settings. This suggests that the notion of parameter markets may be a useful paradigm for improving large-scale model training in the future.

Train 'n Trade: Foundations of Parameter Markets
Tzu-Heng Huang, Harit Vishwakarma, Frederic Sala
Department of Computer Science
University of Wisconsin-Madison
(thuang273, hvishvakarma) edu, fredsalaßcs wisc. edu
Abstract
Organizations typically train large models individually. This is costly and time-conumming, particularly for large-scale foundation models. Such vertical production is known to be suboptimal. Inspired by this economic insight, we ask whether is possible to leverage others' expertise by trading the constituent parts in models, i.e., sets of weights, as if they were market commoditíes. While recent advances in aligning and interpolating models suggest that doing so may be possible, number of fundamental questions must be answered create viable parameter markets. In this work, we address these basic questions, propose a framework containing the infrastructure necessary for market operations to take place, study strategies for exchanging parameters, and offer means for agents to monetize parameters. Excitingly, compared to agents who train siloed models from scratch, we show that it possible to mutually gain by using the market, even in competitive settings. This suggests that the notion of parameter markets may be useful paradigm for improving large-scale model training in the future.
Introduction
Costs to build powerful state-of-the-ane machine learning models, such foundation models (e.g.. GPT-3 [Л] T5 (2], PaLM B. BLOOM-I76B [ and OPT-175B (5)), have increased enormously. These costs can easily reach millions of dollars and even exceed that amount. For example, training the 11 billion-parameter T5 model is estimated to take around 1.3 million dollars for a single training run Unfortunately, few organizations and fewer individuals are sufficiently calittlltd to afford such training costs.
One approach to reduce expense is to broadly distribute training workloads, such as in decentralized training 7 10. However, this is limiting; even the decentralized setting, participants must first agree train shared model and least minimally coordinate the training process. For this reason, such techniques cannot be applied when organizations develop different models for different purposes on different timelines. In these scenarios-the most common solution in large-scale model development-moadik are trained individually regardless of high cost.
A natural question is whether such vertical production can be broken down into parts that can be more easily built, re-used, and exchanged. Taking inspiration from other areas manufacturing, observe that most products not built in vertical silos, but from components that traded in markets. Economic agents, even when competing against each other, buy, sell, and trade such components to leverage the expertise of other agents that production costs can be lowered.
This leads us ask whether subsets of trained weights can thought of constituent parts to be bought and sold parameter markets. Such markets may provide mutual benefits for both buyers and sellers. Buyers are able to purchase well-trained parameter sets directly as commodities to leverage the training expertise of others and then use them improve model performance. Sellers (i.e., owners partially or fully-trained models) are able to monetize parameters as a second profit center, in addition to the downstream activity enabled by using their models.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Challenges and Proposed Approach. How can we build and use such markets? To answer this, we must first overcome several obstacles. An immediate challenge is the notion of alignment: models trained in isolation may not have parameter sets that correspond in any natural way. especially if these models have differing purposes. Excitingly, recent work suggests that is possible to align model B
components and then merge them via linear interpolation [ 12
14 15, 16 17 Similarly, it
known that training data can be potentially recovered from weights or gradients so privacy is an We tackle the orthogonal question:
additional challenge [8 19 20
If model alignment and sufficient privacy can indeed be assured, how can a viable parameter marketplace be designed?
There are three fundamental challenges in doing so:
How should agents (users in the market) decide to perform transactions? Discovering and verifying useful model parameter sets on the market without prior knowledge is challenging for buyers. To address this issue, we introduce a trusted third-party organization, the broker. The broker enables parameters. This is a common approach in the existing works on data marketplaces [21 22 it allows buyers evaluate the quality of data before purchase. Doing so with parameters rather than data presents additional complications that must be resolved.
city-before-purcee
mechanism for buyers to examine the quality of
2.
What rewards may agents gain? An important consideration when using such a framework is whether agents can expect to see any gains. Indeed, if the process of exchanging and validating parameter sets does not yield any improvements when compared to siloed training, there is no reason participate in parameter markets. We provide theoretical and empirical analyses validating the improvements gained from participating in such markets.
3. How to monetize model parameters in competitive marker? In settings where parameters are bought and sold, rather than bartered, it can be challenging to price these assets. Both the seller and the buyer may not have a clear understanding of the other's valuation of the parameters, which makes it difficult for each to maximize their revenues in a trade. To address this issue, apply Bayesian-optimal pricing mechanism of parameter sets and study Nash bargaining 24 to find market prices in negotiation.
2 provide valuation
Results and Contributions. We propose and formulate a viable marketplace to trade parameters for machine learning models. We validate it in both theoretical and empirical settings. Theoretically, in basic scenarios we show how agent training converges faster through purchasing parameters in the market. We offer bounds on the improvement gained via trading when training linear models. Empirically, we conduct experiments effectiveness. We demonstrate that compared to agents who stay outside the market and train models in isolation, participating in parameter trading, even trading subsets of the full set of model parameters, provides benefits efficient model training and better model performance. For example, when training and trading parameters of ResNer20 on Tiny IaggeNe, two agents improve their performance by gaining accuracy improvements of +10.02 % and +15.93% versus separate training. We also demonstrate the success of price estimation to monetize parameters and negotiate prices.
variety of practical scenarios to validate the framework's 's
2 Related Works
First, we describe two related concepts: data and model-as opposed to parameter--marketpaccc. We then give background on model alignment techniques, which are used in our framework.
Data Marketplaces. Data is a key ingredient in machine learning pipelines. There is rich vein of work proposing infrastructure to trade data as commodity [21 22 marketplaces have also been instantiated Services (AWS) Data Exchange, Microsoft's 's Azure Data Marketplace, and Google's Cloud Data Catalog. Such research; however, cannot be directly applied to trading parameters. is relatively easy to evaluate the valuation of a data trade using basic statistical measurements. In contrast, the value of parameters is challenging to measure, as can only be determined after testing and depends on the model's performance.
Such industry, including in services such as Amazon Web
25
26
27
23
29

---

routine to align Model Marketplaces. There have been several efforts to build markets to trade entire model instances trained by centralized broker [QB Two major obstacles that need be overeome are determining the value and pricing models, and safeguarding privacy. To address the former issue, a noise-injection mechanism has been suggested, which involves assessing the accuracy of an optimal model with random Gaussian noise to determine its worth and creating a price-error curve for selling it on the market [30]. The latter issue has been tackled by proposing a system that apply differential privacy while still maximizing revenue BI. In contrast to trading entire model instances for downstream use, parameter markets are far more refined, enabling each user in the market to train their own models for their purposes while gaining from others' training runs.
Model Alignment. Models that are trained with different batch orders or initialization weights may not be properly aligned. Directly merging purchased parameters through interpolation may fail. The process of aligning parameters in model training is therefore critical. Recent studies have explored the geometric relationship between models
13 14 15 and proposed methods for aligning 16 We use such techniques as a building block in our proposed market infrastructure. Through proper model alignments, we expect agents are able find and purchase desired parameter sets in the market.
two sets of neural network parameters
3 A Framework for Parameter Markets
We provide general description of the proposed marketplace and then discuss each component in depth.
3.1 General Marketplace Framework
Prising Purenuenes
Prkdig ParsDittes
Figure [ depicts a two-agent version of the marketplace. Multiple agents training models for potentially different tasks seek to buy sell sets of parameters. Buying well-trained parameter sets enables reaching goal performance faster while selling such parameters produces profits.
Agent
Agent i2
A high-level description of the trading process follows. First, agents send their own parameters to the market !. A third party (the broker) operates a try-before-purchart and merge parameters for buyers 2 The broker privately informs agents of the gains or losses resulting from a potential trade using validation data 3 Based on this information, a buyer values seller's parameters, and then makes a trading decision. If the buyer is willing to purchase, quote request is sent out, and both sides generate and report their valuations to the broker 4 55 The broker helps both parties negotiate the price until they reach an agreement 6. Afterwards, the broker ships parameters to the buyer and transfers the payment to the seller, completing the trade.
Dasazadar
Qastatio
Sand Vhitntions
Try bey-beerre-annee parchese
Price Negotiotice
Figure 1: Overall workflow a two-agent market. Blue and orange blocks represent actions taken by agents and the broker, respectively. In this example, agent A is informed of potential gain through purchasing agent B's parameters. Hence, agent sends quotation request to inquire about purchasing parameters. Then, broker helps both sides negotiate the price of agent B's parameters.
3.2 Market Setup
In the following sections, we discuss each foundational concept in parameter markets. We first fix some notation. For agent u, let D, = {" be the samples drawn from distribution D supported on S. At round t, let 0" E IRd be trained parameters that agent u has access to, and let C. (0t.) be the empirical loss of agent u for the corresponding model measured on data D... The
3

---

empirical loss is defined with the agents' loss function E in a standard fashion,
Nu Σ The i=1
1
(.()%)
E(O Su,e).
While our framework can handle general settings (i.e., any well-trained parameter sets can be traded), we focus on supervised learning for ease of exposition. We assume S = X' x )' where X' and ) are the instance and label spaces respectively. Thus, $u,, = (Iu,s: Fu,),, and also denote Du as (X,, Y,) where X,, is the set of points Tu, and Y, is the set of corresponding labels. We drop the superscript u when it otherwise clear from the context.
Agents and Broker. For simplicity, suppose there are two agents A, B in the market. Agent A has na data points (Xa: Ya), and agent B has n, data points (X,, Y6). There is also trusted third party, the broker Z, helping the two agents operate in the market impartially. The broker has access to validation dataset (X Yz) of size nt with noiseless samples from both distributions.
In the proposed trading framework, the two agents train their models locally first and then trade with each other. At the beginning of round t, agents perform standard gradient descent update using their own datasets. That is, 6% == 0-- (
{a, b), where rj indicates the step size.
Using Purchased Parameters. In the market, parameters can be bought or sold. To use acquired parameters, several model alignment techniques can be employed [16 177 For potential trade, broker aligns seller's parameter sets to match a buyer's. Post-alignment, the parameters can be merged as straightforward convex combination. For instance, if agent A is willing to buy parameters from agent B, and a trade occurs eventually, the post-alignment combination will have the form & == :1 a) + adf, a E (0, 1]. Here a and (for agent B, (9) are weights indicating the portion of trained parameters from the seller side in the merged parameters.
3.3 Try-Before-Purchar Mechanism
However, before making any trading decision, agents are unaware of the potential benefits they can achieve through purchasing. Our proposal is for a neutral broker to implement mechanism which assists agents. It does by helping them align and merge parameters then pre-evaluate their quality by using the broker's dataset (Xe, Yg). In the w--beforr-purchaas mechanism, the broker can pick the optimized weights or 3 for buyer agents by minimizing the empirical loss. We write and B == arg min 2. ((1 v)ge + vτ:).
arg (1 v)0t + vdğ)
Using the optimized purchased weights, the broker calculates and communicates to agents their gain or loss from the trade in confidential manner. We denote the gain-from-trade for agent by A E {a, b). which serves as prior knowledge to help agents make informed decisions about whether to make purchases or not. Generally, the notion of guin-from-tradd relative improvement versus not trading. The trading benefits can be expressed in various ways, such as the difference between agent's 's loss before and after the trade. For example, it might take the form A = 2. (,.) ) - L. (..). Other ways to define the gain-from- from-trade include using the relative improvement ratio the empirical loss improvement ratio the (estimated) parameter recovery error.
to compare
If the trin-from-trade A does not indicate any benefit for buyer agent u, the agent will not be willing to make a purchase. Consequently, no quote request will be sent out, and no trade will take place. In such a scenario, the parameters 03, will remain with agent u until the next round of gradient descent. The final parameters at the end of round I, denoted by 0 can be either 0r or 0r To indicate a trade, use the indicator variable Tr:
{1
if agent u buys parameters, 0 if agent u does not make a purchase.
0 8=:=
(1) 0
1 Øy where 1eg

---

3.4 Valuation and Pricing Mechanism
Once the tuin-from-tradd has been determined and trading decisions have been made, quote requests with purchased weights are circulated throughout the market. Both buyers and sellers begin to assess various parameters in order to negotiate prices. Each agent has their own private valuation function, denoted by Uy Rd + IR+ which quantifies their trading benefits produced by specific parameters 0 E R. The valuation function of each agent is private and is not exposed to others. For instance, agent A's valuation function is denoted by un va(O)) for the value of agent B's parameters for purchasing. and V.. (0%) ) is the value of their own parameters for selling. In order to generate revenue from trade, (f6) is the highest price that agent A is willing to bid for purchase, while Va va((") is the lowest price that agent A is willing to ask for in order to sell.
Once valuations are completed, the broker-in their role as an impartial middleman-assise in bargaining the market price to maximize revenue for both sides. The negotiation process for prices continues until a mutually acceptable market price for parameters is reached. When buyer offers to pay a lower price for certain parameters than the seller is willing to accept, the trade cannot be fulfilled. To analyze this negotiation process, treat it Nash bargaining problem [4] We assume that the broker is knowledgeable about the valuations of both parties (not the valuation functions themselves) and sets the price by maximizing the revenue of both agents impartially. An agent's 's revenue, defined as Un. is derived from two sources: the profit earned from selling self-parameters and the profit gained from buying parameters. We employ the popular Cobb-Dough function to maximize both agents' revenue [32). We denote the market price for agent w's parameters at round / as P E R*, then formulate the problem accordingly. Set (P Pt) == (P' ea (0:) + (va(O)) P2) and Ub( Pg Pd) (Pg vo())) (vb(O)) P.). Then we have the problem
argmax PE
Ua(Pt, Pb) Uo(Pg.PD)
Pd E (va(()))) (b,)
Pj E [vo(6)), va (0)))
s.t.
By solving this problem, the broker can determine the difference in price, denoted by A using
1 APt = Pr. P = 2 (v)(.') + va - v0( 1z (0))
The steps for solving this problem are shown in the Appendix A
The resulting price difference APt. represents the amount of money that needs be transferred between parties.
4 Instantiating the Market: Concrete Examples
Now we give a concrete instantiation of the market that we described in the previous section.
Valuations. There are various ways for agents to define valuation functions (i.e. based on agent's s preference, training budget, or model performance). Here assume that in the market, agents to purchase use gain-from-trane A,, which can be seen as a notion of relative performance improvement, to assess the value of parameters so that u. (0',) A where u' represents their seller agent.
However, assessing the value of eeff-parameters for agent u, who is a seller, is difficult task as there is no clear information available from the broker regarding the quality of such parameters. The best approach for seller to maximize profit is to set price close possible to the buyer's valuation, which is the highest price that the buyer is willing to pay. To arrive at this virtual valuation, we use the Bayesian-optimal pricing mechanism described in Myerson's 's work [3] This mechanism enables the seller to monetize self- saff-prramters. Under this Bayesian mechanism, we assume that the seller is also aware that the buyer's valuation arises from gain- rain-from-trade and that their valuation function is derived from a known probability distribution. We discuss these common priors in Appendix G
Suppose the buyer's valuation has cumulative distribution function F.. If the seller sets a price of P. the probability that the buyer is willing to purchase is P(P < v) = 1 - P( S P) = (P).). The expected revenue for the seller is P x (1 FP))).. Hence, the optimal price to ask for can be
5

---

Algorithm Single Round of Parameter Trading
Input: (Xa, Ya), Y6), (X,. Yz), 0", 0g o0
Output: 86
0g <- 0- -r (0.-),, {a.b)
D agents' local training
0. (1 a1-a) + ab
a = arg min.e L.z ((1 vv)0 + v06)
b broker's my--efoorepurchass
8 = (1 - (1-8)6) + 30
B = arg 1in L. ((1 - v)0i + vd%)
> broker's 's gyy-before-purhhaae
AG [ -9113
u b}
inform agents about gain- from-trradd
[ist --112
if A then
b agent B sends quotation request with agent A's valuation is estimated by the bounds of A
agent A, B provide valuations broker if (6%) 2 va (6,) then
transferred payment for buying 6% after negotiation is set to (vo((t) - va (6%)) /2 return db
ship merged parameters
else
return o
b negotiation fails
else
return 0%
b agent B decides not to buy
found by maximizing the expected revenue, and it satisfies
P - (P))
(2)
FTP)))
Linear Model Case Study. To further illustrate concrete example of the seller's virtual valuation, we consider pricing parameters for training linear models. For simplicity, we assume that the broker knows the parameters thouh this is not necessary in practice, these can be approximated using the broker's validation dataset. Both agents' data is sampled from the same distribution so that true parameters are identical. Additionally, we take the gain-from-trade revealed by broker be the ratio of squared parameter estimation error. We write
# 8"
A
(3)
|
0-
If A 1, the agent's model is able to get closer to the true parameter (0*) by purchasing. We use agent A to illustrate the sale of their parameters O* to agent B. First, we show how agent A determines bounds for the buyer's valuation v% ((%)). Recall that once agent B expresses willingness to purchase, quote request with their purchased weight will be communicated agent through the broker. Therefore, when agent A evaluates self- self-parameters, the broker provides three pieces of information: the buyer's purchased weight 3, agent A's purchased weight a, and the gain-from gain-from-trade Ag. In this setting, we have that
Theorem 4.1. Bounds on Buyer's Gain-from-Truse Trade: In the linear model setting, by knowing A and weights a, 33 agent A can obtain bounds on the gain-from- of agent given by:
1 - /
(1- 1+ V a - 3 2a8)
SASS
(1 +)+ A a 3 2a²)
Discussion. Theorem 4. states that by knowing information disclosed by the broker (including purchased weights a, (), the seller agent A can find the upper and lower bounds of the buyer's gain-from-trade, A This information can then be used to estimate the value that the buyer places on the item. Using the Bayesian aptimal-pricing mechanism, and taking into account the known probability distribution, the seller can estimate the price determine their own virtual valuation.
Furthermore, the optimal scenario occurs when the seller agent values self parammersss exactly the buyer does. In this case, based on Eq. ① the broker will set the transfer payment as AP:, P Pt vb(()) va (.)). Hence, the transfer payment is equal to the difference between the gain-from-tras trrde of the two agents, where A = A -i,
The proof for Theorem 4.T is in the Appendix c We summarize trading steps in single round for this instantiation above (Algorithm) M. where two agents are training and trading parameters for the linear model setting. Here, agents A, B act as seller and buyer, respectively.

---

5 Convergence Analysis
Next, we study theoretical analysis for the effectiveness of buying parameters. In particular, we interested in understanding whether participating in the market leads to faster training convergence (in the worst-case scenario). We show this holds in simplified setting where agent A always leads and never purchases parameters, while agent B always purchases from agent A. This asymmetry could be due to various reasons including lack of training resources for agent B. Here we study setting for general L-smooth functions, which is more practical, as the broker doesn't need to be knowledgeable about the true parameter l". We assume the broker's loss lower than agents' losses, in particular, (0) S (f) and (0) I (0), VO Rd. We take the gain-from-trade A by using the subtraction of empirical loss before and after trade. We write it as
A = (..) - (b.), u E {a,b).
(4)
Theorem 5.1. For all agents u E {aa =). let the loss function Pu be L- smooth and let the samples on all agents be drawnfrom the same distribution D. Let ED| = Lu. and D (,)) L. (8)) Let the algorithm run until round T with step size E (0, h.), and let ób == minee/r E and
T. == minke[T] Elll LI (0)) Then we have the following results.
a) (Always Trade) If A > 0, Vt, and agent B always buys (i.e. 1 = 1, Then T z 2(2(0))-C(03)) implies 96 c.
462,
b) (Never Trade) If the agents never trade i.e. (T) = rg V,V Then Zo S for
T T2 -(118)--((E))
Discussion. We show the convergence rate of always trade for agent B is (11((c2 + 6z)). while never trade is ( The difference between these two scenarios is due to ô ò the minimal gain-from-trade over T runs. More from-traee implies a smaller T (i.e. faster convergence) in the worst-case scenario. In addition, when 6g is M(e), we can get much better convergence rate of (1/e). Our results in this fundamental setting illustrate that participation in the market can lead to faster convergence when there exists mein-from-trees -rade.. The proof for Theorem 5. is in the Appendix D.I In addition to this general setting, we provide convergence analysis for the linear model that we used as case study in Sec. This analysis also shows a better convergence rate when agents trade parameters. See Appendix D.2 for more details.
Experiments
We study the proposed framework empirically. Our goals are to validate (i) trading in the proposed framework results in improvements, (ii) these improvements persist even when trading subsets of parameters, (iii) these persist even when agents are trading different models for different tasks, (iv) trading and pricing are viable in competitive settings, and (v) understand the importance of key components of our framework, such as the need for alignment.
6.1 Collaborative Agents
We first conduct experiments in collaborative setting, where there is no payment for buying parameters. Our goal is to validate the potential improvements from transactions independently of the pricing mechanism, which we explore in a later set of experiments.
6.1.1 Parameter Trading in Neural Networks
Setup. We use MNIST B3 CIFARIO R. and TinyImageNet 355 for training MLPs and ResNet20 [36). Agents have imbalanced datasets where half of the classes contain only 10% of datapoints. Agents are limited to collecting a part of a dataset, making it difficult for them to achieve satisfactory performance without collaborating and trading parameters to leverage each other's strengths.
Models are trained from different random initializations and batch orders over 60 epochs. Agents trade entire parameter sets and join the market after five epochs. The broker discloses gain-frow cain-from-trade to agents. Broker aligns parameters [16), then merge.
7

---

Agent A
Agent B
Testing Acc. (%)
Testing Acc. (%)
out-of-market FedAvg
68.50%
72.97%
MNIST MLP
81.98%
81.98%
w/o alignment
84,
84, 64.64%
w alignment
86.96%
86.96% %
out-of-marker
71.14%
70.56%
CIFARIO ResNet22
FedAvg wlo alignment
70.35%
67.85 97.85%
78.31%
78.31%
w alignment 79.90%
79.90%
out-of-market FedAvg
21.67%
15.89%
TinyImageNet ResNet20
19.95%
19.33%
w/o alignment 31.28%
31.30%
w alignment
31.69% %
31.82%
Table I: Testing accuracies are reported for each combination dataset and model. In FedAvg, there is no broker to assist agents in conducting transactions. The interpolated weight is determined solely based on the proportion of data assets. w/o alignment indicates that broker merges parameters via simple interpolation with the optimized purchased weight. In w alignment, the broker aligns parameters by applying [ and then interpolates.
In addition to the approach that agents train models on their own, we include another baseline method, FedAvg 377 which assumes that there is no broker involved in a trade to help agents align parameters and optimize their purchased weights. FedAvg, the interpolated weight is determined by the portion of data assets that agent is endowed with, which is 0.5 in this setting.
Results. Table I shows the performance of two agents, We find that both agents are able to achieve improved performance by leveraging each other's training expertise, compared to out-of-market agents. Specifically, training and trading ResNet20 with TinylmageNet resulted in improving accuracy by +10.02% and +15.93%, respectively. We measure two ways to merge parameters for buying: with and without model aligmment. With model alignment, the broker is able merge models more effectively, ultimately enhancing performance for both agents. In addition, compared to FedAvg method, results confirm the significance having a trusted broker in parameter trading. Without an intermediary broker to facilitate the trade, the performance of purchased weights can be negatively impacted, as evidenced by the results of CIFARIO 10 + ResNet20 and TinyImageNet ResNet20.
24
Out of Market win alignment alignment
21-
14-
2,10,)
10
22
30
40
50
Agent
Out of Market
21
wio alignment
1.a
w alignment
2,18))13
12.
06-
Finally, Figure 2 displays comparison of testing loss for the MLP on MNIST. Our results demonstrate that trading parameters results in faster convergence compared to siloed training. Using alignment further helps, as expected.
Epoch
Figure 2: Testing loss converges the
fastest by aligning and interpolating.
6.1.2 Parameter Subsets Trading in Neural Networks
Setup. Next, we explore the potential benefits of trading subsets of parameters. This scenario may take place when agents are interested in only certain components or are restricted by trading budgets. We use the same data endowment as in the preceding configuration. We train two 5-layer MLPs on MNIST and align parameters in each layer trade.
Results. Table 2 displays the results by trading parameters from different layers. As expected, trading the entire model gives optimal performance (the last row). Trading subsets is helpful but suboptimal. We observe that purchasing parameters from shallow layers, close to the input layers,

---

Agent A
Ageni B
Testing Acc. (%)
Testing Acc. (%)
out-of-market
71.29%
72.54%
layers (3.4) 4)
66.28%
71.98%
layers (2, 4]
70.73%
73.36%
layers [0, 1,
74.76%
74.16%
layers {0, 1]
78.86%
79.82%
layers [0, 1, 3,4]
86.96 %
86.96 %
Table 2: Testing accuracies for trading parameters from different layers in 5-layer MLPs on MNIST.
Apent Dowentt Buy
Tie Line, y
Dut of Market In the Market
Gut Marcet
the Market
Agent 0
[9,-6'1E
(,18,)
2,08gl
Agert
Dresn't Buy
20
45
100
Epoch
Epoch
Epoch
Epoch
Figure 3: Both agents earn improvements mutually through trading in the market. We visualize trading results two linear regression models with synthesized dataset. Leftmost: trading logs over 100 runs. Second from left: The ratio of squared parameter estimation error between agent A and agent B. A red line represents tied performance. Above the red line, agent B leads, and vice versa. Second from right & rightmost: agent A's and agent I's learning curve compared to out-of-marker agents. Market usage thus produces performance improvements.
offers more benefits than trading with deeper layers. This provides trading guidance for budget-limited agents.
6.1.3 The Effectiveness of Buying Parameters in Controlled Settings
Setup. We use synthetic dataset study trading in fully-controlled environment. We two linear regression models with dimension d' = 1000. Agent A has n. = 500 datapoints in dataset (X,, Y.) and B has n,, 800 datapoints in dataset (X,,, Y,), but the latter's labels are affected by zero-mean Gaussian noise (o² = 0.5). We assume that the broker knows the true parameter 8" and obtains (X,, Y2) and has access to n. 10, 000 datapoints. Both agents start learning function fa, fo with the same initialized weight 80_ We compare the results over 100 runs with agents who obtain the same data endowment but do not participate in the market.
The leftmost Figure 3 displays the trading log over 100 runs. Green dots and red dots show whether an agent purchases the other's ' parameters in a specific run. Next, we show the ratio of squared parameter estimation eгrоr between agents. If the ratio is larger than (red dashed line), agent leads the market. We see that agent A leads the market in the first half of runs, making agent B continue buying parameters from agent A. At the end of few runs, agent B turns to the lead.
Results.
The rightmost plots show convergence. If an agent is involved in the market and trades with the other, the convergence rate is faster compared to never trading. This study demonstrates agents' behaviors in the trading runs and validates the effectiveness buying parameters, leading to more efficient model training. We compute the empirical testing loss at the end. Compared to out-of-markes agents, we find that agent A and agent are able to improve testing loss by 42.84 % and 23.88 88, respectively.
6.1.4 Trading Parameters of Models with Different Purposes
Setup. Next, we validate whether trading makes sense even if agents are training different models for different purposes. This scenario is more realistic it simulates situations where organizations in the market are working on different but potentially related tasks. We model this setting by sweeping the distance between the true parameters 0%, 0, of two linear regression models to observe how it impacts the benefits of trading.
22.5
noMtie 12.5
100
50
20
200 202
300
Trading with parameters that are from related tasks is possible.
Figure 4:
We record the benefits from trading when compared to an agent who does not participate but has the same data as agent
Results.

---

A. We measure the relative improvement in empirical testing loss. The results are shown in Figure 4 which indicates that even though the two agents are not training on the same task, agent A is still able to benefit from trading. Note that the gain exists even when the tasks are quite different (i.e. large distance 10% 12) but strongest as the tasks are most closely related.
6.2
Competitive Agents
Finally, we study agents engaging in competitive scenario. In this case, the transactions require pricing. We validate the proposed pricing mechanism.
6.2.1 The Effectiveness of Bayesian Optimal Pricing Mechanism
Setup. We reuse the synthesized dataset from the collaborative experiment. We set the buyer's valuation to gain-from-trade and estimate the seller's virtual valuation by the lower bound that
Agent
find.
P 0.-
Results. Market prices negotiated by the broker over 100 trading runs are displayed in Figure 5 Based on Eq. I the market price is determined by the average of the buyer's valuation and the seller's virtual valuation. To demonstrate market efficiency, we also study the scenario where the seller sets the valuation be exactly the same as the buyer's. As shown in Figure B (second from left: performance ratio), agent A is initially leading in the first half of the runs, resulting in higher price for their parameters. However, at the end of few runs, agent B takes the lead, causing their parameter price to increase while opponent A' goes down. It is important to note that the gap between the estimated price (the blue line) and the optimal price (the orange line) can be reduced with more information learned through the market, such as via historical transactions. Besides, the resulting negotiated price creates discrepancy with the price in the optimal scenario where both parties report their valuations truthfully, highlighting the significance of revealing accurate parameter values and justifying the need for incentives. At last, this study illustrates the feasibility of using the Bayesian optimal pricing mechanism assist seller agents in monetizing parameters with limited information to make trade.
Estimated Price
Optimal Scenario
20
Agent
Estimated Price
Optimal Scenario
Pb
20
40
160
Epoch
Figure 5: We visualize parameter market price negotiated by broker. We can see in the first half of runs, since agent A's performance dominates the market, making agent A's parameter more valuable compared to opponent agent B. Note that, if there is no trade, market price remains the same as historical price.
7 Conclusion
In this paper, we introduced framework for parameter markets that can serve to reduce the heavy costs of large-scale model training. Borrowing from economic principles, we provided a set of mechanisms that enable the functioning of such markets. Theoretically, for simple settings, analyzed market efficiency and proved that agents can gain from participating. We empirically validated the effectiveness of parameter markets under collaborative and competitive settings and demonstrated when participants in the market earn mutual benefits through market usage.
8 Acknowledgments
We would like to express our gratitude to the Wisconsin Alumni Research Foundation (WARF) for supporting this work. Additionally, we would like to thank all the reviewers for their valuable comments and constructive feedback our work.
10

---

References
[1] Tom Brown, Benjamin Mann, Nick Ryder, Mclanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 13:1877-1901, 2020.
[2] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, al. Exploring the limits of transfer learning with unified lext-to-text transformer. J. Mach. Learn. Res., 21(140):1-67, 2020.
[3] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham. Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.00 0.2000..22112222 2022.
[4] Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilié, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:221 1.05 2022.
[5] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01066 2022.
[6] Or Sharir, Barak Peleg, and Yoav Shoham. The cost of training nlp models: A concise overview. arXiv preprint arXiv:2004.08900 089000 2020.
[7] Lie He, An Bian, and Martin Jaggi. Cola: Decentralized linear learning, Advances in Neural Information Processing Systems, 31, 2018.
[8] Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, and Ji Liu. d: Decentralized training over decentralized data. International Conference on Machine Learning, pages 8848-4856.. PMLR, 2018
[9] Qinyi Luo, Jinkun Lin, Youwei Zhuo, and Xuchai Qian. Hop: Heterogeneity-awann decentralized training. In Proceedings of the Twenty-Fourd International Conference on Architectural Support Programming Languages and Operating Systems, pages 893-907, 2019.
[10] Binhang Yuan, Yongjun He, Jared Quincy Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Re, and Ce Zhang. Decentralized training of foundation models in heterogeneous environments. arXiv preprint arXiv:2206.01288 2022.
[11] Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. Advances in neural information processing systems, 31, 2018.
[12] Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode connectivity and the lottery ticket hypothesis. International Conference on Machine Learning, pages 2269. PMLR, 2020.
[13] Norman Tatro, Pin- Yu-Yu Chen, Payel Das, Igor Melnyk, Prasanna Sattigeri, and Rongjie Lai. Optimizing mode connectivity via neuron alignment. Advances Neural Information Processing Systems, 33:15300-1533 2020.
[14] Seyed Iman Mirzadeh, Mehrdad Farajtabar, Dilan Gorur, Razvan Pascanu, and Hassan Ghasemzadeh. Linear mode connectivity in multitask and continual learning, arXiv preprint arXiv:2010.044995 2020.
[15] Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur. The role of permutation invariance in linear mode connectivity of neural networks. arXiv preprint arXiv:2/10.06990 2021.
[16] Samuel K Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git re-basin: Merging models modulo permutation symmetries. arXiv preprint
2022.
11

---

[17] Keller Jordan, Hanie Sedghi, Olga Saukh, Rahim Entezari, and Behnam Neyshabur. Repair: Renormalizing permuted activations for interpolation repair. arXiv preprint arXiv:2211.08403, 2022.
[18] Min Du, Ruoxi Jia, and Dawn Song. Robust anomaly detection and backdoor attack detection via differential privacy. arXiv preprint arXiv:/911.07116, 2019.
[19] Xudong Zhu and Hui Li. presevyp decentralized federated deep learning. In ACM Turing Award Celebration Conference- (ACM TURC 2021), pages 33-38, 2021.
[20] Dario Pasquini, Mathilde Raynal, and Carmela Troncoso. On the privacy of decentralized machine learning. arXiv preprint arXiv:2205.0844: 08444, 2022.
[21] Qiyang Song, Jíahao Cao, Kun Sun, Qi Li, and Ke Xu. Try before you buy: Privacy-preserving data evaluation on cloud- based machine learning data marketplace. In Annual Computer Security Applications Conference, pages 260-272, 2021.
[22] Santiago Andrés Azcoitia and Nikolaos Laoutaris. Try before you buy: A practical data purchasing algorithm for real-world data marketplaces. In Proceedings of the 1st International Workshop Data Economy, pages 27-33, 2022.
[23] Roger B Myerson. Optimal auction design. Mathematics of operations research, 6(1):58-73, 1981.
[24] John F Nash Jr. The bargaining problem. Econometrica: Journal of the econometric society, pages 155-162, 1950.
[25] Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In International Conference on Machine Learning, pages 1242-2251. PMLR, 2019.
[26] Anish Agarwal, Munther Dahleh, and Tuhin Sarkar. A marketplace for data: An algorithmic solution. In Proceedings of the 2019 ACM Conference on Economics and Computation, pages 701-726, 2019.
[27] Raul Castro Fernandez, Pranav Subramaniam, and Michael J Franklin. Data market platforms: Trading data assets to solve data problems. arXiv preprint arXiv:2002.01047, 2020.
[28] Junjie Chen, Minming Li, and Haifeng Xu. Selling data to a machine learner: Pricing via costly signaling. In International Conference Machine Learning, pages 3336-3355 PMLR, 2022.
[29] Meena Jagadeesan, Michael I Jordan, and Nika Haghtalab. Competition, alignment, and equilibria in digital marketplaces. arXiv preprint arXiv:2208. 2022.
[30] Lingjiao Chen, Paraschos Koutris, and Arun Kumar. Towards model-based pricing for machine learning in a data marketplace. In Proceedings of the 2019 International Conference Management of Data, pages 1535-1552, 2019.
[31] Jinfei Liu, Jian Lou, Junxu Liu, Xiong, Jian Pei, and Jimeng Sun. Dealer: an end-to-end model marketplace with differential privacy. 2021.
[32] Charles W Cobb and Paul H Douglas. A theory of production. 1928.
[33] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffnnr. Gradient-based learning applied to document recognition. Proceedings of the IEEE, (6(11))227822244 1998.
[34] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
[35] Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
[36] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[37] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. coamuunicatioo-ffffcieee learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.
12

---

[38] ABDULLAH YAVAŞ. Economics of brokerage: an overview. Journal of Real Estate Literature, 2(2):169-195, 1994.
[39] Simran Khanuja, Melvin Johnson, and Partha Talukdar. Mergedistill: Merging pre-trained language models using distillation. arXiv preprint urXiv:2106.02834, 2021.
[40] Sebastian Lawrenz, Priyanka Sharma, and Andreas Rausch. Blockchain technology approach for data marketplaces. In Proceedings of the 2019 international conference on blockchain technology, pages 55-59, 2019.
[41] Franziska Boenisch. A systematic review model watermarking for neural networks. Frontiers in big Data, 4:729663, 2021.
[42] Reza Yazdani Aminabadi. Samyam Rajbhandari, Ammar Ahmad Awan, Cheng Li. Du Li, Elton Zheng. Olatunji Ruwase, Shaden Smith, Minjia Zhang. Jeff Rasley, et al. Deepspeed-inference: enabling efficient inference of transformer models at unprecedented scale. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1-15. IEEE, 2022.
[43] Jannik Kossen, Sebastian Farquhar, Yarin Gal, and Tom Rainforth. Active testing: Sample-efficient model evaluation. In International Conference on Machine Learning, pages 5753 5763. PMLR, 2021.
[44] Ante Custié, Vladyslav Sokol, Abraham Punnen, and Binay Bhattacharya. The bilinear assignment problem: complexity and polynomially solvable special cases. Mathematical programming, 166:185-205, 2017.
13


Reviews:
[{"review_id": "G39uPFLCN7", "reviewer": "Reviewer_91tn", "summary": "The paper proposes a framework for collaborative and competitive parameter trading among deep learning agents. The authors conduct experiments to validate the effectiveness of the proposed framework in improving the performance of the agents. The experiments show that even when the agents are training on different tasks, they can still benefit from trading parameters. The authors also validate the proposed pricing mechanism in a competitive scenario. The paper makes contributions in demonstrating the potential of parameter trading to improve the performance of deep learning agents and providing a framework for collaborative and competitive trading.\n", "questions": "- Why is parameter trading necessary?\n- How to resolve the legal concerns on copyrights?\n", "limitations": "More limitations should be discussed.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper proposes a novel framework for collaborative and competitive parameter trading\n- The experimental results are promising.\n- The experiments show that even when the agents are training on different tasks, they can still benefit from trading parameters.", "weaknesses": "- The authors do not provide an analysis of the limitations of the proposed framework, which could help identify potential issues in real-world scenarios.\n- The experiments could benefit from a more extensive evaluation of the proposed framework's performance by comparing it with more baselines.\n- The paper could benefit from a more detailed explanation of the proposed pricing mechanism and its implementation.\n- The authors do not discuss the potential ethical implications of parameter trading, which could be relevant in real-world applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Response to Reviewer 91tn\nThank you for finding our framework novel and the results to be encouraging, particularly in terms of trading parameters with different purposes. We appreciate your thoughtful review! \n* **On limitations.**\n    * There are two primary limitations toward building viable parameter markets. We discuss these in the paper and have included additional clarifications. First, the reliability of the broker is crucial to the success of the market. The broker must report trade gains/losses without any bias to ensure the market remains efficient. One limits of our work is that some entity is willing to serve as a broker with these properties.\n    * On the other hand, such a limitation may not be substantial. In fact, a similar requirement is accepted and used in practice in other settings, such as (non-machine learning) markets [1]. \n    * An additional limitation is that while agents are allowed to train models using various model architectures for a range of downstream tasks, the parameter sets to be traded require a certain alignment. We believe this limitation is not too severe: it can be overcome, for example, by projecting the parameters onto the same space for merging [2].\n* **On a new baseline.**\n    * Thank you for your suggestion. We have conducted another baseline test (FedAvg) and describe the setup in the general response. Results are presented in Table 1 in our attachment, and they confirm the significance of having a trusted broker in parameter trading. Without an intermediary broker to facilitate the trade, the performance of purchased weights is negatively impacted, as evidenced by the CIFAR10 + ResNet20 results.\n* **On further explanation of the pricing mechanism.**\n    * The discussion of pricing mechanisms is in Sec. 3.4, along with Sec. 4, where we provided a practical example to instantiate the market. We include additional explanations here and have updated our draft.\n    * The steps are as follows. Once agents become aware of the gain-from-trade information disclosed by the broker, they assess the value of parameters and negotiate prices accordingly. To negotiate prices, the broker solves a type of Nash bargaining problem. There are several ways to assess parameters, such as according to the agent's preference, training budget, and relative improvements. To provide a concrete study of how to value trades for the seller, we study trading in linear models and introduce virtual valuation for sellers. Finally, buyer's valuation can be bounded, enabling the seller to provide an estimated asking price for negotiation purposes.\n* **On potential ethical implications.**\n    * Under the conditions we require for our framework, we do not foresee ethical concerns around this work. However, if some of the assumptions we make are violated, we could envision certain ethical implications. For example, suppose that the broker acts adversarially, rather than being a neutral third party as intended. In this case, the broker could potentially use models from both parties, could misinform one party of their potential benefit and engage in front-running, etc. We have added a discussion of these possibilities to our updated draft. \n* **On why parameter trading is necessary.**\n    * Our motivation follows from the fact that the costs of building high-quality ML models have become prohibitively high, leaving only a few well-capitalized organizations in a position to build such models. Indeed, we see this in the space of large pre-trained models, where all other individuals can at best fine-tune or adapt these base models. **Our goal is to take a step towards reducing such costs**. The most intuitive approach is to perform distributed training. However, this would require fully open-sourced models, which many users may not be able to adhere to.\n    * How can we reduce costs, even when models aren't open-sourced and organizations may even be competing with each other? **Our solution is a type of parameter trading that can benefit all parties**. Buyers can purchase pre-trained parameter sets instead of building from scratch, reducing training expenses and leveraging the expertise of others. Sellers can profit from selling their parameters as a secondary source of income. The most exciting aspect of this work demonstrates that this simple solution works: we prove the effectiveness of parameter trading and allow buyers having faster convergence of losses. Sec. 5.2 demonstrates the success of our pricing mechanism and the opportunities for sellers to earn a profit.\n* **On legal concerns with copyright.**\n    * This is a great question! In fact, one of our motivations is precisely _the fact that trading data is challenging due to the state of digital property rights_. **Parameter trading, on the other hand, does not suffer from the same limitations**, at least in many present legal frameworks. That is, parameters are typically owned by organizations or individuals training their models. Because of this, voluntarily trading parameters does not require breaking any type of copyright law. Broker is the only additional party to access to these.\n    * One potential additional concern is how to prevent out-of-bounds parameter usage (i.e., how to ensure a buyer does not publicly release the purchased parameters, sell them to another unauthorized party). There is a wide array of existing research on this question. Applicable tools enable users to check the purchased weights by creating transaction hashes [3] and watermarks [4]. We discuss these in our updated draft.\n\n[1] Yavas, \"Economics of Brokerage: An Overview\", Journal of Real Estate Literature, Vol. 2, No. 2, 1994. \\\n[2] Khanuja et al, \"MergeDistill: Merging Language Models using Pre-trained Distillation\", ACL-IJCNLP, 2021. \\\n[3] Lawrenz et al, \"Blockchain Technology as an Approach for Data Marketplaces\", ICBCT 2019. \\\n[4] Boenisch, \"A Systematic Review on Model Watermarking for Neural Networks\", arXiv:2009.12153."}, {"review_id": "pRl6TGisOV", "reviewer": "Reviewer_iN1r", "summary": "The paper investigates how to design a marketplace for model parameters. The marketplace consists of agents training models for potentially different objectives and a trusted third-party broker. The broker receives the model parameters from each agent, assesses (and informs each agent of) the loss achieved by the merged model, and helps determine prices using Nash bargaining. \n\nThe paper provides a theoretical and empirical analysis of this framework. In particular, they prove a bound on the gain from trade for linear models and they analyze the effectiveness of buying parameters in terms of improving training convergence.  They empirically assess their framework on an image classification task. They validate that the merging method of the broker (Ainsworth et al., 2022) improves loss, and they evaluate the effectiveness of their pricing strategy. \n", "questions": "How might the framework be modified to avoid the reliance on a trusted broker, e.g., via a more decentralized approach? \n", "limitations": "The authors adequately addressed the limitations. ", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The idea of trading subsets of parameters in a marketplace seems novel and interesting. Parameter marketplaces seem to contrast with classical approaches of data marketplaces, where agents buy data from each other, or model marketplaces, where agents directly purchase models or model access. ", "weaknesses": "The proposed framework seems to rely on there being a trusted broker facilitating the trades and assessing and reporting the gains of merging parameters to the agents. In reality, this seems to be a strong assumption, as it is not clear if (a) a broker can be trusted with all of the parameters, and (b) if a broker will be able to assess the gain of merging models without significant data and infrastructure. See question below.  \n\nThe technical contribution of the paper seems limited. The paper combines existing work on merging and aligning models (e.g. Ainsworth et al., 2022) and classical approaches for Nash bargaining. However, the paper seems to use these approaches out-of-box and does not seem to present significant new technical contributions. \n\nThe paper is a bit difficult to follow and the exposition could be improved. For example, the results in Section 4 use a lot of notation and the qualitative insights are a bit difficult to infer.    \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Response to Reviewer iN1r\nWe are grateful for your review and for describing our framework as novel and interesting. We address your questions in the response below and have updated our paper!\n* **On trusting brokers.**\n    * Indeed, having a reliable broker is **essential for any trading market**, just as it is in real-life scenarios (for example, stockbrokers help perform huge numbers of trades, and large-scale escrow organizations help aid in billion-dollar transactions). One area of interest is how to motivate third parties to play this role, a question that is frequently studied by economists and business researchers [1]. Examples of this are research into how to set fees for brokers, what other types of incentives can be found, etc.\n    * We view this question as orthogonal to this work, and instead simply presume that such brokers exist. This supposition is commonly employed and has been substantiated by a number of prior research papers on trading markets [2, 3, 4]. We note, in addition, that our market permits the trading of subsets of parameters and transferred parameters only if a trade is made, thereby safeguarding a certain degree of model confidentiality.\n* **On broker infrastructure.**\n    * This is a great question! We agree that it is essential to concern the accessibility of computational resources for the broker, but we do believe this is not a major challenge. The reason for this is that **brokers can be paid from brokerage fees** to a sufficient extent to meet their computational needs. Note that these requirements need not be prohibitive: our trading framework does not ask the broker to train models, only to validate model performance through inference, which typically requires fewer resources [5]. Additionally, the amount of validation data needed does not need to be high [6]. Moreover, in general, the alignment approach we employ is computationally efficient. As a result, the computational requirements for the broker are manageable.\n* **On contributions.**\n    * Our focus is on developing a novel trading framework that contributes to creating parameter markets and demonstrating the feasibility of trading sets of parameters. The techniques used in this framework are flexible and can include more alignment approaches [7, 8, 9, 10] and be generalized to any other utility functions for price negotiation [11]. Our findings are important: they suggest that **the entire approach of parameter trading is viable**. Indeed, this is the major contribution of our work.\n    * In addition, through empirical evidence, we have found that trading parameter sets can also lead to faster convergence and improved model performance (refer to Sec. 5.1.2)---a further contribution. Finally, as far as we are aware, this is the first work that successfully monetizes parameter sets and treats them as commodities to be traded. Such a market can be used in the current large language model community: by engaging in parameter trading, individuals or start-ups are able to purchase pre-trained parameter sets instead of building from scratch.\n* **On clarifying results and takeaways.**\n    * Sec. 4 offers concrete examples of market instances. We highlight two main results and have clarified this further in our draft. \n        * **How does a seller value a trade?** We study the case of linear models and help the seller provide a valuation to the broker by using virtual valuation. Theorem 4.1 reveals that the seller can estimate their own valuation by bounding the buyer's valuation based on information disclosed by the broker. We empirically validate the success of price estimation procedure to monetize parameters in Sec. 5.2.\n        * **How effective is buying parameters from a theoretical perspective?** To investigate the effectiveness of parameter trading, we offer a convergence analysis. Theorem 4.2 states that more gain-from-trade in the market results in faster convergence, which supports the trading incentives to participate in the market. Additionally, we offer a generalized form of convergence analysis with L-smooth functions in Appendix D.2, which also demonstrates similar results that more gain-from-trade leads to quicker convergence. These results can be justified by empirical evidence in Sec. 5.1.\n* **On avoiding a trusted broker through decentralization.**\n    * We agree! A decentralized approach can indeed eliminate the need for a centralized broker. However, there are downsides to this approach as well. We would require new ways to motivate decentralized agents to perform the work. This may cut against our goal of minimizing expenses for individuals. Nevertheless, we think this is an interesting question and consider it for further investigation.\n\n[1] Yavas, \"Economics of Brokerage: An Overview\", Journal of Real Estate Literature, Vol. 2, No. 2, 1994. \\\n[2] Liu et al, \"Dealer: An End-to-End Model Marketplace with Differential Privacy\", Proceedings of the VLDB Endowment, Vol. 14, No. 6, 2021. \\\n[3] Azcoitia and Laoutaris, \"Try Before You Buy: a Practical Data Purchasing Algorithm for Real-World Data Marketplaces\", DE 2022. \\\n[4] Chen et al, \"Towards Model-based Pricing for Machine Learning in a Data Marketplace\", SIGMOD 2019. \\\n[5] Aminabadi et al, \"DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale\", SC 2022. \\\n[6] Kossen et al, \"Active Testing: Sample–Efficient Model Evaluation\", ICML 2021. \\\n[7] Stoica et al, \"ZipIt! Merging Models from Different Tasks without Training\", arXiv:2305.03053. \\\n[8] Wortsman et al, \"Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time\", ICML 2022. \\\n[9] Singh and Jaggi, \"Model Fusion via Optimal Transport\", NeurIPS 2022. \\\n[10] Qin et al, \"Exploring Mode Connectivity for Pre-trained Language Models\", EMNLP 2022. \\\n[11] McFadden, \"Constant Elasticity of Substitution Production Functions\", The Review of Economic Studies, Vol. 30, No. 2, 1963."}, {"review_id": "KgvF3zKYEG", "reviewer": "Reviewer_Tv8J", "summary": "* This paper proposes an economic framework for trading parameters of prediction models.\n* Interaction is modeled as a brokered marketplace - Each agent $u$ trains a model characterized by parameters $\\theta_u$ using gradient descent. At each time-step $t$: \n  * Each agent performs a gradient descent step on the previous parameters $\\theta_u^{t-1}$, to obtain $\\dot{\\theta}_u^t$.\n  * After the GD step, agents relay their model parameters $\\dot{\\theta}_u^t$ to a  trusted broker, which calculates the optimal linear interpolation between them, yielding the possibly-improved combination $\\bar{\\theta}_u$ for each agent. The broker indicates for each agent the gain-from-trade $\\Delta_u^t$.\n  * The agents participate in Nash bargaining to decide on the value exchange, set trading prices, and $\\theta^t_u$ is decided. Each agent has a valuation function $v_u$ for their own parameters, and the other agent’s parameters. Utility function is assumed to be fully known by the broker. \n* Analysis:\n  * Analysis is restricted to the two-agent setting $u\\in\\{A,B\\}$, and a specific form for gain-from-trade $\\Delta_u^t$ is assumed. It is assumed that each seller has complete knowledge of the valuation prior, such that Myerson’s revenue-maximizing pricing mechanism (virtual valuations) can be applied based on the information relayed from the broker ($\\alpha$, $\\beta$, $\\Delta_a^t$ - By Theorem 4.1).\n  * It is assumed that the broker has complete knowledge of the optimal model parameters in advance ($\\theta^*$). Combining with the assumptions about the agents’ utility function, a closed-form expression is obtained for the monetary transfer after Nash bargaining.\n  * In Section 4.1, two upper bounds on convergence rates are presented - One for a scenario where agent $B$ always trades (Theorem 4.2), and one for a scenario where both agents don’t trade (Proposition 4.3). The “always-trade” bound (Thm 4.2) is lower, and the authors claim that this explains why trading is sometimes beneficial. \n* Two sets of experiments are presented:\n  * The first set of experiments is an evaluation of the data-sharing setting, without taking parameters into account (Section 5.1.1 - complete networks, Section 5.1.2 - subsets of networks), on real-world vision datasets (MNIST, CIFAR10, TinyImageNet).\n  * Finally, the value of trading is evaluated using a synthetic model, and positive results are reported for both cooperative and competitive scenarios.\n", "questions": "* How would the model extend to more than 2 agents? Which results can be used as-is, and which need to be generalized? Would the system behave qualitatively differently in any way when the number of agents is increased?\n* What are the computational requirements from the broker?\n* Is it possible to quantify the economic implications of the broker not being able to interpolate successfully?\n", "limitations": "I feel that limitations could be discussed at more depth. The key assumptions are added gradually throughout the paper, and it is hard to keep track. I feel that the paper can greatly benefit from a thorough discussion of limitations.\n", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* Problem is well-motivated.\n* Leverages recent deep insights about the structure of contemporary learning problems.\n* The breadth of work is substantial, and it contains a number of new definitions, theoretical analyses, and empirical evaluations.\n* Code is provided.\n", "weaknesses": "* Some key assumptions are not well supported:\n  * Valuation functions $v(\\theta)$ are assumed to be fully known by the broker. For contemporary neural nets, the dimensionality of $\\theta$ could be in the order of billions (or even trillions, such as in GPT4), and therefore the parameter valuation would be a function $v:\\mathbb{R}^{10^9}\\to\\mathbb{R}$ - Which may be prohibitively expensive to represent, store and compute.\n  * Broker is assumed to know the true model parameters $\\theta^*$ in advance, which in practice may render the whole learning process unnecessary. In L214, it is claimed that knowing $\\theta^*$ “is not necessary in practice”, but I did not find further support of this claim. \n  * Very specific functional forms are assumed for gain-from-trade but are not sufficiently justified.\n  * A common prior on agent valuation is assumed (L208), however it is not discussed whether having such prior is realistic and how it would be implemented.\n* Mathematical soundness concern regarding one of the claims: In section 4.1, authors claim that trading is beneficial by comparing two performance upper bounds (Theorem 4.2, Proposition 4.3). However, the difference between two upper bounds does not indicate the relation between the true quantities unless they are tight (in other words, proving that $a<100$ and $b<50$ does not indicate that $a>b$ unless there are matching lower bounds for $a$ and $b$). Therefore, if this observation is true, the analysis in Section 4.1 does not address the posed question (whether participation leads to better convergence).\n* First set of experiments (Sec 5.1.1, 5.12) only seem to demonstrate the ability to interpolate model parameters, but do contain simulation of parameter trading. Economic behavior is only demonstrated on the synthetic dataset. As the benefit of parameter interpolation has already been demonstrated in the literature to some extent, I did not understand the contribution of the results in sections 5.1.1, 5.1.2 to the understanding of parameter-trading markets.\n* Paper only analyzes the case of two agents $\\{A,B\\}$. Unclear how results extend to multiple agents.\n* Trading parameters during the gradient descent process would require agents to train models simultaneously, or suffer significant delays due to trading. This may make such trading impractical. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Response to Reviewer Tv8J\nThank you for your clear summary and thoughtful review! We appreciate the kind words. We answer your questions below and include two new experimental results: multi-agent market and asynchronous parameter trading.\n* **On valuation function.**\n    * Thanks for pointing this out. In fact, the valuation function is not exposed to anyone in the market (see Ln 173 - Ln 175), but the valuation of trade (a scalar) will be disclosed to the broker for negotiation purposes. Additionally, the valuation function is quantifying the value of parameter sets based on gain-from-trade, which is written in the form of losses, for example in Ln 161. Hence, agents do not need to plug in high-dimensional parameter sets and compute the value. They value a potential trade by knowing the gain-from-trade.\n* **On knowing the true parameter.**\n    * However, it is only for the sake of offering a concrete example of assisting sellers in assessing the transaction and providing theoretical insights (Theorem 4.2). In practice, **there are a variety of ways to determine the value of a trade** based on factors such as preference, budget, and relative returns. For more practical purposes, we present a generalized convergence analysis in Appendix D.2, where we remove the need for the broker to possess knowledge of $\\theta^*$.\n* **On the form of gain-from-trade.**\n    * The gain-from-trade in our case study can be seen as the relative improvement if a trade is made. It should be noted that the gain-from-trade can take different forms and **is not limited to a specific definition**. For instance, it can be defined as the difference between an agent's loss before and after trading (as seen in Ln 161). Excitingly, **any method that quantifies the benefit of trading before and after** can be generalized into our pricing mechanism.\n* **On common prior.**\n    * The common prior (Ln 205): buyer sets a price arising from gain-from-trade and this price is derived from a probability distribution. **Both are inspired by Myerson's auction work, designed to help sellers estimate the values of trades**. The first prior is reasonable, since all agents consider the gain-from-trade while assessing a trade. Second, the notion that buyer's price is drawn from a probability distribution is standard in economic modeling [1]. We clarify this in our updated draft. \n* **On the upper bound of convergence rate.**\n    * We appreciate the comment. We have reviewed and revised our draft accordingly. While the upper bound on the end round $T$ may not necessarily lead to faster convergence, participating in the market provides agents with sufficient incentive and the assurance of achieving a smaller $T$ in the worst-case scenario. The discrepancy arises from the presence of gain-from-trade. We note that this is not unusual: comparing the upper bounds of the convergence rates (worst cases) is standard in the optimization.\n* **On the contribution of Sec. 5.1.1 and Sec. 5.1.2.**\n    * To the best of our knowledge, previous research has mainly focused on merging entire trained models at the end of the training process [2, 3], but there has been limited investigation into aligning parameters or subsets of parameters during training. This is a crucial factor in establishing a functional marketplace, and it is worthwhile to investigate such potential benefits. Hence, we demonstrate the feasibility of parameter trading while training in Sec. 5.1.1 and 5.1.2.\n* **On extension to multiple agents.**\n    * For the sake of simplicity and demonstrate the viability, we presented a two-agent market, which is a commonly used economic model for studying many settings. Nevertheless, it is straightforward to include multiple agents. The trading logic can be generalized to look for trades that enable agents to make the largest advancements. To validate this, **we implemented a three-agent market and present detailed experimental setups** in our general response. Results show that the generalization functions as expected.\n* **On asynchronous trading.**\n    * Thank you for bringing up this important point. Indeed, this area has been extensively studied in the distributed training literature [4]. The typical solution is to permit asynchrony. **We conducted asynchronous parameter trading and offer detailed experimental setups** in the general response. Results show that the parameter market allows delayed agent actions to enhance performance as well. Since the presence of the broker helps agents align parameters and optimize purchase weights, **the potential issues caused by asynchronous training can be mitigated**.\n* **On computational requirements for broker.**\n    * The broker has two main computational needs. Firstly, the broker needs to validate model's performance through inference, which is generally less intensive than training. Second, the broker assists in aligning parameters for both parties using an approximation algorithm, which has been shown to perform efficiently in experiments. In other words, the **computational workload for the broker is not a significant issue and can be managed**.\n* **On quantifying interpolation failure.**\n    * This is a great question, as the weights purchased may not be useful for agents if they are built for very unrelated tasks, which can result in a negative gain-from-trade and lead to agents stopping their purchases, ultimately reducing market efficiency. To measure such failures, one can establish prior knowledge of task interdependence. As mentioned in Section 5.1.4, the advantages of trading diminish when the tasks are not related.\n\n[1] Chawla et al, \"Algorithmic Pricing via Virtual Valuations\", EC 2007. \\\n[2] Stoica et al, \"ZipIt! Merging Models from Different Tasks without Training\", arXiv:2305.03053. \\\n[3] Qin et al, \"Exploring Mode Connectivity for Pre-trained Language Models\", EMNLP 2022. \\\n[4] Wang et al, \"Asynchronous Training Schemes in Distributed Learning with Time Delay\", arXiv:2208.13154."}, {"review_id": "oSQM9Rhref", "reviewer": "Reviewer_Mu3U", "summary": "This paper introduces a novel concept called *parameter markets*, which serves as a platform for exchanging parameters learned by machine learning models. In this framework, agents have the option to engage in parameter trading, to achieve (1) mutual benefits through collaboration or (2) monetary gains by simply selling these parameters. The key contributions of this paper involve the formulation and design of these parameter markets. This encompasses defining the value of parameters, establishing pricing mechanisms, and outlining how transactions can occur. The authors explore both competitive and collaborative scenarios and illustrate the advantages agents can obtain by opting in to trade through the marketplace.", "questions": "1. Refer to Weaknesses. \n2. It's unclear why Cobb-Douglas utility function is used. ", "limitations": "To my best knowledge, there isn't an immediate potential negative societal impact of their work. This paper, however, doesn't have an explicit weakness section (relating to their framework). ", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The idea to trade parameters is novel (to my best knowledge) and is quite different from data markets. The key benefit from parameter markets is that agents can train models for their own tasks and can yet benefit from training runs of each other. This is a very relevant problem to study.\n2. The formulation is thorough and the empirical experiments are adequate to demonstrate the claims made in the paper\n3. The paper is well written and easy to follow.", "weaknesses": "1. This work is a bit existential - this framework works if *model privacy and alignment are assured*. \n2. Theorem 4.1 relies on the broker having access to $\\theta^*$ which is a strong assumption.\n3. There is no discussion relating to incentives (i.e. agent misreporting their valuations to benefit from the trades).Perhaps this paper could benefit from adding a mechanism design angle to this framework.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Response to Reviewer Mu3U\nWe are grateful for the review, the kind words, and the positive assessment. We address your questions and concerns below.\n* **On assurance of model privacy and alignment.**\n    * This is true! We do not view this as a limitation, however. Vast research resources are being expended in assuring privacy [1, 2] and alignment can be performed [3, 4, 5, 6]. The motivation of our work is to discover, _if_ these concerns can be resolved, whether parameter marketplaces are viable at all. We believe that an answer to this question, whether positive or negative, is extremely valuable.\n    * We note, in addition, that the progress required in alignment and privacy may not be out of reach. According to the findings presented in Sec. 5.1, while trading with model alignment yields optimal outcomes, basic convex interpolation is also sufficient in achieving quicker convergence and enhancements. Furthermore, in our trading system, parameters are secured and only exchanged if both parties agree on the agreed-upon price, thus ensuring a type of privacy in the market. \n* **On accessing true parameters.**\n    * We agree! Having knowledge of the true parameter $\\theta^*$ to reveal the gain-from-trade $\\Delta^t_u = ||\\dot\\theta^t_u - \\theta^*||^2_2 / ||\\bar\\theta^t_u - \\theta^*||^2_2$ is a strong assumption. However, the result using it is only a specific example that can assist sellers in assessing the transaction and providing theoretical insights (Theorem 4.2). In practice, **there are a variety of ways to determine the value of a trade** based on factors such as the agent's preference, resource budget, and relative performance improvement. Furthermore, for more practical purposes, we present a generalized convergence analysis in Appendix D.2, where the gain-from-trade is calculated by subtracting the empirical loss before and after a trade, $\\Delta^t_u = \\hat{L}(\\dot\\theta^t_u) - \\hat{L}(\\bar\\theta^t_u)$, removing the need for the broker to possess knowledge of the true parameter $\\theta^*$.\n* **On agent’s incentives and honesty.**\n    * Thank you! We note another finding in Sec. 5.2, where the seller estimates the value solely using the lower bound they found according to Theorem 4.1. The resulting negotiated price creates a discrepancy with the price in the optimal scenario where both parties report their valuations truthfully without any estimation, **highlighting the significance of revealing accurate parameter values and justifying the need for incentives.**\n* **On the perspective of mechanism design.**\n    * Thank you for your suggestion! This is an interesting approach to strengthen our work, and we will include this in our updated draft. \n* **On the clarity of Cobb-Douglas utility function.**\n    * The concept behind price negotiation is to split a fixed joint surplus ($U_a \\times U_b$) while maximizing the revenue for both parties equally (as the power of $U_a$ and $U_b$ is the same). The Cobb-Douglas utility function is ideal for this purpose. In addition, it has several convenient mathematical properties. For instance, it is strictly quasiconcave, which means it has a unique maximizer and an optimal solution. Moreover, it demonstrates the diminishing marginal rate of substitution as the negotiated price changes. Lastly, when expressed in log form, it serves as the first Taylor approximation of a convex production function, thereby eliminating the need for extensive knowledge of the underlying production function.\n\n[1] Zhu et al, \"Privacy-preserving Decentralized Federated Deep Learning\", ACM TURC 2021. \\\n[2] Pasquini et al, \"On the (In)security of Peer-to-Peer Decentralized Machine Learning\", IEEE Symposium on Security and Privacy (SP), 2023. \\\n[3] Stoica et al, \"ZipIt! Merging Models from Different Tasks without Training\", arXiv:2305.03053. \\\n[4] Wortsman et al, \"Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time\", ICML 2022. \\\n[5] Singh and Jaggi, \"Model Fusion via Optimal Transport\", NeurIPS 2022. \\\n[6] Qin et al, \"Exploring Mode Connectivity for Pre-trained Language Models\", EMNLP 2022."}]

Author rebuttal:
### General Response

We are grateful for all the comments and constructive feedback on our work. Reviewers consistently commented that our proposed trading framework is **novel and well-motivated**. Reviewers 91tn, Tv8J, and Mu3U note that our experimental results as **promising and substantial**. In our revision, we adopted several reviewer-suggested clarifications and performed additional experiments, leading to a much stronger draft.

Additional experiments include:
* **A new baseline** [Reviewer 91tn]: we provide another baseline test (FedAvg [1]), which assumes that there is no broker involved in a trade to help agents align parameters and optimize their purchased weights. In the FedAvg method, the interpolated weight is determined by the portion of data assets an agent is endowed with, which is 0.5 in the case of our two-agent trading example. The results are presented in Table 1 in our one-page response. Results confirm **the significance of having a trusted broker** in parameter trading. Without an intermediary broker to facilitate the trade, the performance of purchased weights can be negatively impacted, as evidenced by the CIFAR10 + ResNet20 results.
* **Multi-agent markets** [Reviewer Tv8J]: we generalize our setting to involve more agents, where the broker helps agents seek for parameters that can bring the largest gain-from-trade to purchase. We implement a three-agent market by reusing the same data endowment for Agent $A$ and Agent $B$ and display performance results in Table 2. We make Agent $C$ collect 10% of data points from class 3 ~ 7 in MNIST and CIFAR10, and 10% of data points from class 50 ~ 149. In the three-agent market, results also follow expectations. The proposed parameter trading is able to help agents achieve higher accuracy performance compared to conventional model training without trading (out-of-market).
* **Asynchronous parameter trading** [Reviewer Tv8J]: we conduct a new experiment on asynchronous parameter trading. Both agents are asked to train the model for 60 epochs in total. Agent $B$ trains the model for 5 epochs, trades in the market for 50 epochs, and then trains the model for the remaining 5 epochs. On the other hand, Agent $A$ is instructed to delay trading. Agent $A$ trains the model for 10 epochs and then trades in the market for 50 epochs. In Table 3, our results demonstrate that the parameter market allows both agents to enhance their performance, with trading in alignment yielding optimal accuracy as expected. This emphasizes the crucial role of brokers in aligning parameters and optimizing purchase weights to eliminate differences **not only in synchronous but also in asynchronous parameter trading**. Note as well that in the paper, agents train for 5 epochs and then trade synchronously for 55 epochs, which have additional 5 epochs for training and trading. This explains the degraded performance of Agent $B$.

We have addressed reviewers' concerns and placed our comments in their respective threads below. Thank you for your questions and thoughtful reviews!

[1] McMahan et al, "Communication-Efficient Learning of Deep Networks from Decentralized Data", AISTATS 2017.

Meta-review: This paper makes use of recent advances in model alignment to advocate for the idea of parameter markets, in which learned parameters can be traded by multiple entities. Reviewers found this prospect to be novel and intriguing, offering a viable alternative to existing markets for data or for entire models.

Reviewers have raised several concerns, the foremost being that technical novelty is low. Nonetheless, they acknowledge that the paper combines existing ideas in a useful way and applies them to a new problem domain. A second concern is that the proposed framework relies on strong assumptions, which some viewed as unrealistic - especially regarding the role and knowledge of the broker, but also on trust and infrastructure requirements. The authors' response was somewhat helpful in mitigating these, but reviewers still feel that the limitations and implications of the assumptions were not sufficiently discussed in the paper, with one reviewer feeling that the actual market mechanism was not entirely in line with how it was presented. More generally, the discussion regarding the paper's central "what if" assumptions - privacy, alignment, and idealized broker knowledge - requires more careful treatment than what is currently provided.

The authors are therefore highly encouraged to fix these issues, and to discuss not only the benefits of their approach, but also what can go *wrong* in such markets.

However, overall reviewers believe that the novelty of the idea of a market for parameters and the interest it promotes, coupled with the broad treatment and promising empirical results, are sufficient to merit acceptance. Reviewers were also appreciative of the new results presented in the discussion and in response to their questions. Hence, despite the above drawbacks, we believe this work has potential to spark interest in the community, and can serve to broaden discussion around economic aspects of machine learning.
Decision: Accept (poster)