Year: 2021

Abstract:
In this paper, we focus on effective learning over a collaborative research network involving multiple clients. Each client has its own sample population which may not be shared with other clients due to privacy concerns. The goal is to learn a model for each client, which behaves better than the one learned from its own data, through secure collaborations with other clients in the network. Due to the discrepancies of the sample distributions across different clients, it is not necessarily that collaborating with everyone will lead to the best local models. We propose a learning to collaborate framework, where each client can choose to collaborate with certain members in the network to achieve a “collaboration equilibrium", where smaller collaboration coalitions are formed within the network so that each client can obtain the model with the best utility. We propose the concept of benefit graph which describes how each client can benefit from collaborating with other clients and develop a Pareto optimization approach to obtain it. Finally the collaboration coalitions can be derived from it based on graph operations. Our framework provides a new way of setting up collaborations in a research network. Experiments on both synthetic and real world data sets are provided to demonstrate the effectiveness of our method.

heading: 1 INTRODUCTION
text: Effective learning of machine learning models over a collaborative network of data clients has drawn considerable interest in recent years. Frequently, due to privacy concerns, we cannot simultaneously access the raw data residing on different clients. Therefore, distributed (Li et al., 2014) or federated learning (McMahan et al., 2017) strategies have been proposed, where typically model parameters are updated locally at each client with its own data and the parameter updates, such as gradients, are transmitted out and communicate with other clients. During this process, it is usually assumed that the participation in the network comes at no cost, i.e., every client is willing to participate in the collaboration. However, this is not always true in reality.
One example is the clinical research network (CRN) involving multiple hospitals (Fleurence et al., 2014). Each hospital has its own patient population. The patient data are sensitive and cannot be shared with other hospitals. If we want to build a risk prediction model with the patient data within this network in a privacy-preserving way, the expectation from each hospital is that a better model can be obtained through participating in the CRN compared to the one built from its own data collected from various clinical practice with big efforts. In this scenario, there has been a prior study showing that the model performance can decrease when collaborating with hospitals with very distinct patient populations due to negative transfer induced by sample distribution discrepancies (Wang et al., 2019; Pan & Yang, 2009).
With these considerations, in this paper, we propose a novel learning to collaborate framework. We allow the participating clients in a large collaborative network to form non-overlapping collaboration coalitions. Each coalition includes a subset of clients such that the collaboration among them can benefit their respective model performance. We aim at identifying the collaboration coalitions that can lead to a collaboration equilibrium, i.e., there are no other coalition settings that any of the individual clients can benefit more (i.e., achieve better model performance).
To obtain the coalitions that can lead to a collaboration equilibrium, we propose a Pareto optimization framework to identify the necessary collaborators for each client in the network to achieve its maximum utility. In particular, we optimize a local model associated with a specific client on the
Under review as a conference paper at ICLR 2022
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
<latexit sha1_base64="078TlPinZnbp3G8YDOaCG04E8G4=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI</latexit>
<latexit sha1_base64="rxH3eGL3jlLzhYJGRaokrVTVvjc=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ</latexit>
<latexit sha1_base64="rX31m9pL4TDFitzg9FObuhvEhn0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK</latexit>
<latexit sha1_base64="078TlPinZnbp3G8YDOaCG04E8G4=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI</latexit>
<latexit sha1_base64="rxH3eGL3jlLzhYJGRaokrVTVvjc=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ</latexit>
<latexit sha1_base64="rX31m9pL4TDFitzg9FObuhvEhn0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK</latexit>
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
<latexit sha1_base64="078TlPinZnbp3G8YDOaCG04E8G4=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI</latexit>
<latexit sha1_base64="rxH3eGL3jlLzhYJGRaokrVTVvjc=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ</latexit>
<latexit sha1_base64="rX31m9pL4TDFitzg9FObuhvEhn0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK</latexit>
<latexit sha1_base64="078TlPinZnbp3G8YDOaCG04E8G4=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI</latexit>
<latexit sha1_base64="rxH3eGL3jlLzhYJGRaokrVTVvjc=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ</latexit>
<latexit sha1_base64="rX31m9pL4TDFitzg9FObuhvEhn0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK</latexit>
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
Pareto front of the learning objectives of all clients. Through the analysis of the geometric location of such an optimal model on the Pareto front, we can identify the necessary collaborators of each client. The relationships between each client and its necessary collaborators can be encoded in a benefit graph, in which each node denotes a client and the edge from Ij to Ii represents Ij is one of the necessary collaborators for Ii, as exemplified in Figure 1 (a). Then we can derive the coalitions corresponding to the collaboration equilibrium through an iterative process introduced as follows. Specifically, we define a stable coalition as the minimum set such that its all involved clients can achieve its maximal utility. From the perspective of graph theory, these stable coalitions are actually the strongly connected components of the benefit graph. For example, C = { I1, I2, I3 } in Figure 1 (b) is a stable coalition as all clients can achieve their best performance by collaborating with the clients in C (compared with collaborating with other clients in the network). By removing the stable coalitions and re-building the benefit graph of the remaining client iteratively as shown in Figure 1 (b) and (c), we can identify all coalitions as in Figure 1 (d) and prove that the obtained coalitions can lead to a collaboration equilibrium.
We empirically evaluate our method on synthetic data, UCI adult (Kohavi, 1996), a classical FL benchmark data set CIFAR10 (Krizhevsky et al., 2009), and a real-world electronic health record (EHR) data repository eICU (Pollard et al., 2018), which includes patient EHR data in ICU from multiple hospitals. The results show our method significantly outperforms existing relevant methods. The experiments on eICU data demonstrate that our algorithm is able to derive a good collaboration strategy for the hospitals to collaborate.

heading: 2 RELATED WORK

heading: 2.1 FEDERATED LEARNING
text: Federated learning (FL) (McMahan et al., 2017) refers to the paradigm of learning from fragmented data without sacrificing privacy. In a typical FL setting, a global model is learned from the data residing in multiple distinct local clients. However, a single global model may lead to performance degradation on certain clients due to data heterogeneity. Personalized federated learning (PFL) (Kulkarni et al., 2020), which aims at learning a customized model for each client in the federation, has been proposed to tackle this challenge. For example, Zhang et al. (2020) proposes to adjust the weight of the objectives corresponding to all clients dynamically; Fallah et al. (2020) proposes a meta-learning based method for achieving an effectively shared initialization of all local models followed by a fine-tuning procedure; Shamsian et al. (2021) proposes to learn a central hypernetwork which can generate a set of customized models for each client. FL assumes all clients are willing to participate in the collaboration and existing methods have not considered whether the collaboration can really benefit each client or not. Without benefit, a local client could be reluctant to participate in the collaboration, which is a realistic scenario we investigate in this paper. One specific FL setup that is relevant to our work is clustered federated learning (Sattler et al., 2020; Mansour et al., 2020), which groups the clients with similar data distributions and trains a model for each client group. The scenario we are considering in this paper is to form collaboration coalitions based on the performance gain each client can get for its corresponding model, rather than sample distribution similarities.

heading: 2.2 MULTI-TASK LEARNING AND NEGATIVE TRANSFER
text: Multi-task learning (Caruana, 1997) (MTL) aims at learning shared knowledge across multiple interrelated tasks for mutual benefits. Typical examples include hard model parameter sharing (Kokkinos, 2017), soft parameter sharing (Lu et al., 2017), and neural architecture search (NAS) for a shared
Under review as a conference paper at ICLR 2022
model architecture (Real et al., 2019). However, sharing representations of model structures cannot guarantee model performance gain due to the existence of negative transfer, while we directly consider forming collaboration coalitions according to individual model performance benefits. In addition, MTL usually assumes the data from all tasks are accessible, while our goal is to learn a personalized model for each client through collaborating with other clients without directly accessing their raw data. It is worth mentioning that there are also clustered MTL approaches (Standley et al., 2020; Zamir et al., 2018) which assume the models for the tasks within the same group are similar to each other, while we want the clients within each coalition can benefit each other through collaboration when learning their respective models. Some works extend MTL to a multi-source domain adaptation (Wen et al., 2020; Duan et al., 2009) setting in which unlabeled data exist in some tasks. See detailed discussions in Appendix.

heading: 3 COLLABORATION LEARNING PROBLEM
text: We first introduce the necessary notations and definitions in Section 3.1, and then define the collaboration equilibrium we aim to achieve in Section 3.2.

heading: 3.1 DEFINITIONS AND NOTATIONS
text: Suppose there are N clients I = { Ii }N i=1
in a collaborative network and each client is associated with a specific learning task T i based on its own data Di = { Xi, Y i } , i ∈ {1, 2, ..., N}, where
the input space Xi and the output space Y i may or may not share across all N clients. Each client pursues collaboration with others to learn a personalized model M i by maximizing its utility (i.e., model performance) without sacrificing data privacy. There is no guarantee that one client can always benefit from the collaboration with others, and the client would be reluctant to participate in the collaboration if there is no benefit. In the following, we describe this through a concrete example.
No benefit, no collaboration. Suppose the local data {Di}, i ∈ {1, 2, ..., N} owned by different clients satisfy the following conditions: 1) all local data are from the same distribution Di ∼ P; 2) D1 ⊂ D2 ⊂ D3, ...,⊂ DN . Since DN contains more data than other clients, IN cannot benefit from collaboration with any other clients, so IN will learn a local model using its own data. Once IN refuses to collaboration, IN−1 will also work on its own as IN−1 can only improve its utility by collaborating with IN . IN−2 will learn individually out of the same concerns. Finally, there is no collaboration among any clients.
Due to the discrepancies of the sample distributions across different clients, the best local model for a specific client is very likely to come from collaborating with a subset of clients rather than all of them. Suppose U(Ii, C) denotes the model utility of client Ii when collaborating with the clients in client set C. In the following, we define Umax(Ii, C) as the maximum model utility that Ii can achieve when collaborating with different subsets of C. Definition 1 (Maximum Achievable Utility (MAU)). This is the maximum model utility for a specific client Ii to collaborate with different subsets of client set C: Umax(Ii, C) = maxC′⊂C U(Ii, C ′).
From Definition 1, MAU satisfies Umax(Ii, C ′) ≤ Umax(Ii, C) if C ′ is a subset of C. Each client Ii ∈ I aims to identify its “optimal set" of collaborators from I to maximize its local utility, which is defined as follows. Definition 2 (Optimal Collaborator Set (OCS)). A client set CoptI (I
i) ⊂ I is an optimal collaborator set for Ii if and only if CoptI (I i) satisfies
∀C ⊂ I, U(Ii, CoptI (Ii)) ≥ U(Ii, C); (1a) ∀C ′ & CoptI (Ii), U(Ii, C opt I (I i)) > U(Ii, C ′). (1b)
Eq.(1a) means that Ii can achieve its maximal utility when collaborating with CoptI (I i) and Eq.(1b) means that all clients in CoptI (I i) are necessary. In this way, the relationships between any client Ii ∈ I and its optimal collaborator set CoptI (Ii) can be represented by a graph which is called the benefit graph (BG). Specifically, for a given client set C ⊂ I , we use BG(C) to denote its corresponding BG. For the example in Figure 1 (a), an arrow from Ij to Ii means Ij ∈ CoptI (Ii), e.g., I1 → I3 means I1 ∈ CoptI (I3). For a client set C, if every member can achieve its maximum
Under review as a conference paper at ICLR 2022
model utility through the collaboration with other members within C (without collaboration with other members outside C), then we call C a coalition.
I4
<latexit sha1_base64="Xwl+BCUvBvIgMA7IdpW3uxrkcDU=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hlpKk0zqYF5OJUko3/oBb/TLxD/QvvDOmoBbRCUnOnHvPmbn3eknAU2lZrwVjbn5hcam4XFpZXVvfKG9uNdM4Ez5z/DiIRdtzUxbwiDmSy4C1E8Hc0AtYy7s9VfHWHRMpj6MrOUpYN3SHER9w35VEOee9cW3SK1esqqWXOQvsHFSQr0ZcfsE1+ojhI0MIhgiScAAXKT0d2LCQENfFmDhBiOs4wwQl0maUxSjDJfaWvkPadXI2or3yTLXap1MCegUpTeyRJqY8QVidZup4pp0V+5v3WHuqu43o7+VeIbESN8T+pZtm/lenapEY4FjXwKmmRDOqOj93yXRX1M3NL1VJckiIU7hPcUHY18ppn02tSXXtqreujr/pTMWqvZ/nZnhXt6QB2z/HOQuaB1W7Vj28rFXqJ/moi9jBLvZpnkeo4wwNOOTN8YgnPBsXRmLcG6PPVKOQa7bxbRkPH0LnkQk=</latexit>
I5
<latexit sha1_base64="KKOEGmeLJ8PCjcxMDMM1BdYiV28=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Iq2B6JLoRl1hYoEECWmHASeUtmmnGkLY+ANu9cuMf6B/4Z2xJCoxOk3bM+fec2buvV7ki0Ra1mvOWFhcWl7JrxbW1jc2t4rbO40kTGPGHRb6Ydzy3IT7IuCOFNLnrSjm7sjzedMbnql4847HiQiDazmOeGfkDgLRF8yVRDkX3Ul12i2WrLKllzkP7AyUkK16WHzBDXoIwZBiBI4AkrAPFwk9bdiwEBHXwYS4mJDQcY4pCqRNKYtThkvskL4D2rUzNqC98ky0mtEpPr0xKU0ckCakvJiwOs3U8VQ7K/Y374n2VHcb09/LvEbEStwS+5dulvlfnapFoo8TXYOgmiLNqOpY5pLqrqibm1+qkuQQEadwj+IxYaaVsz6bWpPo2lVvXR1/05mKVXuW5aZ4V7ekAds/xzkPGkdlu1KuXlVKtdNs1HnsYR+HNM9j1HCOOhzyFnjEE56NSyMy7o3xZ6qRyzS7+LaMhw9FSJEK</latexit>
I6
<latexit sha1_base64="dLuWEcCiJIy7Wrla2OQjmor1BOM=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Iq3Bx5LoRl1hYoEECWnLgBNK27RTDSFs/AG3+mXGP9C/8M44JCoxOk3bM+fec2buvV4c8FRY1mvOmJtfWFzKLxdWVtfWN4qbW/U0yhKfOX4UREnTc1MW8JA5gouANeOEuUMvYA1vcCbjjTuWpDwKr8UoZu2h2w95j/uuIMq56IyPJp1iySpbapmzwNagBL1qUfEFN+gigo8MQzCEEIQDuEjpacGGhZi4NsbEJYS4ijNMUCBtRlmMMlxiB/Tt066l2ZD20jNVap9OCehNSGlijzQR5SWE5WmmimfKWbK/eY+Vp7zbiP6e9hoSK3BL7F+6aeZ/dbIWgR5OVA2caooVI6vztUumuiJvbn6pSpBDTJzEXYonhH2lnPbZVJpU1S5766r4m8qUrNz7OjfDu7wlDdj+Oc5ZUD8o25Xy4VWlVD3Vo85jB7vYp3keo4pz1OCQN8cjnvBsXBqxcW+MPlONnNZs49syHj4AR6mRCw==</latexit>
I1
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
I2
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
I3
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
achieve its best performance without collaborating with the clients in C.

heading: 3.2 PROBLEM SETUP
text: As each client in I aims to maximize its local model utility by forming a collaboration coalition with others, all clients in I can form several non-overlapping collaboration coalitions. In order to derive those coalitions, we propose the concept of collaboration equilibrium (CE) as follows.
Suppose we have a set of coalitions S = { C0, C1, ...CK } such that ⋃K k=1 C k = I andCk1 ⋂ Ck2 = ∅ for ∀k1 6= k2, then we say S reaches CE if it satisfies the following two axioms.
Axiom 1 (Inner Agreement). All collaboration coalitions satisfy inner agreement, i.e.,
∀C ∈ S, ∀C ′ & C, ∃Ii ∈ C ′, s.t., Umax(Ii, C ′) < Umax(Ii, C) (2)
From Axiom 1, inner agreement emphasizes that the clients of each coalition agree to form this coalition. It gives the necessary condition for a collaboration coalition to be formed such that any of the subset C ′ & C can benefit from the collaboration with C\C ′. Eq.(2) tells us that there always exists a client in C ′ that opposes leaving C because its utility will go down if C ′ is split from C. In this way, inner agreement guarantees that all coalitions will not fall apart or the clients involved will suffer. For example, S = {{ I1, I2, I3, I4 } , { I5, I6 }} in Figure 2 does not satisfy inner agreement,
because the clients in the subset C ′ = { I1, I2, I3 } achieves their optimal utility in C ′ and can leave
C = { I1, I2, I3, I4 } without any loss.
Axiom 2 (Outer Agreement). The collaboration strategy should satisfy outer agreement, i.e.,
∀C ′ /∈ S, ∃Ii ∈ C ′, s.t. Umax(Ii, C ′) ≤ Umax(Ii, C) (Ii ∈ C ∈ S) (3)
From Axiom 2, outer agreement guarantees that there is no other coalition C ′ which can benefit each client involved more than S achieves. Eq.(3) tells us that if C ′ is a coalition not from S, there always exists a client Ii and a coalition in C ∈ S such that Ii can benefit more. The collaboration strategy S = {{ I1, I2, I3 } , { I4 } , { I5, I6 }} in Figure 2 is a CE in which the clients in { I1, I2, I3 } and{
I5, I6 }
achieve their optimal model utility. Though I4 does not achieve its maximum model utility in { I4 }
, there is no other coalitions which can attract I2 and I5 to form a new coalition with I4. Therefore, all clients have no better choice but agree upon this collaboration strategy.
Our goal is to obtain a collaboration strategy to achieve CE which satisfies Axiom 1 and Axiom 2, so that all clients achieve their optimal model utilities in the collaboration coalition. For the nature of CE, we also share our extended discussions in Appendix. In the next section, we introduce our algorithm in detail on 1) how to derive a collaboration strategy that can achieve CE from the benefit graph and 2) how to construct the benefit graph.

heading: 4 COLLABORATION EQUILIBRIUM
text: In this section, we will introduce our framework on learning to collaborate. Firstly, we propose an iterative graph-theory-based method to achieve CE based on a given benefit graph.
Under review as a conference paper at ICLR 2022

heading: 4.1 ACHIEVING COLLABORATION EQUILIBRIUM GIVEN THE BENEFIT GRAPH
text: In theory, there are BN collaboration strategies for partitioning N clients into several coalitions, where BN is the Bell number which denotes how many solutions for partitioning a set with N elements (Bell, 1934). While exhaustive trying all partitions has exponential time complexity, in this section, we propose an iterative method for deriving a collaboration strategy that achieves CE with polynomial time complexity. Specifically, at each iteration, we search for a stable coalition which is formally defined in Definition 3 below, then we remove the clients in the stable coalition and re-build the benefit graph for the remaining clients. The iterations will continue until all clients are able to identify their own coalitions. Definition 3 (Stable Coalition). Given a client set I , a coalition Cs is stable if it satisfies
1. Each client in Cs achieves its maximal model utility, i.e., Umax(I i, Cs) = Umax(I i, I) ∀Ii ∈ Cs. (4)
2. Any sub coalition C ′ ⊂ Cs cannot achieve the maximal utility for all clients in C ′, i.e., ∀C ′ & Cs, ∃Ii ∈ C ′, s.t., Umax(Ii, C ′) < Umax(Ii, Cs). (5)
From Definition 3, Eq.(4) means that any client in a stable coalition cannot improve its model utility further. Eq.(5) states that this coalition is stable as any sub coalition C ′ can benefit from Cs\C ′. Therefore any sub coalition has no motivation to leave Cs. Eq.(4) implies that a stable coalition will not welcome any other clients to join as others will not benefit the clients in Cs further. In Figure 2,{ I1, I2, I3 } and { I5, I6 } are the two stable coalitions. In order to identify the stable coalitions from the benefit graph, we first introduce the concept of strongly connected component in a directed graph. Definition 4 (Strongly Connected Component (Tarjan, 1972)). A subgraphG′ is a strongly connected component of a given directed graph G if it satisfies: 1) It is strongly connected, which means that there is a path in each direction between each pair of vertices in G′; 2) It is maximal, which means no additional vertices from G can be included in G′ without breaking the property of being strongly connected.
Then we derive a graph-based method to obtain the collaboration coalitions that can achieve collaboration equilibrium by identifying all stable coalitions iteratively according to Theorem 1 below. Theorem 1. (Proof in Appendix) Given a client set I and its BG(I), the stable coalitions are strongly connected components of BG(I).
With Theorem 1, we need to identify all strongly connected components of BG(I), which can be achieved using the Tarjan algorithm (Tarjan, 1972) with time complexity O(V + E), where V is the number of nodes and E is the number of edges. Then following Eq.(4), we judge whether a strongly connected component is a stable coalition by checking whether all clients have achieved their maximal model utility. A stable coalition Cs has no interest to collaborate with other clients, so Cs will be removed and the remaining clients I\Cs will continue to seek collaborations until all N clients find their coalitions. In this way, we can achieve a partitioning strategy, with the details shown in Algorithm 1 in the Appendix. Theorem 2. (Proof in Appendix) The collaboration strategy obtained above achieves collaboration equilibrium.
The clients in all stable coalitions found in each iteration cannot improve their model utility further and will not collaborate with others because there are no additional benefits. Therefore, the collaboration strategy is approved by all clients. The iterative method achieves CE considering the BG varies in each iteration after removing the stable coalitions, which can be time-consuming because we need to redefine BG in each iteration by re-learning an optimal personalized model for each remaining client. Assumption 1. The benefit graph of a subset C ⊂ I (BG(C)) is the subgraph of the BG(I).
Assumption 1 claims that the benefit graph of the remaining clients I\Cs keeps unchanged when the subgraph BG(Cs) is split from BG(I). It implies that for each pair of clients Ii and Ij , whether Ii is one of the optimal collaborators for Ij will not be affected by other clients. In this case, we do not need to re-build the benefit graph and have the following corollary. Corollary 1. (proof in Appendix) When Assumption 1 holds, the strongly connected components of BG(I) leads to a collaboration equilibrium.
Under review as a conference paper at ICLR 2022

heading: 4.2 DETERMINE THE BENEFIT GRAPH BY SPECIFIC PARETO OPTIMIZATION
text: The benefit graph of N clients consists of the clients and their corresponding OCS. However, each client has 2N−1 collaborator sets and it’s hard to determine which one is the OCS. Exhaustive trying all sets to determine the OCS may be impractical, especially when there are many clients. To identify the OCS effectively and efficiently, we propose Specific Pareto Optimization (SPO) to alternately perform the following two steps: 1). learn a Pareto model (defined below) given the weight of all clients d; 2). optimize the weight vector d to search for a best model M∗(Ii) by gradient descent.
Definition 5 (Pareto Solution and Pareto Front). We consider n objectives corresponding to n clients: li : Rn → R+, i = {1, 2, ..., n}. Given a learned hypothesis h, suppose the loss vector l(h) = [l1, l2, ..., ln] represents the utility loss on n clients with hypothesis h ∈ H, we say h is a Pareto Solution if there is no hypothesis h′ that dominates h, often called Pareto optimality, i.e.,
@h′ ∈ H, s.t. ∀i : li (h′) ≤ li(h) and ∃j : lj (h′) < lj(h).
In a collaboration network with N clients {Ii}Ni=1, as each client has its own learning task which can be formulated as a specific objective, we use P ( { I1, I2, ..., IN } ) to represent the Pareto Front (PF) of the client set {Ii}Ni=1 formed by all Pareto hypothesis.
Learning a best model and identify the OCS by SPO To search for an optimal model M∗(Ii) for Ii, we propose to firstly learn the empirical Pareto Front by a hypernetwork denoted as HN 1. As each client owns its objective, given the weight of all clients (objectives), the learned PT by HN outputs a Pareto model M ,
M = HN(d), (6) where d = [d1, d2, ..., dN ] satisfying ∑N
i=1 d i = 1 and di denotes the weight of the objective li.
Each d corresponds to a specific Pareto model M , and all Pareto models M satisfy Pareto optimality that the losses on the training data of all clients cannot be further optimized.
Though the Pareto models maximize the utilization of the training data from all clients, they may not be the best models on the true (test) data distribution. For example, HN(d4) achieves a minimum loss on the training data of I1 shown in Figure 3 (b), but it is not the optimal model on the true data distribution of I1 shown in Figure 3 (c). Therefore, we propose to search for a best Pareto model that achieves the best performance on the true (validation) data set of the target client Ii, i.e.,
M∗(Ii) = HN(d∗), where d∗ = argmax d Per(HN(d), Ii), (7)
where Per(HN(d), Ii) denotes the performance of HN(d) on the validation data set of Ii. We identify the OCS of Ii based on the optimized weight of all clients d∗. For example shown in Figure 3(c), the best model is h∗1 for client I
1 and the corresponding optimized weight d∗ = d3 = [0.7, 0.3, 0]. So the OCS of I1 consists of the clients with non-zero weights, which is { I1, I2 } .
<latexit sha1_base64="cudA1LBluwoZv1calCLXV3a4dA8=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRlxVMG2hlpJMp3VoXiQTpZRu/AG3+mXiH+hfeGdMQS2iE5KcOfeeM3Pv9WJfpNKyXgvGwuLS8kpxtbS2vrG5Vd7eaaZRljDusMiPkrbnptwXIXekkD5vxwl3A8/nLW90ruKtO56kIgqv5Tjm3cAdhmIgmCuJcvzexJ72yhWraullzgM7BxXkqxGVX3CDPiIwZAjAEUIS9uEipacDGxZi4rqYEJcQEjrOMUWJtBllccpwiR3Rd0i7Ts6GtFeeqVYzOsWnNyGliQPSRJSXEFanmTqeaWfF/uY90Z7qbmP6e7lXQKzELbF/6WaZ/9WpWiQGONU1CKop1oyqjuUume6Kurn5pSpJDjFxCvcpnhBmWjnrs6k1qa5d9dbV8TedqVi1Z3luhnd1Sxqw/XOc86B5VLVr1eOrWqV+lo+6iD3s45DmeYI6LtCAQ94Cj3jCs3FpxMa9Mf5MNQq5ZhfflvHwAY9wkSk=</latexit>
<latexit sha1_base64="opHOc9XKYD/lL4gYlWp76hdYl0Y=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVZKi6LLoRlxVMG2hlpKk0zp08iCZKKV04w+41S8T/0D/wjtjCmoRnZDkzLn3nJl7rxcLnkrLei0YC4tLyyvF1dLa+sbmVnl7p5lGWeIzx49ElLQ9N2WCh8yRXArWjhPmBp5gLW90ruKtO5akPAqv5Thm3cAdhnzAfVcS5YjepDbtlStW1dLLnAd2DirIVyMqv+AGfUTwkSEAQwhJWMBFSk8HNizExHUxIS4hxHWcYYoSaTPKYpThEjui75B2nZwNaa88U6326RRBb0JKEwekiSgvIaxOM3U8086K/c17oj3V3cb093KvgFiJW2L/0s0y/6tTtUgMcKpr4FRTrBlVnZ+7ZLor6ubml6okOcTEKdyneELY18pZn02tSXXtqreujr/pTMWqvZ/nZnhXt6QB2z/HOQ+atap9VD2+OqrUz/JRF7GHfRzSPE9QxwUacMib4xFPeDYujdi4N8afqUYh1+zi2zIePgCR0ZEq</latexit>
<latexit sha1_base64="cudA1LBluwoZv1calCLXV3a4dA8=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRlxVMG2hlpJMp3VoXiQTpZRu/AG3+mXiH+hfeGdMQS2iE5KcOfeeM3Pv9WJfpNKyXgvGwuLS8kpxtbS2vrG5Vd7eaaZRljDusMiPkrbnptwXIXekkD5vxwl3A8/nLW90ruKtO56kIgqv5Tjm3cAdhmIgmCuJcvzexJ72yhWraullzgM7BxXkqxGVX3CDPiIwZAjAEUIS9uEipacDGxZi4rqYEJcQEjrOMUWJtBllccpwiR3Rd0i7Ts6GtFeeqVYzOsWnNyGliQPSRJSXEFanmTqeaWfF/uY90Z7qbmP6e7lXQKzELbF/6WaZ/9WpWiQGONU1CKop1oyqjuUume6Kurn5pSpJDjFxCvcpnhBmWjnrs6k1qa5d9dbV8TedqVi1Z3luhnd1Sxqw/XOc86B5VLVr1eOrWqV+lo+6iD3s45DmeYI6LtCAQ94Cj3jCs3FpxMa9Mf5MNQq5ZhfflvHwAY9wkSk=</latexit>
<latexit sha1_base64="opHOc9XKYD/lL4gYlWp76hdYl0Y=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVZKi6LLoRlxVMG2hlpKk0zp08iCZKKV04w+41S8T/0D/wjtjCmoRnZDkzLn3nJl7rxcLnkrLei0YC4tLyyvF1dLa+sbmVnl7p5lGWeIzx49ElLQ9N2WCh8yRXArWjhPmBp5gLW90ruKtO5akPAqv5Thm3cAdhnzAfVcS5YjepDbtlStW1dLLnAd2DirIVyMqv+AGfUTwkSEAQwhJWMBFSk8HNizExHUxIS4hxHWcYYoSaTPKYpThEjui75B2nZwNaa88U6326RRBb0JKEwekiSgvIaxOM3U8086K/c17oj3V3cb093KvgFiJW2L/0s0y/6tTtUgMcKpr4FRTrBlVnZ+7ZLor6ubml6okOcTEKdyneELY18pZn02tSXXtqreujr/pTMWqvZ/nZnhXt6QB2z/HOQ+atap9VD2+OqrUz/JRF7GHfRzSPE9QxwUacMib4xFPeDYujdi4N8afqUYh1+zi2zIePgCR0ZEq</latexit>
<latexit sha1_base64="ggQwfb6hlt8KfQoEypmzZ0xC6Fc=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKt6LLoRlxVMK1QS0mm0zo0L5KJUko3/oBb/TLxD/QvvDOmoBbRCUnOnHvPmbn3erEvUmlZrwVjbn5hcam4XFpZXVvfKG9uNdMoSxh3WORHybXnptwXIXekkD6/jhPuBp7PW97wTMVbdzxJRRReyVHMO4E7CEVfMFcS5fjd8eGkW65YVUsvcxbYOaggX42o/IIb9BCBIUMAjhCSsA8XKT1t2LAQE9fBmLiEkNBxjglKpM0oi1OGS+yQvgPatXM2pL3yTLWa0Sk+vQkpTeyRJqK8hLA6zdTxTDsr9jfvsfZUdxvR38u9AmIlbon9SzfN/K9O1SLRx4muQVBNsWZUdSx3yXRX1M3NL1VJcoiJU7hH8YQw08ppn02tSXXtqreujr/pTMWqPctzM7yrW9KA7Z/jnAXNg6pdqx5d1ir103zURexgF/s0z2PUcY4GHPIWeMQTno0LIzbujdFnqlHINdv4toyHD5QykSs=</latexit>
<latexit sha1_base64="cudA1LBluwoZv1calCLXV3a4dA8=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRlxVMG2hlpJMp3VoXiQTpZRu/AG3+mXiH+hfeGdMQS2iE5KcOfeeM3Pv9WJfpNKyXgvGwuLS8kpxtbS2vrG5Vd7eaaZRljDusMiPkrbnptwXIXekkD5vxwl3A8/nLW90ruKtO56kIgqv5Tjm3cAdhmIgmCuJcvzexJ72yhWraullzgM7BxXkqxGVX3CDPiIwZAjAEUIS9uEipacDGxZi4rqYEJcQEjrOMUWJtBllccpwiR3Rd0i7Ts6GtFeeqVYzOsWnNyGliQPSRJSXEFanmTqeaWfF/uY90Z7qbmP6e7lXQKzELbF/6WaZ/9WpWiQGONU1CKop1oyqjuUume6Kurn5pSpJDjFxCvcpnhBmWjnrs6k1qa5d9dbV8TedqVi1Z3luhnd1Sxqw/XOc86B5VLVr1eOrWqV+lo+6iD3s45DmeYI6LtCAQ94Cj3jCs3FpxMa9Mf5MNQq5ZhfflvHwAY9wkSk=</latexit>
<latexit sha1_base64="opHOc9XKYD/lL4gYlWp76hdYl0Y=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVZKi6LLoRlxVMG2hlpKk0zp08iCZKKV04w+41S8T/0D/wjtjCmoRnZDkzLn3nJl7rxcLnkrLei0YC4tLyyvF1dLa+sbmVnl7p5lGWeIzx49ElLQ9N2WCh8yRXArWjhPmBp5gLW90ruKtO5akPAqv5Thm3cAdhnzAfVcS5YjepDbtlStW1dLLnAd2DirIVyMqv+AGfUTwkSEAQwhJWMBFSk8HNizExHUxIS4hxHWcYYoSaTPKYpThEjui75B2nZwNaa88U6326RRBb0JKEwekiSgvIaxOM3U8086K/c17oj3V3cb093KvgFiJW2L/0s0y/6tTtUgMcKpr4FRTrBlVnZ+7ZLor6ubml6okOcTEKdyneELY18pZn02tSXXtqreujr/pTMWqvZ/nZnhXt6QB2z/HOQ+atap9VD2+OqrUz/JRF7GHfRzSPE9QxwUacMib4xFPeDYujdi4N8afqUYh1+zi2zIePgCR0ZEq</latexit>
<latexit sha1_base64="Q4cVN5ImikMo2cCdUabe79Vka6I=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkUQFyWRii6LblxJBfuQWkuSTtvQvJhMhBKy9Qfc6neJf6B/4Z0xglpEJyQ5c+49Z+bea0eeGwvDeCloc/MLi0vF5dLK6tr6RnlzqxWHCXdY0wm9kHdsK2aeG7CmcIXHOhFnlm97rG1PzmS8fcd47IbBlZhGrOdbo8Aduo4liLoe91Mzu00Psn65YlQNtfRZYOaggnw1wvIzbjBACAcJfDAEEIQ9WIjp6cKEgYi4HlLiOCFXxRkylEibUBajDIvYCX1HtOvmbEB76RkrtUOnePRyUurYI01IeZywPE1X8UQ5S/Y371R5yrtN6W/nXj6xAmNi/9J9Zv5XJ2sRGOJE1eBSTZFiZHVO7pKorsib61+qEuQQESfxgOKcsKOUn33WlSZWtcveWir+qjIlK/dOnpvgTd6SBmz+HOcsaB1WzVr16LJWqZ/moy5iB7vYp3keo45zNNAkbx8PeMSTdqEJLdWyj1StkGu28W1p9++3QpLN</latexit>
<latexit sha1_base64="WbdFWdKM/yAwwLWzlQgo5eqYQJI=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkUQFyUtFV0W3biSCvYhtZZkOm1D8yKZCCVk6w+41e8S/0D/wjtjCmoRnZDkzLn3nJl7rxU4diQM4zWnLSwuLa/kVwtr6xubW8XtnVbkxyHjTeY7ftixzIg7tsebwhYO7wQhN13L4W1rci7j7XseRrbvXYtpwHuuOfLsoc1MQdTNuJ9U07vkKO0XS0bZUEufB5UMlJCthl98wS0G8MEQwwWHB0HYgYmIni4qMBAQ10NCXEjIVnGOFAXSxpTFKcMkdkLfEe26GevRXnpGSs3oFIfekJQ6DkjjU15IWJ6mq3isnCX7m3eiPOXdpvS3Mi+XWIExsX/pZpn/1claBIY4VTXYVFOgGFkdy1xi1RV5c/1LVYIcAuIkHlA8JMyUctZnXWkiVbvsranibypTsnLPstwY7/KWNODKz3HOg1a1XKmVj69qpfpZNuo89rCPQ5rnCeq4QANN8nbxiCc8a5ea0BIt/UzVcplmF9+W9vABuaeSzg==</latexit>
<latexit sha1_base64="UcThipVmL9Q5gOKu6AlY36y2FKQ=">AAAC7XicjVHLSsNAFD3GV31XXboJFqGClMQHuhTduKxga6GRksRpHZwmYTIRpPQX3LkTt/6AW/0N8Q/0L7wzTcEHohMyc+bce87MnRskgqfKcV5HrNGx8YnJwtT0zOzc/EJxcamexpkMWS2MRSwbgZ8ywSNWU1wJ1kgk87uBYKfB5aGOn14xmfI4OlHXCTvr+p2It3noK6JaxXK17AnWVl7PFq2e29/Qy+Zg2erbnuSdC+X17fVWseRUHDPsn8DNQQn5qMbFF3g4R4wQGbpgiKAIC/hI6WvChYOEuDP0iJOEuIkz9DFN2oyyGGX4xF7S3KFdM2cj2mvP1KhDOkXQL0lpY400MeVJwvo028Qz46zZ37x7xlPf7ZrWIPfqEqtwQexfumHmf3W6FoU29kwNnGpKDKOrC3OXzLyKvrn9qSpFDglxGp9TXBIOjXL4zrbRpKZ2/ba+ib+ZTM3qfZjnZnjXt6QGu9/b+RPUNyvudmXneLu0f5C3uoAVrKJM/dzFPo5QRY28b/CIJzxbsXVr3Vn3g1RrJNcs48uwHj4AK52ebQ==</latexit>
<latexit sha1_base64="wXvsXwJbWdeumt7mhrARD3nnRZY=">AAAC5XicjVHLSsNAFD3G97vq0k2wCBWkJKLoUnTjsoLVgiklSafp0GkSJhNBSrfu3Ilbf8Ct/or4B/oX3hkj+EB0QmbOnHvPmblzg1TwTDnO84g1OjY+MTk1PTM7N7+wWFpaPs2SXIasHiYikY3Az5jgMasrrgRrpJL5/UCws6B3qONnF0xmPIlP1GXKmn0/inmHh74iqlWyaxVPsI7yBrZoDdzhpl62hp7kUVd5Q3ujVSo7VccM+ydwC1BGMWpJ6Qke2kgQIkcfDDEUYQEfGX3ncOEgJa6JAXGSEDdxhiFmSJtTFqMMn9gezRHtzgs2pr32zIw6pFME/ZKUNtZJk1CeJKxPs008N86a/c17YDz13S5pDQqvPrEKXWL/0n1k/lena1HoYM/UwKmm1DC6urBwyc2r6Jvbn6pS5JASp3Gb4pJwaJQf72wbTWZq12/rm/iLydSs3odFbo5XfUtqsPu9nT/B6VbV3a7uHG+X9w+KVk9hFWuoUD93sY8j1FAn7yvc4wGPVmRdWzfW7XuqNVJoVvBlWHdv9Dmbuw==</latexit>
<latexit sha1_base64="ADly3S7vcp1rmNa9+P0ASxDZ1Aw=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRlxVMG2hlpJMp3VoXiQTpZRu/AG3+mXiH+hfeGdMQS2iE5KcOfeeM3Pv9WJfpNKyXgvGwuLS8kpxtbS2vrG5Vd7eaaZRljDusMiPkrbnptwXIXekkD5vxwl3A8/nLW90ruKtO56kIgqv5Tjm3cAdhmIgmCuJcvq9iT3tlStW1dLLnAd2DirIVyMqv+AGfURgyBCAI4Qk7MNFSk8HNizExHUxIS4hJHScY4oSaTPK4pThEjui75B2nZwNaa88U61mdIpPb0JKEwekiSgvIaxOM3U8086K/c17oj3V3cb093KvgFiJW2L/0s0y/6tTtUgMcKprEFRTrBlVHctdMt0VdXPzS1WSHGLiFO5TPCHMtHLWZ1NrUl276q2r4286U7Fqz/LcDO/qljRg++c450HzqGrXqsdXtUr9LB91EXvYxyHN8wR1XKABh7wFHvGEZ+PSiI17Y/yZahRyzS6+LePhA3xQkSE=</latexit>
<latexit sha1_base64="SkfLKD59GddKZ+u4nFmzr9ASLv8=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVZKi6LLoRlxVMG2hlpJMp3VoXiQTpZRu/AG3+mXiH+hfeGdMQS2iE5KcOfeeM3Pv9WJfpNKyXgvGwuLS8kpxtbS2vrG5Vd7eaaZRljDusMiPkrbnptwXIXekkD5vxwl3A8/nLW90ruKtO56kIgqv5Tjm3cAdhmIgmCuJcvq9SW3aK1esqqWXOQ/sHFSQr0ZUfsEN+ojAkCEARwhJ2IeLlJ4ObFiIietiQlxCSOg4xxQl0maUxSnDJXZE3yHtOjkb0l55plrN6BSf3oSUJg5IE1FeQlidZup4pp0V+5v3RHuqu43p7+VeAbESt8T+pZtl/lenapEY4FTXIKimWDOqOpa7ZLor6ubml6okOcTEKdyneEKYaeWsz6bWpLp21VtXx990pmLVnuW5Gd7VLWnA9s9xzoNmrWofVY+vjir1s3zURexhH4c0zxPUcYEGHPIWeMQTno1LIzbujfFnqlHINbv4toyHD36xkSI=</latexit>
<latexit sha1_base64="lcvu4y2WcBVG844uabTDNqp3j+k=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKt6LLoRlxVMK1QS0mm0zo0L5KJUko3/oBb/TLxD/QvvDOmoBbRCUnOnHvPmbn3erEvUmlZrwVjbn5hcam4XFpZXVvfKG9uNdMoSxh3WORHybXnptwXIXekkD6/jhPuBp7PW97wTMVbdzxJRRReyVHMO4E7CEVfMFcS5fS648NJt1yxqpZe5iywc1BBvhpR+QU36CECQ4YAHCEkYR8uUnrasGEhJq6DMXEJIaHjHBOUSJtRFqcMl9ghfQe0a+dsSHvlmWo1o1N8ehNSmtgjTUR5CWF1mqnjmXZW7G/eY+2p7jaiv5d7BcRK3BL7l26a+V+dqkWijxNdg6CaYs2o6ljukumuqJubX6qS5BATp3CP4glhppXTPptak+raVW9dHX/TmYpVe5bnZnhXt6QB2z/HOQuaB1W7Vj26rFXqp/moi9jBLvZpnseo4xwNOOQt8IgnPBsXRmzcG6PPVKOQa7bxbRkPH4ESkSM=</latexit>
<latexit sha1_base64="gadRRTH7nCcRdKfbaXipfNSZTNc=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRlxVMG2hlpJMp3VoXiQTpZRu/AG3+mXiH+hfeGdMQS2iE5KcOfeeM3Pv9WJfpNKyXgvGwuLS8kpxtbS2vrG5Vd7eaaZRljDusMiPkrbnptwXIXekkD5vxwl3A8/nLW90ruKtO56kIgqv5Tjm3cAdhmIgmCuJcvq9SW3aK1esqqWXOQ/sHFSQr0ZUfsEN+ojAkCEARwhJ2IeLlJ4ObFiIietiQlxCSOg4xxQl0maUxSnDJXZE3yHtOjkb0l55plrN6BSf3oSUJg5IE1FeQlidZup4pp0V+5v3RHuqu43p7+VeAbESt8T+pZtl/lenapEY4FTXIKimWDOqOpa7ZLor6ubml6okOcTEKdyneEKYaeWsz6bWpLp21VtXx990pmLVnuW5Gd7VLWnA9s9xzoPmUdWuVY+vapX6WT7qIvawj0Oa5wnquEADDnkLPOIJz8alERv3xvgz1Sjkml18W8bDB4NzkSQ=</latexit>
<latexit sha1_base64="Q4cVN5ImikMo2cCdUabe79Vka6I=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkUQFyWRii6LblxJBfuQWkuSTtvQvJhMhBKy9Qfc6neJf6B/4Z0xglpEJyQ5c+49Z+bea0eeGwvDeCloc/MLi0vF5dLK6tr6RnlzqxWHCXdY0wm9kHdsK2aeG7CmcIXHOhFnlm97rG1PzmS8fcd47IbBlZhGrOdbo8Aduo4liLoe91Mzu00Psn65YlQNtfRZYOaggnw1wvIzbjBACAcJfDAEEIQ9WIjp6cKEgYi4HlLiOCFXxRkylEibUBajDIvYCX1HtOvmbEB76RkrtUOnePRyUurYI01IeZywPE1X8UQ5S/Y371R5yrtN6W/nXj6xAmNi/9J9Zv5XJ2sRGOJE1eBSTZFiZHVO7pKorsib61+qEuQQESfxgOKcsKOUn33WlSZWtcveWir+qjIlK/dOnpvgTd6SBmz+HOcsaB1WzVr16LJWqZ/moy5iB7vYp3keo45zNNAkbx8PeMSTdqEJLdWyj1StkGu28W1p9++3QpLN</latexit>
<latexit sha1_base64="J5Yzkk8O6EiWy0ejFA/sMf4PCf8=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoxpVUsA+ptSTTaRuaF8lEKCFbf8Ctfpf4B/oX3hlTUIvohCRnzr3nzNx77dB1YmEYrwVtYXFpeaW4Wlpb39jcKm/vtOIgiRhvssANoo5txdx1fN4UjnB5J4y45dkub9uTcxlv3/ModgL/WkxD3vOske8MHWYJom7Gd6mR9VMz65crRtVQS58HZg4qyFcjKL/gFgMEYEjggcOHIOzCQkxPFyYMhMT1kBIXEXJUnCNDibQJZXHKsIid0HdEu27O+rSXnrFSMzrFpTcipY4D0gSUFxGWp+kqnihnyf7mnSpPebcp/e3cyyNWYEzsX7pZ5n91shaBIU5VDQ7VFCpGVsdyl0R1Rd5c/1KVIIeQOIkHFI8IM6Wc9VlXmljVLntrqfibypSs3LM8N8G7vCUN2Pw5znnQOqqaterxVa1SP8tHXcQe9nFI8zxBHRdooEneHh7xhGftUhNaqmWfqVoh1+zi29IePgDFgJLT</latexit>
<latexit sha1_base64="Q4cVN5ImikMo2cCdUabe79Vka6I=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkUQFyWRii6LblxJBfuQWkuSTtvQvJhMhBKy9Qfc6neJf6B/4Z0xglpEJyQ5c+49Z+bea0eeGwvDeCloc/MLi0vF5dLK6tr6RnlzqxWHCXdY0wm9kHdsK2aeG7CmcIXHOhFnlm97rG1PzmS8fcd47IbBlZhGrOdbo8Aduo4liLoe91Mzu00Psn65YlQNtfRZYOaggnw1wvIzbjBACAcJfDAEEIQ9WIjp6cKEgYi4HlLiOCFXxRkylEibUBajDIvYCX1HtOvmbEB76RkrtUOnePRyUurYI01IeZywPE1X8UQ5S/Y371R5yrtN6W/nXj6xAmNi/9J9Zv5XJ2sRGOJE1eBSTZFiZHVO7pKorsib61+qEuQQESfxgOKcsKOUn33WlSZWtcveWir+qjIlK/dOnpvgTd6SBmz+HOcsaB1WzVr16LJWqZ/moy5iB7vYp3keo45zNNAkbx8PeMSTdqEJLdWyj1StkGu28W1p9++3QpLN</latexit>
1More information please refer to (Navon et al., 2020)
Under review as a conference paper at ICLR 2022
Proposition 1 (Pareto Front Embedding Property). (proof in Appendix) Suppose l′∗ = [li(h′∗)], i ∈ C ′ and l∗ = [li(h∗)], i ∈ C are the loss vectors achieved by the PFs P (C ′) and P (C) whereC ′ ⊂ C, then
∀h′∗ ∈ P (C ′), ∃h∗ ∈ P (C), s.t., li(h′∗) = li(h∗) ∀i ∈ C ′. (8)
From Proposition 1, the loss vectors achieved by the PF of a sub-coalition are embedded in the loss vectors of the full coalition, such as the loss curve of P ({l1, l2}) is in the loss plane of P ({l1, l2, l3}) shown in Figure 3 (a) and (b).
Explanation of SPO from a geometric point of view Intuitively, if the best model M∗(Ii) searched on the PF of all clients P (I) belongs to the PF of a sub coalition P (C) simultaneously, i.e.,
M∗(Ii) ∈ P (C) and M∗(Ii) ∈ P (I), (9) then the clients I\C are not necessary for obtaining M∗(Ii). However, exhaustively trying the PF of all sub-coalitions can have exponential time complexity. From Proposition 1, the PF of full coalition I contains the PF of all sub-coalitions. Therefore, we propose SPO to search for a best model M∗(Ii) on the PF of full coalition I , and identify whether M∗(Ii) belongs to the PF of a sub-coalition using the optimized weight vector d. For example, suppose there are three clients and the three corresponding objectives achieved by the PF are shown in Figure 3(a). The model h∗1 in Figure 3(a) is on the PF P ({l1, l2, l3}). Since the corresponding weight d = [0.7, 0.3, 0], from Figure 3(b), h∗1 is also on the PF P ({l1, l2}), so I3 is not a necessary client and the OCS of I1 is {l1, l2}. More discussion about optimality and implementation details are in Appendix.

heading: 5 EXPERIMENTS
text: To intuitively demonstrate the motivation of collaboration equilibrium and the effectiveness of SPO, we conduct experiments on synthetic data, a real-world UCI dataset Adult (Kohavi, 1996) and a benchmark data set CIFAR10 (LeCun et al., 1998). Moreover, we verify the practicability of our framework on a real-world multiple hospitals collaboration network using the electronic health record (EHR) data set eICU (Pollard et al., 2018). As SPO aims to achieve an optimal model utility by optimizing the personalized model on the PF of all clients, we use SPO to denote the model utility achieved by SPO. According to the OCS determined by SPO we achieve a CE for all clients and the model utility of each client in the CE can be different from the utility achieved by SPO. We use CE to denote the model utility achieved in the CE without causing further confusion. To verify the practicability of our framework, we also compare the time consumption and provide more experimental results in Appendix.

heading: 5.1 SYNTHETIC EXPERIMENTS
text: Synthetic data Suppose there are 6 clients in the collaboration network. The synthetic features owned by each client Ii are generated by x ∼ U [−1.0, 1.0]; the ground-truth weights ui = v+ri are samples as v ∼ U [0.0, 1.0], ri ∼ N (0d, ρ2) where ρ2 represents the client variance (if ρ2 increases, the data distribution discrepancy among clients will increase). Labels of the clients are observed with i.i.d noise ∼ N ( 0, σ2 ) . To generate conflicting learning tasks assigned to different clients, we flip over the label of some clients: y = u>i x+ , i ∈ {0, 1, 2} and y = −u>i x+ , i ∈ {3, 4, 5} . From Table 1, when there are fewer samples (n = 2000) and less distribution discrepancy ρ = 0.1 in the client set { I0, I1, I2 } or { I3, I4, I5 } with similar label generation process, these clients collaborate with others to achieve a low MSE. In this case, the OCS of each client is the clients with similar learning tasks and we achieve CE as S = {{Ii}2i=0, {Ii}5i=3} as shown in the top of Figure 4 (a). With the increase of the number of samples and the distribution discrepancy, collaboration cannot benefit the clients and all clients will learn individually on their own data. Therefore, when n = 20000 and ρ = 1.0, the OCS of each client is itself and the collaboration strategy S = {{I0}, {I1}, ., {I5}} leads to a CE as shown in the bottom of Figure 4 (a).
UCI adult data adult contains more than 40000 adult records and the task is to predict whether an individual earns more than 50K/year given other features (e.g., age, gender, education, etc.). Following the setting in (Li et al., 2019; Mohri et al., 2019), we split the data set into two clients. One is PhD client (I1) in which all individuals are PhDs and the other is non-PhD client (I0). In this
Under review as a conference paper at ICLR 2022
experiment, we implement SPO on this data set and compare the performance with existing relevant methods AFL (Mohri et al., 2019) and q-FFL (Li et al., 2019)2.
The two clients I0 and I1 have different data distribution and non-PhD client has more than 30000 samples while PhD client has about 500 samples. From Table 2, SPO achieves higher accuracy compared to baselines especially on PhD clients (77.0). non-PhD client achieves an optimal accuracy (83.5) by local training. Therefore, PhD client improves its performance by collaborating with non-PhD client while the performance of non-PhD client declines. The benefit graph is shown in the top of Figure 4 (b). The CE is non-collaboration as in the bottom of Figure 4 (b) and the model of both clients in the CE are trained individually.

heading: 5.2 BENCHMARK EXPERIMENTS
text: We compare our method with previous personalized federated learning (PFL) methods on CIFAR10 (Krizhevsky et al., 2009)3. Following the setting in (McMahan et al., 2016), we simulate non-i.i.d environment by randomly assigning two classes to each client among ten total classes. Baselines we evaluate are as follows: (1) Local training on each client; (2) FedAvg (McMahan et al., 2016); (3) Per-FedAvg (Fallah et al., 2020), a meta-learning based PFL algorithm. (4) pFedMe (T Dinh et al., 2020), a PFL approach which adds a Moreau-envelopes loss term; (5) LG-FedAvg (Liang et al., 2020) PFL method with local feature extractor and global output layers; (6) FedPer (Arivazhagan et al., 2019), a PFL approach that learns personal classifier on top of a shared feature extractor; (7) pFedHN (Shamsian et al., 2021), a PFL approach that generates models by training a hyper-network. In all experiments, our target network shares the same architecture as the baseline models. For each client, we split 87% of the training data for learning a Pareto Front by collaborating with the others and the remaining 13% of the training data for optimizing the vector d to reach an optimal model as shown in Figure 3 (c). More implementation details are in Appendix.
Table 3 reports the results of all methods. FedAve achieves a lower accuracy (51.4) compared to local training (86.46) which means that training a global model can hurt the performance of each client. Compared to other PFL methods in Table 3, SPO reaches an optimal model on the PF of all objectives and achieves a higher accuracy (92.47). As the features learned from the images are transferable though there is a label shift among all clients, the collaboration among all clients leads to a more efficient feature extractor for each client. Therefore, the benefit graph of this collaboration network is a fully connected graph and the collaboration equilibrium is that all clients form a full coalition for collaboration as shown in Figure 4 (c). In this experiment, the accuracy model of each clients in CE equals to the accuracy achieved by SPO.
Table 1: Synthetic
Table 2: Adult
Table 3: CIFAR10
<latexit sha1_base64="xkZinkB3fXx+z+jqDlELdUOZnwY=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2JpclytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zcAkQQ=</latexit>
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
<latexit sha1_base64="078TlPinZnbp3G8YDOaCG04E8G4=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI</latexit>
<latexit sha1_base64="rxH3eGL3jlLzhYJGRaokrVTVvjc=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ</latexit>
<latexit sha1_base64="xkZinkB3fXx+z+jqDlELdUOZnwY=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2JpclytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zcAkQQ=</latexit>
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
<latexit sha1_base64="078TlPinZnbp3G8YDOaCG04E8G4=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI</latexit>
<latexit sha1_base64="rxH3eGL3jlLzhYJGRaokrVTVvjc=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ</latexit>
<latexit sha1_base64="Ur/sI7S0Zw7wTwgN7bh/9weye80=">AAAC2XicjVHLSsNAFD2Nr1pf8bFzEyyCCymTUtGNUHTjsoJ9QC0lSUcNpkmYTIRaXLgTt/6AW/0h8Q/0L7wzpqAW0QlJzj33njNz57px4CeSsdecMTE5NT2Tny3MzS8sLpnLK40kSoXH614URKLlOgkP/JDXpS8D3ooFd/puwJvu5aHKN6+4SPwoPJGDmHf6znnon/meI4nqmmuhtW+VGWPb1qm4iChgJbtrFlmJ6WWNAzsDRWSrFpkvOEUPETyk6IMjhCQcwEFCTxs2GGLiOhgSJwj5Os9xgwJpU6riVOEQe0nfc4raGRtSrDwTrfZol4BeQUoLm6SJqE4QVrtZOp9qZ8X+5j3UnupsA/q7mVefWIkLYv/SjSr/q1O9SJxhT/fgU0+xZlR3XuaS6ltRJ7e+dCXJISZO4R7lBWFPK0f3bGlNontXd+vo/JuuVKyKvaw2xbs6JQ3Y/jnOcdAol+xKaee4UqweZKPOYx0b2KJ57qKKI9RQJ+9rPOIJz0bbuDXujPvPUiOXaVbxbRkPH/NalKY=</latexit>
<latexit sha1_base64="xkZinkB3fXx+z+jqDlELdUOZnwY=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2JpclytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zcAkQQ=</latexit>
<latexit sha1_base64="agBvV6VbSguQi+hSyDEufUlb/1g=">AAACyXicjVHLTsJAFD3UF+ILdemmkZi4Iq3B6JLoRuMGE3kkiKYtA1ZKW6dTIxJW/oBb/THjH+hfeGccEpUYnabtmXPvOTP3XjcO/ERY1mvGmJqemZ3LzucWFpeWV/Kra7UkSrnHql4URLzhOgkL/JBVhS8C1og5c/puwOpu71DG67eMJ34UnolBzFp9pxv6Hd9zBFG144uhPcpd5gtW0VLLnAS2BgXoVYnyLzhHGxE8pOiDIYQgHMBBQk8TNizExLUwJI4T8lWcYYQcaVPKYpThENujb5d2Tc2GtJeeiVJ7dEpALyeliS3SRJTHCcvTTBVPlbNkf/MeKk95twH9Xe3VJ1bgiti/dOPM/+pkLQId7KsafKopVoysztMuqeqKvLn5pSpBDjFxErcpzgl7Sjnus6k0iapd9tZR8TeVKVm593Ruind5Sxqw/XOck6C2U7RLxd3TUqF8oEedxQY2sU3z3EMZR6igSt7XeMQTno0T48a4M+4/U42M1qzj2zIePgB4uZEZ</latexit>
<latexit sha1_base64="xkZinkB3fXx+z+jqDlELdUOZnwY=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2JpclytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zcAkQQ=</latexit>
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="xkZinkB3fXx+z+jqDlELdUOZnwY=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2JpclytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zcAkQQ=</latexit>
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
<latexit sha1_base64="078TlPinZnbp3G8YDOaCG04E8G4=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI</latexit>
<latexit sha1_base64="rxH3eGL3jlLzhYJGRaokrVTVvjc=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ</latexit>
<latexit sha1_base64="rX31m9pL4TDFitzg9FObuhvEhn0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK</latexit>
<latexit sha1_base64="TPeHe+EdHuk9nU8nbYxS2EGTCv0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVgcEl0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljbn5hcSm/XFhZXVvfKG5uNZMwjRm3WeiFcdt1Eu6JgNtSSI+3o5g7vuvxljs8UfHWHY8TEQaXchTxru8MAtEXzJFE2WfX49rkpliyypZe5iyoZKCEbDXC4guu0EMIhhQ+OAJIwh4cJPR0UIGFiLguxsTFhISOc0xQIG1KWZwyHGKH9B3QrpOxAe2VZ6LVjE7x6I1JaWKPNCHlxYTVaaaOp9pZsb95j7WnutuI/m7m5RMrcUvsX7pp5n91qhaJPo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMI9iseEmVZO+2xqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zlnQPChXquXDi2qpfpyNOo8d7GKf5llDHadowCZvgUc84dk4NyLj3hh9phq5TLONb8t4+ABHp5EL</latexit>
<latexit sha1_base64="yLKmKiLjSNF7gQFV1rqUemEXJus=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVgZEl0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljbn5hcSm/XFhZXVvfKG5uNZMwjRm3WeiFcdt1Eu6JgNtSSI+3o5g7vuvxljs8UfHWHY8TEQaXchTxru8MAtEXzJFE2WfX49rkpliyypZe5iyoZKCEbDXC4guu0EMIhhQ+OAJIwh4cJPR0UIGFiLguxsTFhISOc0xQIG1KWZwyHGKH9B3QrpOxAe2VZ6LVjE7x6I1JaWKPNCHlxYTVaaaOp9pZsb95j7WnutuI/m7m5RMrcUvsX7pp5n91qhaJPmq6BkE1RZpR1bHMJdVdUTc3v1QlySEiTuEexWPCTCunfTa1JtG1q946Ov6mMxWr9izLTfGubkkDrvwc5yxoHpQr1fLhRbVUP85GnccOdrFP8zxCHadowCZvgUc84dk4NyLj3hh9phq5TLONb8t4+ABKCJEM</latexit>
<latexit sha1_base64="4cXIA0l05KwFp/J1B3eamNy7Rh4=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVg1B3RjbrCxAIJommHASf0lXaqIYSNP+BWv8z4B/oX3hlLohKj07Q9c+49Z+be60aeSKRlveaMmdm5+YX8YmFpeWV1rbi+0UjCNGbcZqEXxi3XSbgnAm5LIT3eimLu+K7Hm+7gRMWbdzxORBhcymHEO77TD0RPMEcSZZ9dj47GN8WSVbb0MqdBJQMlZKseFl9whS5CMKTwwRFAEvbgIKGnjQosRMR1MCIuJiR0nGOMAmlTyuKU4RA7oG+fdu2MDWivPBOtZnSKR29MShM7pAkpLyasTjN1PNXOiv3Ne6Q91d2G9HczL59YiVti/9JNMv+rU7VI9HCoaxBUU6QZVR3LXFLdFXVz80tVkhwi4hTuUjwmzLRy0mdTaxJdu+qto+NvOlOxas+y3BTv6pY04MrPcU6Dxl65Ui3vX1RLteNs1HlsYRu7NM8D1HCKOmzyFnjEE56NcyMy7o3hZ6qRyzSb+LaMhw9MaZEN</latexit>
<latexit sha1_base64="ZbsyFj/zkkCygShlWDI2kppK3uo=">AAAC2nicjVHLSsNAFD2Nr1pfVXHlJlgEF1KSUtGNUHTjsoJ9QFtKkk7b0LyYTIRS3LgTt/6AW/0g8Q/0L7wzpqAW0RuSnDn3njNz59qR58bCMF4z2tz8wuJSdjm3srq2vpHf3KrHYcIdVnNCL+RN24qZ5wasJlzhsWbEmeXbHmvYo3OZb1wzHrthcCXGEev41iBw+65jCaK6+Z1AP9VLBsWh3ubDkFZm0ejmC0bRUKHPAjMFBaRRDfMvaKOHEA4S+GAIIAh7sBDT04IJAxFxHUyI44RclWe4QY60CVUxqrCIHdF3QKtWyga0lp6xUju0i0cvJ6WOfdKEVMcJy910lU+Us2R/854oT3m2Mf3t1MsnVmBI7F+6aeV/dbIXgT5OVA8u9RQpRnbnpC6JuhV5cv1LV4IcIuIk7lGeE3aUcnrPutLEqnd5t5bKv6lKycq1k9YmeJenpAGbP8c5C+qlolkuHl2WC5WzdNRZ7GIPBzTPY1RwgSpq5D3BI57wrLW1W+1Ou/8s1TKpZhvfQnv4AI9dlOA=</latexit>

heading: 5.3 HOSPITAL COLLABORATION
text: eICU (Pollard et al., 2018) is a clinical data set collecting the patients about their admissions to ICUs with hospital information. Each instance is a specific ICU stay. We follow the data pre-processing
2The results of baselines are from (Li et al., 2019) 3The results of baselines are from (Shamsian et al., 2021)
Under review as a conference paper at ICLR 2022
procedure in (Sheikhalishahi et al., 2019) and naturally treat different hospitals as local clients. We conduct the task of predicting in-hospital mortality which is defined as the patient’s outcome at the hospital discharge. This is a binary classification task, where each data sample spans a 1-hour window. In this experiment, we select 5 hospitals with more patient samples (about 1000) { Ii }4 i=0 and 5
hospitals with less patient samples { Ii }9 i=5
. Due to label imbalance (more than 90% samples have negative labels), we use AUC to measure the utility for each client as in (Sheikhalishahi et al., 2019). For all methods, we use the ANN as the network structure as in (Sheikhalishahi et al., 2019).
The model AUC of each hospital is reported in Table 4. Because of the lack of patient data for each hospital, Local achieves a relatively lower AUC compared to FedAve and SPO. While patient populations vary substantially from hospital to hospital, SPO learns a personalized model for each hospital and outperforms FedAve from Table 4.
<latexit sha1_base64="xkZinkB3fXx+z+jqDlELdUOZnwY=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2JpclytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zcAkQQ=</latexit>
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
<latexit sha1_base64="078TlPinZnbp3G8YDOaCG04E8G4=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI</latexit>
<latexit sha1_base64="rxH3eGL3jlLzhYJGRaokrVTVvjc=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ</latexit>
<latexit sha1_base64="TPeHe+EdHuk9nU8nbYxS2EGTCv0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVgcEl0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljbn5hcSm/XFhZXVvfKG5uNZMwjRm3WeiFcdt1Eu6JgNtSSI+3o5g7vuvxljs8UfHWHY8TEQaXchTxru8MAtEXzJFE2WfX49rkpliyypZe5iyoZKCEbDXC4guu0EMIhhQ+OAJIwh4cJPR0UIGFiLguxsTFhISOc0xQIG1KWZwyHGKH9B3QrpOxAe2VZ6LVjE7x6I1JaWKPNCHlxYTVaaaOp9pZsb95j7WnutuI/m7m5RMrcUvsX7pp5n91qhaJPo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMI9iseEmVZO+2xqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zlnQPChXquXDi2qpfpyNOo8d7GKf5llDHadowCZvgUc84dk4NyLj3hh9phq5TLONb8t4+ABHp5EL</latexit>
<latexit sha1_base64="yLKmKiLjSNF7gQFV1rqUemEXJus=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVgZEl0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljbn5hcSm/XFhZXVvfKG5uNZMwjRm3WeiFcdt1Eu6JgNtSSI+3o5g7vuvxljs8UfHWHY8TEQaXchTxru8MAtEXzJFE2WfX49rkpliyypZe5iyoZKCEbDXC4guu0EMIhhQ+OAJIwh4cJPR0UIGFiLguxsTFhISOc0xQIG1KWZwyHGKH9B3QrpOxAe2VZ6LVjE7x6I1JaWKPNCHlxYTVaaaOp9pZsb95j7WnutuI/m7m5RMrcUvsX7pp5n91qhaJPmq6BkE1RZpR1bHMJdVdUTc3v1QlySEiTuEexWPCTCunfTa1JtG1q946Ov6mMxWr9izLTfGubkkDrvwc5yxoHpQr1fLhRbVUP85GnccOdrFP8zxCHadowCZvgUc84dk4NyLj3hh9phq5TLONb8t4+ABKCJEM</latexit>
<latexit sha1_base64="4cXIA0l05KwFp/J1B3eamNy7Rh4=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVg1B3RjbrCxAIJommHASf0lXaqIYSNP+BWv8z4B/oX3hlLohKj07Q9c+49Z+be60aeSKRlveaMmdm5+YX8YmFpeWV1rbi+0UjCNGbcZqEXxi3XSbgnAm5LIT3eimLu+K7Hm+7gRMWbdzxORBhcymHEO77TD0RPMEcSZZ9dj47GN8WSVbb0MqdBJQMlZKseFl9whS5CMKTwwRFAEvbgIKGnjQosRMR1MCIuJiR0nGOMAmlTyuKU4RA7oG+fdu2MDWivPBOtZnSKR29MShM7pAkpLyasTjN1PNXOiv3Ne6Q91d2G9HczL59YiVti/9JNMv+rU7VI9HCoaxBUU6QZVR3LXFLdFXVz80tVkhwi4hTuUjwmzLRy0mdTaxJdu+qto+NvOlOxas+y3BTv6pY04MrPcU6Dxl65Ui3vX1RLteNs1HlsYRu7NM8D1HCKOmzyFnjEE56NcyMy7o3hZ6qRyzSb+LaMhw9MaZEN</latexit>
<latexit sha1_base64="rX31m9pL4TDFitzg9FObuhvEhn0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK</latexit>
<latexit sha1_base64="xkZinkB3fXx+z+jqDlELdUOZnwY=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2JpclytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zcAkQQ=</latexit>
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
<latexit sha1_base64="078TlPinZnbp3G8YDOaCG04E8G4=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI</latexit>
<latexit sha1_base64="rxH3eGL3jlLzhYJGRaokrVTVvjc=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ</latexit>
<latexit sha1_base64="TPeHe+EdHuk9nU8nbYxS2EGTCv0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVgcEl0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljbn5hcSm/XFhZXVvfKG5uNZMwjRm3WeiFcdt1Eu6JgNtSSI+3o5g7vuvxljs8UfHWHY8TEQaXchTxru8MAtEXzJFE2WfX49rkpliyypZe5iyoZKCEbDXC4guu0EMIhhQ+OAJIwh4cJPR0UIGFiLguxsTFhISOc0xQIG1KWZwyHGKH9B3QrpOxAe2VZ6LVjE7x6I1JaWKPNCHlxYTVaaaOp9pZsb95j7WnutuI/m7m5RMrcUvsX7pp5n91qhaJPo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMI9iseEmVZO+2xqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zlnQPChXquXDi2qpfpyNOo8d7GKf5llDHadowCZvgUc84dk4NyLj3hh9phq5TLONb8t4+ABHp5EL</latexit>
<latexit sha1_base64="yLKmKiLjSNF7gQFV1rqUemEXJus=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVgZEl0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljbn5hcSm/XFhZXVvfKG5uNZMwjRm3WeiFcdt1Eu6JgNtSSI+3o5g7vuvxljs8UfHWHY8TEQaXchTxru8MAtEXzJFE2WfX49rkpliyypZe5iyoZKCEbDXC4guu0EMIhhQ+OAJIwh4cJPR0UIGFiLguxsTFhISOc0xQIG1KWZwyHGKH9B3QrpOxAe2VZ6LVjE7x6I1JaWKPNCHlxYTVaaaOp9pZsb95j7WnutuI/m7m5RMrcUvsX7pp5n91qhaJPmq6BkE1RZpR1bHMJdVdUTc3v1QlySEiTuEexWPCTCunfTa1JtG1q946Ov6mMxWr9izLTfGubkkDrvwc5yxoHpQr1fLhRbVUP85GnccOdrFP8zxCHadowCZvgUc84dk4NyLj3hh9phq5TLONb8t4+ABKCJEM</latexit>
<latexit sha1_base64="4cXIA0l05KwFp/J1B3eamNy7Rh4=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVg1B3RjbrCxAIJommHASf0lXaqIYSNP+BWv8z4B/oX3hlLohKj07Q9c+49Z+be60aeSKRlveaMmdm5+YX8YmFpeWV1rbi+0UjCNGbcZqEXxi3XSbgnAm5LIT3eimLu+K7Hm+7gRMWbdzxORBhcymHEO77TD0RPMEcSZZ9dj47GN8WSVbb0MqdBJQMlZKseFl9whS5CMKTwwRFAEvbgIKGnjQosRMR1MCIuJiR0nGOMAmlTyuKU4RA7oG+fdu2MDWivPBOtZnSKR29MShM7pAkpLyasTjN1PNXOiv3Ne6Q91d2G9HczL59YiVti/9JNMv+rU7VI9HCoaxBUU6QZVR3LXFLdFXVz80tVkhwi4hTuUjwmzLRy0mdTaxJdu+qto+NvOlOxas+y3BTv6pY04MrPcU6Dxl65Ui3vX1RLteNs1HlsYRu7NM8D1HCKOmzyFnjEE56NcyMy7o3hZ6qRyzSb+LaMhw9MaZEN</latexit>
<latexit sha1_base64="rX31m9pL4TDFitzg9FObuhvEhn0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK</latexit>
<latexit sha1_base64="xkZinkB3fXx+z+jqDlELdUOZnwY=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2JpclytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zcAkQQ=</latexit>
<latexit sha1_base64="ZroCL0JJbplYYxKF0TuIbB7Z7xk=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU=</latexit>
<latexit sha1_base64="2++cL5TEXumvU+oahUsTVndZR8E=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG</latexit>
<latexit sha1_base64="G06mXSCW+K11xSPKJ+cv3bGQhME=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH</latexit>
<latexit sha1_base64="078TlPinZnbp3G8YDOaCG04E8G4=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI</latexit>
<latexit sha1_base64="rxH3eGL3jlLzhYJGRaokrVTVvjc=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ</latexit>
<latexit sha1_base64="TPeHe+EdHuk9nU8nbYxS2EGTCv0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVgcEl0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljbn5hcSm/XFhZXVvfKG5uNZMwjRm3WeiFcdt1Eu6JgNtSSI+3o5g7vuvxljs8UfHWHY8TEQaXchTxru8MAtEXzJFE2WfX49rkpliyypZe5iyoZKCEbDXC4guu0EMIhhQ+OAJIwh4cJPR0UIGFiLguxsTFhISOc0xQIG1KWZwyHGKH9B3QrpOxAe2VZ6LVjE7x6I1JaWKPNCHlxYTVaaaOp9pZsb95j7WnutuI/m7m5RMrcUvsX7pp5n91qhaJPo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMI9iseEmVZO+2xqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zlnQPChXquXDi2qpfpyNOo8d7GKf5llDHadowCZvgUc84dk4NyLj3hh9phq5TLONb8t4+ABHp5EL</latexit>
<latexit sha1_base64="yLKmKiLjSNF7gQFV1rqUemEXJus=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVgZEl0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljbn5hcSm/XFhZXVvfKG5uNZMwjRm3WeiFcdt1Eu6JgNtSSI+3o5g7vuvxljs8UfHWHY8TEQaXchTxru8MAtEXzJFE2WfX49rkpliyypZe5iyoZKCEbDXC4guu0EMIhhQ+OAJIwh4cJPR0UIGFiLguxsTFhISOc0xQIG1KWZwyHGKH9B3QrpOxAe2VZ6LVjE7x6I1JaWKPNCHlxYTVaaaOp9pZsb95j7WnutuI/m7m5RMrcUvsX7pp5n91qhaJPmq6BkE1RZpR1bHMJdVdUTc3v1QlySEiTuEexWPCTCunfTa1JtG1q946Ov6mMxWr9izLTfGubkkDrvwc5yxoHpQr1fLhRbVUP85GnccOdrFP8zxCHadowCZvgUc84dk4NyLj3hh9phq5TLONb8t4+ABKCJEM</latexit>
<latexit sha1_base64="4cXIA0l05KwFp/J1B3eamNy7Rh4=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVg1B3RjbrCxAIJommHASf0lXaqIYSNP+BWv8z4B/oX3hlLohKj07Q9c+49Z+be60aeSKRlveaMmdm5+YX8YmFpeWV1rbi+0UjCNGbcZqEXxi3XSbgnAm5LIT3eimLu+K7Hm+7gRMWbdzxORBhcymHEO77TD0RPMEcSZZ9dj47GN8WSVbb0MqdBJQMlZKseFl9whS5CMKTwwRFAEvbgIKGnjQosRMR1MCIuJiR0nGOMAmlTyuKU4RA7oG+fdu2MDWivPBOtZnSKR29MShM7pAkpLyasTjN1PNXOiv3Ne6Q91d2G9HczL59YiVti/9JNMv+rU7VI9HCoaxBUU6QZVR3LXFLdFXVz80tVkhwi4hTuUjwmzLRy0mdTaxJdu+qto+NvOlOxas+y3BTv6pY04MrPcU6Dxl65Ui3vX1RLteNs1HlsYRu7NM8D1HCKOmzyFnjEE56NcyMy7o3hZ6qRyzSb+LaMhw9MaZEN</latexit>
<latexit sha1_base64="rX31m9pL4TDFitzg9FObuhvEhn0=">AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK</latexit>
Collaboration Equilibrium The benefit graph of all hospitals determined by SPO and its corresponding strongly connected components are shown in Figure 5(a) and 5(b). From Figure 5(a), I0 and I3 are the unique necessary collaborator for each other, C1 = { I0, I3 } is the first identified
stable coalition as shown in Figure 5(b); I9 is a tiny clinic that cannot contribute to any hospitals, so no hospital is willing to collaborate with it and I9 learns a local model with its own data by forming a simple coalition C2 = { I9 }
. The benefit graph of the remaining clients are shown in Figure 5(b). On the one hand they cannot benefit I3 or I0 so they cannot form coalitions with them, on the other hand they refuse to contribute I9 without any charge. They choose form the coalition C3 = { I1, I2, I4, I5, I6, I7, I8 } to maximize their AUC. Therefore, the CE in this hospital collaboration network is achieved by the collaboration strategy S = {C1, C2, C3} and the model AUC of each client in the CE is in Table 4.
Comparing Figure 5(b) and 5(c), the strongly connected components of the benefit graph leads to a CE. This empirically verifies that Assumption 1 is reasonable. Meanwhile, CE guarantees that every client in its coalition will not collaborate with harmful clients, so the client may achieve a higher utility in a CE compared to collaborating with everyone such as the AUC of I0 in CE (77.93) is higher than in SPO (76.35).

heading: 6 CONCLUSION
text: In this paper, we propose a learning to collaborate framework to achieve collaboration equilibrium such that any of the individual clients cannot improve their performance further. Comprehensive experiments on benchmark and real-world data sets demonstrated the validity of our proposed framework. In our study, some small clients could be isolated as they cannot benefit others. Our framework can quantify both the benefit to and the contribution from each client in a network. In practice, such information can be utilized to either provide incentives or to impose charges on each client, to facilitate and enhance the foundation of the network or coalition.
Under review as a conference paper at ICLR 2022

heading: ETHICS STATEMENT
text: We propose a novel collaboration framework for multiple clients, which is based the principle that each member in the coalition benefits others while aims to maximize its utility. We have read the codes of ethics and make sure this paper conforms to them. The procedures of data processing and experiments comply with requirements. Moreover, we also share our extended discussions about the applications and limitations of our method in Appendix.

heading: REPRODUCIBILITY STATEMENT
text: Our all instructions and training details needed to reproduce the experimental results are in Appendix. The links of public data sets are also provided. We report the resources used in Appendix. The source code of our method will be made public when it is published. For all theoretical results, we also give rigorous analysis, detailed proofs, and comprehensive discussions in Appendix.

heading: A PROOFS OF ALL THEORETICAL RESULTS

heading: A.1 PROOF OF THEOREM 1
text: To prove Theorem 1, we first prove that the benefit graph of a stable coalition is strongly connected shown in Lemma 1.
Lemma 1. For a given client set I , the benefit graph of each stable coalitionCs is strongly connected, which means that there is a path in each direction between each pair of vertices in BG(Cs).
Proof. We will prove the Lemma 1 by contradiction. Given a stable coalition Cs, suppose there exsits a pair of vertices I1, I2 ∈ BG(Cs) such that there is no path from I1 to I2, which is denoted as @p, s.t., I1 → I2 for expressive clearly. We split Cs into two sub-coalitions C ′ and Cs\C ′ depending on whether the clients have paths to I2. The clients in C ′ have no path to I2, and the clients in Cs\C ′ have paths to I2.
∀Ii ∈ C ′,@p, s.t., Ii → I2 ∀Ii ∈ Cs\C ′,∃p, s.t., Ii → I2. (10)
Obviously, C ′ and Cs\C ′ are not empty because I1 ∈ C ′ and and I2 ∈ Cs\C ′. From Eq.(5) in the main text in the Definition of stable coalition, each client Ii ∈ Cs achieves its maximal utility by collaborating with others in Cs which means that
∀Ii ∈ Cs, ∀Ij ∈ CoptI (Ii), Ij ∈ Cs. (11)
From the Definition of Optimal Collaborator Set (OCS), each client Ii in the OCS of I2 has an edge from Ii to I2,
∀Ii ∈ CoptI (I2), ∃p, s.t., Ii → I2, (12)
Therefore, because each client Ii ∈ CoptI (I2) is in Cs\C ′, I2 can achieve its maximal utility in Cs\C ′. Morever, we can prove that Ii ∈ CoptI (I2) achieves its maximal utiltiy in Cs\C ′ because each client Ii ∈ CoptI (I2) and its corresponding OCS are in Cs\C ′,
∀Ii ∈ CoptI (I2), ∀Ij ∈ C opt I (I i), ∃p, Ij → Ii, (13a) ∀Ii ∈ CoptI (I2), ∃p, s.t., Ii → I2, (13b) ⇒ ∀Ii ∈ CoptI (I2), ∀Ij ∈ C opt I (I i), ∃p, Ij → Ii → I2 (13c)
Under review as a conference paper at ICLR 2022
From Eq.(13), the clients in the OCS of Ii (Ii ∈ CoptI (I2)) are in Cs\C ′ and Ii can achieve its optimal utility. Based on the same analysis, for each client Ii ∈ Cs\C ′, its OCS CoptI (Ii) is in Cs\C ′ because all clients in CoptI (Ii) have a path to I2. So we have the conclusion that all clients in Cs\C ′ achieves its optimal utility by collaborating with others in Cs\C ′. However, this contradicts Eq.(6) in the main text in the Definition of stable coalition. Therefore, we prove that the benefit graph BG(Cs) of the stable coalition Cs is strongly connected as in Lemma 1.
From Lemma 1, the BG(Cs) is strongly connected, then we will prove that BG(Cs) is a strongly connected component of BG(I) by pointing out that BG(Cs) is maximal which means that no additional vertices from I can be included inBG(Cs) without breaking the property of being strongly connected.
Proof. We will prove that BG(Cs) is maximal by contradiction. Suppose there is another client I0 ∈ I\Cs which can be added inBG(Cs) without breaking the property of being strongly connected. Therefore, there exists Ii ∈ Cs which has an edge from I0 to Ii. This means that I0 is one of the necessary collaborators of Ii (I0 ∈ CoptI (Ii)) and I0 cannot achieve its optimal utility in Cs without collaborating with I0. However, this contradicts Eq.(5) in the main text in the Definition of stable coalition that all clients in Cs can achieve its optimal utility by collaborating with others in Cs. Therefore, we prove that BG(Cs) is a strongly connected component as stated in Theorem 1.

heading: A.2 PROOF OF THEOREM 2
text: Proof. 1. All coalitions in S = {C0, C1, ..., Ck} satisfies Inner Agreement as in Axiom 1. As each coalition Ci ∈ S is a stable coalition, from Eq.(6) of the Definition of stable coalition, we have
∀Ci ∈ S, ∀C ′ & Ci, ∃Ii ∈ C ′, s.t., Umax(Ii, C ′) < Umax(Ii, Ci). (14)
Then Eq.(14) is the definition of Inner Agreement in Eq.(3) in the main text. Therefore, we prove that all collaboration coalitions in S satisfy inner agreement.

heading: 2. The collaboration strategy S satisfies Outer Agreement as in Axiom 2.
text: We will prove it by contradiction. Suppose there exists a new nonempty coalition C ′ = {Ii0 , Ii1 , ..., Iik} /∈ S which satisfies
∀Ii ∈ C ′, Umax(Ii, C ′) > Umax(Ii, Cj) (Ii ∈ Cj ∈ S). (15)
Suppose that C0 denotes the coalition in the first iteration. Then ∀Ii ∈ C0, Ii achieves its maximal utility in the client set I and cannot increase its model utility further. So we have
∀Ii ∈ C ′, Ii /∈ C0. (16)
The clients in the coalition C0 ∈ S identified in the first iteration have no interest in collaborating with others and will be removed. Suppose C1 denotes the coalition in the first iteration. ∀Ii ∈ C1, Ii achieves its maximal utility in the client set I\C0 and cannot increase its model utility further. So we have
∀Ii ∈ C ′, Ii /∈ C1. (17)
Then C1 will be removed and we have ∀Ii ∈ C ′, Ii /∈ I2.. Based on the same analysis, we finally have
∀Ii ∈ C ′, ∀Cj ∈ S, Ii /∈ Cj , ⇒ C ′ = ∅.
(18)
Therefore, we prove that there is no nonempty C ′ satisfying Eq.(15) and S satisfies Outer Agreement.
Under review as a conference paper at ICLR 2022

heading: A.3 PROOF OF COROLLARY 1
text: Firstly, we prove that all strongly connected components are disjoint as in Lemma 2.
Lemma 2. Given a directed graph G and all its strongly connected components {C0, ...Ck}, each pair of its strongly connected components Ci and Cj satisfies Ci ∩ Cj = ∅.
Proof. We will prove it by contradiction. Suppose there exists a pair of the strongly connected components which satisfies I0 ∈ (Ci ∩ Cj) and C = Ci ∪ Cj . For each pair of vertices Ii, Ij , we have
∃p, Ii → I0 → Ij , ∃p, Ij → I0 → Ii.
(19)
From Eq.(19), C is strongly connected which conflicts the definition of strongly connected components that each strongly connected component is maximal. Therefore, we prove thatCi∩Cj = ∅.
Theorem 2 states that the collaboration coalitions identified iteratively leads to a collaboration equilibrium. To prove the Corollary 1, we will demostrate that the collaboration coalitions identified iteratively are all strongly connected components of BG(I) when Assumption 1 holds.
Proof. Suppose the coalitions strategy S = {C0, C1, ...Ck} is determined iteratively and Ci is identified in the i-th iteration. From Theorem 1, C0 is one of the strongly connected components of BG(I). C1 is a stable coalition which is one of the strongly connected componts of BG(I\C0). When Assumption 1 holds, BG(I\C0) is the subgraph ofBG(I) and any edge inBG(I\C0) is also the edge in BG(I) and vice versa. Therefore, BG(C1) is strongly connected in BG(I). Suppose BG(C1) is not a strongly connected component of BG(I) and there exists an additional client Ii which can be added in C1 without breaking the strongly connected property. From Lemma 2, Ii cannot be from another strongly connected component, so we have
Ii /∈ C0. (20)
If Ii is from I\(C0∪C1) and can be added into C1 without breaking the strongly connected property, it conflicts the definition that C1 is a strongly connected component of I\C0. So we have
Ii /∈ I\(C0 ∪ C1). (21)
From Eq.(20) and Eq.(21), there is no additional Ii which can be added in C1 without breaking the strongly connected property. So C1 is a strongly connected component of BG(I). For the same analysis, we can prove that ∀Ci ∈ S, Ci is a strongly connected component of BG(I) when Assumption 1 holds. Therefore, we prove that the strongly connected components of BG(I) are the stable coalitions {C0, C1, ...Ck} identified iteratively by Algorithm 1 and these strongly connected components lead to a CE.

heading: A.4 PROOF OF PROPOSITION 1
text: Proof. For any hypothesis h′∗ ∈ P (C ′), h′∗ satisfies
@h′ ∈ H, s.t. ∀i ∈ C ′ : li (h′) ≤ li(h′∗) and ∃j : lj (h′) < lj(h′∗). (22)
1) If h∗ ∈ P (C) which satisfies
@h′ ∈ H, s.t. ∀i ∈ C : li (h′) ≤ li(h) and ∃j : lj (h′) < lj(h), (23)
then we let h∗ = h′∗ in Eq.(8) so Eq.(8) holds;
2) if h′∗ /∈ P (C), there exists h′ ∈ P (C) which satisfies
Under review as a conference paper at ICLR 2022
∀i ∈ C : li (h′) ≤ li(h′∗) and ∃j : lj (h′) < lj(h′∗), (24)
combining Eq.(22), we have
∀i ∈ C ′ : li (h′) = li(h′∗) and ∃j ∈ C\C ′ : lj (h′) < lj(h′∗). (25)
then we let h∗ = h′ in Eq.(8) so Eq.(8) holds.
B IMPLEMENTATION
In this section, we will introduce how to learn the Pareto Front of all objectives corresponding to all clients. Then, we will describe how to determine the optimal collaborator set for each client.

heading: B.1 LEARNING PARETO FRONT
text: Multi-objective optimization (MOO) problems have a set of optimal solutions, and these optimal solutions form the Pareto front, where each point on the front represents a different trade-off among possibly conflicting objectives. We construct a personalized model for each client which we call target network and it has the same architecture as the baselines. To determine the parameters of the target network of each client, in our experiments, we learn the entire Pareto Front simultaneously using a hypernetwork which receives as input a vector d and returns all parameters of the target network. The input vector is N-dimension bmd in which each entry di corresponds to the client Ii and is sampled from the convex hull D,
D = { d, |∀i, 1 ≤ i ≤ N, di ≥ 0, and
N∑ i=1 di = 1
} . (26)
Specifically, we construct the hypernetwork following in the architecture introduced in (Navon et al., 2020). By a n-layer (n ≤ 3) MLP network with the activation function ReLU, we inference the parameters of the target network. Then the obtained parameters will be assigned to the target network. We evaluate the performance of the parameters on the training set and the gradient information will be returned for updating the hypernetwork.
B.2 IDENTIFYING THE OPTIMAL COLLABORATOR SET
From the statement above, we train a hypernetwork HN for learning the whole Pareto front. HN bridges the mapping from the vector d to the corresponding model parameters of the target network. To obtain the optimal target network parameters that achieve the minimal value loss of the target objective on the validation set, we optimize the vector d by gradient descent. Specifically, given an initial direction d0, we firstly obtain the parameters of the target network using the learned hypernetwork HN(d0). Like training the hypernetwork, the obtained parameters will be assigned to the target network. Then we evaluate the performance of the generated parameters on the validation set and compute the gradient of the input direction d0. Finally, the gradient information will be used for updating the vector d iteratively until convergence.
di+1 = di − η · ∇di l∗ di+1 ← Clip(di+1) di+1 ← Normalization(di+1),
(27)
where η denotes the learning rate, Clip(di+1) means that we clip each values d j i+1 ∈ di+1 to satisfy
0 ≤ dji+1 ≤ 1 and Normalization(di+1) is as follows,
Under review as a conference paper at ICLR 2022
∀j (1 ≤ j ≤ N), d′ji+1 = dji+1∑N j=1 d j i+1 . (28)
Finally, we reach the optimal direction d∗ and its corresponding Pareto model M∗ = HN(d∗).
For each client, we determine the optimal collaborator set by the optimal direction d∗ and the loss value of all objectives l = [l1(M∗), l2(M∗), ..., lN (M∗)]. From the Pareto Front Embedded property described in Proposition 1, an ideal direction d∗ is a sparse vector in which the indexes of the non-zero values in d∗ are the necessary collaborators for the target client. In our experiments, d∗ usually is not a sparse vector, but there are values in d∗ which are significantly small. Therefore, we set a threshold for d∗ to determine which clients are necessary/unnecessary.
Algorithm 1: Achieving collaboration equilibrium Input: N institutions I = {Ii}Ni=1 seeking collaborating with others Set original client set C ← I; Set collaboration strategy S ← ∅; while C 6= ∅ do
forall client Ii ∈ C do Determine the OCS of Ii by SPO; end Construct the benefit graph BG(C); Search for all strongly connected components { C1, C2, ...Ck } of BG(C) using Tarjan
algorithm; forall i = 1, 2, 3,... k do
if Ci is stable coalition then C ← C\Ci ; S ← S ∪ {Ci};
end end
end Output: collaboration strategy S
When Assumption 1 holds, from Corollary 1 proposed in the main text, the framework for obtaining a collaboration strategy that leads to a CE is simplified as in Algorithm 2.
Algorithm 2: Achieving collaboration equilibrium under Assumption 1 Input: N institutions I = {Ii}Ni=1 seeking collaborating with others Set original client set C ← I; Set collaboration strategy S ← ∅; forall client Ii ∈ C do
Determine the OCS of Ii by SPO; end Construct the benefit graph BG(C); Search for all strongly connected components S = { C1, C2, ...Ck } of BG(C) using Tarjan
algorithm; Output: collaboration strategy S

heading: C EXPERIMENTAL DETAILS

heading: C.1 SYNTHETIC DATA EXPERIMENTS
text: The synthetic data generation process is illustrated in the main text. In this experiment, we construct the hypernetwork using 1-layer hidden MLP for generating the target network parameters and the
Under review as a conference paper at ICLR 2022
target network is a 1-layer MLP. The experiments have two settings, 1) 2000 training samples and ρ = 0.1; 2) 20000 training samples and ρ = 1.0.
We plot the vector d = [d0, d1, d2, d3, d4, d5] varying in each iteration as we learn a personalized model for I0. When n = 2000, ρ = 0.1 and the target client is I0, from Figure 6(a), {d3, d4, d5} decrease until reach the minimum and {d0, d1, d2} increase in each iteration. As ρ 6= 0, the data in I1 (I2) and I0 does not satisfy i.i.d, so d0 is slightly larger than d1 and d2. This means that the optimal personalized model of I0 is on PF (I0, I1, I2) and the OCS of I0 is {I0, I1, I2}. When n = 20000, ρ = 1.0 and the target client is I0, from Figure 6(b), {d1, d2, d3, d4, d5} decrease and reach the minimum and d0 increases in each iteration. This means that the optimal personalized model of I0 is on PF (I0) and the OCS of I0 is {I0}.
After identifying all coalitions, all clients will re-learn a personalized model by collaborating with other clients in its coalition.

heading: C.2 UCI ADULT DATA EXPERIMENTS
text: UCI Adult is a public dataset (Kohavi, 1996) which is split as training set and test set. We follow the dataset preprocessing procedure as in (Li et al., 2019; Mohri et al., 2019) and split the data into two clients PhD client and Non-PhD client. In this experiment, we construct the hypernetwork using a 1-layer hidden MLP. The target network is a Logistic Regression model as in (Li et al., 2019; Mohri et al., 2019). We split 83% training data for learning the PF and the remaining 17% training data for determining the optimal vector d. All hyperparameters are determined by the performance on the validation data set.

heading: C.3 CIFAR10 DATA EXPERIMENTS
text: CIFAR10 is a public dataset (LeCun et al., 1998) which contains 50000 images for training and 10000 images for testing. In our experiments, we following the setting in (McMahan et al., 2016). We split the data into 10 clients and simulate a non-i.i.d environment by randomly assigning two classes to each client among ten total classes. The training data of each client will be divided into training set (83%) and validation set(17%). We construct the hypernetwork using 3-layer hidden MLP for generating the parameters of the target network and the target network is constructed following the work (Shamsian et al., 2021). All baselines share the same target network structure for each client. To determine the hyperparameters (e.g., learning rate), we evaluate the performance of different hyperparameter combinations on the validation set and choose the hyperparameter combination which achieves the optimal accuracy.

heading: C.4 EICU DATA EXPERIMENTS
text: eICU Data Asset eICU dataset (Pollard et al., 2018) is a dataset for which permission is required. We followed the procedure on the website https://eicu-crd.mit.edu and got the approval
Under review as a conference paper at ICLR 2022
to the dataset. In this experiment, we follow the data preprocessing as in (Sheikhalishahi et al., 2019) and select 10 hospitals for collaboration as introduced in the main text. As all hospitals own relatively insufficient patient data samples (100 ≤ n ≤ 1300), dividing the data into training set and test set in different ways may lead to different CE. We construct the hypernetwork by a 1-layer MLP for training the Pareto Front of all objectives. The target network is an ANN model following the work in (Sheikhalishahi et al., 2019). We determine the hyperparameters by choosing the hyperparameter combination which achieves the optimal performance on the validation set. Full experiment results on eICU is in Table 5

heading: C.5 TRAINING RESOURCES
text: We run our experiments on a local Linux server that has two physical CPU chips (Inter(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz) and 32 logical kernels. We use Pytorch framework to implement our model and train all models on GeForce RTX 2080 Ti GPUs.

heading: D DETAILED DISCUSSIONS ABOUT MULTI-DOMAIN ADAPTATION
text: Multi-domain adaptation refers to the learning scenario that the target domains have no labeled data and researchers transfer the knowledge by aligning the feature spaces between the source domains and the target domains (Mansour et al., 2008; Crammer et al., 2008), while the federated learning scenario in our work considers that all clients own labeled data and the feature spaces are shared. The differences between the methods in federated learning and multi-domain adaptation are as follows:
1. the core goal of the methods in multi-domain adaptation is to align the feature spaces between the source domains and the target domains (Wilson & Cook, 2020), which may be redundant for the problem we focus on because the feature space is shared across all domains;
2. in the paradigm of federated learning/multi-task learning, the data of the target clients are labeled and it allows researchers to learn an optimal model by supervised training while methods in multi-domain adaptation cannot;
3. federated learning requires that learning a model across multi-client cannot have access to the raw data of the local clients, while methods in multi-domain adaptation can.

heading: E EXTENDED DISCUSSIONS

heading: E.1 COLLABORATIVE EQUILIBRIUM
text: The collaboration equilibrium of a collaborated network is affected by 1) the selected model structure; 2) the data distribution of the clients. For 1), different model structures utilize the data in different ways. For example, non-linear models can capture the non-linear mapping relations in the data among different local clients while linear models cannot. For 2), the collaboration equilibrium is affected by the data distribution of the clients given the model structure. The data distribution of one client determines whether it can benefit another when learning together and this was verified
Under review as a conference paper at ICLR 2022
by the experiments. From the experimental results shown in Table 1, when the data distribution discrepancies increases ( from 0.1 to 1.0), I0 and I1 could not benefit I2 anymore. Therefore, the collaboration equilibrium varies as shown in Figure 4(a).

heading: E.2 OPTIMALITY
text: Identifying stable coalition to reach a CE Our method guarantees the optimality of achieving a CE given a benefit graph. In particular, the stable coalitions (or collaboration equilibrium) are determined uniquely given the benefit graph of the client set. Based upon the constructed benefit graph, our approach guarantees the optimality of the obtained stable coalitions theoretically as the identified stable coalitions strictly satisfy the two conditions in Definition 3 as proved in Sec A.2 in Appendix. Moreover, we point out that the strongly connected components of the benefit graph lead to a CE under Assumption 1 as proved in A.3;
Constructing benefit graph by SPO Given the client set and the model structure, the true optimal collaborator sets (OCS) of each client may be defined uniquely. However, the true OCS of each client is actually agnostic without any prior assumption and we have to train an optimal model for each subset and then determine the OCS by observing which model performs best. Due to its exponential time complexity, we develop SPO to identify OCS by alternately optimizing the model and the collaborator set for each client. Roughly speaking, this procedure is similar to the ExpectationMaximization procedure performed in the mixture-of-experts (Masoudnia & Ebrahimpour, 2014) approaches, in which the model for each expert and the ensemble weights for each sample are also learned alternately. We verify its effectiveness on both synthetic and benchmark datasets. Specifically, from the experimental results shown in Figure 4, SPO finds the true OCS on synthetic, adult and cifar10.

heading: E.3 TIME COMPLEXITY
text: The analysis about the time complexity is as follows:
1. to find an OCS efficiently, we propose SPO to learn an optimal model by gradient descent and identify the OCS based upon the geometric location of the learned model on the Pareto Front according to Proposition 1, which avoids exhaustively trying;
2. to find the stable coalitions of a given client set, we propose to identify the OCS of all clients to construct the benefit graph firstly. Then we propose a graph-based method to recognize the stable coalitions which has O(V+E) time complexity as stated in Section 4.1;
3. to achieve a collaboration equilibrium, we propose to look for the stable coalitions and removes them iteratively. Combining (1) and (2), our method to reach CE has polynomial time complexity.
The run time of SPO is listed as in the following table. We also present the runtime of FedAve for comparison. As SPO learns the Pareto Front and optimizes the vector d to search for the OCS by gradient descent, there is no significant additional time cost compared to baselines.

heading: E.4 APPLICATIONS
text: One cannot know which one should collaborate with unless it knows the result of the collaboration. This requires that all clients agree to collaborate to obtain the results of the collaboration, before finalizing the collaboration equilibrium. The premise of achieving the collaboration equilibrium is that all clients agree to collaborate first to construct the benefit graph. In practice, we are inclined
Under review as a conference paper at ICLR 2022
to the view that this would be done by an impartial and authoritative third-party (e.g., the industry association) in the paradigm of federated learning. The approved third-party firstly determines the benefit graphs by learning the optimal models from multiple clients without direct access to the data of the local clients. Then the approved third-party derives the collaboration equilibrium based upon the benefit graphs and publishes the collaboration strategies of all clients.
As our framework quantifies the benefit and the contribution of each client in a collaborative network, on the one hand, this promotes a more equitable collaboration such as some big clients may no longer collaborate with small clients without any charge; on the other hand, it also leads to a more efficient collaboration as all clients will collaborate with the necessary collaborators rather than all participants.

heading: E.5 LIMITATIONS AND FUTURE WORK
text: In this work, we study the collaborative learning problem and propose the core goal that a collaboration strategy should lead to a collaborative equilibrium (CE). In theory, it is a combinatorial optimization problem with exponential complexity. There are plenty of works on addressing combinatorial optimization problems by reinforcement learning (Mazyavkina et al., 2021), though reinforcement learning methods are mostly time-consuming and hard to obtain stable results. Though we propose an efficient alternate-optimization method for CE in polynomial complexity, the optimality of the OCS (or the benefit graph) is still an unsolved problem that will be explored in our future work.

Review summaries:
This paper considers the problem of federated learning where different clients/entities share their datasets with other clients in order to obtain a model that performs best for the local loss functions. Standard federated learning setting usually share all datasets together to obtain a central model and might not be the best choice for each client separately. The authors propose a notion of collaboration equilibrium where each client shares data with a subset of all the available clients. In particular, a coalition of agents belong to a collaboration equilibrium if each agent gets maximum benefit by collaborating with all the agents in the subset, and the subset is maximal.  In order to obtain such a collaboration equilibrium the authors propose an iterative algorithm which under certain assumptions, is equivalent to finding strongly connected components in the graph. Finding the collaboration equilibrium requires identifying the benefit graph among the client. Under a very specific model of agents' utility functions, a pareto optimization based algorithm is proposed to find the graph. Finally, though experiments, the authors show that finding collaboration equilibrium does provide improvement compared to standard federated learning based methods. 

Given a network of clients each with his own private dataset, the locally learned model can be improved by obtaining data from other clients (assuming no domain mismatch). The paper models this problem by introducing the benefit graph which models how each client may benefit another. The concept of collaboration equilibrium is introduced and optimization algorithms to solve it are given. Some theoretical guarantees are shown along with experimental results.  

The paper considers the problem of agents sharing training data to improve the accuracy of the model obtained on their own population.  The high-level goal is to find "stable" collaboration patterns where no agents want to deviate (e.g., stop sharing or form their own groups of sharing).  The authors first consider an abstract formulation where each agent has a utility function over all sets of other agents to collaborate with.  Assuming that one can find the most beneficial set of collaborators for each agent, the authors present an efficient algorithm to divide agents into groups that form a stable collaboration pattern (aka a collaboration equilibrium).  Since the problem of finding this optimal set of collaborators for each agent is nontrivial in general, the authors come back to the setting of sharing training data, where they propose a theoretically principled approach to build the benefit graph (which encodes most beneficial collaborators) by optimizing on the Pareto frontier of hypotheses / models of accuracy of all agents.  The authors then evaluate their approach on synthetic and real data, and compare that against several benchmark methods for personalized federated learning.  Empirical results suggest that the collaboration equilibrium behaves roughly as one would expect.  Also, the model found by optimizing on the Pareto frontier appears to achieve remarkable performance, often comparable to or better than the best benchmark.  The accuracy at collaboration equilibria is somewhat worse, which is natural given that there might be a "price of stability".