Learning to Collaborate

1 INTRODUCTION . Effective learning of machine learning models over a collaborative network of data clients has drawn considerable interest in recent years . Frequently , due to privacy concerns , we can not simultaneously access the raw data residing on different clients . Therefore , distributed ( Li et al. , 2014 ) or federated learning ( McMahan et al. , 2017 ) strategies have been proposed , where typically model parameters are updated locally at each client with its own data and the parameter updates , such as gradients , are transmitted out and communicate with other clients . During this process , it is usually assumed that the participation in the network comes at no cost , i.e. , every client is willing to participate in the collaboration . However , this is not always true in reality . One example is the clinical research network ( CRN ) involving multiple hospitals ( Fleurence et al. , 2014 ) . Each hospital has its own patient population . The patient data are sensitive and can not be shared with other hospitals . If we want to build a risk prediction model with the patient data within this network in a privacy-preserving way , the expectation from each hospital is that a better model can be obtained through participating in the CRN compared to the one built from its own data collected from various clinical practice with big efforts . In this scenario , there has been a prior study showing that the model performance can decrease when collaborating with hospitals with very distinct patient populations due to negative transfer induced by sample distribution discrepancies ( Wang et al. , 2019 ; Pan & Yang , 2009 ) . With these considerations , in this paper , we propose a novel learning to collaborate framework . We allow the participating clients in a large collaborative network to form non-overlapping collaboration coalitions . Each coalition includes a subset of clients such that the collaboration among them can benefit their respective model performance . We aim at identifying the collaboration coalitions that can lead to a collaboration equilibrium , i.e. , there are no other coalition settings that any of the individual clients can benefit more ( i.e. , achieve better model performance ) . To obtain the coalitions that can lead to a collaboration equilibrium , we propose a Pareto optimization framework to identify the necessary collaborators for each client in the network to achieve its maximum utility . In particular , we optimize a local model associated with a specific client on the Under review as a conference paper at ICLR 2022 < latexit sha1_base64= '' ZroCL0JJbplYYxKF0TuIbB7Z7xk= '' > AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU= < /latexit > < latexit sha1_base64= '' 2++cL5TEXumvU+oahUsTVndZR8E= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG < /latexit > < latexit sha1_base64= '' G06mXSCW+K11xSPKJ+cv3bGQhME= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH < /latexit > < latexit sha1_base64= '' 078TlPinZnbp3G8YDOaCG04E8G4= '' > AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI < /latexit > < latexit sha1_base64= '' rxH3eGL3jlLzhYJGRaokrVTVvjc= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ < /latexit > < latexit sha1_base64= '' rX31m9pL4TDFitzg9FObuhvEhn0= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK < /latexit > < latexit sha1_base64= '' 078TlPinZnbp3G8YDOaCG04E8G4= '' > AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI < /latexit > < latexit sha1_base64= '' rxH3eGL3jlLzhYJGRaokrVTVvjc= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ < /latexit > < latexit sha1_base64= '' rX31m9pL4TDFitzg9FObuhvEhn0= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK < /latexit > < latexit sha1_base64= '' ZroCL0JJbplYYxKF0TuIbB7Z7xk= '' > AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU= < /latexit > < latexit sha1_base64= '' 2++cL5TEXumvU+oahUsTVndZR8E= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG < /latexit > < latexit sha1_base64= '' G06mXSCW+K11xSPKJ+cv3bGQhME= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH < /latexit > < latexit sha1_base64= '' ZroCL0JJbplYYxKF0TuIbB7Z7xk= '' > AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU= < /latexit > < latexit sha1_base64= '' 2++cL5TEXumvU+oahUsTVndZR8E= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG < /latexit > < latexit sha1_base64= '' G06mXSCW+K11xSPKJ+cv3bGQhME= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH < /latexit > < latexit sha1_base64= '' 078TlPinZnbp3G8YDOaCG04E8G4= '' > AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI < /latexit > < latexit sha1_base64= '' rxH3eGL3jlLzhYJGRaokrVTVvjc= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ < /latexit > < latexit sha1_base64= '' rX31m9pL4TDFitzg9FObuhvEhn0= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK < /latexit > < latexit sha1_base64= '' 078TlPinZnbp3G8YDOaCG04E8G4= '' > AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF8lEKaUbf8Ctfpn4B/oX3hlHUIvohCRnzr3nzNx7vSTgmbCsl4IxMzs3v1BcLC0tr6yuldc3mlmcpz5z/DiI07bnZizgEXMEFwFrJylzQy9gLW94LOOtW5ZmPI4uxChh3dAdRLzPfVcQ5ZxejWuT63LFqlpqmdPA1qACvRpx+RmX6CGGjxwhGCIIwgFcZPR0YMNCQlwXY+JSQlzFGSYokTanLEYZLrFD+g5o19FsRHvpmSm1T6cE9KakNLFDmpjyUsLyNFPFc+Us2d+8x8pT3m1Ef097hcQK3BD7l+4z8786WYtAH4eqBk41JYqR1fnaJVddkTc3v1QlyCEhTuIexVPCvlJ+9tlUmkzVLnvrqvirypSs3Ps6N8ebvCUN2P45zmnQ3Kvater+ea1SP9KjLmIL29ileR6gjhM04JA3xwMe8WScGYlxZ4w+Uo2C1mzi2zLu3wFAhJEI < /latexit > < latexit sha1_base64= '' rxH3eGL3jlLzhYJGRaokrVTVvjc= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsVAdEl0o64wsUCCaNoy4IS+Mp1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3urHPE2lZrzljbn5hcSm/XFhZXVvfKG5uNZMoFR6zvciPRNt1EubzkNmSS5+1Y8GcwPVZyx2eqHjrjomER+GlHMWsGziDkPe550ii7LPrcW1yUyxZZUsvcxZUMlBCthpR8QVX6CGChxQBGEJIwj4cJPR0UIGFmLguxsQJQlzHGSYokDalLEYZDrFD+g5o18nYkPbKM9Fqj07x6RWkNLFHmojyBGF1mqnjqXZW7G/eY+2p7jaiv5t5BcRK3BL7l26a+V+dqkWijyNdA6eaYs2o6rzMJdVdUTc3v1QlySEmTuEexQVhTyunfTa1JtG1q946Ov6mMxWr9l6Wm+Jd3ZIGXPk5zlnQPChXquXaRbVUP85GnccOdrFP8zxEHadowCZvjkc84dk4N2Lj3hh9phq5TLONb8t4+ABC5ZEJ < /latexit > < latexit sha1_base64= '' rX31m9pL4TDFitzg9FObuhvEhn0= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IsXgY0l0o64wsUCCaNphwAl9pZ1qCGHjD7jVLzP+gf6Fd8aSqMToNG3PnHvPmbn3upEnEmlZrzljZnZufiG/WFhaXlldK65vNJIwjRm3WeiFcct1Eu6JgNtSSI+3opg7vuvxpjs4UfHmHY8TEQaXchjxju/0A9ETzJFE2WfXo4PxTbFklS29zGlQyUAJ2aqHxRdcoYsQDCl8cASQhD04SOhpowILEXEdjIiLCQkd5xijQNqUsjhlOMQO6NunXTtjA9orz0SrGZ3i0RuT0sQOaULKiwmr00wdT7WzYn/zHmlPdbch/d3MyydW4pbYv3STzP/qVC0SPRzpGgTVFGlGVccyl1R3Rd3c/FKVJIeIOIW7FI8JM62c9NnUmkTXrnrr6PibzlSs2rMsN8W7uiUNuPJznNOgsVeuVMv7F9VS7TgbdR5b2MYuzfMQNZyiDpu8BR7xhGfj3IiMe2P4mWrkMs0mvi3j4QNFRpEK < /latexit > < latexit sha1_base64= '' ZroCL0JJbplYYxKF0TuIbB7Z7xk= '' > AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU= < /latexit > < latexit sha1_base64= '' 2++cL5TEXumvU+oahUsTVndZR8E= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG < /latexit > < latexit sha1_base64= '' G06mXSCW+K11xSPKJ+cv3bGQhME= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH < /latexit > Pareto front of the learning objectives of all clients . Through the analysis of the geometric location of such an optimal model on the Pareto front , we can identify the necessary collaborators of each client . The relationships between each client and its necessary collaborators can be encoded in a benefit graph , in which each node denotes a client and the edge from Ij to Ii represents Ij is one of the necessary collaborators for Ii , as exemplified in Figure 1 ( a ) . Then we can derive the coalitions corresponding to the collaboration equilibrium through an iterative process introduced as follows . Specifically , we define a stable coalition as the minimum set such that its all involved clients can achieve its maximal utility . From the perspective of graph theory , these stable coalitions are actually the strongly connected components of the benefit graph . For example , C = { I1 , I2 , I3 } in Figure 1 ( b ) is a stable coalition as all clients can achieve their best performance by collaborating with the clients in C ( compared with collaborating with other clients in the network ) . By removing the stable coalitions and re-building the benefit graph of the remaining client iteratively as shown in Figure 1 ( b ) and ( c ) , we can identify all coalitions as in Figure 1 ( d ) and prove that the obtained coalitions can lead to a collaboration equilibrium . We empirically evaluate our method on synthetic data , UCI adult ( Kohavi , 1996 ) , a classical FL benchmark data set CIFAR10 ( Krizhevsky et al. , 2009 ) , and a real-world electronic health record ( EHR ) data repository eICU ( Pollard et al. , 2018 ) , which includes patient EHR data in ICU from multiple hospitals . The results show our method significantly outperforms existing relevant methods . The experiments on eICU data demonstrate that our algorithm is able to derive a good collaboration strategy for the hospitals to collaborate . 2 RELATED WORK . 2.1 FEDERATED LEARNING . Federated learning ( FL ) ( McMahan et al. , 2017 ) refers to the paradigm of learning from fragmented data without sacrificing privacy . In a typical FL setting , a global model is learned from the data residing in multiple distinct local clients . However , a single global model may lead to performance degradation on certain clients due to data heterogeneity . Personalized federated learning ( PFL ) ( Kulkarni et al. , 2020 ) , which aims at learning a customized model for each client in the federation , has been proposed to tackle this challenge . For example , Zhang et al . ( 2020 ) proposes to adjust the weight of the objectives corresponding to all clients dynamically ; Fallah et al . ( 2020 ) proposes a meta-learning based method for achieving an effectively shared initialization of all local models followed by a fine-tuning procedure ; Shamsian et al . ( 2021 ) proposes to learn a central hypernetwork which can generate a set of customized models for each client . FL assumes all clients are willing to participate in the collaboration and existing methods have not considered whether the collaboration can really benefit each client or not . Without benefit , a local client could be reluctant to participate in the collaboration , which is a realistic scenario we investigate in this paper . One specific FL setup that is relevant to our work is clustered federated learning ( Sattler et al. , 2020 ; Mansour et al. , 2020 ) , which groups the clients with similar data distributions and trains a model for each client group . The scenario we are considering in this paper is to form collaboration coalitions based on the performance gain each client can get for its corresponding model , rather than sample distribution similarities . 2.2 MULTI-TASK LEARNING AND NEGATIVE TRANSFER . Multi-task learning ( Caruana , 1997 ) ( MTL ) aims at learning shared knowledge across multiple interrelated tasks for mutual benefits . Typical examples include hard model parameter sharing ( Kokkinos , 2017 ) , soft parameter sharing ( Lu et al. , 2017 ) , and neural architecture search ( NAS ) for a shared Under review as a conference paper at ICLR 2022 model architecture ( Real et al. , 2019 ) . However , sharing representations of model structures can not guarantee model performance gain due to the existence of negative transfer , while we directly consider forming collaboration coalitions according to individual model performance benefits . In addition , MTL usually assumes the data from all tasks are accessible , while our goal is to learn a personalized model for each client through collaborating with other clients without directly accessing their raw data . It is worth mentioning that there are also clustered MTL approaches ( Standley et al. , 2020 ; Zamir et al. , 2018 ) which assume the models for the tasks within the same group are similar to each other , while we want the clients within each coalition can benefit each other through collaboration when learning their respective models . Some works extend MTL to a multi-source domain adaptation ( Wen et al. , 2020 ; Duan et al. , 2009 ) setting in which unlabeled data exist in some tasks . See detailed discussions in Appendix . 3 COLLABORATION LEARNING PROBLEM . We first introduce the necessary notations and definitions in Section 3.1 , and then define the collaboration equilibrium we aim to achieve in Section 3.2 . 3.1 DEFINITIONS AND NOTATIONS . Suppose there are N clients I = { Ii } N i=1 in a collaborative network and each client is associated with a specific learning task T i based on its own data Di = { Xi , Y i } , i ∈ { 1 , 2 , ... , N } , where the input space Xi and the output space Y i may or may not share across all N clients . Each client pursues collaboration with others to learn a personalized model M i by maximizing its utility ( i.e. , model performance ) without sacrificing data privacy . There is no guarantee that one client can always benefit from the collaboration with others , and the client would be reluctant to participate in the collaboration if there is no benefit . In the following , we describe this through a concrete example . No benefit , no collaboration . Suppose the local data { Di } , i ∈ { 1 , 2 , ... , N } owned by different clients satisfy the following conditions : 1 ) all local data are from the same distribution Di ∼ P ; 2 ) D1 ⊂ D2 ⊂ D3 , ... , ⊂ DN . Since DN contains more data than other clients , IN can not benefit from collaboration with any other clients , so IN will learn a local model using its own data . Once IN refuses to collaboration , IN−1 will also work on its own as IN−1 can only improve its utility by collaborating with IN . IN−2 will learn individually out of the same concerns . Finally , there is no collaboration among any clients . Due to the discrepancies of the sample distributions across different clients , the best local model for a specific client is very likely to come from collaborating with a subset of clients rather than all of them . Suppose U ( Ii , C ) denotes the model utility of client Ii when collaborating with the clients in client set C. In the following , we define Umax ( Ii , C ) as the maximum model utility that Ii can achieve when collaborating with different subsets of C. Definition 1 ( Maximum Achievable Utility ( MAU ) ) . This is the maximum model utility for a specific client Ii to collaborate with different subsets of client set C : Umax ( Ii , C ) = maxC′⊂C U ( Ii , C ′ ) . From Definition 1 , MAU satisfies Umax ( Ii , C ′ ) ≤ Umax ( Ii , C ) if C ′ is a subset of C. Each client Ii ∈ I aims to identify its “ optimal set '' of collaborators from I to maximize its local utility , which is defined as follows . Definition 2 ( Optimal Collaborator Set ( OCS ) ) . A client set CoptI ( I i ) ⊂ I is an optimal collaborator set for Ii if and only if CoptI ( I i ) satisfies ∀C ⊂ I , U ( Ii , CoptI ( Ii ) ) ≥ U ( Ii , C ) ; ( 1a ) ∀C ′ & CoptI ( Ii ) , U ( Ii , C opt I ( I i ) ) > U ( Ii , C ′ ) . ( 1b ) Eq . ( 1a ) means that Ii can achieve its maximal utility when collaborating with CoptI ( I i ) and Eq . ( 1b ) means that all clients in CoptI ( I i ) are necessary . In this way , the relationships between any client Ii ∈ I and its optimal collaborator set CoptI ( Ii ) can be represented by a graph which is called the benefit graph ( BG ) . Specifically , for a given client set C ⊂ I , we use BG ( C ) to denote its corresponding BG . For the example in Figure 1 ( a ) , an arrow from Ij to Ii means Ij ∈ CoptI ( Ii ) , e.g. , I1 → I3 means I1 ∈ CoptI ( I3 ) . For a client set C , if every member can achieve its maximum Under review as a conference paper at ICLR 2022 model utility through the collaboration with other members within C ( without collaboration with other members outside C ) , then we call C a coalition . I4 < latexit sha1_base64= '' Xwl+BCUvBvIgMA7IdpW3uxrkcDU= '' > AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hlpKk0zqYF5OJUko3/oBb/TLxD/QvvDOmoBbRCUnOnHvPmbn3eknAU2lZrwVjbn5hcam4XFpZXVvfKG9uNdM4Ez5z/DiIRdtzUxbwiDmSy4C1E8Hc0AtYy7s9VfHWHRMpj6MrOUpYN3SHER9w35VEOee9cW3SK1esqqWXOQvsHFSQr0ZcfsE1+ojhI0MIhgiScAAXKT0d2LCQENfFmDhBiOs4wwQl0maUxSjDJfaWvkPadXI2or3yTLXap1MCegUpTeyRJqY8QVidZup4pp0V+5v3WHuqu43o7+VeIbESN8T+pZtm/lenapEY4FjXwKmmRDOqOj93yXRX1M3NL1VJckiIU7hPcUHY18ppn02tSXXtqreujr/pTMWqvZ/nZnhXt6QB2z/HOQuaB1W7Vj28rFXqJ/moi9jBLvZpnkeo4wwNOOTN8YgnPBsXRmLcG6PPVKOQa7bxbRkPH0LnkQk= < /latexit > I5 < latexit sha1_base64= '' KKOEGmeLJ8PCjcxMDMM1BdYiV28= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Iq2B6JLoRl1hYoEECWmHASeUtmmnGkLY+ANu9cuMf6B/4Z2xJCoxOk3bM+fec2buvV7ki0Ra1mvOWFhcWl7JrxbW1jc2t4rbO40kTGPGHRb6Ydzy3IT7IuCOFNLnrSjm7sjzedMbnql4847HiQiDazmOeGfkDgLRF8yVRDkX3Ul12i2WrLKllzkP7AyUkK16WHzBDXoIwZBiBI4AkrAPFwk9bdiwEBHXwYS4mJDQcY4pCqRNKYtThkvskL4D2rUzNqC98ky0mtEpPr0xKU0ckCakvJiwOs3U8VQ7K/Y374n2VHcb09/LvEbEStwS+5dulvlfnapFoo8TXYOgmiLNqOpY5pLqrqibm1+qkuQQEadwj+IxYaaVsz6bWpPo2lVvXR1/05mKVXuW5aZ4V7ekAds/xzkPGkdlu1KuXlVKtdNs1HnsYR+HNM9j1HCOOhzyFnjEE56NSyMy7o3xZ6qRyzS7+LaMhw9FSJEK < /latexit > I6 < latexit sha1_base64= '' dLuWEcCiJIy7Wrla2OQjmor1BOM= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Iq3Bx5LoRl1hYoEECWnLgBNK27RTDSFs/AG3+mXGP9C/8M44JCoxOk3bM+fec2buvV4c8FRY1mvOmJtfWFzKLxdWVtfWN4qbW/U0yhKfOX4UREnTc1MW8JA5gouANeOEuUMvYA1vcCbjjTuWpDwKr8UoZu2h2w95j/uuIMq56IyPJp1iySpbapmzwNagBL1qUfEFN+gigo8MQzCEEIQDuEjpacGGhZi4NsbEJYS4ijNMUCBtRlmMMlxiB/Tt066l2ZD20jNVap9OCehNSGlijzQR5SWE5WmmimfKWbK/eY+Vp7zbiP6e9hoSK3BL7F+6aeZ/dbIWgR5OVA2caooVI6vztUumuiJvbn6pSpBDTJzEXYonhH2lnPbZVJpU1S5766r4m8qUrNz7OjfDu7wlDdj+Oc5ZUD8o25Xy4VWlVD3Vo85jB7vYp3keo4pz1OCQN8cjnvBsXBqxcW+MPlONnNZs49syHj4AR6mRCw== < /latexit > I1 < latexit sha1_base64= '' ZroCL0JJbplYYxKF0TuIbB7Z7xk= '' > AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRKp6LLoRl1VMG2hVknSaR2aF5OJUko3/oBb/TLxD/QvvDNGUIvohCRnzr3nzNx7vSTgqbSsl4IxMzs3v1BcLC0tr6yuldc3mmmcCZ85fhzEou25KQt4xBzJZcDaiWBu6AWs5Q2PVbx1y0TK4+hCjhLWDd1BxPvcdyVRzunV2J5clytW1dLLnAZ2DirIVyMuP+MSPcTwkSEEQwRJOICLlJ4ObFhIiOtiTJwgxHWcYYISaTPKYpThEjuk74B2nZyNaK88U6326ZSAXkFKEzukiSlPEFanmTqeaWfF/uY91p7qbiP6e7lXSKzEDbF/6T4z/6tTtUj0cahr4FRTohlVnZ+7ZLor6ubml6okOSTEKdyjuCDsa+Vnn02tSXXtqreujr/qTMWqvZ/nZnhTt6QB2z/HOQ2ae1W7Vt0/r1XqR/moi9jCNnZpngeo4wQNOOTN8YBHPBlnRmLcGaOPVKOQazbxbRn37zlhkQU= < /latexit > I2 < latexit sha1_base64= '' 2++cL5TEXumvU+oahUsTVndZR8E= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4Ii3B6JLoRl1hYoEE0bTDgBNK27RTDSFs/AG3+mXGP9C/8M5YEpUYnabtmXPvOTP3Xi/yRSIt6zVnzM0vLC7llwsrq2vrG8XNrUYSpjHjDgv9MG55bsJ9EXBHCunzVhRzd+j5vOkNTlS8ecfjRITBpRxFvDN0+4HoCeZKopyz63FlclMsWWVLL3MW2BkoIVv1sPiCK3QRgiHFEBwBJGEfLhJ62rBhISKugzFxMSGh4xwTFEibUhanDJfYAX37tGtnbEB75ZloNaNTfHpjUprYI01IeTFhdZqp46l2Vuxv3mPtqe42or+XeQ2Jlbgl9i/dNPO/OlWLRA9HugZBNUWaUdWxzCXVXVE3N79UJckhIk7hLsVjwkwrp302tSbRtaveujr+pjMVq/Ysy03xrm5JA7Z/jnMWNCplu1o+uKiWasfZqPPYwS72aZ6HqOEUdTjkLfCIJzwb50Zk3Bujz1Qjl2m28W0ZDx87wpEG < /latexit > I3 < latexit sha1_base64= '' G06mXSCW+K11xSPKJ+cv3bGQhME= '' > AAACyHicjVHLTsJAFD3UF+ILdemmkZi4IkUxuiS6UVeYWCBBNO0w4IS+0k41hLDxB9zqlxn/QP/CO2NJVGJ0mrZnzr3nzNx73cgTibSs15wxMzs3v5BfLCwtr6yuFdc3GkmYxozbLPTCuOU6CfdEwG0ppMdbUcwd3/V40x2cqHjzjseJCINLOYx4x3f6gegJ5kii7LPr0f74pliyypZe5jSoZKCEbNXD4guu0EUIhhQ+OAJIwh4cJPS0UYGFiLgORsTFhISOc4xRIG1KWZwyHGIH9O3Trp2xAe2VZ6LVjE7x6I1JaWKHNCHlxYTVaaaOp9pZsb95j7SnutuQ/m7m5RMrcUvsX7pJ5n91qhaJHo50DYJqijSjqmOZS6q7om5ufqlKkkNEnMJdiseEmVZO+mxqTaJrV711dPxNZypW7VmWm+Jd3ZIGXPk5zmnQ2CtXquWDi2qpdpyNOo8tbGOX5nmIGk5Rh03eAo94wrNxbkTGvTH8TDVymWYT35bx8AE+I5EH < /latexit > achieve its best performance without collaborating with the clients in C .

Summary: Given a network of clients each with his own private dataset, the locally learned model can be improved by obtaining data from other clients (assuming no domain mismatch). The paper models this problem by introducing the benefit graph which models how each client may benefit another. The concept of collaboration equilibrium is introduced and optimization algorithms to solve it are given. Some theoretical guarantees are shown along with experimental results.