pwc-archive/papers-with-abstracts huggingface dataset:
Self-Supervised Graph Representation Learning for Neuronal Morphologies
Authors: Marissa A. Weis, Laura Hansel, Timo Lüddecke, Alexander S. Ecker
Date: 2021-12-23 00:00:00
Tasks: Data Augmentation, Graph Representation Learning, Representation Learning, Self-Supervised Learning

Unsupervised graph representation learning has recently gained interest in several application domains such as neuroscience, where modeling the diverse morphology of cell types in the brain is one of the key challenges. It is currently unknown how many excitatory cortical cell types exist and what their defining morphological features are. Here we present GraphDINO, a purely data-driven approach to learn low-dimensional representations of 3D neuronal morphologies from unlabeled large-scale datasets. GraphDINO is a novel transformer-based representation learning method for spatially-embedded graphs. To enable self-supervised learning on transformers, we (1) developed data augmentation strategies for spatially-embedded graphs, (2) adapted the positional encoding and (3) introduced a novel attention mechanism, AC-Attention, which combines attention-based global interaction between nodes and classic graph convolutional processing. We show, in two different species and across multiple brain areas, that this method yields morphological cell type clusterings that are on par with manual feature-based classification by experts, but without using prior knowledge about the structural features of neurons. Moreover, it outperforms previous approaches on quantitative benchmarks predicting expert labels. Our method could potentially enable data-driven discovery of novel morphological features and cell types in large-scale datasets. It is applicable beyond neuroscience in settings where samples in a dataset are graphs and graph-level embeddings are desired.
The interplay between ranking and communities in networks
Authors: Laura Iacovissi, Caterina De Bacco
Date: 2021-12-23 00:00:00
Tasks: Community Detection

Community detection and hierarchy extraction are usually thought of as separate inference tasks on networks. Considering only one of the two when studying real-world data can be an oversimplification. In this work, we present a generative model based on an interplay between community and hierarchical structures. It assumes that each node has a preference in the interaction mechanism and nodes with the same preference are more likely to interact, while heterogeneous interactions are still allowed. The sparsity of the network is exploited for implementing a more efficient algorithm. We demonstrate our method on synthetic and real-world data and compare performance with two standard approaches for community detection and ranking extraction. We find that the algorithm accurately retrieves the overall node's preference in different scenarios, and we show that it can distinguish small subsets of nodes that behave differently than the majority. As a consequence, the model can recognize whether a network has an overall preferred interaction mechanism. This is relevant in situations where there is no clear "a priori" information about what structure explains the observed network datasets well. Our model allows practitioners to learn this automatically from the data.
Intra-Household Management of Joint Resources: Evidence from Malawi
Authors: Anna Josephson
Date: 2021-12-23 00:00:00
Tasks: Decision Making, Management

In models of intra-household resource allocation, the earnings from joint work between two or more household members are often omitted. I test assumptions about complete pooling of resources within a household, by accounting for income earned jointly by multiple household members, in addition to income earned individually by men and women. Applied in the case of Malawi, I find that by explicitly including intra-household collaboration, I find evidence of partial income pooling and partial insurance within the household, specifically for expenditures on food. Importantly, including joint income reveals dynamics between household members, as well as opportunities and vulnerabilities which may previously be obfuscated in simpler, binary specifications. Contrasting with previous studies and empirical practice, my findings suggest that understanding detailed intra-household interactions and their outcomes on household behavior have important consequences for household resource allocation and decision making.
Analysis of Langevin Monte Carlo from Poincaré to Log-Sobolev
Authors: Sinho Chewi, Murat A. Erdogdu, Mufan Bill Li, Ruoqi Shen, Matthew Zhang
Date: 2021-12-23 00:00:00

Classically, the continuous-time Langevin diffusion converges exponentially fast to its stationary distribution $\pi$ under the sole assumption that $\pi$ satisfies a Poincar\'e inequality. Using this fact to provide guarantees for the discrete-time Langevin Monte Carlo (LMC) algorithm, however, is considerably more challenging due to the need for working with chi-squared or R\'enyi divergences, and prior works have largely focused on strongly log-concave targets. In this work, we provide the first convergence guarantees for LMC assuming that $\pi$ satisfies either a Lata\l{}a--Oleszkiewicz or modified log-Sobolev inequality, which interpolates between the Poincar\'e and log-Sobolev settings. Unlike prior works, our results allow for weak smoothness and do not require convexity or dissipativity conditions.
DENSE: Data-Free One-Shot Federated Learning
Authors: Jie Zhang, Chen Chen, Bo Li, Lingjuan Lyu, Shuang Wu, Shouhong Ding, Chunhua Shen, Chao Wu
Date: 2021-12-23 00:00:00
Tasks: Federated Learning

One-shot Federated Learning (FL) has recently emerged as a promising approach, which allows the central server to learn a model in a single communication round. Despite the low communication cost, existing one-shot FL methods are mostly impractical or face inherent limitations, \eg a public dataset is required, clients' models are homogeneous, and additional data/model information need to be uploaded. To overcome these issues, we propose a novel two-stage \textbf{D}ata-fre\textbf{E} o\textbf{N}e-\textbf{S}hot federated l\textbf{E}arning (DENSE) framework, which trains the global model by a data generation stage and a model distillation stage. DENSE is a practical one-shot FL method that can be applied in reality due to the following advantages: (1) DENSE requires no additional information compared with other methods (except the model parameters) to be transferred between clients and the server; (2) DENSE does not require any auxiliary dataset for training; (3) DENSE considers model heterogeneity in FL, \ie different clients can have different model architectures. Experiments on a variety of real-world datasets demonstrate the superiority of our method.For example, DENSE outperforms the best baseline method Fed-ADI by 5.08\% on CIFAR10 dataset.
FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation
Authors: Xingyu Li, Zhe Qu, Bo Tang, Zhuo Lu
Date: 2021-12-22 00:00:00
Tasks: Federated Learning

Federated Learning (FL) is a decentralized machine learning architecture, which leverages a large number of remote devices to learn a joint model with distributed training data. However, the system-heterogeneity is one major challenge in a FL network to achieve robust distributed learning performance, which comes from two aspects: i) device-heterogeneity due to the diverse computational capacity among devices; ii) data-heterogeneity due to the non-identically distributed data across the network. Prior studies addressing the heterogeneous FL issue, e.g., FedProx, lack formalization and it remains an open problem. This work first formalizes the system-heterogeneous FL problem and proposes a new algorithm, called FedLGA, to address this problem by bridging the divergence of local model updates via gradient approximation. To achieve this, FedLGA provides an alternated Hessian estimation method, which only requires extra linear complexity on the aggregator. Theoretically, we show that with a device-heterogeneous ratio $\rho$, FedLGA achieves convergence rates on non-i.i.d. distributed FL training data for the non-convex optimization problems with $\mathcal{O} \left( \frac{(1+\rho)}{\sqrt{ENT}} + \frac{1}{T} \right)$ and $\mathcal{O} \left( \frac{(1+\rho)\sqrt{E}}{\sqrt{TK}} + \frac{1}{T} \right)$ for full and partial device participation respectively, where $E$ is the number of local learning epoch, $T$ is the number of total communication round, $N$ is the total device number and $K$ is the number of selected device in one communication round under partially participation scheme. The results of comprehensive experiments on multiple datasets show that FedLGA outperforms current FL methods against the system-heterogeneity.
FLoBC: A Decentralized Blockchain-Based Federated Learning Framework
Authors: Mohamed Ghanem, Fadi Dawoud, Habiba Gamal, Eslam Soliman, Hossam Sharara, Tamer El-Batt
Date: 2021-12-22 00:00:00
Tasks: BIG-bench Machine Learning, Federated Learning

The rapid expansion of data worldwide invites the need for more distributed solutions in order to apply machine learning on a much wider scale. The resultant distributed learning systems can have various degrees of centralization. In this work, we demonstrate our solution FLoBC for building a generic decentralized federated learning system using blockchain technology, accommodating any machine learning model that is compatible with gradient descent optimization. We present our system design comprising the two decentralized actors: trainer and validator, alongside our methodology for ensuring reliable and efficient operation of said system. Finally, we utilize FLoBC as an experimental sandbox to compare and contrast the effects of trainer-to-validator ratio, reward-penalty policy, and model synchronization schemes on the overall system performance, ultimately showing by example that a decentralized federated learning system is indeed a feasible alternative to more centralized architectures.
On-the-fly Resource-Aware Model Aggregation for Federated Learning in Heterogeneous Edge
Authors: Hung T. Nguyen, Roberto Morabito, Kwang Taik Kim, Mung Chiang
Date: 2021-12-21 00:00:00
Tasks: BIG-bench Machine Learning, Edge-computing, Federated Learning

Edge computing has revolutionized the world of mobile and wireless networks world thanks to its flexible, secure, and performing characteristics. Lately, we have witnessed the increasing use of it to make more performing the deployment of machine learning (ML) techniques such as federated learning (FL). FL was debuted to improve communication efficiency compared to conventional distributed machine learning (ML). The original FL assumes a central aggregation server to aggregate locally optimized parameters and might bring reliability and latency issues. In this paper, we conduct an in-depth study of strategies to replace this central server by a flying master that is dynamically selected based on the current participants and/or available resources at every FL round of optimization. Specifically, we compare different metrics to select this flying master and assess consensus algorithms to perform the selection. Our results demonstrate a significant reduction of runtime using our flying master FL framework compared to the original FL from measurements results conducted in our EdgeAI testbed and over real 5G networks using an operational edge testbed.
Hierarchical Over-the-Air Federated Edge Learning
Authors: Ozan Aygün, Mohammad Kazemi, Deniz Gündüz, Tolga M. Duman
Date: 2021-12-21 00:00:00
Tasks: Federated Learning

Federated learning (FL) over wireless communication channels, specifically, over-the-air (OTA) model aggregation framework is considered. In OTA wireless setups, the adverse channel effects can be alleviated by increasing the number of receive antennas at the parameter server (PS), which performs model aggregation. However, the performance of OTA FL is limited by the presence of mobile users (MUs) located far away from the PS. In this paper, to mitigate this limitation, we propose hierarchical over-the-air federated learning (HOTAFL), which utilizes intermediary servers (IS) to form clusters near MUs. We provide a convergence analysis for the proposed setup, and demonstrate through theoretical and experimental results that local aggregation in each cluster before global aggregation leads to a better performance and faster convergence than OTA FL.
FedPOIRec: Privacy Preserving Federated POI Recommendation with Social Influence
Authors: Vasileios Perifanis, George Drosatos, Giorgos Stamatelatos, Pavlos S. Efraimidis
Date: 2021-12-21 00:00:00
Tasks: Federated Learning, Privacy Preserving, Transfer Learning

With the growing number of Location-Based Social Networks, privacy preserving location prediction has become a primary task for helping users discover new points-of-interest (POIs). Traditional systems consider a centralized approach that requires the transmission and collection of users' private data. In this work, we present FedPOIRec, a privacy preserving federated learning approach enhanced with features from users' social circles for top-$N$ POI recommendations. First, the FedPOIRec framework is built on the principle that local data never leave the owner's device, while the local updates are blindly aggregated by a parameter server. Second, the local recommenders get personalized by allowing users to exchange their learned parameters, enabling knowledge transfer among friends. To this end, we propose a privacy preserving protocol for integrating the preferences of a user's friends after the federated computation, by exploiting the properties of the CKKS fully homomorphic encryption scheme. To evaluate FedPOIRec, we apply our approach into five real-world datasets using two recommendation models. Extensive experiments demonstrate that FedPOIRec achieves comparable recommendation quality to centralized approaches, while the social integration protocol incurs low computation and communication overhead on the user side.
Distributed Machine Learning and the Semblance of Trust
Authors: Dmitrii Usynin, Alexander Ziller, Daniel Rueckert, Jonathan Passerat-Palmbach, Georgios Kaissis
Date: 2021-12-21 00:00:00
Tasks: BIG-bench Machine Learning, Federated Learning, Privacy Preserving

The utilisation of large and diverse datasets for machine learning (ML) at scale is required to promote scientific insight into many meaningful problems. However, due to data governance regulations such as GDPR as well as ethical concerns, the aggregation of personal and sensitive data is problematic, which prompted the development of alternative strategies such as distributed ML (DML). Techniques such as Federated Learning (FL) allow the data owner to maintain data governance and perform model training locally without having to share their data. FL and related techniques are often described as privacy-preserving. We explain why this term is not appropriate and outline the risks associated with over-reliance on protocols that were not designed with formal definitions of privacy in mind. We further provide recommendations and examples on how such algorithms can be augmented to provide guarantees of governance, security, privacy and verifiability for a general ML audience without prior exposure to formal privacy techniques.
Mitigating Leakage from Data Dependent Communications in Decentralized Computing using Differential Privacy
Authors: Riad Ladjel, Nicolas Anciaux, Aurélien Bellet, Guillaume Scerri
Date: 2021-12-23 00:00:00

Imagine a group of citizens willing to collectively contribute their personal data for the common good to produce socially useful information, resulting from data analytics or machine learning computations. Sharing raw personal data with a centralized server performing the computation could raise concerns about privacy and a perceived risk of mass surveillance. Instead, citizens may trust each other and their own devices to engage into a decentralized computation to collaboratively produce an aggregate data release to be shared. In the context of secure computing nodes exchanging messages over secure channels at runtime, a key security issue is to protect against external attackers observing the traffic, whose dependence on data may reveal personal information. Existing solutions are designed for the cloud setting, with the goal of hiding all properties of the underlying dataset, and do not address the specific privacy and efficiency challenges that arise in the above context. In this paper, we define a general execution model to control the data-dependence of communications in user-side decentralized computations, in which differential privacy guarantees for communication patterns in global execution plans can be analyzed by combining guarantees obtained on local clusters of nodes. We propose a set of algorithms which allow to trade-off between privacy, utility and efficiency. Our formal privacy guarantees leverage and extend recent results on privacy amplification by shuffling. We illustrate the usefulness of our proposal on two representative examples of decentralized execution plans with data-dependent communications.
Person Detection in Collaborative Group Learning Environments Using Multiple Representations
Authors: Wenjing Shi, Marios S. Pattichis, Sylvia Celedón-Pattichis, Carlos LópezLeiva
Date: 2021-12-22 00:00:00
Tasks: Face Detection, Human Detection

We introduce the problem of detecting a group of students from classroom videos. The problem requires the detection of students from different angles and the separation of the group from other groups in long videos (one to one and a half hours). We use multiple image representations to solve the problem. We use FM components to separate each group from background groups, AM-FM components for detecting the back-of-the-head, and YOLO for face detection. We use classroom videos from four different groups to validate our approach. Our use of multiple representations is shown to be significantly more accurate than the use of YOLO alone.
Beyond Low Earth Orbit: Biomonitoring, Artificial Intelligence, and Precision Space Health
Authors: Ryan T. Scott, Erik L. Antonsen, Lauren M. Sanders, Jaden J. A. Hastings, Seung-min Park, Graham Mackintosh, Robert J. Reynolds, Adrienne L. Hoarfrost, Aenor Sawyer, Casey S. Greene, Benjamin S. Glicksberg, Corey A. Theriot, Daniel C. Berrios, Jack Miller, Joel Babdor, Richard Barker, Sergio E. Baranzini, Afshin Beheshti, Stuart Chalk, Guillermo M. Delgado-Aparicio, Melissa Haendel, Arif A. Hamid, Philip Heller, Daniel Jamieson, Katelyn J. Jarvis, John Kalantari, Kia Khezeli, Svetlana V. Komarova, Matthieu Komorowski, Prachi Kothiyal, Ashish Mahabal, Uri Manor, Hector Garcia Martin, Christopher E. Mason, Mona Matar, George I. Mias, Jerry G. Myers, Jr., Charlotte Nelson, Jonathan Oribello, Patricia Parsons-Wingerter, R. K. Prabhu, Amina Ann Qutub, Jon Rask, Amanda Saravia-Butler, Suchi Saria, Nitin Kumar Singh, Frank Soboczenski, Michael Snyder, Karthik Soman, David Van Valen, Kasthuri Venkateswaran, Liz Warren, Liz Worthey, Jason H. Yang, Marinka Zitnik, Sylvain V. Costes
Date: 2021-12-22 00:00:00
Tasks: Management

Human space exploration beyond low Earth orbit will involve missions of significant distance and duration. To effectively mitigate myriad space health hazards, paradigm shifts in data and space health systems are necessary to enable Earth-independence, rather than Earth-reliance. Promising developments in the fields of artificial intelligence and machine learning for biology and health can address these needs. We propose an appropriately autonomous and intelligent Precision Space Health system that will monitor, aggregate, and assess biomedical statuses; analyze and predict personalized adverse health outcomes; adapt and respond to newly accumulated data; and provide preventive, actionable, and timely insights to individual deep space crew members and iterative decision support to their crew medical officer. Here we present a summary of recommendations from a workshop organized by the National Aeronautics and Space Administration, on future applications of artificial intelligence in space biology and health. In the next decade, biomonitoring technology, biomarker science, spacecraft hardware, intelligent software, and streamlined data management must mature and be woven together into a Precision Space Health system to enable humanity to thrive in deep space.
A meta-analysis of residential PV adoption: the important role of perceived benefits, intentions and antecedents in solar energy acceptance
Authors: Emily Schulte, Fabian Scheller, Daniel Sloot, Thomas Bruckner
Date: 2021-12-23 00:00:00

The adoption of residential photovoltaic systems (PV) is seen as an important part of the sustainable energy transition. To facilitate this process, it is crucial to identify the determinants of solar adoption. This paper follows a meta-analytical structural equation modeling approach, presenting a meta-analysis of studies on residential PV adoption intention, and assessing four behavioral models based on the theory of planned behavior to advance theory development. Of 653 initially identified studies, 110 remained for full-text screening. Only eight studies were sufficiently homogeneous, provided bivariate correlations, and could thus be integrated into the meta-analysis.The pooled correlations across primary studies revealed medium to large correlations between environmental concern, novelty seeking, perceived benefits, subjective norm and intention to adopt a residential PV system, whereas socio-demographic variables were uncorrelated with intention. Meta-analytical structural equation modeling revealed a model (N = 1,714) in which adoption intention was predicted by benefits and perceived behavioral control, and benefits in turn could be explained by environmental concern, novelty seeking, and subjective norm. Our results imply that measures should primarily focus on enhancing the perception of benefits. Based on obstacles we encountered within the analysis, we suggest guidelines to facilitate the future aggregation of scientific evidence, such as the systematic inclusion of key variables and reporting of bivariate correlations.
Dual Path Structural Contrastive Embeddings for Learning Novel Objects
Authors: Bingbin Li, Elvis Han Cui, Yanan Li, Donghui Wang, Weng Kee Wong
Date: 2021-12-23 00:00:00
Tasks: Meta-Learning, Transfer Learning

Learning novel classes from a very few labeled samples has attracted increasing attention in machine learning areas. Recent research on either meta-learning based or transfer-learning based paradigm demonstrates that gaining information on a good feature space can be an effective solution to achieve favorable performance on few-shot tasks. In this paper, we propose a simple but effective paradigm that decouples the tasks of learning feature representations and classifiers and only learns the feature embedding architecture from base classes via the typical transfer-learning training strategy. To maintain both the generalization ability across base and novel classes and discrimination ability within each class, we propose a dual path feature learning scheme that effectively combines structural similarity with contrastive feature construction. In this way, both inner-class alignment and inter-class uniformity can be well balanced, and result in improved performance. Experiments on three popular benchmarks show that when incorporated with a simple prototype based classifier, our method can still achieve promising results for both standard and generalized few-shot problems in either an inductive or transductive inference setting.