OpenRaiser/Intern-Atlas huggingface dataset:
Towards Federated Learning-Enabled Visible Light Communication in 6G Systems
Venue: arXiv cs.AI 2021
Year: 2021

Visible light communication (VLC) technology was introduced as a key enabler for the next generation of wireless networks, mainly thanks to its simple and low-cost implementation. However, several challenges prohibit the realization of the full potentials of VLC, namely, limited modulation bandwidth, ambient light interference, optical diffuse reflection effects, devices non-linearity, and random receiver orientation. On the contrary, centralized machine learning (ML) techniques have demonstrated a significant potential in handling different challenges relating to wireless communication systems. Specifically, it was shown that ML algorithms exhibit superior capabilities in handling complicated network tasks, such as channel equalization, estimation and modeling, resources allocation, and opportunistic spectrum access control, to name a few. Nevertheless, concerns pertaining to privacy and communication overhead when sharing raw data of the involved clients with a server constitute major bottlenecks in the implementation of centralized ML techniques. This has motivated the emergence of a new distributed ML paradigm, namely federated learning (FL), which can reduce the cost associated with transferring raw data, and preserve privacy by training ML models locally and collaboratively at the clients' side. Hence, it becomes evident that integrating FL into VLC networks can provide ubiquitous and reliable implementation of VLC systems. With this motivation, this is the first in-depth review in the literature on the application of FL in VLC networks. To that end, besides the different architectures and related characteristics of FL, we provide a thorough overview on the main design aspects of FL based VLC systems. Finally, we also highlight some potential future research directions of FL that are envisioned to substantially enhance the performance and robustness of VLC systems.
MSHCNet: Multi-Stream Hybridized Convolutional Networks with Mixed Statistics in Euclidean/Non-Euclidean Spaces and Its Application to Hyperspectral Image Classification
Venue: arXiv cs.CV 2021
Year: 2021

It is well known that hyperspectral images (HSI) contain rich spatial-spectral contextual information, and how to effectively combine both spectral and spatial information using DNN for HSI classification has become a new research hotspot. Compared with CNN with square kernels, GCN have exhibited exciting potential to model spatial contextual structure and conduct flexible convolution on arbitrarily irregular image regions. However, current GCN only using first-order spectral-spatial signatures can result in boundary blurring and isolated misclassification. To address these, we first designed the graph-based second-order pooling (GSOP) operation to obtain contextual nodes information in non-Euclidean space for GCN. Further, we proposed a novel multi-stream hybridized convolutional network (MSHCNet) with combination of first and second order statistics in Euclidean/non-Euclidean spaces to learn and fuse multi-view complementary information to segment HSIs. Specifically, our MSHCNet adopted four parallel streams, which contained G-stream, utilizing the irregular correlation between adjacent land covers in terms of first-order graph in non-Euclidean space; C-stream, adopting convolution operator to learn regular spatial-spectral features in Euclidean space; N-stream, combining first and second order features to learn representative and discriminative regular spatial-spectral features of Euclidean space; S-stream, using GSOP to capture boundary correlations and obtain graph representations from all nodes in graphs of non-Euclidean space. Besides, these feature representations learned from four different streams were fused to integrate the multi-view complementary information for HSI classification. Finally, we evaluated our proposed MSHCNet on three hyperspectral datasets, and experimental results demonstrated that our method significantly outperformed state-of-the-art eight methods.
Analysis of the influence of political polarization in the vaccination stance: the Brazilian COVID-19 scenario
Venue: arXiv cs.SI 2021
Year: 2021

The outbreak of COVID-19 had a huge global impact, and non-scientific beliefs and political polarization have significantly influenced the population's behavior. In this context, COVID vaccines were made available in an unprecedented time, but a high level of hesitance has been observed that can undermine community immunization. Traditionally, anti-vaccination attitudes are more related to conspiratorial thinking rather than political bias. In Brazil, a country with an exemplar tradition in large-scale vaccination programs, all COVID-related topics have also been discussed under a strong political bias. In this paper, we use a multidimensional analysis framework to understand if anti/pro-vaccination stances expressed by Brazilians in social media are influenced by political polarization. The analysis framework incorporates techniques to automatically infer from users their political orientation, topic modeling to discover their concerns, network analysis to characterize their social behavior, and the characterization of information sources and external influence. Our main findings confirm that anti/pro stances are biased by political polarization, right and left, respectively. While a significant proportion of pro-vaxxers display haste for an immunization program and criticize the government's actions, the anti-vaxxers distrust a vaccine developed in a record time. Anti-vaccination stance is also related to prejudice against China (anti-sinovaxxers), revealing conspiratorial theories related to communism. All groups display an "echo chamber behavior, revealing they are not open to distinct views.
Beam Search with Bidirectional Strategies for Neural Response Generation
Venue: arXiv cs.CL 2021
Year: 2021

Sequence-to-sequence neural networks have been widely used in language-based applications as they have flexible capabilities to learn various language models. However, when seeking for the optimal language response through trained neural networks, current existing approaches such as beam-search decoder strategies are still not able reaching to promising performances. Instead of developing various decoder strategies based on a "regular sentence order" neural network (a trained model by outputting sentences from left-to-right order), we leveraged "reverse" order as additional language model (a trained model by outputting sentences from right-to-left order) which can provide different perspectives for the path finding problems. In this paper, we propose bidirectional strategies in searching paths by combining two networks (left-to-right and right-to-left language models) making a bidirectional beam search possible. Besides, our solution allows us using any similarity measure in our sentence selection criterion. Our approaches demonstrate better performance compared to the unidirectional beam search strategy.
SLASH: Embracing Probabilistic Circuits into Neural Answer Set Programming
Venue: arXiv cs.AI 2021
Year: 2021

The goal of combining the robustness of neural networks and the expressivity of symbolic methods has rekindled the interest in neuro-symbolic AI. Recent advancements in neuro-symbolic AI often consider specifically-tailored architectures consisting of disjoint neural and symbolic components, and thus do not exhibit desired gains that can be achieved by integrating them into a unifying framework. We introduce SLASH -- a novel deep probabilistic programming language (DPPL). At its core, SLASH consists of Neural-Probabilistic Predicates (NPPs) and logical programs which are united via answer set programming. The probability estimates resulting from NPPs act as the binding element between the logical program and raw input data, thereby allowing SLASH to answer task-dependent logical queries. This allows SLASH to elegantly integrate the symbolic and neural components in a unified framework. We evaluate SLASH on the benchmark data of MNIST addition as well as novel tasks for DPPLs such as missing data prediction and set prediction with state-of-the-art performance, thereby showing the effectiveness and generality of our method.
What motivates people to telework? Exploratory study in a post-confinement context
Venue: arXiv cs.CY 2021
Year: 2021

After the considerable excitement caused by COVID-19 and the first telework measures, many management issues became apparent and some questions quickly arose, especially about efficiency of employees' work and conditions for successful telework adoption. This study focuses on the following questions: what is the new interest to telework for employees and what are the potential reasons for this? How much employees feel able to do their work remotely now and why? To answer these questions, we conducted a survey over several weeks, that involved employees coming from different industries (N=170), in order to collect their experience, skills and motivations for teleworking. The results show that adoption of telework is real for the respondents. Those who have experienced it present a strong motivation and a real capacity to use it, with almost no technological barriers. The theoretical model that we used, based on Technology Acceptance Models (TAM), has highlighted an important new factor of telework adoption: time saved in commuting. The study points out that adoption of telework could be sustainable for both public and private organizations, and this requires a critical examination and discussion.
Efficient and Private Federated Learning with Partially Trainable Networks
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning is used for decentralized training of machine learning models on a large number (millions) of edge mobile devices. It is challenging because mobile devices often have limited communication bandwidth and local computation resources. Therefore, improving the efficiency of federated learning is critical for scalability and usability. In this paper, we propose to leverage partially trainable neural networks, which freeze a portion of the model parameters during the entire training process, to reduce the communication cost with little implications on model performance. Through extensive experiments, we empirically show that Federated learning of Partially Trainable neural networks (FedPT) can result in superior communication-accuracy trade-offs, with up to $46\times$ reduction in communication cost, at a small accuracy cost. Our approach also enables faster training, with a smaller memory footprint, and better utility for strong differential privacy guarantees. The proposed FedPT method can be particularly interesting for pushing the limitations of over-parameterization in on-device learning.
Privacy-preserving methods for smart-meter-based network simulations
Venue: arXiv cs.CR 2021
Year: 2021

Smart-meters are a key component of energy transition. The large amount of data collected in near real-time allows grid operators to observe and simulate network states. However, privacy-preserving rules forbid the use of such data for any applications other than network operation and billing. Smart-meter measurements must be anonymised to transmit these sensitive data to a third party to perform network simulation and analysis. This work proposes two methods for data anonymisation that enable the use of raw active power measurements for network simulation and analysis. The first is based on an allocation of an externally sourced load database. The second consists of grouping smart-meter data with similar electric characteristics, then performing a random permutation of the network load-bus assignment. A benchmark of these two methods highlights that both provide similar results in bus-voltage magnitude estimation concerning ground-truth voltage.
On the relationship between disentanglement and multi-task learning
Venue: arXiv cs.LG 2021
Year: 2021

One of the main arguments behind studying disentangled representations is the assumption that they can be easily reused in different tasks. At the same time finding a joint, adaptable representation of data is one of the key challenges in the multi-task learning setting. In this paper, we take a closer look at the relationship between disentanglement and multi-task learning based on hard parameter sharing. We perform a thorough empirical study of the representations obtained by neural networks trained on automatically generated supervised tasks. Using a set of standard metrics we show that disentanglement appears naturally during the process of multi-task neural network training.
Surrogate-Based Black-Box Optimization Method for Costly Molecular Properties
Venue: arXiv cs.LG 2021
Year: 2021

AI-assisted molecular optimization is a very active research field as it is expected to provide the next-generation drugs and molecular materials. An important difficulty is that the properties to be optimized rely on costly evaluations. Machine learning methods are investigated with success to predict these properties, but show generalization issues on less known areas of the chemical space. We propose here a surrogate-based black box optimization method, to tackle jointly the optimization and machine learning problems. It consists in optimizing the expected improvement of the surrogate of a molecular property using an evolutionary algorithm. The surrogate is defined as a Gaussian Process Regression (GPR) model, learned on a relevant area of the search space with respect to the property to be optimized. We show that our approach can successfully optimize a costly property of interest much faster than a purely metaheuristic approach.
mRAT-SQL+GAP:A Portuguese Text-to-SQL Transformer
Venue: arXiv cs.CL 2021
Year: 2021

The translation of natural language questions to SQL queries has attracted growing attention, in particular in connection with transformers and similar language models. A large number of techniques are geared towards the English language; in this work, we thus investigated translation to SQL when input questions are given in the Portuguese language. To do so, we properly adapted state-of-the-art tools and resources. We changed the RAT-SQL+GAP system by relying on a multilingual BART model (we report tests with other language models), and we produced a translated version of the Spider dataset. Our experiments expose interesting phenomena that arise when non-English languages are targeted; in particular, it is better to train with original and translated training datasets together, even if a single target language is desired. This multilingual BART model fine-tuned with a double-size training dataset (English and Portuguese) achieved 83% of the baseline, making inferences for the Portuguese test dataset. This investigation can help other researchers to produce results in Machine Learning in a language different from English. Our multilingual ready version of RAT-SQL+GAP and the data are available, open-sourced as mRAT-SQL+GAP at: https://github.com/C4AI/gap-text2sql
Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators
Venue: arXiv cs.LG 2021
Year: 2021

Discrete and especially binary random variables occur in many machine learning models, notably in variational autoencoders with binary latent states and in stochastic binary networks. When learning such models, a key tool is an estimator of the gradient of the expected loss with respect to the probabilities of binary variables. The straight-through (ST) estimator gained popularity due to its simplicity and efficiency, in particular in deep networks where unbiased estimators are impractical. Several techniques were proposed to improve over ST while keeping the same low computational complexity: Gumbel-Softmax, ST-Gumbel-Softmax, BayesBiNN, FouST. We conduct a theoretical analysis of bias and variance of these methods in order to understand tradeoffs and verify the originally claimed properties. The presented theoretical results allow for better understanding of these methods and in some cases reveal serious issues.
On the Complexity of Inductively Learning Guarded Rules
Venue: arXiv cs.CC 2021
Year: 2021

We investigate the computational complexity of mining guarded clauses from clausal datasets through the framework of inductive logic programming (ILP). We show that learning guarded clauses is NP-complete and thus one step below the $σ^P_2$-complete task of learning Horn clauses on the polynomial hierarchy. Motivated by practical applications on large datasets we identify a natural tractable fragment of the problem. Finally, we also generalise all of our results to $k$-guarded clauses for constant $k$.
Boxhead: A Dataset for Learning Hierarchical Representations
Venue: arXiv cs.LG 2021
Year: 2021

Disentanglement is hypothesized to be beneficial towards a number of downstream tasks. However, a common assumption in learning disentangled representations is that the data generative factors are statistically independent. As current methods are almost solely evaluated on toy datasets where this ideal assumption holds, we investigate their performance in hierarchical settings, a relevant feature of real-world data. In this work, we introduce Boxhead, a dataset with hierarchically structured ground-truth generative factors. We use this novel dataset to evaluate the performance of state-of-the-art autoencoder-based disentanglement models and observe that hierarchical models generally outperform single-layer VAEs in terms of disentanglement of hierarchically arranged factors.
Developing Medical AI : a cloud-native audio-visual data collection study
Venue: arXiv cs.HC 2021
Year: 2021

Designing Artificial Intelligence (AI) solutions that can operate in real-world situations is a highly complex task. Deploying such solutions in the medical domain is even more challenging. The promise of using AI to improve patient care and reduce cost has encouraged many companies to undertake such endeavours. For our team, the goal has been to improve early identification of deteriorating patients in the hospital. Identifying patient deterioration in lower acuity wards relies, to a large degree on the attention and intuition of clinicians, rather than on the presence of physiological monitoring devices. In these care areas, an automated tool which could continuously observe patients and notify the clinical staff of suspected deterioration, would be extremely valuable. In order to develop such an AI-enabled tool, a large collection of patient images and audio correlated with corresponding vital signs, past medical history and clinical outcome would be indispensable. To the best of our knowledge, no such public or for-pay data set currently exists. This lack of audio-visual data led to the decision to conduct exactly such study. The main contributions of this paper are, the description of a protocol for audio-visual data collection study, a cloud-architecture for efficiently processing and consuming such data, and the design of a specific data collection device.
SVG-Net: An SVG-based Trajectory Prediction Model
Venue: arXiv cs.CV 2021
Year: 2021

Anticipating motions of vehicles in a scene is an essential problem for safe autonomous driving systems. To this end, the comprehension of the scene's infrastructure is often the main clue for predicting future trajectories. Most of the proposed approaches represent the scene with a rasterized format and some of the more recent approaches leverage custom vectorized formats. In contrast, we propose representing the scene's information by employing Scalable Vector Graphics (SVG). SVG is a well-established format that matches the problem of trajectory prediction better than rasterized formats while being more general than arbitrary vectorized formats. SVG has the potential to provide the convenience and generality of raster-based solutions if coupled with a powerful tool such as CNNs, for which we introduce SVG-Net. SVG-Net is a Transformer-based Neural Network that can effectively capture the scene's information from SVG inputs. Thanks to the self-attention mechanism in its Transformers, SVG-Net can also adequately apprehend relations amongst the scene and the agents. We demonstrate SVG-Net's effectiveness by evaluating its performance on the publicly available Argoverse forecasting dataset. Finally, we illustrate how, by using SVG, one can benefit from datasets and advancements in other research fronts that also utilize the same input format. Our code is available at https://vita-epfl.github.io/SVGNet/.
HABCSm: A Hamming Based t-way Strategy Based on Hybrid Artificial Bee Colony for Variable Strength Test Sets Generation
Venue: arXiv cs.SE 2021
Year: 2021

Search-based software engineering that involves the deployment of meta-heuristics in applicable software processes has been gaining wide attention. Recently, researchers have been advocating the adoption of meta-heuristic algorithms for t-way testing strategies (where t points the interaction strength among parameters). Although helpful, no single meta-heuristic based t-way strategy can claim dominance over its counterparts. For this reason, the hybridization of meta-heuristic algorithms can help to ascertain the search capabilities of each by compensating for the limitations of one algorithm with the strength of others. Consequently, a new meta-heuristic based t-way strategy called Hybrid Artificial Bee Colony (HABCSm) strategy, based on merging the advantages of the Artificial Bee Colony (ABC) algorithm with the advantages of a Particle Swarm Optimization (PSO) algorithm is proposed in this paper. HABCSm is the first t-way strategy to adopt Hybrid Artificial Bee Colony (HABC) algorithm with Hamming distance as its core method for generating a final test set and the first to adopt the Hamming distance as the final selection criterion for enhancing the exploration of new solutions. The experimental results demonstrate that HABCSm provides superior competitive performance over its counterparts. Therefore, this finding contributes to the field of software testing by minimizing the number of test cases required for test execution.
Optimizing Oil and Gas Acquisitions Using Recommender Systems
Venue: arXiv cs.IR 2021
Year: 2021

Well acquisition in the oil and gas industry can often be a hit or miss process, with a poor purchase resulting in substantial loss. Recommender systems suggest items (wells) that users (companies) are likely to buy based on past activity, and applying this system to well acquisition can increase company profits. While traditional recommender systems are impactful enough on their own, they are not optimized. This is because they ignore many of the complexities involved in human decision-making, and frequently make subpar recommendations. Using a preexisting Python implementation of a Factorization Machine results in more accurate recommendations based on a user-level ranking system. We train a Factorization Machine model on oil and gas well data that includes features such as elevation, total depth, and location. The model produces recommendations by using similarities between companies and wells, as well as their interactions. Our model has a hit rate of 0.680, reciprocal rank of 0.469, precision of 0.229, and recall of 0.463. These metrics imply that while our model is able to recommend the correct wells in a general sense, it does not match exact wells to companies via relevance. To improve the model's accuracy, future models should incorporate additional features such as the well's production data and ownership duration as these features will produce more accurate recommendations.