OpenRaiser/Intern-Atlas huggingface dataset:
Gaussian Experts Selection using Graphical Models
Venue: arXiv cs.LG 2021
Year: 2021

Local approximations are popular methods to scale Gaussian processes (GPs) to big data. Local approximations reduce time complexity by dividing the original dataset into subsets and training a local expert on each subset. Aggregating the experts' prediction is done assuming either conditional dependence or independence between the experts. Imposing the \emph{conditional independence assumption} (CI) between the experts renders the aggregation of different expert predictions time efficient at the cost of poor uncertainty quantification. On the other hand, modeling dependent experts can provide precise predictions and uncertainty quantification at the expense of impractically high computational costs. By eliminating weak experts via a theory-guided expert selection step, we substantially reduce the computational cost of aggregating dependent experts while ensuring calibrated uncertainty quantification. We leverage techniques from the literature on undirected graphical models, using sparse precision matrices that encode conditional dependencies between experts to select the most important experts. Moreov
Clickbait Headline Detection in Indonesian News Sites using Multilingual Bidirectional Encoder Representations from Transformers (M-BERT)
Venue: arXiv cs.CL 2021
Year: 2021

Click counts are related to the amount of money that online advertisers paid to news sites. Such business models forced some news sites to employ a dirty trick of click-baiting, i.e., using a hyperbolic and interesting words, sometimes unfinished sentence in a headline to purposefully tease the readers. Some Indonesian online news sites also joined the party of clickbait, which indirectly degrade other established news sites' credibility. A neural network with a pre-trained language model M-BERT that acted as a embedding layer is then combined with a 100 nodes hidden layer and topped with a sigmoid classifier was trained to detect clickbait headlines. With a total of 6632 headlines as a training dataset, the classifier performed remarkably well. Evaluated with 5-fold cross validation, it has an accuracy score of 0.914, f1-score of 0.914, precision score of 0.916, and ROC-AUC of 0.92. The usage of multilingual BERT in Indonesian text classification task was tested and is possible to be enhanced further. Future possibilities, societal impact, and limitations of the clickbait detection are discussed.
Targeted Branching for the Maximum Independent Set Problem
Venue: arXiv cs.DS 2021
Year: 2021

Finding a maximum independent set is a fundamental NP-hard problem that is used in many real-world applications. Given an unweighted graph, this problem asks for a maximum cardinality set of pairwise non-adjacent vertices. Some of the most successful algorithms for this problem are based on the branch-and-bound or branch-and-reduce paradigms. In particular, branch-and-reduce algorithms, which combine branch-and-bound with reduction rules, achieved substantial results, solving many previously infeasible instances. These results were to a large part achieved by developing new, more practical reduction rules. However, other components that have been shown to have an impact on the performance of these algorithms have not received as much attention. One of these is the branching strategy, which determines what vertex is included or excluded in a potential solution. The most commonly used strategy selects vertices based on their degree and does not take into account other factors that contribute to the performance. In this work, we develop and evaluate several novel branching strategies for both branch-and-bound and branch-and-reduce algorithms. Our strategies are based on one of two approaches. They either (1) aim to decompose the graph into two or more connected components which can then be solved independently, or (2) try to remove vertices that hinder the application of a reduction rule. Our experiments on a large set of real-world instances indicate that our strategies are able to improve the performance of the state-of-the-art branch-and-reduce algorithms. To be more specific, our reduction-based packing branching rule is able to outperform the default branching strategy of selecting a vertex of highest degree on 65% of all instances tested. Furthermore, our decomposition-based strategy based on edge cuts is able to achieve a speedup of 2.29 on sparse networks (1.22 on all instances).
Tree trace reconstruction using subtraces
Venue: arXiv cs.DS 2021
Year: 2021

Tree trace reconstruction aims to learn the binary node labels of a tree, given independent samples of the tree passed through an appropriately defined deletion channel. In recent work, Davies, Rácz, and Rashtchian used combinatorial methods to show that $\exp(\mathcal{O}(k \log_{k} n))$ samples suffice to reconstruct a complete $k$-ary tree with $n$ nodes with high probability. We provide an alternative proof of this result, which allows us to generalize it to a broader class of tree topologies and deletion models. In our proofs, we introduce the notion of a subtrace, which enables us to connect with and generalize recent mean-based complex analytic algorithms for string trace reconstruction.
Real-time detection of uncalibrated sensors using Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Nowadays, sensors play a major role in several contexts like science, industry and daily life which benefit of their use. However, the retrieved information must be reliable. Anomalies in the behavior of sensors can give rise to critical consequences such as ruining a scientific project or jeopardizing the quality of the production in industrial production lines. One of the more subtle kind of anomalies are uncalibrations. An uncalibration is said to take place when the sensor is not adjusted or standardized by calibration according to a ground truth value. In this work, an online machine-learning based uncalibration detector for temperature, humidity and pressure sensors was developed. This solution integrates an Artificial Neural Network as main component which learns from the behavior of the sensors under calibrated conditions. Then, after trained and deployed, it detects uncalibrations once they take place. The obtained results show that the proposed solution is able to detect uncalibrations for deviation values of 0.25 degrees, 1% RH and 1.5 Pa, respectively. This solution can be adapted to different contexts by means of transfer learning, whose application allows for the addition of new sensors, the deployment into new environments and the retraining of the model with minimum amounts of data.
Super-klust: Another Way of Piecewise Linear Classification
Venue: arXiv cs.LG 2021
Year: 2021

With our previous study, the Super-k algorithm, we have introduced a novel way of piecewise-linear classification. While working on the Super-k algorithm, we have found that there is a similar, and simpler way to explain for obtaining a piecewise-linear classifier based on Voronoi tessellations. Replacing the multidimensional voxelization and expectation-maximization stages of the algorithm with a distance-based clustering algorithm, preferably k-means, works as well as the prior approach. Since we are replacing the voxelization with the clustering, we have found it meaningful to name the modified algorithm, with respect to Super-k, as Supervised k Clusters or in short Super-klust. Similar to the Super-k algorithm, the Super-klust algorithm covers data with a labeled Voronoi tessellation, and uses resulting tessellation for classification. According to the experimental results, the Super-klust algorithm has similar performance characteristics with the Super-k algorithm.
FEDZIP: A Compression Framework for Communication-Efficient Federated Learning
Venue: arXiv cs.LG 2021
Year: 2021

Federated Learning marks a turning point in the implementation of decentralized machine learning (especially deep learning) for wireless devices by protecting users' privacy and safeguarding raw data from third-party access. It assigns the learning process independently to each client. First, clients locally train a machine learning model based on local data. Next, clients transfer local updates of model weights and biases (training data) to a server. Then, the server aggregates updates (received from clients) to create a global learning model. However, the continuous transfer between clients and the server increases communication costs and is inefficient from a resource utilization perspective due to the large number of parameters (weights and biases) used by deep learning models. The cost of communication becomes a greater concern when the number of contributing clients and communication rounds increases. In this work, we propose a novel framework, FedZip, that significantly decreases the size of updates while transferring weights from the deep learning model between clients and their servers. FedZip implements Top-z sparsification, uses quantization with clustering, and implements compression with three different encoding methods. FedZip outperforms state-of-the-art compression frameworks and reaches compression rates up to 1085x, and preserves up to 99% of bandwidth and 99% of energy for clients during communication.
Model-based multi-parameter mapping
Venue: arXiv cs.CV 2021
Year: 2021

Quantitative MR imaging is increasingly favoured for its richer information content and standardised measures. However, computing quantitative parameter maps, such as those encoding longitudinal relaxation rate (R1), apparent transverse relaxation rate (R2*) or magnetisation-transfer saturation (MTsat), involves inverting a highly non-linear function. Many methods for deriving parameter maps assume perfect measurements and do not consider how noise is propagated through the estimation procedure, resulting in needlessly noisy maps. Instead, we propose a probabilistic generative (forward) model of the entire dataset, which is formulated and inverted to jointly recover (log) parameter maps with a well-defined probabilistic interpretation (e.g., maximum likelihood or maximum a posteriori). The second order optimisation we propose for model fitting achieves rapid and stable convergence thanks to a novel approximate Hessian. We demonstrate the utility of our flexible framework in the context of recovering more accurate maps from data acquired using the popular multi-parameter mapping protocol. We also show how to incorporate a joint total variation prior to further decrease the noise in the maps, noting that the probabilistic formulation allows the uncertainty on the recovered parameter maps to be estimated. Our implementation uses a PyTorch backend and benefits from GPU acceleration. It is available at https://github.com/balbasty/nitorch.
D2D-Aided Multi-Antenna Multicasting under Generalized CSIT
Venue: arXiv cs.IT 2021
Year: 2021

Multicasting, where a base station (BS) wishes to convey the same message to several user equipments (UEs), represents a common yet highly challenging wireless scenario. In fact, guaranteeing decodability by the whole UE population proves to be a major performance bottleneck since the UEs in poor channel conditions ultimately determine the achievable rate. To overcome this issue, two-phase cooperative multicasting schemes, which use conventional multicasting in a first phase and leverage device-to-device (D2D) communications in a second phase to effectively spread the message, have been extensively studied. However, most works are limited either to the simple case of single-antenna BS or to a specific channel state information at the transmitter (CSIT) setup. This paper proposes a general two-phase framework that is applicable to the cases of perfect, statistical, and topological CSIT in the presence of multiple antennas at the BS. The proposed method exploits the precoding capabilities at the BS, which enable targeting specific UEs that can effectively serve as D2D relays towards the remaining UEs, and maximize the multicast rate under some outage constraint. Numerical results show that our schemes bring substantial gains over traditional single-phase multicasting and overcome the worst-UE bottleneck behavior in all the considered CSIT configurations.
Ambient Backscatter-Assisted Wireless-Powered Relaying
Venue: arXiv cs.IT 2021
Year: 2021

Internet-of-Things (IoT) is featured with low-power communications among a massive number of ubiquitously-deployed and energy-constrained electronics, e.g., sensors and actuators. To cope with the demand, wireless-powered cooperative relaying emerges as a promising communication paradigm to extend data transmission coverage and solve energy scarcity for the IoT devices. In this paper, we propose a novel hybrid relaying strategy by combining wireless-powered communication and ambient backscattering functions to improve applicability and performance of data transfer. In particular, the hybrid relay can harvest energy from radio frequency (RF) signals and use the energy for active transmission. Alternatively, the hybrid relay can choose to perform ambient backscattering of incident RF signals for passive transmission. To efficiently utilize the ambient RF resource, we design mode selection protocols to coordinate between the active and passive relaying in circumstances with and without instantaneous channel gain. With different mode selection protocols, we characterize the success probability and ergodic capacity of a dual-hop relaying system with the hybrid relay in the field of randomly located ambient transmitters. The analytical and the numerical results demonstrate the effectiveness of the mode selection protocols in adapting the hybrid relaying into the network environment and reveal the impacts of system parameters on the performance gain of the hybrid relaying. As applications of our analytical framework which is computationally tractable, we formulate optimization problems based on the derived expressions to optimize the system parameters with different objectives. The optimal solutions exhibit a tradeoff between the maximum energy efficiency and target success probability.
Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search
Venue: arXiv cs.NE 2021
Year: 2021

In this research work we present CLIP-GLaSS, a novel zero-shot framework to generate an image (or a caption) corresponding to a given caption (or image). CLIP-GLaSS is based on the CLIP neural network, which, given an image and a descriptive caption, provides similar embeddings. Differently, CLIP-GLaSS takes a caption (or an image) as an input, and generates the image (or the caption) whose CLIP embedding is the most similar to the input one. This optimal image (or caption) is produced via a generative network, after an exploration by a genetic algorithm. Promising results are shown, based on the experimentation of the image Generators BigGAN and StyleGAN2, and of the text Generator GPT2
A Novel Use of Discrete Wavelet Transform Features in the Prediction of Epileptic Seizures from EEG Data
Venue: arXiv cs.CE 2021
Year: 2021

This paper demonstrates the predictive superiority of discrete wavelet transform (DWT) over previously used methods of feature extraction in the diagnosis of epileptic seizures from EEG data. Classification accuracy, specificity, and sensitivity are used as evaluation metrics. We specifically show the immense potential of 2 combinations (DWT-db4 combined with SVM and DWT-db2 combined with RF) as compared to others when it comes to diagnosing epileptic seizures either in the balanced or the imbalanced dataset. The results also highlight that MFCC performs less than all the DWT used in this study and that, The mean-differences are statistically significant respectively in the imbalanced and balanced dataset. Finally, either in the balanced or the imbalanced dataset, the feature extraction techniques, the models, and the interaction between them have a statistically significant effect on the classification accuracy.
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Venue: arXiv cs.CL 2021
Year: 2021

We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics. Measuring progress in NLG relies on a constantly evolving ecosystem of automated metrics, datasets, and human evaluation standards. Due to this moving target, new models often still evaluate on divergent anglo-centric corpora with well-established, but flawed, metrics. This disconnect makes it challenging to identify the limitations of current models and opportunities for progress. Addressing this limitation, GEM provides an environment in which models can easily be applied to a wide set of tasks and in which evaluation strategies can be tested. Regular updates to the benchmark will help NLG research become more multilingual and evolve the challenge alongside models. This paper serves as the description of the data for which we are organizing a shared task at our ACL 2021 Workshop and to which we invite the entire NLG community to participate.
Generacion de voces artificiales infantiles en castellano con acento costarricense
Venue: arXiv cs.SD 2021
Year: 2021

This article evaluates a first experience of generating artificial children's voices with a Costa Rican accent, using the technique of statistical parametric speech synthesis based on Hidden Markov Models. The process of recording the voice samples used for learning the models, the fundamentals of the technique used and the subjective evaluation of the results through the perception of a group of people is described. The results show that the intelligibility of the results, evaluated in isolated words, is lower than the voices recorded by the group of participating children. Similarly, the detection of the age and gender of the speaking person is significantly affected in artificial voices, relative to recordings of natural voices. These results show the need to obtain larger amounts of data, in addition to becoming a numerical reference for future developments resulting from new data or from processes to improve results in the same technique.
Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluation
Venue: arXiv cs.CL 2021
Year: 2021

Public datasets are often used to evaluate the efficacy and generalizability of state-of-the-art methods for many tasks in natural language processing (NLP). However, the presence of overlap between the train and test datasets can lead to inflated results, inadvertently evaluating the model's ability to memorize and interpreting it as the ability to generalize. In addition, such data sets may not provide an effective indicator of the performance of these methods in real world scenarios. We identify leakage of training data into test data on several publicly available datasets used to evaluate NLP tasks, including named entity recognition and relation extraction, and study them to assess the impact of that leakage on the model's ability to memorize versus generalize.
Deep CNNs for large scale species classification
Venue: arXiv cs.CV 2021
Year: 2021

Large Scale image classification is a challenging problem within the field of computer vision. As the real world contains billions of different objects, understanding the performance of popular techniques and models is vital in order to apply them to real world tasks. In this paper, we evaluate techniques and popular CNN based deep learning architectures to perform large scale species classification on the dataset from iNaturalist 2019 Challenge. Methods utilizing dataset pruning and transfer learning are shown to outperform models trained without either of the two techniques. The ResNext based classifier outperforms other model architectures over 10 epochs and achieves a top-one validation error of 0.68 when classifying amongst the 1,010 species.
Multi-color balancing for correctly adjusting the intensity of target colors
Venue: arXiv cs.MM 2021
Year: 2021

In this paper, we propose a novel multi-color balance method for reducing color distortions caused by lighting effects. The proposed method allows us to adjust three target-colors chosen by a user in an input image so that each target color is the same as the corresponding destination (benchmark) one. In contrast, white balancing is a typical technique for reducing the color distortions, however, they cannot remove lighting effects on colors other than white. In an experiment, the proposed method is demonstrated to be able to remove lighting effects on selected three colors, and is compared with existing white balance adjustments.
Length Learning for Planar Euclidean Curves
Venue: arXiv cs.GR 2021
Year: 2021

In this work, we used deep neural networks (DNNs) to solve a fundamental problem in differential geometry. One can find many closed-form expressions for calculating curvature, length, and other geometric properties in the literature. As we know these concepts, we are highly motivated to reconstruct them by using deep neural networks. In this framework, our goal is to learn geometric properties from examples. The simplest geometric object is a curve. Therefore, this work focuses on learning the length of planar sampled curves created by a sine waves dataset. For this reason, the fundamental length axioms were reconstructed using a supervised learning approach. Following these axioms a simplified DNN model, we call ArcLengthNet, was established. The robustness to additive noise and discretization errors were tested.
Space Shift Keying with Reconfigurable Intelligent Surfaces: Phase Configuration Designs and Performance Analysis
Venue: arXiv cs.IT 2021
Year: 2021

Reconfigurable intelligent surface (RIS)-assisted transmission and space shift keying (SSK) appear as promising candidates for future energy-efficient wireless systems. In this paper, two RIS-based SSK schemes are proposed to efficiently improve the error and throughput performance of conventional SSK systems, respectively. The first one, termed RIS-SSK with passive beamforming (RIS-SSK-PB), employs an RIS for beamforming and targets the maximization of the minimum squared Euclidean distance between any two decision points. The second one, termed RIS-SSK with Alamouti space-time block coding (RIS-SSK-ASTBC), employs an RIS for ASTBC and enables the RIS to transmit its own Alamouti-coded information while reflecting the incident SSK signals to the destination. A low-complexity beamformer and an efficient maximum-likelihood (ML) detector are designed for RIS-SSK-PB and RIS-SSK-ASTBC, respectively. Approximate expressions for the average bit error probabilities of the source and/or the RIS are derived in closed-form assuming ML detection. Extensive computer simulations are conducted to verify the performance analysis. Results show that RIS-SSK-PB significantly outperforms the existing RIS-free and RIS-based SSK schemes, and RIS-SSK-ASTBC enables highly reliable transmission with throughput improvement.