Miracle-dz/UniFAR-FTUs huggingface dataset:
Multidimensional Index Modulation for 5G and Beyond Wireless Networks

Index modulation (IM) provides a novel way for the transmission of additional data bits via the indices of the available transmit entities compared with classical communication schemes. This study examines the flexible utilization of existing IM techniques in a comprehensive manner to satisfy the challenging and diverse requirements of 5G and beyond services. After spatial modulation (SM), which transmits information bits through antenna indices, application of IM to orthogonal frequency-division multiplexing (OFDM) subcarriers has opened the door for the extension of IM into different dimensions, such as radio frequency (RF) mirrors, time slots, codes, and dispersion matrices. Recent studies have introduced the concept of multidimensional IM by various combinations of 1-D IM techniques to provide higher spectral efficiency (SE) and better bit error rate (BER) performance at the expense of higher transmitter (Tx) and receiver (Rx) complexity. Despite the ongoing research on the design of new IM techniques and their implementation challenges, proper use of the available IM techniques to address different requirements of 5G and beyond networks is an open research area in the literature. For this reason, we first provide the dimensional-based categorization of available IM domains and review the existing IM types regarding this categorization. Then, we develop a framework that investigates the efficient utilization of these techniques and establishes a link between the IM schemes and 5G services, namely, enhanced mobile broadband (eMBB), massive machine-type communications (mMTCs), and ultrareliable low-latency communication (URLLC). In addition, this work defines key performance indicators (KPIs) to quantify the advantages and disadvantages of IM techniques in time, frequency, space, and code dimensions. Finally, future recommendations are given regarding the design of flexible IM-based communication systems for 5G and beyond wireless networks.
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems

Natural language generation (NLG) is an essential component of task-oriented dialog systems. Despite the recent success of neural approaches for NLG, they are typically developed in an offline manner for particular domains. To better fit real-life applications where new data come in a stream, we study NLG in a “continual learning” setting to expand its knowledge to new domains or functionalities incrementally. The major challenge towards this goal is catastrophic forgetting, meaning that a continually trained model tends to forget the knowledge it has learned before. To this end, we propose a method called ARPER (Adaptively Regularized Prioritized Exemplar Replay) by replaying prioritized historical exemplars, together with an adaptive regularization technique based on Elastic Weight Consolidation. Extensive experiments to continually learn new domains and intents are conducted on MultiWoZ-2.0 to benchmark ARPER with a wide range of techniques. Empirical results demonstrate that ARPER significantly outperforms other methods by effectively mitigating the detrimental catastrophic forgetting issue.
SST-BERT at SemEval-2020 Task 1: Semantic Shift Tracing by Clustering in BERT-based Embedding Spaces

Lexical semantic change detection (also known as semantic shift tracing) is a task of identifying words that have changed their meaning over time. Unsupervised semantic shift tracing, focal point of SemEval2020, is particularly challenging. Given the unsupervised setup, in this work, we propose to identify clusters among different occurrences of each target word, considering these as representatives of different word meanings. As such, disagreements in obtained clusters naturally allow to quantify the level of semantic shift per each target word in four target languages. To leverage this idea, clustering is performed on contextualized (BERT-based) embeddings of word occurrences. The obtained results show that our approach performs well both measured separately (per language) and overall, where we surpass all provided SemEval baselines.
Including Everyone, Everywhere: Understanding Opportunities and Challenges of Geographic Gender-Inclusion in OSS

The gender gap is a significant concern facing the software industry as the development becomes more geographically distributed. Widely shared reports indicate that gender differences may be specific to each region. However, how complete can these reports be with little to no research reflective of the Open Source Software (OSS) process and communities software is now commonly developed in? Our study presents a multi-region geographical analysis of gender inclusion on GitHub. This mixed-methods approach includes quantitatively investigating differences in gender inclusion in projects across geographic regions and investigate these trends over time using data from contributions to 21,456 project repositories. We also qualitatively understand the unique experiences of developers contributing to these projects through a survey that is strategically targeted to developers in various regions worldwide. Our findings indicate that gender diversity is low across all parts of the world, with no substantial difference across regions. However, there has been statistically significant improvement in diversity worldwide since 2014, with certain regions such as Africa improving at faster pace. We also find that most motivations and barriers to contributions (e.g., lack of resources to contribute and poor working environment) were shared across regions, however, some insightful differences, such as how to make projects more inclusive, did arise. From these findings, we derive and present implications for tools that can foster inclusion in open source software communities and empower contributions from everyone, everywhere.
Spreading height and critical conditions for the collapse of turbulent fountains in stratified media

Axisymmetric fountains in stratified environments rise until reaching a maximum height, where the vertical momentum vanishes, and then falls and spread radially as an annular plume following a well-known top-hat profile. Here, firstly, we generalize the model of Morton et al. (Proc. R. Soc. Lond. A \textbf{234}, 1, 1956), in order to correctly determine the dependence of the maximum height and the spreading height with the parameters involved. We obtain the critical conditions for the collapse of the fountain, \textit i.e. when the jet falls up to the source level, and show that the spreading height must be expressed as a function of at least two parameters. To improve the quantitative agreement with the experiments we modify the criterion to take the mixing process in the down flow into account. Numerical simulations were implemented to estimate the parameter values that characterizes this merging. We show that our generalized model agrees very well with the experimental measurements.
CARLA: A Convolution Accelerator With a Reconfigurable and Low-Energy Architecture

Convolutional Neural Networks (CNNs) have proven to be extremely accurate for image recognition, even outperforming human recognition capability. When deployed on battery–powered mobile devices, efficient computer architectures are required to enable fast and energy-efficient computation of costly convolution operations. Despite recent advances in hardware accelerator design for CNNs, two major problems have not yet been addressed effectively, particularly when the convolution layers have highly diverse structures: (1) minimizing energy-hungry off-chip DRAM data movements; (2) maximizing the utilization factor of processing resources to perform convolutions. This work thus proposes an energy-efficient architecture equipped with several optimized dataflows to support the structural diversity of modern CNNs. The proposed approach is evaluated on convolutional layers of VGGNet-16 and ResNet-50. Results show that the architecture achieves a Processing Element (PE) utilization factor of 98% for the majority of $3\times 3$ and $1\times 1$ convolutional layers, while limiting latency to 396.9 ms and 92.7 ms when performing convolutional layers of VGGNet-16 and ResNet-50, respectively. In addition, the proposed architecture benefits from the structured sparsity in ResNet-50 to reduce the latency to 42.5 ms when half of the channels are pruned.
Bridging the gaps in statistical models of protein alignment

Abstract Summary Sequences of proteins evolve by accumulating substitutions together with insertions and deletions (indels) of amino acids. However, it remains a common practice to disconnect substitutions and indels, and infer approximate models for each of them separately, to quantify sequence relationships. Although this approach brings with it computational convenience (which remains its primary motivation), there is a dearth of attempts to unify and model them systematically and together. To overcome this gap, this article demonstrates how a complete statistical model quantifying the evolution of pairs of aligned proteins can be constructed using a time-parameterized substitution matrix and a time-parameterized alignment state machine. Methods to derive all parameters of such a model from any benchmark collection of aligned protein sequences are described here. This has not only allowed us to generate a unified statistical model for each of the nine widely used substitution matrices (PAM, JTT, BLOSUM, JO, WAG, VTML, LG, MIQS and PFASUM), but also resulted in a new unified model, MMLSUM. Our underlying methodology measures the Shannon information content using each model to explain losslessly any given collection of alignments, which has allowed us to quantify the performance of all the above models on six comprehensive alignment benchmarks. Our results show that MMLSUM results in a new and clear overall best performance, followed by PFASUM, VTML, BLOSUM and MIQS, respectively, amongst the top five. We further analyze the statistical properties of MMLSUM model and contrast it with others. Supplementary information Supplementary data are available at Bioinformatics online.
Splay trees on trees

Search trees on trees (STTs) are a far-reaching generalization of binary search trees (BSTs), allowing the efficient exploration of tree-structured domains. (BSTs are the special case in which the underlying domain is a path.) Trees on trees have been extensively studied under various guises in computer science and discrete mathematics. Recently Bose, Cardinal, Iacono, Koumoutsos, and Langerman (SODA 2020) considered adaptive STTs and observed that, apart from notable exceptions, the machinery developed for BSTs in the past decades does not readily transfer to STTs. In particular, they asked whether the optimal STT can be efficiently computed or approximated (by analogy to Knuth's algorithm for optimal BSTs), and whether natural self-adjusting BSTs such as Splay trees (Sleator, Tarjan, 1983) can be extended to this more general setting. We answer both questions affirmatively. First, we show that a $(1 + \frac{1}{t})$-approximation of an optimal size-$n$ STT for a given search distribution can be computed in time $O(n^{2t + 1})$ for all integers $t \geq 1$. Second, we identify a broad family of STTs with linear rotation-distance, allowing the generalization of Splay trees to the STT setting. We show that our generalized Splay satisfies a static optimality theorem, asymptotically matching the cost of the optimal STT in an online fashion, i.e. without knowledge of the search distribution. Our results suggest an extension of the dynamic optimality conjecture for Splay trees to the broader setting of trees on trees.
Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning

Meta-learning is a powerful tool for learning policies that can adapt efficiently when deployed in new tasks. If however the meta-training tasks have sparse rewards, the need for exploration during meta-training is exacerbated given that the agent has to explore and learn across many tasks. We show that current meta-learning methods can fail catastrophically in such environments. To address this problem, we propose HyperX, a novel method for meta-learning in sparse reward tasks. Using novel reward bonuses for meta-training, we incentivise the agent to explore in approximate hyper-state space, i.e., the joint state and approximate belief space, where the beliefs are over tasks. We show empirically that these bonuses allow an agent to successfully learn to solve sparse reward tasks where existing meta-learning methods fail.
TeRo: A Time-aware Knowledge Graph Embedding via Temporal Rotation

In the last few years, there has been a surge of interest in learning representations of entities and relations in knowledge graph (KG). However, the recent availability of temporal knowledge graphs (TKGs) that contain time information for each fact created the need for reasoning over time in such TKGs. In this regard, we present a new approach of TKG embedding, TeRo, which defines the temporal evolution of entity embedding as a rotation from the initial time to the current time in the complex vector space. Specially, for facts involving time intervals, each relation is represented as a pair of dual complex embeddings to handle the beginning and the end of the relation, respectively. We show our proposed model overcomes the limitations of the existing KG embedding models and TKG embedding models and has the ability of learning and inferring various relation patterns over time. Experimental results on three different TKGs show that TeRo significantly outperforms existing state-of-the-art models for link prediction. In addition, we analyze the effect of time granularity on link prediction over TKGs, which as far as we know has not been investigated in previous literature.
Numerical Methods to Compute the Coriolis Matrix and Christoffel Symbols for Rigid-Body Systems

This article presents methods to efficiently compute the Coriolis matrix and underlying Christoffel symbols (of the first kind) for tree-structure rigid-body systems. The algorithms can be executed purely numerically, without requiring partial derivatives as in unscalable symbolic techniques. The computations share a recursive structure in common with classical methods such as the composite-rigid-body algorithm and are of the lowest possible order: O(Nd) for the Coriolis matrix and O(Nd2) for the Christoffel symbols, where N is the number of bodies and d is the depth of the kinematic tree. Implementation in C/C++ shows computation times of the order of 10–20 μs for the Coriolis matrix and 40–120 μs for the Christoffel symbols on systems with 20-degrees-of-freedom (DoF). The results demonstrate feasibility for the adoption of these algorithms within high-rate (>1 kHz) loops for model-based control applications.
Neural Thompson Sampling

Thompson Sampling (TS) is one of the most effective algorithms for solving contextual multi-armed bandit problems. In this paper, we propose a new algorithm, called Neural Thompson Sampling, which adapts deep neural networks for both exploration and exploitation. At the core of our algorithm is a novel posterior distribution of the reward, where its mean is the neural network approximator, and its variance is built upon the neural tangent features of the corresponding neural network. We prove that, provided the underlying reward function is bounded, the proposed algorithm is guaranteed to achieve a cumulative regret of $\mathcal{O}(T^{1/2})$, which matches the regret of other contextual bandit algorithms in terms of total round number $T$. Experimental comparisons with other benchmark bandit algorithms on various data sets corroborate our theory.
The CAMELS Project: Cosmology and Astrophysics with Machine-learning Simulations

We present the Cosmology and Astrophysics with MachinE Learning Simulations (CAMELS) project. CAMELS is a suite of 4233 cosmological simulations of 25h−1Mpc3 volume each: 2184 state-of-the-art (magneto)hydrodynamic simulations run with the AREPO and GIZMO codes, employing the same baryonic subgrid physics as the IllustrisTNG and SIMBA simulations, and 2049 N-body simulations. The goal of the CAMELS project is to provide theory predictions for different observables as a function of cosmology and astrophysics, and it is the largest suite of cosmological (magneto)hydrodynamic simulations designed to train machine-learning algorithms. CAMELS contains thousands of different cosmological and astrophysical models by way of varying Ω m , σ 8, and four parameters controlling stellar and active galactic nucleus feedback, following the evolution of more than 100 billion particles and fluid elements over a combined volume of (400h−1Mpc)3 . We describe the simulations in detail and characterize the large range of conditions represented in terms of the matter power spectrum, cosmic star formation rate density, galaxy stellar mass function, halo baryon fractions, and several galaxy scaling relations. We show that the IllustrisTNG and SIMBA suites produce roughly similar distributions of galaxy properties over the full parameter space but significantly different halo baryon fractions and baryonic effects on the matter power spectrum. This emphasizes the need for marginalizing over baryonic effects to extract the maximum amount of information from cosmological surveys. We illustrate the unique potential of CAMELS using several machine-learning applications, including nonlinear interpolation, parameter estimation, symbolic regression, data generation with Generative Adversarial Networks, dimensionality reduction, and anomaly detection.
Determinação do trajeto do canal mandibular por meio de tomografia computadorizada em dez mandíbulas de cadáveres de cães mesaticefálicos

Sabe-se que durante qualquer procedimento cirurgico na regiao da mandibula no homem, um dano iatrogenico ao feixe vasculonervoso que percorre o canal mandibular (CM) poderia causar desde parestesia ate dor constante. Na odontologia veterinaria, diferentes procedimentos cirurgicos sao realizados no tecido osseo adjacente ao CM, o que implica no conhecimento acurado da localizacao do mesmo. O objetivo desta pesquisa foi determinar por meio da Tomografia Computadorizada (TC) o trajeto do CM em relacao as faces da mandibula: lingual, vestibular, ventral e crista alveolar em dez mandibulas de cadaveres de caes mesaticefalicos. Os tomogramas foram realizados no plano transversal, tomando como referencia para cada corte o forame mandibular, as raizes dentarias dos dentes molares, pre-molares e forame mentoniano medio. No tomografo foram realizadas medidas desde o CM ate as diferentes faces. Conclui-se que a partir do dente 3o molar, no sentido rostral, o CM aumenta gradualmente a distância em relacao a crista alveolar, alcancando a sua maxima profundidade na regiao dente 1o molar. Em relacao as faces vestibular e lingual, o CM apresentou-se lingualizado na regiao dos dentes molares. O CM continua rostralmente ocupando a regiao ventral do corpo da mandibula, mantendo uma distância similar entre a superficie vestibular e lingual. Ja na regiao do dente 3o pre-molar o canal mandibular aumenta ligeiramente a sua distância com relacao a face ventral e lingual da mandibula, antes do seu fim no forame mentoniano medial na face vestibular da mandibula.
Coded Stochastic ADMM for Decentralized Consensus Optimization With Edge Computing

Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones, and vehicles. Due to the limitations of communication costs and security requirements, it is of paramount importance to analyze information in a decentralized manner instead of aggregating data to a fusion center. To train large-scale machine learning models, edge/fog computing is often leveraged as an alternative to centralized learning. We consider the problem of learning model parameters in a multiagent system with data locally processed via distributed edge nodes. A class of minibatch stochastic alternating direction method of multipliers (ADMMs) algorithms is explored to develop the distributed learning model. To address two main critical challenges in distributed learning systems, i.e., communication bottleneck and straggler nodes (nodes with slow responses), error-control-coding-based stochastic incremental ADMM is investigated. Given an appropriate minibatch size, we show that the minibatch stochastic ADMM-based method converges in a rate of $O(1/\sqrt {k})$ , where $k$ denotes the number of iterations. Through numerical experiments, it is revealed that the proposed algorithm is communication efficient, rapidly responding, and robust in the presence of straggler nodes compared with state-of-the-art algorithms.
Energy-flux vector in anisotropic turbulence: application to rotating turbulence

Abstract Energy flux plays a key role in analyses of energy-cascading turbulence. In isotropic turbulence, the flux is given by a scalar as a function of the magnitude of the wavenumber. On the other hand, the flux in anisotropic turbulence should be a geometric vector that has a direction as well as a magnitude, and depends not only on the magnitude of the wavenumber but also on its direction. The energy-flux vector in anisotropic turbulence cannot be uniquely determined in a way used for the isotropic flux. In this work, introducing two ansatzes, net locality and efficiency of nonlinear energy transfer, we propose a way to determine the energy-flux vector in anisotropic turbulence using the Moore–Penrose inverse. The energy-flux vector in strongly rotating turbulence is demonstrated based on the energy transfer rate obtained by direct numerical simulations. It is found that the direction of the energy-flux vector is consistent with the prediction of the weak turbulence theory in the wavenumber range dominated by inertial waves. However, the energy flux along the critical wavenumbers predicted by the critical balance in the buffer range between the weak turbulence range and the isotropic Kolmogorov turbulence range is not observed in the present simulations. This discrepancy between the critical balance and the present numerical results is discussed and dissipation is found to play an important role in the energy flux in the buffer range.
Beyond The Text: Analysis of Privacy Statements through Syntactic and Semantic Role Labeling

This paper formulates a new task of extracting privacy parameters from a privacy policy, through the lens of Contextual Integrity (CI), an established social theory framework for reasoning about privacy norms. Through extensive experiments, we further show that incorporating CI-based domain-specific knowledge into a BERT-based SRL model results in the highest precision and recall, achieving an F1 score of 84%. With our work, we would like to motivate new research in building NLP applications for the privacy domain.