Miracle-dz/UniFAR-FTUs huggingface dataset:
Estimating Multiplicative Relations in Neural Networks

Universal approximation theorem suggests that a shallow neural network can approximate any function. The input to neurons at each layer is a weighted sum of previous layer neurons and then an activation is applied. These activation functions perform very well when the output is a linear combination of input data. When trying to learn a function which involves product of input data, the neural networks tend to overfit the data to approximate the function. In this paper we will use properties of logarithmic functions to propose a pair of activation functions which can translate products into linear expression and learn using backpropagation. We will try to generalize this approach for some complex arithmetic functions and test the accuracy on a disjoint distribution with the training set.
GWTC-2: Compact Binary Coalescences Observed by LIGO and Virgo during the First Half of the Third Observing Run

We report on gravitational-wave discoveries from compact binary coalescences detected by Advanced LIGO and Advanced Virgo in the first half of the third observing run (O3a) between 1 April 2019 15∶00 UTC and 1 October 2019 15∶00 UTC. By imposing a false-alarm-rate threshold of two per year in each of the four search pipelines that constitute our search, we present 39 candidate gravitational-wave events. At this threshold, we expect a contamination fraction of less than 10%. Of these, 26 candidate events were reported previously in near-real time through gamma-ray coordinates network notices and circulars; 13 are reported here for the first time. The catalog contains events whose sources are black hole binary mergers up to a redshift of approximately 0.8, as well as events whose components cannot be unambiguously identified as black holes or neutron stars. For the latter group, we are unable to determine the nature based on estimates of the component masses and spins from gravitational-wave data alone. The range of candidate event masses which are unambiguously identified as binary black holes (both objects ≥ 3 M⊙) is increased compared to GWTC-1, with total masses from approximately 14 M⊙ for GW190924_021846 to approximately 150 M⊙ for GW190521. For the first time, this catalog includes binary systems with significantly asymmetric mass ratios, which had not been observed in data taken before April 2019. We also find that 11 of the 39 events detected since April 2019 have positive effective inspiral spins under our default prior (at 90% credibility), while none exhibit negative effective inspiral spin. Given the increased sensitivity of Advanced LIGO and Advanced Virgo, the detection of 39 candidate events in approximately 26 weeks of data (approximately 1.5 per week) is consistent with GWTC-1.
Mirror symmetry and mixed Chern-Simons levels for Abelian 3D 
N=2
 theories

In 3d $\N=2$ gauge theories, mirror symmetry as Fourier transformation of sphere partition functions can be used to check and derive dualities. In this notes, we study Abelian 3d $N=2$ theories with mixed Chern-Simons levels, in particular $(U(1)-[1])^{N}_{k_{ij}}$, which is $N$ copies of $U(1)-[1]$ coupled together by mixed Chern-Simons levels $k_{ij}$. It turns out that mirror symmetry gives rise to many mirror dual theories with different mixed CS levels and FI parameters, and mirror transformations form a finite group with nice mathematical structure. We also discuss the mirror transformations of $U(1)_k- [N] $ theories with brane constructions. The results obtained match with vortex partition functions calculated from open topological strings.
Explore-Before-Talk: Multichannel Selection Diversity for Uplink Transmissions in Machine-Type Communication

Improving the data rate of machine-type communication (MTC) is essential in supporting emerging Internet of things (IoT) applications ranging from real-time surveillance to edge machine learning. To this end, in this paper we propose a resource allocation approach for uplink transmissions within a random access procedure in MTC by exploiting multichannel selection diversity, coined explore-before-talk (EBT). Each user in EBT first sends pilot signals through multiple channels that are initially allocated by a base station (BS) for exploration, and then the BS informs a subset of initially allocated channels that are associated with high signal-to-noise ratios (SNRs) for data packet transmission by the user while releasing the rest of the channels for other users. Consequently, EBT exploits a multichannel selection diversity gain during data packet transmission, at the cost of exploration during pilot transmission. We optimize this exploration-exploitation trade-off, by deriving closed-form mean data rate and resource outage probability expressions. Numerical results corroborate that EBT achieves a higher mean data rate while satisfying the same outage constraint, compared to a conventional MTC protocol without exploration.
Learning Contextualised Cross-lingual Word Embeddings and Alignments for Extremely Low-Resource Languages Using Parallel Corpora

We propose a new approach for learning contextualised cross-lingual word embeddings based on a small parallel corpus (e.g. a few hundred sentence pairs). Our method obtains word embeddings via an LSTM encoder-decoder model that simultaneously translates and reconstructs an input sentence. Through sharing model parameters among different languages, our model jointly trains the word embeddings in a common cross-lingual space. We also propose to combine word and subword embeddings to make use of orthographic similarities across different languages. We base our experiments on real-world data from endangered languages, namely Yongning Na, Shipibo-Konibo, and Griko. Our experiments on bilingual lexicon induction and word alignment tasks show that our model outperforms existing methods by a large margin for most language pairs. These results demonstrate that, contrary to common belief, an encoder-decoder translation model is beneficial for learning cross-lingual representations even in extremely low-resource conditions. Furthermore, our model also works well on high-resource conditions, achieving state-of-the-art performance on a German-English word-alignment task.
EC-SVC: Secure CAN Bus In-Vehicle Communications With Fine-Grained Access Control Based on Edge Computing

In-vehicle communications are not designed for message exchange between the vehicles and outside systems originally. Thus, the security design of message protection is insufficient. Moreover, the internal devices do not have enough resources to process the additional security operations. Nonetheless, due to the characteristic of the in-vehicle network in which messages are broadcast, secure message transmission to specific receivers must be ensured. With consideration of the facts aforementioned, this work addresses resource problems by offloading secure operations to high-performance devices, and uses attribute-based access control to ensure the confidentiality of messages from attackers and unauthorized users. In addition, we reconfigure existing access control based cryptography to address new vulnerabilities arising from the use of edge computing and attribute-based access control. Thus, this paper proposes an edge computing-based security protocol with fine-grained attribute-based encryption using a hash function, symmetric-based cryptography, and reconfigured cryptographic scheme. In addition, this work formally proves the reconfigured cryptographic scheme and security protocol, and evaluates the feasibility of the proposed security protocol in various aspects using the CANoe software.
Sparse Symmetric Tensor Regression for Functional Connectivity Analysis

Tensor regression models, such as CP regression and Tucker regression, have many successful applications in neuroimaging analysis where the covariates are of ultrahigh dimensionality and possess complex spatial structures. The high-dimensional covariate arrays, also known as tensors, can be approximated by low-rank structures and fit into the generalized linear models. The resulting tensor regression achieves a significant reduction in dimensionality while remaining efficient in estimation and prediction. Brain functional connectivity is an essential measure of brain activity and has shown significant association with neurological disorders such as Alzheimer's disease. The symmetry nature of functional connectivity is a property that has not been explored in previous tensor regression models. In this work, we propose a sparse symmetric tensor regression that further reduces the number of free parameters and achieves superior performance over symmetrized and ordinary CP regression, under a variety of simulation settings. We apply the proposed method to a study of Alzheimer's disease (AD) and normal ageing from the Berkeley Aging Cohort Study (BACS) and detect two regions of interest that have been identified important to AD.
Transferable Universal Adversarial Perturbations Using Generative Models

Deep neural networks tend to be vulnerable to adversarial perturbations, which by adding to a natural image can fool a respective model with high confidence. Recently, the existence of image-agnostic perturbations, also known as universal adversarial perturbations (UAPs), were discovered. However, existing UAPs still lack a sufficiently high fooling rate, when being applied to an unknown target model. In this paper, we propose a novel deep learning technique for generating more transferable UAPs. We utilize a perturbation generator and some given pretrained networks so-called source models to generate UAPs using the ImageNet dataset. Due to the similar feature representation of various model architectures in the first layer, we propose a loss formulation that focuses on the adversarial energy only in the respective first layer of the source models. This supports the transferability of our generated UAPs to any other target model. We further empirically analyze our generated UAPs and demonstrate that these perturbations generalize very well towards different target models. Surpassing the current state of the art in both, fooling rate and model-transferability, we can show the superiority of our proposed approach. Using our generated non-targeted UAPs, we obtain an average fooling rate of 93.36% on the source models (state of the art: 82.16%). Generating our UAPs on the deep ResNet-152, we obtain about a 12% absolute fooling rate advantage vs. cutting-edge methods on VGG-16 and VGG-19 target models.
Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport

This paper presents a light-weight, high-quality texture synthesis algorithm that easily generalizes to other applications such as style transfer and texture mixing. We represent texture features through the deep neural activation vectors within the bottleneck layer of an auto-encoder and frame the texture synthesis problem as optimal transport between the activation values of the image being synthesized and those of an exemplar texture. To find this optimal transport mapping, we utilize an N-dimensional probability density function (PDF) transfer process that iterates over multiple random rotations of the PDF basis and matches the 1D marginal distributions across each dimension. This achieves quality and flexibility on par with expensive back-propagation based neural texture synthesis methods, but with the potential of achieving interactive rates. We demonstrate that first order statistics offer a more robust representation for texture than the second order statistics that are used today. We propose an extension of this algorithm that reduces the dimensionality of the neural feature space. We utilize a multi-scale coarse-to-fine synthesis pyramid to capture and preserve larger image features; unify color and style transfer under one framework; and further augment this system with a novel masking scheme that re-samples and re-weights the feature distribution for user-guided texture painting and targeted style transfer.
TopicModel4J: A Java Package for Topic Models

Topic models provide a flexible and principled framework for exploring hidden structure in high-dimensional co-occurrence data and are commonly used natural language processing (NLP) of text. In this paper, we design and implement a Java package, TopicModel4J, which contains 13 kinds of representative algorithms for fitting topic models. The TopicModel4J in the Java programming environment provides an easy-to-use interface for data analysts to run the algorithms, and allow to easily input and output data. In addition, this package provides a few unstructured text preprocessing techniques, such as splitting textual data into words, lowercasing the words, preforming lemmatization and removing the useless characters, URLs and stop words.
No Perfect Outdoors: Towards A Deep Profiling of GNSS-based Location Contexts

While both outdoor and indoor localization methods are flourishing, how to properly marry them to offer pervasive localizability in urban areas remains open. Recently proposals on indoor-outdoor detection make the first step towards such an integration, yet complicated urban environments render such a binary classification incompetent. In this paper, we intend to fully explore raw GNSS measurements in order to better characterize the diversified urban environments. Essentially, we tackle the challenges introduced by the complex GNSS data and apply a deep learning model to identify representations for respective location contexts. We further develop two preliminary applications of our deep profiling. On one hand, we offer a more fine-grained semantic classification than binary indoor-outdoor detection. On the other hand, we derive a GPS error indicator more meaningful than that provided by Google Maps. These results are all corroborated by our extensive data collection and trace-driven evaluations.
Dissolution of donor-vacancy clusters in heavily doped n-type germanium

The n-type doping of Ge is a self-limiting process due to the formation of vacancy-donor complexes (D n V with n ⩽ 4) that deactivate the donors. This work unambiguously demonstrates that the dissolution of the dominating P4V clusters in heavily phosphorus-doped Ge epilayers can be achieved by millisecond-flash lamp annealing at about 1050 K. The P4V cluster dissolution increases the carrier concentration by more than three-fold together with a suppression of phosphorus diffusion. Electrochemical capacitance–voltage measurements in conjunction with secondary ion mass spectrometry, positron annihilation lifetime spectroscopy and theoretical calculations enabled us to address and understand a fundamental problem that has hindered so far the full integration of Ge with complementary-metal-oxide-semiconductor technology.
Displacement-Invariant Matching Cost Learning for Accurate Optical Flow Estimation

Learning matching costs has been shown to be critical to the success of the state-of-the-art deep stereo matching methods, in which 3D convolutions are applied on a 4D feature volume to learn a 3D cost volume. However, this mechanism has never been employed for the optical flow task. This is mainly due to the significantly increased search dimension in the case of optical flow computation, ie, a straightforward extension would require dense 4D convolutions in order to process a 5D feature volume, which is computationally prohibitive. This paper proposes a novel solution that is able to bypass the requirement of building a 5D feature volume while still allowing the network to learn suitable matching costs from data. Our key innovation is to decouple the connection between 2D displacements and learn the matching costs at each 2D displacement hypothesis independently, ie, displacement-invariant cost learning. Specifically, we apply the same 2D convolution-based matching net independently on each 2D displacement hypothesis to learn a 4D cost volume. Moreover, we propose a displacement-aware projection layer to scale the learned cost volume, which reconsiders the correlation between different displacement candidates and mitigates the multi-modal problem in the learned cost volume. The cost volume is then projected to optical flow estimation through a 2D soft-argmin layer. Extensive experiments show that our approach achieves state-of-the-art accuracy on various datasets, and outperforms all published optical flow methods on the Sintel benchmark.
ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications

To train deep learning models for vision-based action recognition of elders’ daily activities, we need large-scale activity datasets acquired under various daily living environments and conditions. However, most public datasets used in human action recognition either differ from or have limited coverage of elders’ activities in many aspects, making it challenging to recognize elders’ daily activities well by only utilizing existing datasets. Recently, such limitations of available datasets have actively been compensated by generating synthetic data from realistic simulation environments and using those data to train deep learning models. In this paper, based on these ideas we develop ElderSim, an action simulation platform that can generate synthetic data on elders’ daily activities. For 55 kinds of frequent daily activities of the elders, ElderSim generates realistic motions of synthetic characters with various adjustable data-generating options and provides different output modalities including RGB videos, two- and three-dimensional skeleton trajectories. We then generate KIST SynADL, a large-scale synthetic dataset of elders’ activities of daily living, from ElderSim and use the data in addition to real datasets to train three state-of-the-art human action recognition models. From the experiments following several newly proposed scenarios that assume different real and synthetic dataset configurations for training, we observe a noticeable performance improvement by augmenting our synthetic data. We also offer guidance with insights for the effective utilization of synthetic data to help recognize elders’ daily activities.
ORBBuf: A Robust Buffering Method for Collaborative Visual SLAM

Collaborative simultaneous localization and mapping (SLAM) approaches provide a solution for autonomous robots based on embedded devices. On the other hand, visual SLAM systems rely on correlations between visual frames. As a result, the loss of visual frames from an unreliable wireless network can easily damage the results of collaborative visual SLAM systems. From our experiment, a loss of less than 1 second of data can lead to the failure of visual SLAM algorithms. We present a novel buffering method, ORBBuf, to reduce the impact of data loss on collaborative visual SLAM systems. We model the buffering problem into an optimization problem. We use an efficient greedy-like algorithm, and our buffering method drops the frame that results in the least loss to the quality of the SLAM results. We implement our ORBBuf method on ROS, a widely used middleware framework. Through an extensive evaluation on real-world scenarios and tens of gigabytes of datasets, we demonstrate that our ORBBuf method can be applied to different algorithms, different sensor data (both monocular images and stereo images), different scenes (both indoor and outdoor), and different network environments (both WiFi networks and 4G networks). Experimental results show that the network interruptions indeed affect the SLAM results, and our ORBBuf method can reduce the RMSE up to 50 times.
Fixed-Length Protein Embeddings using Contextual Lenses

The Basic Local Alignment Search Tool (BLAST) is currently the most popular method for searching databases of biological sequences. BLAST compares sequences via similarity defined by a weighted edit distance, which results in it being computationally expensive. As opposed to working with edit distance, a vector similarity approach can be accelerated substantially using modern hardware or hashing techniques. Such an approach would require fixed-length embeddings for biological sequences. There has been recent interest in learning fixed-length protein embeddings using deep learning models under the hypothesis that the hidden layers of supervised or semi-supervised models could produce potentially useful vector embeddings. We consider transformer (BERT) protein language models that are pretrained on the TrEMBL data set and learn fixed-length embeddings on top of them with contextual lenses. The embeddings are trained to predict the family a protein belongs to for sequences in the Pfam database. We show that for nearest-neighbor family classification, pretraining offers a noticeable boost in performance and that the corresponding learned embeddings are competitive with BLAST. Furthermore, we show that the raw transformer embeddings, obtained via static pooling, do not perform well on nearest-neighbor family classification, which suggests that learning embeddings in a supervised manner via contextual lenses may be a compute-efficient alternative to fine-tuning.
Geometric Fabrics for the Acceleration-based Design of Robotic Motion

This paper describes the pragmatic design and construction of geometric fabrics for shaping a robot's task-independent nominal behavior, capturing behavioral components such as obstacle avoidance, joint limit avoidance, redundancy resolution, global navigation heuristics, etc. Geometric fabrics constitute the most concrete incarnation of a new mathematical formulation for reactive behavior called optimization fabrics. Fabrics generalize recent work on Riemannian Motion Policies (RMPs); they add provable stability guarantees and improve design consistency while carrying forward the intuitive acceleration-based principles of modular design that make RMPs successful. We describe a suite of mathematical modeling tools that practitioners can employ in practice and demonstrate both how to mitigate system complexity by constructing behaviors layer-wise and how to employ these tools to design robust strongly-generalizing policies that solve practical problems one would expect to find in industry applications. Our experimental system exhibits intelligent global navigation behaviors expressed entirely as fabrics with zero planning or state machine governance.
Matrix Product Density Operators: when do they have a local parent Hamiltonian?.

We study whether one can write a Matrix Product Density Operator (MPDO) as the Gibbs state of a quasi-local parent Hamiltonian. We conjecture this is the case for generic MPDOs and give evidences to support it. To investigate the locality of the parent Hamiltonian, we take the approach of checking whether the quantum conditional mutual information decays exponentially. The MPDOs we consider are constructed from a chain of 1-input/2-output (`Y-shaped') completely-positive channels, guaranteeing the MPDO has a local purification. We derive an upper bound on the conditional mutual information for bistochastic channels and strictly positive channels, and show that it decays exponentially if the correctable algebra of the channel is trivial. We also introduce a conjecture on a quantum data processing inequality that implies the exponential decay of the conditional mutual information for every Y-shaped channel with trivial correctable algebra. We additionally investigate a close but nonequivalent cousin: MPDOs measured in a local basis. We provide sufficient conditions for the exponential decay of the conditional mutual information of the measured states, and numerically confirmed they are generically true for certain random MPDO.