Miracle-dz/UniFAR-FTUs huggingface dataset:
Hierarchical, rotation‐equivariant neural networks to select structural models of protein complexes

Predicting the structure of multi‐protein complexes is a grand challenge in biochemistry, with major implications for basic science and drug discovery. Computational structure prediction methods generally leverage predefined structural features to distinguish accurate structural models from less accurate ones. This raises the question of whether it is possible to learn characteristics of accurate models directly from atomic coordinates of protein complexes, with no prior assumptions. Here we introduce a machine learning method that learns directly from the 3D positions of all atoms to identify accurate models of protein complexes, without using any precomputed physics‐inspired or statistical terms. Our neural network architecture combines multiple ingredients that together enable end‐to‐end learning from molecular structures containing tens of thousands of atoms: a point‐based representation of atoms, equivariance with respect to rotation and translation, local convolutions, and hierarchical subsampling operations. When used in combination with previously developed scoring functions, our network substantially improves the identification of accurate structural models among a large set of possible models. Our network can also be used to predict the accuracy of a given structural model in absolute terms. The architecture we present is readily applicable to other tasks involving learning on 3D structures of large atomic systems.
Quantization of Acoustic Model Parameters in Automatic Speech Recognition Framework

State-of-the-art hybrid automatic speech recognition (ASR) system exploits deep neural network (DNN) based acoustic models (AM) trained with Lattice Free-Maximum Mutual Information (LF-MMI) criterion and n-gram language models. The AMs typically have millions of parameters and require significant parameter reduction to operate on embedded devices. The impact of parameter quantization on the overall word recognition performance is studied in this paper. Following approaches are presented: (i) AM trained in Kaldi framework with conventional factorized TDNN (TDNN-F) architecture, (ii) the TDNN AM built in Kaldi loaded into the PyTorch toolkit using a C++ wrapper for post-training quantization, (iii) quantization-aware training in PyTorch for Kaldi TDNN model, (iv) quantization-aware training in Kaldi. Results obtained on standard Librispeech setup provide an interesting overview of recognition accuracy w.r. t. applied quantization scheme.
EMU: Efficient Muscle Simulation in Deformation Space

EMU is an efficient and scalable model to simulate bulk musculoskeletal motion with heterogenous materials. First, EMU requires no model reductions, or geometric coarsening, thereby producing results visually accurate when compared to an FEM simulation. Second, EMU is efficient and scales much better than state‐of‐the‐art FEM with the number of elements in the mesh, and is more easily parallelizable. Third, EMU can handle heterogeneously stiff meshes with an arbitrary constitutive model, thus allowing it to simulate soft muscles, stiff tendons and even stiffer bones all within one unified system. These three key characteristics of EMU enable us to efficiently orchestrate muscle activated skeletal movements. We demonstrate the efficacy of our approach via a number of examples with tendons, muscles, bones and joints.
Deep Multimodal Transfer-Learned Regression in Data-Poor Domains

In many real-world applications of deep learning, estimation of a target may rely on various types of input data modes, such as audio-video, image-text, etc. This task can be further complicated by a lack of sufficient data. Here we propose a Deep Multimodal Transfer-Learned Regressor (DMTL-R) for multimodal learning of image and feature data in a deep regression architecture effective at predicting target parameters in data-poor domains. Our model is capable of fine-tuning a given set of pre-trained CNN weights on a small amount of training image data, while simultaneously conditioning on feature information from a complimentary data mode during network training, yielding more accurate single-target or multi-target regression than can be achieved using the images or the features alone. We present results using phase-field simulation microstructure images with an accompanying set of physical features, using pre-trained weights from various well-known CNN architectures, which demonstrate the efficacy of the proposed multimodal approach.
Projection-Free Optimization on Uniformly Convex Sets

The Frank-Wolfe method solves smooth constrained convex optimization problems at a generic sublinear rate of $\mathcal{O}(1/T)$, and it (or its variants) enjoys accelerated convergence rates for two fundamental classes of constraints: polytopes and strongly-convex sets. Uniformly convex sets non-trivially subsume strongly convex sets and form a large variety of \textit{curved} convex sets commonly encountered in machine learning and signal processing. For instance, the $\ell_p$-balls are uniformly convex for all $p > 1$, but strongly convex for $p\in]1,2]$ only. We show that these sets systematically induce accelerated convergence rates for the original Frank-Wolfe algorithm, which continuously interpolate between known rates. Our accelerated convergence rates emphasize that it is the curvature of the constraint sets -- not just their strong convexity -- that leads to accelerated convergence rates. These results also importantly highlight that the Frank-Wolfe algorithm is adaptive to much more generic constraint set structures, thus explaining faster empirical convergence. Finally, we also show accelerated convergence rates when the set is only locally uniformly convex and provide similar results in online linear optimization.
Improving the Performance of Deep Learning for Wireless Localization

Indoor localization systems are most commonly based on Received Signal Strength Indicator (RSSI) measurements of either WiFi or Bluetooth-Low-Energy (BLE) beacons. In such systems, the two most common techniques are trilateration and fingerprinting, with the latter providing higher accuracy. In the fingerprinting technique, Deep Learning (DL) algorithms are often used to predict the location of the receiver based on the RSSI measurements of multiple beacons received at the receiver. In this paper, we address two practical issues with applying Deep Learning to wireless localization -- transfer of solution from one wireless environment to another \emph{and} small size of labelled data set. First, we apply automatic hyperparameter optimization to a deep neural network (DNN) system for indoor wireless localization, which makes the system easy to port to new wireless environments. Second, we show how to augment a typically small labelled data set using the unlabelled data set. We observed improved performance in DL by applying the two techniques. Additionally, all relevant code has been made freely available.
Prior knowledge distillation based on financial time series

One of the major characteristics of financial time series is that they contain a large amount of nonstationary noise, which is challenging for deep neural networks. People normally use various features to address this problem. However, the performance of these features depends on the choice of hyper-parameters. In this paper, we propose to use neural networks to represent these indicators and train a large network constructed of smaller networks as feature layers to fine-tune the prior knowledge represented by the indicators. During back propagation, prior knowledge is transferred from human logic to machine logic via gradient descent. Prior knowledge is the neural network's deep belief and teaches the network to not be affected by non-stationary noise. Moreover, co-distillation is applied to distill the structure into a much smaller size to reduce redundant features and the risk of overfitting. In addition, the decisions of the smaller networks in terms of gradient descent are more robust and cautious than those of large networks. In numerical experiments, we find that our algorithm is faster and more accurate than traditional methods on real financial datasets. We also conduct experiments to verify the method.
Triggerflow: trigger-based orchestration of serverless workflows

As more applications are being moved to the Cloud thanks to serverless computing, it is increasingly necessary to support native life cycle execution of those applications in the data center. But existing systems either focus on short-running workflows (like IBM Composer or Amazon Express Workflows) or impose considerable overheads for synchronizing massively parallel jobs (Azure Durable Functions, Amazon Step Functions, Google Cloud Composer). None of them are open systems enabling extensible interception and optimization of custom workflows. We present Triggerflow: an extensible Trigger-based Orchestration architecture for serverless workflows built on top of Knative Eventing and Kubernetes technologies. We demonstrate that Triggerflow is a novel serverless building block capable of constructing different reactive schedulers (State Machines, Directed Acyclic Graphs, Workflow as code). We also validate that it can support high-volume event processing workloads, auto-scale on demand and transparently optimize scientific workflows.
MCRapper: Monte-Carlo Rademacher Averages for Poset Families and Approximate Pattern Mining

We present MCRapper, an algorithm for efficient computation of Monte-Carlo Empirical Rademacher Averages (MCERA) for families of functions exhibiting poset (e.g., lattice) structure, such as those that arise in many pattern mining tasks. The MCERA allows us to compute upper bounds to the maximum deviation of sample means from their expectations, thus it can be used to find both statistically-significant functions (i.e., patterns) when the available data is seen as a sample from an unknown distribution, and approximations of collections of high-expectation functions (e.g., frequent patterns) when the available data is a small sample from a large dataset. This feature is a strong improvement over previously proposed solutions that could only achieve one of the two. MCRapper uses upper bounds to the discrepancy of the functions to efficiently explore and prune the search space, a technique borrowed from pattern mining itself. To show the practical use of MCRapper, we employ it to develop an algorithm TFP-R for the task of True Frequent Pattern (TFP) mining. TFP-R gives guarantees on the probability of including any false positives (precision) and exhibits higher statistical power (recall) than existing methods offering the same guarantees. We evaluate MCRapper and TFP-R and show that they outperform the state-of-the-art for their respective tasks.
Neural mass modeling of slow-fast dynamics of seizure initiation and abortion

Epilepsy is a dynamic and complex neurological disease affecting about 1% of the worldwide population, among which 30% of the patients are drug-resistant. Epilepsy is characterized by recurrent episodes of paroxysmal neural discharges (the so-called seizures), which manifest themselves through a large-amplitude rhythmic activity observed in depth-EEG recordings, in particular in local field potentials (LFPs). The signature characterizing the transition to seizures involves complex oscillatory patterns, which could serve as a marker to prevent seizure initiation by triggering appropriate therapeutic neurostimulation methods. To investigate such protocols, neurophysiological lumped-parameter models at the mesoscopic scale, namely neural mass models, are powerful tools that not only mimic the LFP signals but also give insights on the neural mechanisms related to different stages of seizures. Here, we analyze the multiple time-scale dynamics of a neural mass model and explain the underlying structure of the complex oscillations observed before seizure initiation. We investigate population-specific effects of the stimulation and the dependence of stimulation parameters on synaptic timescales. In particular, we show that intermediate stimulation frequencies (>20 Hz) can abort seizures if the timescale difference is pronounced. Those results have the potential in the design of therapeutic brain stimulation protocols based on the neurophysiological properties of tissue.
Dense Non-Rigid Structure from Motion: A Manifold Viewpoint

Non-Rigid Structure-from-Motion (NRSfM) problem aims to recover 3D geometry of a deforming object from its 2D feature correspondences across multiple frames. Classical approaches to this problem assume a small number of feature points and, ignore the local non-linearities of the shape deformation, and therefore, struggles to reliably model non-linear deformations. Furthermore, available dense NRSfM algorithms are often hurdled by scalability, computations, noisy measurements and, restricted to model just global deformation. In this paper, we propose algorithms that can overcome these limitations with the previous methods and, at the same time, can recover a reliable dense 3D structure of a non-rigid object with higher accuracy. Assuming that a deforming shape is composed of a union of local linear subspace and, span a global low-rank space over multiple frames enables us to efficiently model complex non-rigid deformations. To that end, each local linear subspace is represented using Grassmannians and, the global 3D shape across multiple frames is represented using a low-rank representation. We show that our approach significantly improves accuracy, scalability, and robustness against noise. Also, our representation naturally allows for simultaneous reconstruction and clustering framework which in general is observed to be more suitable for NRSfM problems. Our method currently achieves leading performance on the standard benchmark datasets.
Using Mobility for Electrical Load Forecasting During the COVID-19 Pandemic

The novel coronavirus (COVID-19) pandemic has posed unprecedented challenges for the utilities and grid operators around the world. In this work, we focus on the problem of load forecasting. With strict social distancing restrictions, power consumption profiles around the world have shifted both in magnitude and daily patterns. These changes have caused significant difficulties in short-term load forecasting. Typically algorithms use weather, timing information and previous consumption levels as input variables, yet they cannot capture large and sudden changes in socioeconomic behavior during the pandemic. In this paper, we introduce mobility as a measure of economic activities to complement existing building blocks of forecasting algorithms. Mobility data acts as good proxies for the population-level behaviors during the implementation and subsequent easing of social distancing measures. The major challenge with such dataset is that only limited mobility records are associated with the recent pandemic. To overcome this small data problem, we design a transfer learning scheme that enables knowledge transfer between several different geographical regions. This architecture leverages the diversity across these regions and the resulting aggregated model can boost the algorithm performance in each region's day-ahead forecast. Through simulations for regions in the US and Europe, we show our proposed algorithm can outperform conventional forecasting methods by more than three-folds. In addition, we demonstrate how the proposed model can be used to project how electricity consumption would recover based on different mobility scenarios.
Flatness is a False Friend

Hessian based measures of flatness, such as the trace, Frobenius and spectral norms, have been argued, used and shown to relate to generalisation. In this paper we demonstrate that for feed forward neural networks under the cross entropy loss, we would expect low loss solutions with large weights to have small Hessian based measures of flatness. This implies that solutions obtained using $L2$ regularisation should in principle be sharper than those without, despite generalising better. We show this to be true for logistic regression, multi-layer perceptrons, simple convolutional, pre-activated and wide residual networks on the MNIST and CIFAR-$100$ datasets. Furthermore, we show that for adaptive optimisation algorithms using iterate averaging, on the VGG-$16$ network and CIFAR-$100$ dataset, achieve superior generalisation to SGD but are $30 \times$ sharper. This theoretical finding, along with experimental results, raises serious questions about the validity of Hessian based sharpness measures in the discussion of generalisation. We further show that the Hessian rank can be bounded by the a constant times number of neurons multiplied by the number of classes, which in practice is often a small fraction of the network parameters. This explains the curious observation that many Hessian eigenvalues are either zero or very near zero which has been reported in the literature.
Effect of a strong magnetic field on gravity-mode period spacings in red giant stars

When a star evolves into a red giant, the enhanced coupling between core-based gravity modes and envelope-based pressure modes forms mixed modes, allowing its deep interior to be probed by asteroseismology. The ability to obtain information about stellar interiors is important for constraining theories of stellar structure and evolution, for which the origin of various discrepancies between prediction and observation are still under debate. Ongoing speculation surrounds the possibility that some red giant stars may harbour strong (dynamically significant) magnetic fields in their cores, but interpretation of the observational data remains controversial. In part, this is tied to shortfalls in our understanding of the effects of strong fields on the seismic properties of gravity modes, which lies beyond the regime of standard perturbative methods. Here we seek to investigate the effect of a strong magnetic field on the asymptotic period spacings of gravity modes. We use a Hamiltonian ray approach to measure the volume of phase space occupied by mode-forming rays, this being roughly proportional to the average density of modes (number of modes per unit frequency interval). A strong field appears to systematically increase this by about 10%, which predicts a ~10% smaller period spacing. Evidence of near integrability in the ray dynamics hints that the gravity-mode spectrum may still exhibit pseudo-regularities under a strong field.
SL(2, R)
 lattices as information processors

Black holes past their Page times should act as efficient scramblers and information mirrors. The information of the infalling bits are rapidly encoded by the old black hole in the Hawking quanta, but it should take time that is exponential in the Page time entropy to decode the interior. Motivated by the features of fragmentation instability of near-extremal black holes, we construct a simple phenomenological model of the black hole as a lattice of interacting nearly $AdS_2$ throats with gravitational hair charges propagating over the lattice. We study the microstate solutions and their response to shocks. The energy of the shocks are almost wholly absorbed by the total ADM mass of the $AdS_2$ throats, but the information of their locations and time-ordering come out in the hair oscillations, which decouple from the final microstate to which the full system quickly relaxes. We discuss the Hayden-Preskill protocol of decoding infalling information. We also construct generalizations of our model involving a lattice of $AdS_2$ throats networked via wormholes and their analogues in the form of tensor networks of SYK spin-states.
Calibrated Top-1 Uncertainty estimates for classification by score based models

While the accuracy of modern deep learning models has significantly improved in recent years, the ability of these models to generate uncertainty estimates has not progressed to the same degree. Uncertainty methods are designed to provide an estimate of class probabilities when predicting class assignment. While there are a number of proposed methods for estimating uncertainty, they all suffer from a lack of calibration: predicted probabilities can be off from empirical ones by a few percent or more. By restricting the scope of our predictions to only the probability of Top-1 error, we can decrease the calibration error of existing methods to less than one percent. As a result, the scores of the methods also improve significantly over benchmarks.
Measuring the engagement level in encrypted group conversations by using temporal networks

Chat groups are well-known for their capacity to promote viral political and marketing campaigns, spread fake news, and create rallies by hundreds of thousands on the streets. Also, with the increasing public awareness regarding privacy and surveillance, many platforms have started to deploy end-to-end encrypted protocols. In this context, the group’s conversations are not accessible in plain text or readable format by third-party organizations or even the platform owner. Then, the main challenge that emerges is related to getting insights from users’ activity of those groups, but without accessing the messages. Previous approaches evaluated the user engagement by assessing user’s activity, however, on limited conditions where the data is encrypted, they cannot be applied. In this work, we present a framework for measuring the level of engagement of group conversations and users, without reading the messages. Our framework creates an ensemble of interaction networks that represent the temporal evolution of the conversation, then, we apply the proposed Engagement Index (EI) for each interval of conversations to asses users’ participation. Our results in five datasets from real-world WhatsApp Groups indicate that, based on the EI, it is possible to identify the most engaged users within a time interval, create rankings and group users according to their engagement and monitor their performance over time.
Identifying the BLE Advertising Channel for Reliable Distance Estimation on Smartphones

Estimating the distance between two smartphones plays an important role in a host of applications. For this purpose, smartphones emit and scan for Bluetooth Low Energy (BLE) signals. When a device is detected, the distance is estimated by evaluating the received strength of these signals. The main insight that is exploited for distance estimation is that the attenuation of a signal increases with the distance along which it has traveled. However, besides distance, there are multiple additional factors that impact the attenuation and hence disturb the distance estimation procedure. Among them, frequency-selective hardware and signal propagation belong to the most significant ones. For example, a BLE device transmits packets on three different frequencies (channels), while the transmit power and the receiver sensitivity depend on the frequency. As a result, the received signal strength varies for each channel, even when the distance remains constant. However, the information on which wireless channel a packet has been received is not made available to a smartphone. Hence, this error cannot be compensated, e.g., by calibration. In this paper, we for the first time provide a solution to detect the wireless channel on which a packet has been received by a smartphone application. We experimentally evaluate our proposed technique on multiple different smartphone models. Our results help to make distance estimation on smartphones more robust and accurate.