Miracle-dz/UniFAR-FTUs huggingface dataset:
Information Bottleneck: Exact Analysis of (Quantized) Neural Networks

The information bottleneck (IB) principle has been suggested as a way to analyze deep neural networks. The learning dynamics are studied by inspecting the mutual information (MI) between the hidden layers and the input and output. Notably, separate fitting and compression phases during training have been reported. This led to some controversy including claims that the observations are not reproducible and strongly dependent on the type of activation function used as well as on the way the MI is estimated. Our study confirms that different ways of binning when computing the MI lead to qualitatively different results, either supporting or refusing IB conjectures. To resolve the controversy, we study the IB principle in settings where MI is non-trivial and can be computed exactly. We monitor the dynamics of quantized neural networks, that is, we discretize the whole deep learning system so that no approximation is required when computing the MI. This allows us to quantify the information flow without measurement errors. In this setting, we observed a fitting phase for all layers and a compression phase for the output layer in all experiments; the compression in the hidden layers was dependent on the type of activation function. Our study shows that the initial IB results were not artifacts of binning when computing the MI. However, the critical claim that the compression phase may not be observed for some networks also holds true.
Momentum density spectroscopy of Pd: Comparison of 2D-ACAR and Compton scattering using a 1D-to-2D reconstruction method

Josef Ketels,1, ∗ David Billington,2 Stephen B. Dugdale,3 Michael Leitner,1, 4 and Christoph P. Hugenschmidt1, 4, † 1Physik-Department, Technische Universität München, James-Franck-Str. 1, 85748 Garching, Germany 2School of Physics and Astronomy, Cardiff University, Queen’s Building, The Parade, Cardiff, CF24 3AA, United Kingdom 3H. H. Wills Physics Laboratory, University of Bristol, Tyndall Avenue, Bristol, BS8 1TL, United Kingdom 4Heinz Maier-Leibnitz Zentrum (MLZ), Technische Universität München, Lichtenbergstr. 1, 85748 Garching, Germany
GNMR: A provable one-line algorithm for low rank matrix recovery

Low rank matrix recovery problems, including matrix completion and matrix sensing, appear in a broad range of applications. In this work we present GNMR -- an extremely simple iterative algorithm for low rank matrix recovery, based on a Gauss-Newton linearization. On the theoretical front, we derive recovery guarantees for GNMR in both the matrix sensing and matrix completion settings. Some of these results improve upon the best currently known for other methods. A key property of GNMR is that it implicitly keeps the factor matrices approximately balanced throughout its iterations. On the empirical front, we show that for matrix completion with uniform sampling, GNMR performs better than several popular methods, especially when given very few observations close to the information limit.
Robust identification of topological phase transition by self-supervised machine learning approach

We propose a systematic methodology to identify the topological phase transition through a self-supervised machine learning model, which is trained to correlate system parameters to the non-local observables in time-of-flight experiments of ultracold atoms. Different from the conventional supervised learning approach, where the predicted phase transition point is very sensitive to the training region and data labeling, our self-supervised learning approach identifies the phase transition point by the largest deviation of the predicted results from the known system parameters and by the highest confidence through a systematic shift of the training regions. We demonstrate the robust application of this approach results in various 1D and 2D exactly solvable models, using different input features (time-of-flight images, spatial correlation function or density–density correlation function). As a result, our self-supervised approach should be a very general and reliable method for many condensed matter or solid state systems to observe new states of matters solely based on experimental measurements, even without a priori knowledge of the phase transition models.
Stock Market Analysis with Text Data: A Review

Stock market movements are influenced by public and private information shared through news articles, company reports, and social media discussions. Analyzing these vast sources of data can give market participants an edge to make profit. However, the majority of the studies in the literature are based on traditional approaches that come short in analyzing unstructured, vast textual data. In this study, we provide a review on the immense amount of existing literature of text-based stock market analysis. We present input data types and cover main textual data sources and variations. Feature representation techniques are then presented. Then, we cover the analysis techniques and create a taxonomy of the main stock market forecast models. Importantly, we discuss representative work in each category of the taxonomy, analyzing their respective contributions. Finally, this paper shows the findings on unaddressed open problems and gives suggestions for future work. The aim of this study is to survey the main stock market analysis models, text representation techniques for financial market prediction, shortcomings of existing techniques, and propose promising directions for future research.
SofaMyRoom: a fast and multiplatform "shoebox" room simulator for binaural room impulse response dataset generation

This paper introduces a shoebox room simulator able to systematically generate synthetic datasets of binaural room impulse responses (BRIRs) given an arbitrary set of head-related transfer functions (HRTFs). The evaluation of machine hearing algorithms frequently requires BRIR datasets in order to simulate the acoustics of any environment. However, currently available solutions typically consider only HRTFs measured on dummy heads, which poorly characterize the high variability in spatial sound perception. Our solution allows to integrate a room impulse response (RIR) simulator with different HRTF sets represented in Spatially Oriented Format for Acoustics (SOFA). The source code and the compiled binaries for different operating systems allow to both advanced and non-expert users to benefit from our toolbox, see https://github.com/spatialaudiotools/sofamyroom/ .
DeepAuditor: Distributed Online Intrusion Detection System for IoT Devices via Power Side-channel Auditing

As the number of IoT devices has increased rapidly, IoT botnets have exploited the vulnerabilities of IoT devices. However, it is still challenging to detect the initial intrusion on IoT devices prior to massive attacks. Recent studies have utilized power side-channel in-formation to identify this intrusion behavior on IoT devices but still lack accurate models in real-time for ubiquitous botnet detection. We propose the first online intrusion detection system called DeepAuditor for multiple IoT devices via power auditing. To de-velop the real-time system, we propose a lightweight power auditing device called Power Auditor. We also design a distributed CNN classifier for online inference in a laboratory setting. In order to protect data leakage and reduce networking redundancy, we then propose a privacy-preserved inference protocol via Packed Homo-morphic Encryption and a sliding window protocol in our system. The classification accuracy and processing time are measured, and the proposed classifier outperforms a baseline classifier, especially against unseen patterns. We also demonstrate that the distributed CNN design is secure against any distributed components. Over-all, the measurements are shown to the feasibility of our real-time distributed system for intrusion detection on IoT devices.
A comprehensive empirical analysis on cross-domain semantic enrichment for detection of depressive language

We analyze the process of creating word embedding feature representations designed for a learning task when annotated data is scarce, for example, in depressive language detection from Tweets. We start with a rich word embedding pre-trained from a large general dataset, which is then augmented with embeddings learned from a much smaller and more specific domain dataset through a simple non-linear mapping mechanism. We also experimented with several other more sophisticated methods of such mapping including, several auto-encoder based and custom loss-function based methods that learn embedding representations through gradually learning to be close to the words of similar semantics and distant to dissimilar semantics. Our strengthened representations better capture the semantics of the depression domain, as it combines the semantics learned from the specific domain coupled with word coverage from the general language. We also present a comparative performance analyses of our word embedding representations with a simple bag-of-words model, well known sentiment and psycholinguistic lexicons, and a general pre-trained word embedding. When used as feature representations for several different machine learning methods, including deep learning models in a depressive Tweets identification task, we show that our augmented word embedding representations achieve a significantly better F1 score than the others, specially when applied to a high quality dataset. Also, we present several data ablation tests which confirm the efficacy of our augmentation techniques.
Design of a fretboard using the stiff string equation

Guitar fretboards are designed based on the equation of the ideal string. That is, it neglecs several factors as nonlinearities and bending stiffness of the strings. Due to this fact, intonation of guitars along the whole neck is not perfect, and guitars have right tuning just in an \emph{average} sense. There are commercially available fretboards that differ from the tradictional design. \footnote{One example is the \cite{patent} by the Company True Temperament AB, where each fretboard is made using CNC processes. } As a final application of this work we would like to redesign the fretboard layout considering the effects of bending stiffness. The main goal of this project is to analyze the differences between the differences in the solution for vibrations of the ideal string and a stiff string. These differences should lead to changes in the fret distribution for a guitar, and, hopefully improve the overall intonation of the instrument. We will start analyzing the ideal string equation and after a good understanding of this analytical solution we will proceed with the, more complex, stiff equation. Topics like separation of variables, Fourier transforms, and Perturbation analysis might prove useful during the course of this project
FaDIV-Syn: Fast Depth-Independent View Synthesis

We introduce FaDIV-Syn, a fast depth-independent method for novel view synthesis. Related methods are often limited by their depth estimation stage, where incorrect depth predictions can lead to large projection errors. To avoid this issue, we efﬁciently warp input images into the target frame for a range of assumed depth planes. The resulting plane sweep volume (PSV) is directly fed into our network, which ﬁrst estimates soft PSV masks in a self-supervised manner, and then directly produces the novel output view. We therefore side-step explicit depth estimation. This improves efﬁciency and performance on transparent, reﬂective, thin, and feature-less scene parts. FaDIV-Syn can perform both interpolation and extrapolation tasks and outperforms state-of-the-art extrapolation methods on the large-scale RealEstate10k dataset. In contrast to comparable methods, it achieves real-time performance due to its lightweight architecture. We thoroughly evaluate ablations, such as removing the Soft-Masking network, training from fewer examples as well as generalization to higher resolutions and stronger depth discretization.
In-depth analysis of evolving binary systems that produce nova eruptions

This study is the direct continuation of a previous work performed by Hillman et al., where they used their feedback dominated numerical simulations to model the evolution of four initial models with white dwarf (WD) masses of 0.7 and $1.0\, \mathrm{M}_\odot$ and red dwarf (RD) masses of 0.45 and $0.7\, \mathrm{M}_\odot$ from first Roche lobe contact of the donor RD, over a few times 109 yr, until the RD was eroded down to below $0.1\, \mathrm{M}_\odot$. This study presents an in-depth analysis of their four models complimented by three models with a higher WD mass of $1.25\, \mathrm{M}_\odot$, one of which comprises an oxygen-neon (ONe) core. Common features were found for all seven models on a secular time-scale as well as on a cyclic time-scale. On the other hand, certain features were found that are strongly dependent either on the WD or the RD mass but are indifferent to the other of the two. Additionally, a model with a WD composed of an ONe core was compared with its corresponding carbon oxygen (CO) core WD model and found to have a significant impact on the heavy element abundances in the ejecta composition.
Regression trees and ensembles for cumulative incidence functions

Abstract The use of cumulative incidence functions for characterizing the risk of one type of event in the presence of others has become increasingly popular over the past two decades. The problems of modeling, estimation and inference have been treated using parametric, nonparametric and semi-parametric methods. Efforts to develop suitable extensions of machine learning methods, such as regression trees and ensemble methods, have begun comparatively recently. In this paper, we propose a novel approach to estimating cumulative incidence curves in a competing risks setting using regression trees and associated ensemble estimators. The proposed methods use augmented estimators of the Brier score risk as the primary basis for building and pruning trees, and lead to methods that are easily implemented using existing R packages. Data from the Radiation Therapy Oncology Group (trial 9410) is used to illustrate these new methods.
A Transformer-based Cross-modal Fusion Model with Adversarial Training for VQA Challenge 2021

In this paper, inspired by the successes of vision-language pre-trained models and the benefits from training with adversarial attacks, we present a novel Transformer-based cross-modal fusion modeling by incorporating the both notions for VQA challenge 2021. Specifically, the proposed model is on top of the architecture of VinVL model [19], and the adversarial training strategy [4] is applied to make the model robust and generalized. Moreover, two implementation tricks are also used in our system to obtain better results. The experiments demonstrate that the novel framework can achieve 76.72% on VQAv2 test-std set.
SGTBN: Generating Dense Depth Maps From Single-Line LiDAR

Depth completion aims to generate a dense depth map from the sparse depth map and aligned RGB image. However, current depth completion methods use extremely expensive 64-line LiDAR(about $\$ $ 100,000) to obtain sparse depth maps, which will limit their application scenarios. Compared with the 64-line LiDAR, the single-line LiDAR is much less expensive and much more robust. Therefore, we propose a method to tackle the problem of single-line depth completion, in which we aim to generate a dense depth map from the single-line LiDAR info and the aligned RGB image. A single-line depth completion dataset is proposed based on the existing 64-line depth completion dataset(KITTI). A network called Semantic Guided Two-Branch Network(SGTBN) which contains global and local branches to extract and fuse global and local info is proposed for this task. A Semantic guided depth upsampling module is used in our network to make full use of the semantic info in RGB images. Except for the usual MSE loss, we add the virtual normal loss to increase the constraint of high-order 3D geometry in our network. Our network outperforms the state-of-the-art in the single-line depth completion task. Besides, compared with the monocular depth estimation, our method also has significant advantages in precision and model size.
Control of a Mixed Autonomy Signalised Urban Intersection: An Action-Delayed Reinforcement Learning Approach

We consider a mixed autonomy scenario where the traffic intersection controller decides whether the traffic light will be green or red at each lane for multiple traffic-light blocks. The objective of the traffic intersection controller is to minimize the queue length at each lane and maximize the outflow of vehicles over each block. We consider that the traffic intersection controller informs the autonomous vehicle (AV) whether the traffic light will be green or red for the future traffic-light block. Thus, the AV can adapt its dynamics by solving an optimal control problem. We model the decision process of the traffic intersection controller as a deterministic delayed Markov decision process owing to the delayed action by the traffic controller. We propose Reinforcement Learning based model-free algorithm to obtain the optimal policy. We show - by extensive simulations - that our algorithm converges and drastically reduces the energy costs of AVs as the traffic controller communicates with the AVs.
A Declarative Goal-oriented Framework for Smart Environments with LPaaS

Smart environments powered by the Internet of Things aim at improving our daily lives by automatically tuning ambient parameters (e.g. temperature, interior light) and by achieving energy savings through self-managing cyber-physical systems. Commercial solutions, however, only permit setting simple target goals on those parameters and do not consider mediating conflicting goals among different users and/or system administrators, and feature limited compatibility across different IoT verticals. In this article, we propose a declarative framework to represent smart environments, user-set goals and customisable mediation policies to reconcile contrasting goals encompassing multiple IoT systems. An open-source Prolog prototype of the framework is showcased over two lifelike motivating examples.
Non-local boundary value problem for a mixed-type equation involving the bi-ordinal Hilfer fractional differential operators

In this paper, we consider a nonlocal boundary-value problem for a mixed-type equation involving the bi-ordinal Hilfer fractional derivative in a rectangular domain. The main target of this work is to analyze the uniqueness and the existence of the solution of the considered problem by means of eigenfunctions. Moreover, we construct the solution of the ordinary fractional differential equation with the right-sided bi-ordinal Hilfer derivative by the method of reduction to the Volterra integral equation. Then, we present sufficient conditions for given data in order to show the existence of the solution.
Advancing biological super-resolution microscopy through deep learning: a brief review

Biological super-resolution microscopy is a new generation of imaging techniques that overcome the ~200 nm diffraction limit of conventional light microscopy in spatial resolution. By providing novel spatial or spatiotemporal information on biological processes at nanometer resolution with molecular specificity, it plays an increasingly important role in biomedical sciences. However, its technical constraints also require trade-offs to balance its spatial resolution, temporal resolution, and light exposure of samples. Recently, deep learning has achieved breakthrough performance in many image processing and computer vision tasks. It has also shown great promise in pushing the performance envelope of biological super-resolution microscopy. In this brief review, we survey recent advances in using deep learning to enhance the performance of biological super-resolution microscopy, focusing primarily on computational reconstruction of super-resolution images. Related key technical challenges are discussed. Despite the challenges, deep learning is expected to play an important role in the development of biological super-resolution microscopy. We conclude with an outlook into the future of this new research area.
Minimum sharpness: Scale-invariant parameter-robustness of neural networks

Toward achieving robust and defensive neural networks, the robustness against the weight parameters perturbations, i.e., sharpness, attracts attention in recent years (Sun et al., 2020). However, sharpness is known to remain a critical issue,"scale-sensitivity. "In this paper, we propose a novel sharpness measure, Minimum Sharpness. It is known that NNs have a specific scale transformation that constitutes equivalent classes where functional properties are completely identical, and at the same time, their sharpness could change unlimitedly. We define our sharpness through a minimization problem over the equivalent NNs being invariant to the scale transformation. We also develop an efficient and exact technique to make the sharpness tractable, which reduces the heavy computational costs involved with Hessian. In the experiment, we observed that our sharpness has a valid correlation with the generalization of NNs and runs with less computational cost than existing sharpness measures.
Multi-frequency characterisation of remnant radio galaxies in the Lockman Hole field

Remnant radio galaxies represent an important phase in the life-cycle of radio active galactic nuclei. It is suggested that in this phase, the jets have switched off and the extended emission is fading rapidly. This phase is not well-studied due to the lack of statistical samples observed at both low and high frequencies. In this work, we study a sample of 23 candidate remnant radio galaxies previously selected using the Low Frequency Array at 150 MHz in the Lockman Hole field. We examine their morphologies and study their spectral properties to confirm their remnant nature and revise the morphological and spectral criteria used to define the initial sample. We present new observations with the Karl G. Jansky Very Large Array at 6000 MHz at both high and low resolution. These observations allowed us to observe the presence or absence of cores and study the spectral curvature and steepness of the spectra of the total emission expected at these high frequencies for the remnant candidates. We confirm 13 out of 23 candidates as remnant radio sources. This corresponds to 7% of the full sample of active, restarted, and remnant candidates from the Lockman Hole field. Surprisingly, only a minority of remnants reside in a cluster (23%). The remnant radio galaxies show a range of properties and morphologies. The majority do not show detection of the core at 6000 MHz and their extended emission often shows ultra-steep spectra (USS). However, there are also remnants with USS total emission and a detection of the core at 6000 MHz, possibly indicating a variety of evolutionary stages in the remnant phase. We confirm the importance of the combination of morphological and spectral criteria and this needs to be taken into consideration when selecting a sample of remnant radio sources.
Next-Day Bitcoin Price Forecast Based on Artificial intelligence Methods

In recent years, Bitcoin price prediction has attracted the interest of researchers and investors. However, the accuracy of previous studies is not well enough. Machine learning and deep learning methods have been proved to have strong prediction ability in this area. This paper proposed a method combined with Ensemble Empirical Mode Decomposition (EEMD) and a deep learning method called long short-term memory (LSTM) to research the problem of next-day Bitcoin price forecast.