Miracle-dz/UniFAR-FTUs huggingface dataset:
Gaussian RBF Centered Kernel Alignment (CKA) in the Large-Bandwidth Limit

Centered kernel alignment (CKA), also known as centered kernel-target alignment, is useful as a similarity measure between kernels and as a kernel-based similarity measure between feature representations. We prove that CKA based on a Gaussian RBF kernel converges to linear CKA in the large-bandwidth limit. The result relies on mean-centering of the feature maps and on a Hilbert-Schmidt Independence Criterion (HSIC) identity. We show that convergence onset is sensitive to the geometry of the feature representations, and that a notion of representation eccentricity, <inline-formula><tex-math notation="LaTeX">$\rho$</tex-math><alternatives><mml:math><mml:mi>ρ</mml:mi></mml:math><inline-graphic xlink:href="alvarez-ieq1-3216518.gif"/></alternatives></inline-formula>, constrains the bandwidth range for which Gaussian CKA can differ noticeably from linear CKA. Our experimental results suggest that Gaussian bandwidths less than <inline-formula><tex-math notation="LaTeX">$\rho$</tex-math><alternatives><mml:math><mml:mi>ρ</mml:mi></mml:math><inline-graphic xlink:href="alvarez-ieq2-3216518.gif"/></alternatives></inline-formula> should be selected in order to enable nonlinear modeling.
Learning Stabilizing Controllers of Linear Systems via Discount Policy Gradient

Stability is one of the most fundamental requirements for systems synthesis. In this paper, we address the stabilization problem for unknown linear systems via policy gradient (PG) methods. We leverage a key feature of PG for Linear Quadratic Regulator (LQR), i.e., it drives the policy away from the boundary of the unstabilizing region along the descent direction, provided with an initial policy with finite cost. To this end, we discount the LQR cost with a factor, by adaptively increasing which gradient leads the policy to the stabilizing set while maintaining a finite cost. Based on the Lyapunov theory, we design an update rule for the discount factor which can be directly computed from data, rendering our method purely model-free. Compared to recent work \citep{perdomo2021stabilizing}, our algorithm allows the policy to be updated only once for each discount factor. Moreover, the number of sampled trajectories and simulation time for gradient descent is significantly reduced to $\mathcal{O}(\log(1/\epsilon))$ for the desired accuracy $\epsilon$. Finally, we conduct simulations on both small-scale and large-scale examples to show the efficiency of our discount PG method.
A Multimodal Approach for Automatic Mania Assessment in Bipolar Disorder

Bipolar disorder is a mental health disorder that causes mood swings that range from depression to mania. Diagnosis of bipolar disorder is usually done based on patient interviews, and reports obtained from the caregivers of the patients. Subsequently, the diagnosis depends on the experience of the expert, and it is possible to have confusions of the disorder with other mental disorders. Automated processes in the diagnosis of bipolar disorder can help providing quantitative indicators, and allow easier observations of the patients for longer periods. Furthermore, the need for remote treatment and diagnosis became especially important during the COVID-19 pandemic. In this thesis, we create a multimodal decision system based on recordings of the patient in acoustic, linguistic, and visual modalities. The system is trained on the Bipolar Disorder corpus. Comprehensive analysis of unimodal and multimodal systems, as well as various fusion techniques are performed. Besides processing entire patient sessions using unimodal features, a task-level investigation of the clips is studied. Using acoustic, linguistic, and visual features in a multimodal fusion system, we achieved a 64.8% unweighted average recall score, which improves the state-of-the-art performance achieved on this dataset.
Statistical inference on representational geometries

Neuroscience has recently made much progress, expanding the complexity of both neural activity measurements and brain-computational models. However, we lack robust methods for connecting theory and experiment by evaluating our new big models with our new big data. Here, we introduce new inference methods enabling researchers to evaluate and compare models based on the accuracy of their predictions of representational geometries: A good model should accurately predict the distances among the neural population representations (e.g. of a set of stimuli). Our inference methods combine novel 2-factor extensions of crossvalidation (to prevent overfitting to either subjects or conditions from inflating our estimates of model accuracy) and bootstrapping (to enable inferential model comparison with simultaneous generalization to both new subjects and new conditions). We validate the inference methods on data where the ground-truth model is known, by simulating data with deep neural networks and by resampling of calcium-imaging and functional MRI data. Results demonstrate that the methods are valid and conclusions generalize correctly. These data analysis methods are available in an open-source Python toolbox (rsatoolbox.readthedocs.io).
Distortion Mitigation in Millimeter-Wave Interferometric Radar Angular Velocity Estimation Using Signal Response Decomposition

A new method of distortion mitigation for multitarget interferometric angular velocity estimation in millimeter-wave radar is presented. In general, when multiple targets are present, the response of a correlation interferometer is corrupted by intermodulation distortion, making it difficult to estimate individual target angular velocities. We present a distortion mitigation method that works by decomposing the responses at each antenna element into the responses from the individual targets. Data association is performed to match individual target responses at each antenna such that cross-correlation is performed only between associated targets. Thus, the intermodulation distortion (cross-terms) from correlating unlike targets are eliminated, and the result is a frequency response whose individual frequencies are proportional to the angular velocities of the targets. We demonstrate the approach with a custom 40 GHz interferometric radar, a high-accuracy motion capture system which provides ground-truth position measurements, and two robotic platforms. The multitarget experiments consist of three scenarios, designed to represent easy, medium, and difficult cases for the distortion mitigation technique. We show that the reduction in distortion yields angular velocity estimation errors in the three cases of less than 0.008 rad/s, 0.020 rad/s, and 0.033 rad/s for the easy, medium, and hard cases, respectively.
Transcribing Natural Languages for The Deaf via Neural Editing Programs

This work studies the task of glossification, of which the aim is to em transcribe natural spoken language sentences for the Deaf (hard-of-hearing) community to ordered sign language glosses. Previous sequence-to-sequence language models trained with paired sentence-gloss data often fail to capture the rich connections between the two distinct languages, leading to unsatisfactory transcriptions. We observe that despite different grammars, glosses effectively simplify sentences for the ease of deaf communication, while sharing a large portion of vocabulary with sentences. This has motivated us to implement glossification by executing a collection of editing actions, e.g. word addition, deletion, and copying, called editing programs, on their natural spoken language counterparts. Specifically, we design a new neural agent that learns to synthesize and execute editing programs, conditioned on sentence contexts and partial editing results. The agent is trained to imitate minimal editing programs, while exploring more widely the program space via policy gradients to optimize sequence-wise transcription quality. Results show that our approach outperforms previous glossification models by a large margin, improving the BLEU-4 score from 16.45 to 18.89 on RWTH-PHOENIX-WEATHER-2014T and from 18.38 to 21.30 on CSL-Daily.
An Online Orchestration Mechanism for General-Purpose Edge Computing

In recent years, the fast development of mobile communications and cloud systems has substantially promoted edge computing. By pushing server resources to the edge, mobile service providers can deliver their content and services with enhanced performance, and mobile-network carriers can alleviate congestion in the core networks. Although edge computing has been attracting much interest, most current research is application-specific, and analysis is lacking from a business perspective of edge cloud providers (ECPs) that provide general-purpose edge cloud services to mobile service providers and users. In this article, we present a vision of general-purpose edge computing realized by multiple interconnected edge clouds, analyzing the business model from the viewpoint of ECPs and identifying the main issues to address to maximize benefits for ECPs. Specifically, we formalize the long-term revenue of ECPs as a function of server-resource allocation and public data-placement decisions subject to the amount of physical resources and inter-cloud data-transportation cost constraints. To optimize the long-term objective, we propose an online framework that integrates the drift-plus-penalty and primal-dual methods. With theoretical analysis and simulations, we show that the proposed method approximates the optimal solution in a challenging environment without having future knowledge of the system.
Min-cost-flow preserving bijection between subgraphs and orientations

Consider an undirected graph $G=(V,E)$. A subgraph of $G$ is a subset of its edges, while an orientation of $G$ is an assignment of a direction to each of its edges. Provided with an integer circulation--demand $d:V\to \mathbb{Z}$, we show an explicit and efficiently computable bijection between subgraphs of $G$ on which a $d$-flow exists and orientations on which a $d$-flow exists. Moreover, given a cost function $w:E\to (0,\infty)$ we can find such a bijection which preserves the $w$-min-cost-flow. In 2013, Kozma and Moran [Electron. J. Comb. 20(3)] showed, using dimensional methods, that the number of subgraphs $k$-edge-connecting a vertex $s$ to a vertex $t$ is the same as the number of orientations $k$-edge-connecting $s$ to $t$. An application of our result is an efficient, bijective proof of this fact.
Concatenated Code Design for Constrained DNA Data Storage with Asymmetric Errors

DNA Data storage has recently attracted much attention due to its durable preservation and extremely high information density (bits per gram) properties. In this work, we propose a hybrid coding strategy comprising of generalized constrained codes to tackle homopolymer (run-length) limit and a protograph based low-density parity-check (LDPC) code to correct asymmetric nucleotide level (i.e., A/T/C/G) substitution errors that may occur in the process of DNA sequencing. Two sequencing techniques namely, Nanopore sequencer and Illumina sequencer with their equivalent channel models and capacities are analyzed. A coding architecture is proposed to potentially eliminate the catastrophic errors caused by the error-propagation in the constrained decoding while enabling high coding potential. We also show the log likelihood ratio (LLR) calculation method for the belief propagation decoding with this coding architecture. The simulation results and the theoretical analysis show that the proposed coding scheme exhibits good bit-error rate (BER) performance and high coding potential ($\sim1.98$ bits per nucleotide).
Three-dimensional deep learning-based reduced order model for unsteady flow dynamics with variable Reynolds number

In this article, we present a deep learning-based reduced order model (DL-ROM) for predicting the fluid forces and unsteady vortex shedding patterns. We consider the flow past a sphere to examine the accuracy of our DL-ROM predictions. The proposed DL-ROM methodology relies on a three-dimensional convolutional recurrent autoencoder network (3D CRAN) to extract the low-dimensional flow features from the full-order snapshots in an unsupervised manner. The low-dimensional features are evolved in time using a long short-term memory-based recurrent neural network and reconstructed back to the full-order as flow voxels. These flow voxels are introduced as static and uniform query probes in the point cloud domain to reduce the unstructured mesh complexity while providing convenience in the 3D CRAN training. We introduce a novel procedure to recover the interface description and the instantaneous force quantities from these 3D flow voxels. To evaluate the 3D flow reconstruction and inference, the 3D CRAN methodology is first applied to an external flow past a static sphere at the single Reynolds number of Re = 300. We provide an assessment of the computing requirements in terms of the memory usage, training, and testing cost of the 3D CRAN framework. Subsequently, variable Re-based flow information is infused in one 3D CRAN to learn a symmetry-breaking flow regime (280 [Formula: see text] Re [Formula: see text] 460) for the flow past a sphere. Effects of transfer learning are analyzed for training this complex 3D flow regime on a relatively smaller time series dataset. The 3D CRAN framework learns the flow regime nearly 20 times faster than the parallel full-order model and predicts this flow regime in time with a reasonable accuracy. Based on the predicted flow fields, the network demonstrates an [Formula: see text] accuracy of 98.58% for the drag and 76.43% for the lift over the sphere in this flow regime. The proposed framework aligns with the development of a digital twin for 3D unsteady flow field and instantaneous force predictions with variable Re-based effects.
Knowledge graph enhanced recommender system

Knowledge Graphs (KGs) have shown great success in recommendation. This is attributed to the rich attribute information contained in KG to improve item and user representations as side information. However, existing knowledge-aware methods leverage attribute information at a coarse-grained level both in item and user side. In this paper, we proposed a novel attentive knowledge graph attribute network(AKGAN) to learn item attributes and user interests via attribute information in KG. Technically, AKGAN adopts a heterogeneous graph neural network framework, which has a different design between the first layer and the latter layer. With one attribute placed in the corresponding range of element-wise positions, AKGAN employs a novel interest-aware attention network, which releases the limitation that the sum of attention weight is 1, to model the complexity and personality of user interests towards attributes. Experimental results on three benchmark datasets show the effectiveness and explainability of AKGAN.
Automated Stability Testing of Elastic Rods With Helical Centerlines Using a Robotic System

Experimental analysis of the mechanics of a deformable object, and particularly its stability, requires repetitive testing and, depending on the complexity of the object’s shape, a testing setup that can manipulate many degrees of freedom at the object’s boundary. Motivated by recent advancements in robotic manipulation of deformable objects, this letter addresses these challenges by constructing a method for automated stability testing of a slender elastic rod — a canonical example of a deformable object — using a robotic system. We focus on rod configurations with helical centerlines since the stability of a helical rod can be described using only three parameters, but experimentally determining the stability requires manipulation of both the position and orientation at one end of the rod, which is not possible using traditional experimental methods that only actuate a limited number of degrees of freedom. Using a recent geometric characterization of stability for helical rods, we construct and implement a manipulation scheme to explore the space of stable helices, and we use a vision system to detect the onset of instabilities within this space. The experimental results obtained by our automated testing system show good agreement with numerical simulations of elastic rods in helical configurations. The methods described in this letter lay the groundwork for automation to grow within the field of experimental mechanics.
Image registration for multi-band images taken by ONC-T onboard Hayabusa2

Hayabusa2, a Japanese sample-return mission to a C-type asteroid, arrived at its target 162173 Ryugu in June 2018. The optical navigation cameras (ONC-T, ONCW1, ONC-W2) successfully obtained numerous images of Ryugu. ONC-T is a telescopic framing camera with a charge-coupled device (CCD), has seven filter bands in ultraviolet, visible and near infrared wavelength ranges, and were used to map the spectral distribution of the Ryugu surface. Since the locations of a target seen in ONC-T images are slightly different among different wavelength images in one multi-band observation sequence due to changes in spacecraft positions and attitudes during the filter-changing sequence, one of the image processing issues is image co-registration among images for different wavelength bands. To quickly complete the image co-registration to meet a limited mission schedule, we combined conventional image co-registration techniques with several improvements based on previous planetary missions. The results of our analysis using actual ONC-T images indicate that image co-registration can reach accuracy on the order of 0.1 pixels, which is sufficient for many spectral mapping applications for Ryugu analyses.
Measuring the Metallicity of Early-type Galaxies. I. Composite Region

We present the data of 9739 early-type galaxies (ETGs), cross-matching Galaxy Zoo 1 with our sample selected from the catalog of the Sloan Digital Sky Survey Data Release 7 of MPA-JHU emission-line measurements. We first investigate the divisor between ETGs with and without star formation (SF), and find the best separator of W2–W3 = 2.0 is added. We explore the ETG sample by refusing a variety of ionization sources, and derive 5376 ETGs with SF by utilizing a diagnostic tool of the division line of W2–W3 = 2.0. We measure their metallicities with four abundance calibrators. We find that our composite ETG sample has similar distributions of M * and star formation rate as star-forming galaxies (SFGs) do, that most of them lie on the “main sequence,” and that our fit is a slightly steeper slope than that derived in Renzini & Peng. Compared with the distributions between different metallicities calibrated by four abundance indicators, we find that the Curti17 method is the most accurate calibrator for composite ETGs among the four abundance indicators. We present a weak positive correlation of SFR and metallicity only when the metallicity is calibrated by the PP04, Curti17, and T04 indicators. The correlation is not consistent with the negative correlation of both parameters in SFGs. We suggest that the weak correlation is due to the dilution effect of gas inflow driven by minor mergers.
Effective prevention of semantic drift as angular distance in memory-less continual deep neural networks

Lifelong machine learning or continual learning models attempt to learn incrementally by accumulating knowledge across a sequence of tasks. Therefore, these models learn better and faster. They are used in various intelligent systems that have to interact with humans or any dynamic environment e.g., chatbots and self-driving cars. Memory-less approach is more often used with deep neural networks that accommodates incoming information from tasks within its architecture. It allows them to perform well on all the seen tasks. These models suffer from semantic drift or the plasticity-stability dilemma. The existing models use Minkowski distance measures to decide which nodes to freeze, update or duplicate. These distance metrics do not provide better separation of nodes as they are susceptible to high dimensional sparse vectors. In our proposed approach, we use angular distance to evaluate the semantic drift in individual nodes that provide better separation of nodes and thus better balancing between stability and plasticity. The proposed approach outperforms state-of-the art models by maintaining higher accuracy on standard datasets.
On the existence of global minima and convergence analyses for gradient descent methods in the training of deep neural networks

In this article we study fully-connected feedforward deep ReLU ANNs with an arbitrarily large number of hidden layers and we prove convergence of the risk of the GD optimization method with random initializations in the training of such ANNs under the assumption that the unnormalized probability density function of the probability distribution of the input data of the considered supervised learning problem is piecewise polynomial, under the assumption that the target function (describing the relationship between input data and the output data) is piecewise polynomial, and under the assumption that the risk function of the considered supervised learning problem admits at least one regular global minimum. In addition, in the special situation of shallow ANNs with just one hidden layer and one-dimensional input we also verify this assumption by proving in the training of such shallow ANNs that for every Lipschitz continuous target function there exists a global minimum in the risk landscape. Finally, in the training of deep ANNs with ReLU activation we also study solutions of gradient flow (GF) differential equations and we prove that every non-divergent GF trajectory converges with a polynomial rate of convergence to a critical point (in the sense of limiting Fr\'echet subdifferentiability). Our mathematical convergence analysis builds up on ideas from our previous article Eberle et al., on tools from real algebraic geometry such as the concept of semi-algebraic functions and generalized Kurdyka-Lojasiewicz inequalities, on tools from functional analysis such as the Arzel\`a-Ascoli theorem, on tools from nonsmooth analysis such as the concept of limiting Fr\'echet subgradients, as well as on the fact that the set of realization functions of shallow ReLU ANNs with fixed architecture forms a closed subset of the set of continuous functions revealed by Petersen et al.
Implementation of a Binary Neural Network on a Passive Array of Magnetic Tunnel Junctions

Magnetic tunnel junctions (MTJs) provide an attractive platform for implementing neural networks because of their simplicity, non-volatility, and scalability. However, in hardware realizations, device variations, write errors, and parasitic resistance degrade performance. To quantify such effects, we perform inference experiments on a 2-layer perceptron constructed from a 15 x 15 passive array of MTJs, examining classification accuracy and write fidelity. Despite imperfections, we achieve median accuracy of 95.3% with proper tuning of network parameters. The success of this tuning process shows that new metrics are needed to characterize and optimize networks reproduced in mixed signal hardware.