Miracle-dz/UniFAR-FTUs huggingface dataset:
Bayesian Calibration of imperfect computer models using Physics-informed priors

We introduce a computational efficient data-driven framework suitable for quantifying the uncertainty in physical parameters and model formulation of computer models, represented by differential equations. We construct physics-informed priors, which are multi-output GP priors that encode the model's structure in the covariance function. This is extended into a fully Bayesian framework that quantifies the uncertainty of physical parameters and model predictions. Since physical models often are imperfect descriptions of the real process, we allow the model to deviate from the observed data by considering a discrepancy function. For inference, Hamiltonian Monte Carlo is used. Further, approximations for big data are developed that reduce the computational complexity from $\mathcal{O}(N^3)$ to $\mathcal{O}(N\cdot m^2),$ where $m \ll N. $ Our approach is demonstrated in simulation and real data case studies where the physics are described by time-dependent ODEs describe (cardiovascular models) and space-time dependent PDEs (heat equation). In the studies, it is shown that our modelling framework can recover the true parameters of the physical models in cases where 1) the reality is more complex than our modelling choice and 2) the data acquisition process is biased while also producing accurate predictions. Furthermore, it is demonstrated that our approach is computationally faster than traditional Bayesian calibration methods.
Recommendation Unlearning

Recommender systems provide essential web services by learning users’ personal preferences from collected data. However, in many cases, systems also need to forget some training data. From the perspective of privacy, users desire a tool to erase the impacts of their sensitive data from the trained models. From the perspective of utility, if a system’s utility is damaged by some bad data, the system needs to forget such data to regain utility. While unlearning is very important, it has not been well-considered in existing recommender systems. Although there are some researches have studied the problem of machine unlearning, existing methods can not be directly applied to recommendation as they are unable to consider the collaborative information. In this paper, we propose RecEraser, a general and efficient machine unlearning framework tailored to recommendation tasks. The main idea of RecEraser is to divide the training set into multiple shards and train submodels with these shards. Specifically, to keep the collaborative information of the data, we first design three novel data partition algorithms to divide training data into balanced groups. We then further propose an adaptive aggregation method to improve the global model utility. Experimental results on three public benchmarks show that RecEraser can not only achieve efficient unlearning but also outperform the state-of-the-art unlearning methods in terms of model utility. The source code can be found at https://github.com/chenchongthu/Recommendation-Unlearning
Dilated Convolutional Neural Networks for Lightweight Diacritics Restoration

Diacritics restoration has become a ubiquitous task in the Latin-alphabet-based English-dominated Internet language environment. In this paper, we describe a small footprint 1D dilated convolution-based approach which operates on a character-level. We find that neural networks based on 1D dilated convolutions are competitive alternatives to solutions based on recurrent neural networks or linguistic modeling for the task of diacritics restoration. Our approach surpasses the performance of similarly sized models and is also competitive with larger models. A special feature of our solution is that it even runs locally in a web browser. We also provide a working example of this browser-based implementation. Our model is evaluated on different corpora, with emphasis on the Hungarian language. We performed comparative measurements about the generalization power of the model in relation to three Hungarian corpora. We also analyzed the errors to understand the limitation of corpus-based self-supervised training.
Addressing the Challenges of Cross-Lingual Hate Speech Detection

The goal of hate speech detection is to filter negative online content aiming at certain groups of people. Due to the easy accessibility of social media platforms it is crucial to protect everyone which requires building hate speech detection systems for a wide range of languages. However, the available labeled hate speech datasets are limited making it problematic to build systems for many languages. In this paper we focus on cross-lingual transfer learning to support hate speech detection in low-resource languages. We leverage cross-lingual word embeddings to train our neural network systems on the source language and apply it to the target language, which lacks labeled examples, and show that good performance can be achieved. We then incorporate unlabeled target language data for further model improvements by bootstrapping labels using an ensemble of different model architectures. Furthermore, we investigate the issue of label imbalance of hate speech datasets, since the high ratio of non-hate examples compared to hate examples often leads to low model performance. We test simple data undersampling and oversampling techniques and show their effectiveness.
Hyperplane bounds for neural feature mappings

Deep learning methods minimise the empirical risk using loss functions such as the cross entropy loss. When minimising the empirical risk, the generalisation of the learnt function still depends on the performance on the training data, the Vapnik-Chervonenkis(VC)-dimension of the function and the number of training examples. Neural networks have a large number of parameters, which correlates with their VC-dimension that is typically large but not infinite, and typically a large number of training instances are needed to effectively train them. In this work, we explore how to optimize feature mappings using neural network with the intention to reduce the effective VC-dimension of the hyperplane found in the space generated by the mapping. An interpretation of the results of this study is that it is possible to define a loss that controls the VC-dimension of the separating hyperplane. We evaluate this approach and observe that the performance when using this method improves when the size of the training set is small.
ON RIGHT-ANGLED POLYHEDRA IN RIEMANNIAN 3-MANIFOLDS WITH NON-NEGATIVE SCALAR CURVATURE

We give a complete characterization of the combinatorial types of all the 3-dimensional right-angled mean curvature convex (or totally geodesic) polyhedra in Riemannian 3-manifolds with non-negative scalar curvature. This result can be considered as an analogue of Andreev’s theorem on right-angled polyhedra in the 3-dimensional hyperbolic space.
Deep Optimal Transport on SPD Manifolds for Domain Adaptation

The domain adaption (DA) problem on symmetric positive deﬁnite (SPD) manifolds has raised interest in the machine learning community because of the growing potential for the SPD-matrix representations across many non-stationary applicable scenarios. This paper generalizes the joint distribution adaption (JDA) to align the source and target domains on SPD manifolds and proposes a deep network architecture, Deep Optimal Transport (DOT), using the generalized JDA and the existing deep network architectures on SPD manifolds. The speciﬁc architecture in DOT enables it to learn an approximate optimal transport (OT) solution to the DA problems on SPD manifolds. In the experiments, DOT exhibits a 2.32% and 2.92% increase on the average accuracy in two highly non-stationary cross-session scenarios in brain-computer interfaces (BCIs), respectively. The visualizational results of the source and target domains before and after the transformation also demonstrate the validity of DOT.
Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems

Large transformer models can highly improve Answer Sentence Selection (AS2) tasks, but their high computational costs prevent their use in many real-world applications. In this paper, we explore the following research question: How can we make the AS2 models more accurate without significantly increasing their model complexity? To address the question, we propose a Multiple Heads Student architecture (named CERBERUS), an efficient neural network designed to distill an ensemble of large transformers into a single smaller model. CERBERUS consists of two components: a stack of transformer layers that is used to encode inputs, and a set of ranking heads; unlike traditional distillation technique, each of them is trained by distilling a different large transformer architecture in a way that preserves the diversity of the ensemble members. The resulting model captures the knowledge of heterogeneous transformer models by using just a few extra parameters. We show the effectiveness of CERBERUS on three English datasets for AS2; our proposed approach outperforms all single-model distillations we consider, rivaling the state-of-the-art large AS2 models that have 2.7x more parameters and run 2.5x slower. Code for our model is available at https://github.com/amazon-research/wqa-cerberus
UTILIZING STANDARD CMOS PROCESS FLOATING GATE DEVICES FOR ANALOG DESIGN

This thesis examines a floating gate device (FGD) structure available under standard (digital) CMOS manufacturing processes and puts forth two applications for these devices. The first application is the creation of a tunable current mirror. Inclusion of the FGD structure allows the legs of the mirror to be electronically tweaked to compensate for mismatch. Experimental data is presented on this device structure? s performance. The second application explores using the FGD structure as a tunable resistor. Operation of the FGD in this manner creates the possibility of an electrically tunable beta-multiplier current reference. This tunability allows theoretical adjustment of both the generated reference current as well as a selectable temperature performance. Experimental data of obtained resistor values is presented with simulation results of the entire circuit.
Out-of-Time Ordered Correlators in Kicked Coupled Tops: Information Scrambling in Mixed Phase Space and the Role of Conserved Quantities

We investigate operator growth in a bipartite kicked coupled tops (KCT) system with out-of-time ordered correlators (OTOC). Due to the conservation of total magnetization, the system admits a decomposition into distinct invariant subspaces. Initially focusing on the largest invariant subspace, we observe that, under strong coupling, the OTOC growth rate correlates remarkably well with the classical Lyapunov exponent. For the case of scrambling in the mixed phase space, we invoke Percival's conjecture to partition the eigenstates of the Floquet map into regular and chaotic. We notice that the scrambling rate in the chaotic subspace is quantified by OTOCs calculated with respect to a random state constructed from the mixture of chaotic eigenstates described above. We then consider the total system and study OTOCs for different types of initial operators, including the case of random operators where the operators are chosen randomly from the Gaussian unitary ensemble. We observe that the presence of a conserved quantity results in different types of scrambling behaviors for various choices of initial operators depending on whether the operators commute with the symmetry operator. When the operators are random, the averaged OTOC is related to the linear entanglement entropy of the Floquet operator, as found in earlier works. More importantly, we derive a simple expression for the averaged OTOC when random operators commute with the symmetry operator. Our results offer (i) fresh insights into scrambling in mixed-phase space - a domain that has not been comprehensively explored before and (ii) implications of the conserved quantities on the scrambling of information.
Automatic Lexical Simplification for Turkish

In this paper, we present the first automatic lexical simplification system for the Turkish language. Recent text simplification efforts rely on manually crafted simplified corpora and comprehensive NLP tools that can analyse the target text both in word and sentence levels. Turkish is a morphologically rich agglutinative language that requires unique considerations such as the proper handling of inflectional cases. Being a low-resource language in terms of available resources and industrial-strength tools, it makes the text simplification task harder to approach. We present a new text simplification pipeline based on pretrained representation model BERT together with morphological features to generate grammatically correct and semantically appropriate word-level simplifications.
Treatment Effect Risk: Bounds and Inference

Since the average treatment effect (ATE) measures the change in social welfare, even if positive, there is a risk of negative effect on, say, some 10% of the population. Assessing such risk is difficult, however, because any one individual treatment effect (ITE) is never observed so the 10% worst-affected cannot be identified, while distributional treatment effects only compare the first deciles within each treatment group, which does not correspond to any 10%-subpopulation. In this paper we consider how to nonetheless assess this important risk measure, formalized as the conditional value at risk (CVaR) of the ITE-distribution. We leverage the availability of pre-treatment covariates and characterize the tightest-possible upper and lower bounds on ITE-CVaR given by the covariate-conditional average treatment effect (CATE) function. We then proceed to study how to estimate these bounds efficiently from data and construct confidence intervals. This is challenging even in randomized experiments as it requires understanding the distribution of the unknown CATE function, which can be very complex if we use rich covariates so as to best control for heterogeneity. We develop a debiasing method that overcomes this and prove it enjoys favorable statistical properties even when CATE and other nuisances are estimated by black-box machine learning or even inconsistently. Studying a hypothetical change to French job-search counseling services, our bounds and inference demonstrate a small social benefit entails a negative impact on a substantial subpopulation.
Indirect adaptive control of nonlinearly parameterized nonlinear dissipative systems

In this note we address the problem of indirect adaptive (regulation or tracking) control of nonlinear, input affine dissipative systems. It is assumed that the supply rate, the storage and the internal dissipation functions may be expressed as nonlinearly parameterized regression equations where the mappings (depending on the unknown parameters) satisfy a monotonicity condition—this encompasses a large class of physical systems, including passive systems. We propose to estimate the system parameters using the “power‐balance” equation, which is the differential version of the classical dissipation inequality, with a new estimator that ensures global, exponential, parameter convergence under the very weak assumption of interval excitation of the power‐balance equation regressor. To design the indirect adaptive controller we make the standard assumption of existence of an asymptotically stabilizing controller that depends—possibly nonlinearly—on the unknown plant parameters, and apply a certainty‐equivalent control law. The benefits of the proposed approach, with respect to other existing solutions, are illustrated with examples.
Explicit and Efficient Constructions of Linear Codes Against Adversarial Insertions and Deletions

In this work, we study linear error-correcting codes against adversarial insertion-deletion (insdel) errors, a topic that has recently gained a lot of attention. We construct linear codes over <inline-formula> <tex-math notation="LaTeX">$\mathbb {F}_{q}$ </tex-math></inline-formula>, for <inline-formula> <tex-math notation="LaTeX">$q= {\mathrm {poly}}(1/\varepsilon)$ </tex-math></inline-formula>, that can efficiently decode from a <inline-formula> <tex-math notation="LaTeX">$\delta $ </tex-math></inline-formula> fraction of insdel errors and have rate <inline-formula> <tex-math notation="LaTeX">$(1-4\delta)/8-\varepsilon $ </tex-math></inline-formula>. We also show that by allowing codes over <inline-formula> <tex-math notation="LaTeX">$\mathbb {F}_{q^{2}}$ </tex-math></inline-formula> that are linear over <inline-formula> <tex-math notation="LaTeX">$\mathbb {F}_{q}$ </tex-math></inline-formula>, we can improve the rate to <inline-formula> <tex-math notation="LaTeX">$(1-\delta)/4-\varepsilon $ </tex-math></inline-formula> while not sacrificing efficiency. Using this latter result, we construct fully linear codes over <inline-formula> <tex-math notation="LaTeX">$\mathbb {F}_{2}$ </tex-math></inline-formula> that can efficiently correct up to <inline-formula> <tex-math notation="LaTeX">$\delta < 1/54$ </tex-math></inline-formula> fraction of deletions and have rate <inline-formula> <tex-math notation="LaTeX">$R = (1-54\cdot \delta)/1216$ </tex-math></inline-formula>. Cheng <italic>et al.</italic> (2021) constructed codes with (extremely small) rates bounded away from zero that can correct up to a <inline-formula> <tex-math notation="LaTeX">$\delta < 1/400$ </tex-math></inline-formula> fraction of insdel errors. They also posed the problem of constructing linear codes that get close to the <italic>half-Singleton bound</italic> [proved in Cheng <italic>et al.</italic> (2021)] over small fields. Thus, our results significantly improve their construction and get much closer to the bound.
VAQF: Fully Automatic Software-hardware Co-design Framework for Low-bit Vision Transformer

The transformer architectures with attention mechanisms have obtained success in Nature Language Processing (NLP), and Vision Transformers (ViTs) have recently extended the application domains to various vision tasks. While achieving high performance, ViTs suffer from large model size and high computation complexity that hinders the deployment of them on edge devices. To achieve high throughput on hardware and preserve the model accuracy simultaneously, we propose VAQF, a framework that builds inference accelerators on FPGA platforms for quantized ViTs with binary weights and low-precision activations. Given the model structure and the desired frame rate, VAQF will automatically output the required quantization precision for activations as well as the optimized parameter settings of the accelerator that fulfill the hardware requirements. The implementations are developed with Vivado High-Level Synthesis (HLS) on the Xilinx ZCU102 FPGA board, and the evaluation results with the DeiT-base model indicate that a frame rate requirement of 24 frames per second (FPS) is satisfied with 8-bit activation quantization, and a target of 30 FPS is met with 6-bit activation quantization. To the best of our knowledge, this is the first time quantization has been incorporated into ViT acceleration on FPGAs with the help of a fully automatic framework to guide the quantization strategy on the software side and the accelerator implementations on the hardware side given the target frame rate. Very small compilation time cost is incurred compared with quantization training, and the generated accelerators show the capability of achieving real-time execution for state-of-the-art ViT models on FPGAs.
A Brief Survey of Machine Learning Methods for Emotion Prediction using Physiological Data

Emotion prediction is a key emerging research area that focuses on identifying and forecasting the emotional state of a human from multiple modalities. Among other data sources, physiological data can serve as an indicator for emotions with an added advantage that it cannot be masked/tampered by the individual and can be easily collected. This paper surveys multiple machine learning methods that deploy smartphone and physiological data to predict emotions in real-time, using self-reported ecological momentary assessments (EMA) scores as ground-truth. Comparing regression, long short-term memory (LSTM) networks, convolutional neural networks (CNN), reinforcement online learning (ROL), and deep belief networks (DBN), we showcase the variability of machine learning methods employed to achieve accurate emotion prediction. We compare the state-of-the-art methods and highlight that experimental performance is still not very good. The performance can be improved in future works by considering the following issues: improving scalability and generalizability, synchronizing multimodal data, optimizing EMA sampling, integrating adaptability with sequence prediction, collecting unbiased data, and leveraging sophisticated feature engineering techniques.
Optimal Single-User Interactive Beam Alignment with Feedback Delay

Communication in Millimeter wave (mmWave) band relies on narrow beams due to directionality, high path loss, and shadowing. One can use beam alignment (BA) techniques to find and adjust the direction of these narrow beams. In this paper, BA at the base station (BS) is considered, where the BS sends a set of BA packets to scan different angular regions while the user listens to the channel and sends feedback to the BS for each received packet. It is assumed that the packets and feedback received at the user and BS, respectively, can be correctly decoded. Motivated by practical constraints such as propagation delay, a feedback delay for each BA packet is considered. At the end of the BA, the BS allocates a narrow beam to the user including its angle of departure for data transmission and the objective is to maximize the resulting expected beamforming gain. A general framework for studying this problem is proposed based on which a lower bound on the optimal performance as well as an optimality achieving scheme are obtained. Simulation results reveal significant performance improvements over the state-of-the-art BA methods in the presence of feedback delay.
From MEMRISTOR to MEMImpedance device

The behavior of the capacitance switching of HfO2 Resistive non-volatile Memories is investigated in view of realizing a MEMImpedance (MEM-Z) device. In such a Metal Insulator Metal structure, the impedance value can be tuned by the adjustment of both resistance and capacitance values. We observe a strong variation of capacitance from positive to negative values in a single layer Metal Insulator Metal device made of HfO2 deposited by Atomic Layer Deposition, but unfortunately no memory effect is observed. However, in the case of a two layer structure, a device has been obtained with a memory effect where both resistance and capacitance values can be tuned simultaneously, with a variation of capacitance down to negative values to get an inductive behavior. Negative capacitance values are observed for voltage values near SET voltage. A schematic model based on shaped oxygen vacancy density is proposed to account for this capacitance variation. The oxygen vacancies can be either isolated or connected in the .
Integrated CIrCuIt ManufaCture

Printmaking is a fine art practice that encompasses a variety of media including intaglio, relief, lithography and screen-printing. In this collaborative research project the authors extend the traditional boundaries of printmaking to create editions of micro-scale prints on the surface of silicon integrated circuits using the layers of materials normally used for making transistors and electrical interconnections. The process by which the images are printed on the silicon surface is discussed, alongside some of the conceptual and technical issues related to creating printed images using this technology. article Frontispiece. erik brunvand and al Denyer, MemChip3, chip detail, 2009. (© erik brunvand and al Denyer)