Miracle-dz/UniFAR-FTUs huggingface dataset:
Lasagne: a static binary translator for weak memory model architectures

The emergence of new architectures create a recurring challenge to ensure that existing programs still work on them. Manually porting legacy code is often impractical. Static binary translation (SBT) is a process where a program’s binary is automatically translated from one architecture to another, while preserving their original semantics. However, these SBT tools have limited support to various advanced architectural features. Importantly, they are currently unable to translate concurrent binaries. The main challenge arises from the mismatches of the memory consistency model specified by the different architectures, especially when porting existing binaries to a weak memory model architecture. In this paper, we propose Lasagne, an end-to-end static binary translator with precise translation rules between x86 and Arm concurrency semantics. First, we propose a concurrency model for Lasagne’s intermediate representation (IR) and formally proved mappings between the IR and the two architectures. The memory ordering is preserved by introducing fences in the translated code. Finally, we propose optimizations focused on raising the level of abstraction of memory address calculations and reducing the number of fences. Our evaluation shows that Lasagne reduces the number of fences by up to about 65%, with an average reduction of 45.5%, significantly reducing their runtime overhead.
The Improved COTD Technique for Hardware Trojan Detection in Gate-level Netlist

Hardware Trojans (HTs) have introduced serious security concerns into the integrated circuit design flow as they can undermine circuit operations by leaking sensitive information, causing malfunction, or similar attacks. An earlier-introduced HT detection technique in gate-level netlist, the Controllability and Observability for hardware Trojan Detection (COTD) technique detects HTs based on controllability and observability signals in a circuit and presents a static analysis based on an unsupervised machine learning model to identify HT signals in the circuit. While COTD detects the existence of HTs in a circuit, some work has highlighted the shortcoming of COTD in detecting some HT signals that present similar features as genuine signals. To address this shortcoming, this paper presents an improved COTD technique. The improved COTD technique introduces an iterative unsupervised machine-learning technique to isolate HT signals. Furthermore, the improved COTD is equipped with the Gradual-N-Justification (GNJ) technique to reduce false-positive rates in detecting HT signals. The improved COTD technique is applied to several different combinations of full-scan and partial scan circuits tampered with hard-to-detect sequential HTs. To realize valid and hard-to-detect HTs, a configurable HT insertion platform is utilized. The comprehensive results have shown that the improved COTD is highly scalable. Furthermore, the improved COTD technique does not miss a HT circuit if exists and it offers a false-positive rate as low as 3.4%, on average.
Age-Dependent Differential Privacy

The proliferation of real-time applications has motivated extensive research on analyzing and optimizing data freshness in the context of age of information. However, classical frameworks of privacy (e.g., differential privacy (DP)) have overlooked the impact of data freshness on privacy guarantees, which may provide a new tool for time-varying databases. In this work, we introduce age-dependent DP, taking into account the underlying stochastic nature of a time-varying database. In this new framework, we assume knowledge of the data process’s statistical information and establish a connection between classical DP and age-dependent DP. We use this connection to characterize the impact of data staleness and temporal correlation on privacy guarantees. Our characterization reveals that the total variation distance is the sole essential statistical information. Moreover, we demonstrate that aging, which involves utilizing stale data inputs and/or delaying the release of outputs, can serve as a novel strategy for safeguarding data privacy, in addition to the traditional approach of injecting noise in the DP framework. Furthermore, to generalize our results to a multi-query scenario, we present a sequential composition result for age-dependent DP under any publishing and aging policies. We then characterize the optimal tradeoffs between privacy risk and utility and show how this can be achieved. Finally, case studies show that to achieve an arbitrarily small privacy risk in a single-query case, combing aging and noise injection only leads to a bounded accuracy loss, whereas using noise injection only (as in the benchmark case of DP) will lead to an unbounded accuracy loss.
Survey of Machine Learning for Electronic Design Automation

An increase in demand for semiconductor ICs, recent advancements in machine learning, and the slowing down of Moore's law have all contributed to the increased interest in using Machine Learning (ML) to enhance Electronic Design Automation (EDA) and Computer-Aided Design (CAD) tools and processes. This paper provides a comprehensive survey of available EDA and CAD tools, methods, processes, and techniques for Integrated Circuits (ICs) that use machine learning algorithms. The ML-based EDA/CAD tools are classified based on the IC design steps. They are utilized in Synthesis, Physical Design (Floorplanning, Placement, Clock Tree Synthesis, Routing), IR drop analysis, Static Timing Analysis (STA), Design for Test (DFT), Power Delivery Network analysis, and Sign-off. The current landscape of ML-based VLSI-CAD tools, current trends, and future perspectives of ML in VLSI-CAD are also discussed.
Optimal Region-based Mixed-Cell-Height Detailed Placement Considering Complex Minimum-Implant-Area Constraints

We propose a minimum-implant-area (MIA) aware detailed placement algorithm for multi-row-height standard cells. Specifically, (1) we calculate the optimal regions for all the cells, and (2) we cluster each group of cells with the same threshold voltage and width less than the minimum implant width and then reshape each cluster. (4) We develop an enhanced legalization algorithm to minimize the total wirelength. (5) We solve the remaining inter-row violations by greedily shifting the concerning cells with minimum displacement. Compared with the state-of-the-art work [6], the experimental results show that on average of all the ISPD 2014 benchmarks A [1] our algorithm reduces the wirelength by 6% and runs 6.02x faster with all the MIA violations resolved.
How Infinitely Wide Neural Networks Can Benefit from Multi-task Learning -- an Exact Macroscopic Characterization

In practice, multi-task learning (through learning features shared among tasks) is an essential property of deep neural networks (NNs). While inﬁnite-width limits of NNs can provide a good intuition for their generalization behavior, the well-known inﬁnite-width limits of NNs in the literature (e.g., neural tangent kernels) assume speciﬁc settings in which wide ReLU-NNs behave like shallow Gaussian Processes with a ﬁxed kernel. Consequently, in such settings, these NNs lose their ability to beneﬁt from multi-task learning in the inﬁnite-width limit. In contrast, we prove that optimizing wide ReLU neural networks with at least one hidden layer using (cid:96) 2 -regularization on the parameters enforces multi-task learning due to representation-learning – also in the limiting regime where the network width tends to inﬁnity. We present an exact quantitative characterization of this inﬁnite width limit in an appropriate function space that neatly describes multi-task learning.
Bridging the Gap: Unifying the Training and Evaluation of Neural Network Binary Classifiers

While neural network binary classifiers are often evaluated on metrics such as Accuracy and $F_1$-Score, they are commonly trained with a cross-entropy objective. How can this training-evaluation gap be addressed? While specific techniques have been adopted to optimize certain confusion matrix based metrics, it is challenging or impossible in some cases to generalize the techniques to other metrics. Adversarial learning approaches have also been proposed to optimize networks via confusion matrix based metrics, but they tend to be much slower than common training methods. In this work, we propose a unifying approach to training neural network binary classifiers that combines a differentiable approximation of the Heaviside function with a probabilistic view of the typical confusion matrix values using soft sets. Our theoretical analysis shows the benefit of using our method to optimize for a given evaluation metric, such as $F_1$-Score, with soft sets, and our extensive experiments show the effectiveness of our approach in several domains.
Supervised Contrastive Learning for Interpretable Long-Form Document Matching

Recent advancements in deep learning techniques have transformed the area of semantic text matching (STM). However, most state-of-the-art models are designed to operate with short documents such as tweets, user reviews, comments, and so on. These models have fundamental limitations when applied to long-form documents such as scientific papers, legal documents, and patents. When handling such long documents, there are three primary challenges: (i) the presence of different contexts for the same word throughout the document, (ii) small sections of contextually similar text between two documents, but dissimilar text in the remaining parts (this defies the basic understanding of “similarity”), and (iii) the coarse nature of a single global similarity measure which fails to capture the heterogeneity of the document content. In this article, we describe CoLDE: Contrastive Long Document Encoder—a transformer-based framework that addresses these challenges and allows for interpretable comparisons of long documents. CoLDE uses unique positional embeddings and a multi-headed chunkwise attention layer in conjunction with a supervised contrastive learning framework to capture similarity at three different levels: (i) high-level similarity scores between a pair of documents, (ii) similarity scores between different sections within and across documents, and (iii) similarity scores between different chunks in the same document and across other documents. These fine-grained similarity scores aid in better interpretability. We evaluate CoLDE on three long document datasets namely, ACL Anthology publications, Wikipedia articles, and USPTO patents. Besides outperforming the state-of-the-art methods on the document matching task, CoLDE is also robust to changes in document length and text perturbations and provides interpretable results. The code for the proposed model is publicly available at https://github.com/InterDigitalInc/CoLDE.
Dynamic Privacy Budget Allocation Improves Data Efficiency of Differentially Private Gradient Descent

Protecting privacy in learning while maintaining the model performance has become increasingly critical in many applications that involve sensitive data. A popular private learning framework is differentially private learning composed of many privatized gradient iterations by noising and clipping. Under the privacy constraint, it has been shown that the dynamic policies could improve the final iterate loss, namely the quality of published models. In this talk, we will introduce these dynamic techniques for learning rate, batch size, noise magnitude and gradient clipping. Also, we discuss how the dynamic policy could change the convergence bounds which further provides insight of the impact of dynamic methods.
Residual Multiplicative Filter Networks for Multiscale Reconstruction

Coordinate networks like Multiplicative Filter Networks (MFNs) and BACON offer some control over the frequency spectrum used to represent continuous signals such as images or 3D volumes. Yet, they are not readily applicable to problems for which coarse-to-fine estimation is required, including various inverse problems in which coarse-to-fine optimization plays a key role in avoiding poor local minima. We introduce a new coordinate network architecture and training scheme that enables coarse-to-fine optimization with fine-grained control over the frequency support of learned reconstructions. This is achieved with two key innovations. First, we incorporate skip connections so that structure at one scale is preserved when fitting finer-scale structure. Second, we propose a novel initialization scheme to provide control over the model frequency spectrum at each stage of optimization. We demonstrate how these modifications enable multiscale optimization for coarse-to-fine fitting to natural images. We then evaluate our model on synthetically generated datasets for the the problem of single-particle cryo-EM reconstruction. We learn high resolution multiscale structures, on par with the state-of-the art.
Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning

The $Q$-learning algorithm is a simple and widely-used stochastic approximation scheme for reinforcement learning, but the basic protocol can exhibit instability in conjunction with function approximation. Such instability can be observed even with linear function approximation. In practice, tools such as target networks and experience replay appear to be essential, but the individual contribution of each of these mechanisms is not well understood theoretically. This work proposes an exploration variant of the basic $Q$-learning protocol with linear function approximation. Our modular analysis illustrates the role played by each algorithmic tool that we adopt: a second order update rule, a set of target networks, and a mechanism akin to experience replay. Together, they enable state of the art regret bounds on linear MDPs while preserving the most prominent feature of the algorithm, namely a space complexity independent of the number of step elapsed. We show that the performance of the algorithm degrades very gracefully under a novel and more permissive notion of approximation error. The algorithm also exhibits a form of instance-dependence, in that its performance depends on the"effective"feature dimension.
EfficientNeRF: Efficient Neural Radiance Fields

Neural Radiance Fields (NeRF) has been wildly applied to various tasks for its high-quality representation of 3D scenes. It takes long per-scene training time and per-image testing time. In this paper, we present EfficientNeRF as an efficient NeRF-based method to represent 3D scene and synthesize novel-view images. Although several ways exist to accelerate the training or testing process, it is still difficult to much reduce time for both phases simultaneously. We analyze the density and weight distribution of the sampled points then propose valid and pivotal sampling at the coarse and fine stage, respectively, to significantly improve sampling efficiency. In addition, we design a novel data structure to cache the whole scene during testing to accelerate the rendering speed. Overall, our method can reduce over 88\% of training time, reach rendering speed of over 200 FPS, while still achieving competitive accuracy. Experiments prove that our method promotes the practicality of NeRF in the real world and enables many applications.
Merlin-Arthur Classifiers: Formal Interpretability with Interactive Black Boxes

We present a new theoretical framework for making black box classiﬁers such as Neural Networks interpretable, basing our work on clear assumptions and guarantees. In our setting, which is inspired by the Merlin-Arthur protocol from Interactive Proof Systems, two functions cooperate to achieve a classiﬁcation together: the prover selects a small set of features as a certiﬁcate and presents it to the classiﬁer . Including a second, adversarial prover allows us to connect a game-theoretic equilibrium to information-theoretic guarantees on the exchanged features. We deﬁne notions of completeness and soundness that enable us to lower bound the mutual information between features and class. To demonstrate good agreement between theory and practice, we support our framework by providing numerical experiments for Neural Network classiﬁers, explicitly calculating the mutual information of features with respect to the class.
MentSum: A Resource for Exploring Summarization of Mental Health Online Posts

Mental health remains a significant challenge of public health worldwide. With increasing popularity of online platforms, many use the platforms to share their mental health conditions, express their feelings, and seek help from the community and counselors. Some of these platforms, such as Reachout, are dedicated forums where the users register to seek help. Others such as Reddit provide subreddits where the users publicly but anonymously post their mental health distress. Although posts are of varying length, it is beneficial to provide a short, but informative summary for fast processing by the counselors. To facilitate research in summarization of mental health online posts, we introduce Mental Health Summarization dataset, MentSum, containing over 24k carefully selected user posts from Reddit, along with their short user-written summary (called TLDR) in English from 43 mental health subreddits. This domain-specific dataset could be of interest not only for generating short summaries on Reddit, but also for generating summaries of posts on the dedicated mental health forums such as Reachout. We further evaluate both extractive and abstractive state-of-the-art summarization baselines in terms of Rouge scores, and finally conduct an in-depth human evaluation study of both user-written and system-generated summaries, highlighting challenges in this research.
Inclusive Thermodynamics of Computational Machines

We introduce a framework designed to analyze the thermodynamics of an abstractly deﬁned logical computer like a deterministic ﬁnite automaton (DFA) or a Turing machine, without specifying any extraneous parameters (like rate matrices, Hamiltonians, etc.) of a physical process that implements the computer. Earlier investigations of how to do this were based on the continuous-time Markov chain (CTMC) formulation of stochastic thermodynamics. These investigations either assumed that there was exactly zero irreversible entropy production (EP) generated by the physical system implementing the computation, or allowed the EP to be nonzero but only considered the “mismatch cost” component of the EP. In addition, they only applied to a single type of computer. Our framework neither requires that EP equal zero nor restricts attention to the mismatch cost component of EP, and is designed to apply to all types of computational machines. In contrast to earlier investigations using the CTMC-based formulation, our framework is based on the inclusive Hamiltonian formulation, in which the combination of the system of interest and the baths evolve in a Hamiltonian (or unitary) dynamics. Here, we use our framework to derive an integral ﬂuctuation theorem for computers, in which the expectation value is strictly less than 1. We also derive an exchange ﬂuctuation theorem, and a mismatch cost formula involving ﬁrst-passage times. We analyze the EP generated by a DFA, a Markov information source, and a noisy communication channel. In particular, we use the Myhill-Nerode theorem of computer science to prove that out of all DFAs which recognize the same language, the “minimal complexity DFA” is the one with minimal EP for all dynamics and at all iterations.
Semantic Instance Segmentation of 3D Scenes Through Weak Bounding Box Supervision

Current 3D segmentation methods heavily rely on large-scale point-cloud datasets, which are notoriously laborious to annotate. Few attempts have been made to circumvent the need for dense per-point annotations. In this work, we look at weakly-supervised 3D instance semantic segmentation. The key idea is to leverage 3D bounding box labels which are easier and faster to annotate. Indeed, we show that it is possible to train dense segmentation models using only weak bounding box labels. At the core of our method, Box2Mask, lies a deep model, inspired by classical Hough voting, that directly votes for bounding box parameters, and a clustering method specifically tailored to bounding box votes. This goes beyond commonly used center votes, which would not fully exploit the bounding box annotations. On ScanNet test, our weakly supervised model attains leading performance among other weakly supervised approaches (+18 mAP 50 ). Remarkably, it also achieves 97% of the performance of fully supervised models. To proof the practicality of our approach, we show segmentation results on the recently released ARKitScenes dataset which is annotated with 3D bounding boxes only and obtain, for the first time, compelling 3D instance segmentation results.
RoCourseNet: Robust Training of a Prediction Aware Recourse Model

Counterfactual (CF) explanations for machine learning (ML) models are preferred by end-users, as they explain the predictions of ML models by providing a recourse (or contrastive) case to individuals who are adversely impacted by predicted outcomes. Existing CF explanation methods generate recourses under the assumption that the underlying target ML model remains stationary over time. However, due to commonly occurring distributional shifts in training data, ML models constantly get updated in practice, which might render previously generated recourses invalid and diminish end-users trust in our algorithmic framework. To address this problem, we propose RoCourseNet, a training framework that jointly optimizes predictions and recourses that are robust to future data shifts. This work contains four key contributions: (1) We formulate the robust recourse generation problem as a tri-level optimization problem which consists of two sub-problems: (i) a bi-level problem that finds the worst-case adversarial shift in the training data, and (ii) an outer minimization problem to generate robust recourses against this worst-case shift. (2) We leverage adversarial training to solve this tri-level optimization problem by: (i) proposing a novel virtual data shift (VDS) algorithm to find worst-case shifted ML models via explicitly considering the worst-case data shift in the training dataset, and (ii) a block-wise coordinate descent procedure to optimize for prediction and corresponding robust recourses. (3) We evaluate RoCourseNet's performance on three real-world datasets, and show that RoCourseNet consistently achieves more than 96% robust validity and outperforms state-of-the-art baselines by at least 10% in generating robust CF explanations. (4) Finally, we generalize the RoCourseNet framework to accommodate any parametric post-hoc methods for improving robust validity.
Contamination Bias in Linear Regressions

We study regressions with multiple treatments and a set of controls that is flexible enough to purge omitted variable bias. We show that these regressions generally fail to estimate convex averages of heterogeneous treatment effects -- instead, estimates of each treatment's effect are contaminated by non-convex averages of the effects of other treatments. We discuss three estimation approaches that avoid such contamination bias, including the targeting of easiest-to-estimate weighted average effects. A re-analysis of nine empirical applications finds economically and statistically meaningful contamination bias in observational studies; contamination bias in experimental studies is more limited due to idiosyncratic effect heterogeneity.