Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning to Fill the Seam by Vision: Sub-millimeter Peg-in-hole on Unseen Shapes in Real World

In the peg insertion task, human pays attention to the seam between the peg and the hole and tries to fill it continuously with visual feedback. By imitating the human's behavior, we design architectures with position and orientation estimators based on the seam representation for pose alignment, which proves to be general to the unseen peg geometries. By putting the estimators into the closed-loop control with reinforcement learning, we further achieve higher or comparable success rate, efficiency, and robustness compared with the baseline methods. The policy is trained totally in simulation without any manual intervention. To achieve sim-to-real, a learnable segmentation module with automatic data collecting and labeling can be easily trained to decouple the perception and the policy, which helps the model trained in simulation quickly adapting to the real world with negligible effort. Results are presented in simulation and on a physical robot. Code, videos, and supplemental material are available at https://github.com/xieliang555/SFN.git
Digital Storytelling Teaching Robotic Basics in a Surgical Robotic Curriculum

Digital Storytelling (DST) is a powerful tool to teach complex concepts. DSTs are typically used in humanistic sciences, but several researches showed that are a wonderful tool also for scientific sciences because they are more involving, contextualized and it is easier that lead to deep reflections. The classical use of DSTs plans that the story is the content, while the digital medium is the way to tell it. Our approach is slightly different, the story is not the content but a glue for the main contents, while the digital medium remains the way to explain the lesson. We propose to use DST to teach to surgeons the robotics basics, using a story that should be involving for our target population (a surgical operation) to teach specific concepts about robotics in surgery.
End-to-end Dense Video Captioning as Sequence Generation

Dense video captioning aims to identify the events of interest in an input video, and generate descriptive captions for each event. Previous approaches usually follow a two-stage generative process, which first proposes a segment for each event, then renders a caption for each identified segment. Recent advances in large-scale sequence generation pretraining have seen great success in unifying task formulation for a great variety of tasks, but so far, more complex tasks such as dense video captioning are not able to fully utilize this powerful paradigm. In this work, we show how to model the two subtasks of dense video captioning jointly as one sequence generation task, and simultaneously predict the events and the corresponding descriptions. Experiments on YouCook2 and ViTT show encouraging results and indicate the feasibility of training complex tasks such as end-to-end dense video captioning integrated into large-scale pretrained models.
FedCau: A Proactive Stop Policy for Communication and Computation Efficient Federated Learning

This paper investigates efficient distributed training of a Federated Learning~(FL) model over a wireless network of wireless devices. The communication iterations of the distributed training algorithm may be substantially deteriorated or even blocked by the effects of the devices' background traffic, packet losses, congestion, or latency. We abstract the communication-computation impacts as an `iteration cost' and propose a cost-aware causal FL algorithm~(FedCau) to tackle this problem. We propose an iteration-termination method that trade-offs the training performance and networking costs. We apply our approach when clients use the slotted-ALOHA, the carrier-sense multiple access with collision avoidance~(CSMA/CA), and the orthogonal frequency-division multiple access~(OFDMA) protocols. We show that, given a total cost budget, the training performance degrades as either the background communication traffic or the dimension of the training problem increases. Our results demonstrate the importance of proactively designing optimal cost-efficient stopping criteria to avoid unnecessary communication-computation costs to achieve only a marginal FL training improvement. We validate our method by training and testing FL over the MNIST dataset. Finally, we apply our approach to existing communication efficient FL methods from the literature, achieving further efficiency. We conclude that cost-efficient stopping criteria are essential for the success of practical FL over wireless networks.
Topology-Based Phase Identification of Bulk, Interface, and Confined Water Using an Edge-Conditioned Convolutional Graph Neural Network

Water plays a significant role in various physicochemical and biological processes. Understanding and identifying water phases in various systems such as bulk, interface, and confined water is crucial in improving and engineering state-of-the-art nano-devices. Various order parameters have been developed to distinguish water phases, including bond-order parameters, local structure index, and tetrahedral order parameters. These order parameters are often developed with the assumption of homogenous bulk systems, while most applications involve heterogeneous and non-bulk systems, thus, limiting their generalizability. Our study develops a methodology based on the graph neural network to distinguish water phases directly from data and to learn features instead of predefining them. We provide comparisons between baseline methods trained using conventional order-parameters as features, and a graph neural network model trained using radial distance and hydrogen-bonding information for phase classification and phase transition of water in bulk, interface, and confined systems with continuous and discontinuous phase transitions. VI, VII, VIII, and IX Ices as well as liquid water. For interface and confined systems, we use similar input as for a bulk system, but the output is a binary value indicating whether water is liquid or solid. GNNs are trained to predict whether a particular configuration of atoms is liquid-like or solid-like inside CNTs or at interface for various temperatures. We study CNT 10x10, inside which both continuous and discontinuous melting temperature of 275.0 ± 2.5 K. The behavior of GNN is monotonic within the temperature range of our study, showing an increasing number of liquid-like molecules, while RF shows a non-monotonic and inconsistent behavior with temperature increase.
Extracting Targeted Training Data from ASR Models, and How to Mitigate It

Recent work has designed methods to demonstrate that model updates in ASR training can leak potentially sensitive attributes of the utterances used in computing the updates. In this work, we design the first method to demonstrate information leakage about training data from trained ASR models. We design Noise Masking, a fill-in-the-blank style method for extracting targeted parts of training data from trained ASR models. We demonstrate the success of Noise Masking by using it in four settings for extracting names from the LibriSpeech dataset used for training a state-of-the-art Conformer model. In particular, we show that we are able to extract the correct names from masked training utterances with 11.8% accuracy, while the model outputs some name from the train set 55.2% of the time. Further, we show that even in a setting that uses synthetic audio and partial transcripts from the test set, our method achieves 2.5% correct name accuracy (47.7% any name success rate). Lastly, we design Word Dropout, a data augmentation method that we show when used in training along with Multistyle TRaining (MTR), provides comparable utility as the baseline, along with significantly mitigating extraction via Noise Masking across the four evaluated settings.
Iterative Hard Thresholding with Adaptive Regularization: Sparser Solutions Without Sacrificing Runtime

We propose a simple modification to the iterative hard thresholding (IHT) algorithm, which recovers asymptotically sparser solutions as a function of the condition number. When aiming to minimize a convex function $f(x)$ with condition number $\kappa$ subject to $x$ being an $s$-sparse vector, the standard IHT guarantee is a solution with relaxed sparsity $O(s\kappa^2)$, while our proposed algorithm, regularized IHT, returns a solution with sparsity $O(s\kappa)$. Our algorithm significantly improves over ARHT which also finds a solution of sparsity $O(s\kappa)$, as it does not require re-optimization in each iteration (and so is much faster), is deterministic, and does not require knowledge of the optimal solution value $f(x^*)$ or the optimal sparsity level $s$. Our main technical tool is an adaptive regularization framework, in which the algorithm progressively learns the weights of an $\ell_2$ regularization term that will allow convergence to sparser solutions. We also apply this framework to low rank optimization, where we achieve a similar improvement of the best known condition number dependence from $\kappa^2$ to $\kappa$.
Four-operator splitting algorithms for solving monotone inclusions

Monotone inclusions involving the sum of three maximally monotone operators or more have received much attention in recent years. In this paper, we propose three splitting algorithms for ﬁnding a zero of the sum of four monotone operators, which are two maximally monotone operators, one monotone Lipschitz operator, and one cocoercive operator. These three splitting algorithms are based on the forward-reﬂected-Douglas-Rachford splitting algorithm, backward-forward-reﬂected-backward splitting algorithm, and backward-reﬂected-forward-backward splitting algorithm, respectively. As applications, we apply the proposed algorithms to solve the monotone inclusions problem involving a ﬁnite sum of maximally monotone operators. Numerical results on the Projection on Minkowski sums of convex sets demonstrate the eﬀectiveness of the proposed algorithms.
Introduction of Integrated Image Deep Learning Solution and how it brought laboratorial level heart rate and blood oxygen results to everyone (preprint)

The general public and medical professionals recognized the importance of accurately measuring and storing blood oxygen levels and heart rate during the COVID-19 pandemic. The demand for accurate contact-less devices was motivated by the need for cross-infection reduction and the shortage of conventional oximeters, partially due to the global supply chain issue. This paper evaluated a contact-less mini-program HealthyPai's heart rate (HR) and oxygen saturation (SpO2) measurements compared with other wearable devices. In the HR study of 185 samples (81 in the laboratory environment, 104 in the real-life environment), the mean absolute error (MAE) $\pm$ standard deviation was $1.4827 \pm 1.7452$ in the lab, $6.9231 \pm 5.6426$ in the real-life setting. In the SpO2 study of 24 samples, the mean absolute error (MAE) $\pm$ standard deviation of the measurement was $1.0375 \pm 0.7745$. Our results validated that HealthyPai utilizing the Integrated Image Deep Learning Solution (IIDLS) framework can accurately measure HR and SpO2, providing the test quality at least comparable to other FDA-approved wearable devices in the market and surpassing the consumer-grade and research-grade wearable standards.
Guaranteed Nonlinear Tracking in the Presence of DNN-Learned Dynamics With Contraction Metrics and Disturbance Estimation

This paper presents an approach to trajectory-centric learning control based on contraction metrics and disturbance estimation for nonlinear systems subject to matched uncertainties. The approach uses deep neural networks to learn uncertain dynamics while still providing guarantees of transient tracking performance throughout the learning phase. Within the proposed approach, a disturbance estimation law is adopted to estimate the pointwise value of the uncertainty, with pre-computable estimation error bounds (EEBs). The learned dynamics, the estimated disturbances, and the EEBs are then incorporated in a robust Riemann energy condition to compute the control law that guarantees exponential convergence of actual trajectories to desired ones throughout the learning phase, even when the learned model is poor. On the other hand, with improved accuracy, the learned model can help improve the robustness of the tracking controller, e.g., against input delays, and can be incorporated to plan better trajectories with improved performance, e.g., lower energy consumption and shorter travel time.The proposed framework is validated on a planar quadrotor example.
On-Demand Delivery from Stores: Dynamic Dispatching and Routing with Random Demand

Problem definition: On-demand delivery has become increasingly popular around the world. Motivated by a large grocery chain store who offers fast on-demand delivery services, we model and solve a stochastic dynamic driver dispatching and routing problem for last-mile delivery systems where on-time performance is the main target. The system operator needs to dispatch a set of drivers and specify their delivery routes facing random demand that arrives over a fixed number of periods. The resulting stochastic dynamic program is challenging to solve because of the curse of dimensionality. Methodology/results: We propose a novel structured approximation framework to approximate the value function via a parametrized dispatching and routing policy. We analyze the structural properties of the approximation framework and establish its performance guarantee under large-demand scenarios. We then develop efficient exact algorithms for the approximation problem based on Benders decomposition and column generation, which deliver verifiably optimal solutions within minutes. Managerial implications: The evaluation results on a real-world data set show that our framework outperforms the current policy of the company by 36.53% on average in terms of delivery time. We also perform several policy experiments to understand the value of dynamic dispatching and routing with varying fleet sizes and dispatch frequencies. Funding: This work was supported by the National Natural Science Foundation of China [Grants 72222011 and 72171112], China Association for Science and Technology [Grant 2019QNRC001], and the Natural Sciences and Engineering Research Council of Canada [Grant RGPIN-2022-04950]. Supplemental Material: The online appendices are available at https://doi.org/10.1287/msom.2022. 1171 .
Assessing Differentially Private Variational Autoencoders under Membership Inference

We present an approach to quantify and compare the privacy-accuracy trade-off for differentially private Variational Autoencoders. Our work complements previous work in two aspects. First, we evaluate the the strong reconstruction MI attack against Variational Autoencoders under differential privacy. Second, we address the data scientist's challenge of setting privacy parameter epsilon, which steers the differential privacy strength and thus also the privacy-accuracy trade-off. In our experimental study we consider image and time series data, and three local and central differential privacy mechanisms. We find that the privacy-accuracy trade-offs strongly depend on the dataset and model architecture. We do rarely observe favorable privacy-accuracy trade-off for Variational Autoencoders, and identify a case where LDP outperforms CDP.
Unsupervised Attention-based Sentence-Level Meta-Embeddings from Contextualised Language Models

A variety of contextualised language models have been proposed in the NLP community, which are trained on diverse corpora to produce numerous Neural Language Models (NLMs). However, different NLMs have reported different levels of performances in downstream NLP applications when used as text representations. We propose a sentence-level meta-embedding learning method that takes independently trained contextualised word embedding models and learns a sentence embedding that preserves the complementary strengths of the input source NLMs. Our proposed method is unsupervised and is not tied to a particular downstream task, which makes the learnt meta-embeddings in principle applicable to different tasks that require sentence representations. Specifically, we first project the token-level embeddings obtained by the individual NLMs and learn attention weights that indicate the contributions of source embeddings towards their token-level meta-embeddings. Next, we apply mean and max pooling to produce sentence-level meta-embeddings from token-level meta-embeddings. Experimental results on semantic textual similarity benchmarks show that our proposed unsupervised sentence-level meta-embedding method outperforms previously proposed sentence-level meta-embedding methods as well as a supervised baseline.
Strengthening Subcommunities: Towards Sustainable Growth in AI Research

AI’s rapid growth has been felt acutely by scholarly venues, leading to growing pains within the peer review process. These challenges largely center on the inability of specific subareas to identify and evaluate work that is appropriate according to criteria relevant to each subcommunity as determined by stakeholders of that subarea. We set forth a proposal that re-focuses efforts within these subcommunities through a decentralization of the reviewing and publication process. Through this re-centering effort, we hope to encourage each subarea to confront the issues specific to their process of academic publication and incentivization. This model has historically been successful for several subcommunities in AI, and we highlight those instances as examples for how the broader field can continue to evolve despite its continually growing size. 1 CHALLENGES IN THE EXISTING SYSTEM 1.1 EXPONENTIAL FIELD GROWTH AI’s rapid growth has been felt acutely by scholarly venues, particularly large field-wise conferences. A 2018 study noted that the number of submissions increased by “47% for ICML, by 50% for NeurIPS, and by almost 100% for ICLR” (Sculley et al., 2018). More recent estimates by NeurIPS in 2020 put their year over year growth rate at a similarly high 40% (Neural Information Processing Systems Conference, 2020). This rapid growth has led to growing pains within the peer review process. While conferences in the past would use recommendations from meta-reviewers to generate new reviewer invitations, conferences in most recent years have resorted to soliciting reviewers over social media to fill the reviewer gap. In 2021, NeurIPS asked reviewers to offer to review (positively bid) 30-40 papers out of an estimated 12,000 submissions. These bids resulted in reviewers matched with 6-7 papers each, with instances of AAAI reviewers being asked to review as many as 10 papers. 1.2 LOW REVIEWER QUALITY Relieving reviewer burden has resulted in an increasing number of reviewers with limited reviewing experience. Lack of experience degrades reviewer quality. In 2019, the shortest reviews for ICLR 2020 were as little as 17 words (Sun, 2019). These lackluster reviews recommended rejection, yet often the reviewer admitted a lack of experience in the subject. Inexperienced reviewers also often express biased reviews within the reviewing process; researchers have noted that inexperienced reviewers tend to more easily reject conference re-submissions, yet one could argue these re-submissions are, on average, higher quality than first submissions (Stelmakh et al., 2021). 1.3 REVIEWER INCONSISTENCY This poor quality of reviews is compounded by poor calibration and inter-rater reliability among reviewer pools. A recent analysis of the 2014 NeurIPS reviewing process, in which the number of submissions to the conference are a fraction to today’s, notes that when papers were submitted to two separate pools of reviewers, 25% of papers received differing recommendations among review ∗Equal contribution.
Using HCI to Tackle Race and Gender Bias in ADHD Diagnosis

Attention Deficit Hyperactivity Disorder (ADHD) is a behavioral disorder that impacts an individual’s education, relationships, career, and ability to acquire fair and just police interrogations. Yet, traditional methods used to diagnose ADHD in children and adults are known to have racial and gender bias. In recent years, diagnostic technology has been studied by both HCI and ML researchers. However, these studies fail to take into consideration racial and gender stereotypes that may impact the accuracy of their results. We highlight the importance of taking race and gender into consideration when creating diagnostic technology for ADHD and provide HCI researchers with suggestions for future studies.
Using Statistical Emulation and Knowledge of Grain-surface Diffusion for Bayesian Inference of Reaction Rate Parameters: An Application to a Glycine Network

There exists much uncertainty surrounding interstellar grain-surface chemistry. One of the major reaction mechanisms is grain-surface diffusion for which the binding energy parameter for each species needs to be known. However, these values vary significantly across the literature which can lead to debate as to whether or not a particular reaction takes place via diffusion. In this work we employ Bayesian inference to use available ice abundances to estimate the reaction rates of the reactions in a chemical network that produces glycine. Using this we estimate the binding energy of a variety of important species in the network, by assuming that the reactions take place via diffusion. We use our understanding of the diffusion mechanism to reduce the dimensionality of the inference problem from 49 to 14, by demonstrating that reactions can be separated into classes. This dimensionality reduction makes the problem computationally feasible. A neural network statistical emulator is used to also help accelerate the Bayesian inference process substantially. The binding energies of most of the diffusive species of interest are found to match some of the disparate literature values, with the exceptions of atomic and diatomic hydrogen. The discrepancies between these two species are related to the limitations of the physical and chemical models. However, the use of a dummy reaction of the form H + X ⟶ HX is found to somewhat reduce the discrepancy with the binding energy of atomic hydrogen. Using the inferred binding energies in the full gas–grain version of UCLCHEM results in almost all the molecular abundances being recovered.
Detect Rumors in Microblog Posts for Low-Resource Domains via Adversarial Contrastive Learning

Massive false rumors emerging along with breaking news or trending topics severely hinder the truth. Existing rumor detection approaches achieve promising performance on the yesterday's news, since there is enough corpus collected from the same domain for model training. However, they are poor at detecting rumors about unforeseen events especially those propagated in different languages due to the lack of training data and prior knowledge (i.e., low-resource regimes). In this paper, we propose an adversarial contrastive learning framework to detect rumors by adapting the features learned from well-resourced rumor data to that of the low-resourced. Our model explicitly overcomes the restriction of domain and/or language usage via language alignment and a novel supervised contrastive training paradigm. Moreover, we develop an adversarial augmentation mechanism to further enhance the robustness of low-resource rumor representation. Extensive experiments conducted on two low-resource datasets collected from real-world microblog platforms demonstrate that our framework achieves much better performance than state-of-the-art methods and exhibits a superior capacity for detecting rumors at early stages.