Miracle-dz/UniFAR-FTUs huggingface dataset:
LCP-dropout: Compression-based Multiple Subword Segmentation for Neural Machine Translation

In this study, we propose a simple and effective preprocessing method for subword segmentation based on a data compression algorithm. Compression-based subword segmentation has recently attracted significant attention as a preprocessing method for training data in Neural Machine Translation. Among them, BPE/BPE-dropout is one of the fastest and most effective method compared to conventional approaches. However, compression-based approach has a drawback in that generating multiple segmentations is difficult due to the determinism. To overcome this difficulty, we focus on a probabilistic string algorithm, called locally-consistent parsing (LCP), that has been applied to achieve optimum compression. Employing the probabilistic mechanism of LCP, we propose LCP-dropout for multiple subword segmentation that improves BPE/BPE-dropout, and show that it outperforms various baselines in learning from especially small training data.
Evoking realistic affective touch experiences in virtual reality

This study aims to better understand the emotional and physiological correlates of being caressed in VR depending on the type of multisensory feedback provided and the animate or inanimate nature of the virtual representation that touches an embodied virtual body. We evaluated how pleasure, arousal, embodiment, and the illusion of being touched in VR were influenced by the inclusion of only visual feedback compared to visuotactile stimulation conditions, where participants, in addition to seeing an avatar or feather caressing their virtual bodies, also perceived congruent mid-air ultrasonic tactile stimulation or real interpersonal touch. We found that visuotactile feedback, either based on ultrasound or real interpersonal touch, boosts the illusion of being affectively touched and embodied in a virtual body compared to conditions only based on visual feedback. However, real interpersonal touch led to the strongest behavioral and emotional responses compared to the other conditions. Moreover, arousal and the desire to withdraw the caressed hand was highest when being touched by a female avatar compared to a virtual feather. Female participants reported a stronger illusion of being caressed in VR compared to males. Overall, this study advances knowledge of the emotional and physiological impact of affective touch in VR.
An EF2X Allocation Protocol for Restricted Additive Valuations

We study the problem of fairly allocating a set of indivisible goods to a set of n agents. Envy-freeness up to any good (EFX) criterion (which requires that no agent prefers the bundle of another agent after the removal of any single good) is known to be a remarkable analogue of envy-freeness when the resource is a set of indivisible goods. In this paper, we investigate EFX for restricted additive valuations, that is, every good has a non-negative value, and every agent is interested in only some of the goods. We introduce a natural relaxation of EFX called EFkX which requires that no agent envies another agent after the removal of any k goods. Our main contribution is an algorithm that finds a complete (i.e., no good is discarded) EF2X allocation for restricted additive valuations. In our algorithm we devise new concepts, namely configuration and envy-elimination that might be of independent interest. We also use our new tools to find an EFX allocation for restricted additive valuations that discards at most n/2 -1 goods.
MSCTD: A Multimodal Sentiment Chat Translation Dataset

Multimodal machine translation and textual chat translation have received considerable attention in recent years. Although the conversation in its natural form is usually multimodal, there still lacks work on multimodal machine translation in conversations. In this work, we introduce a new task named Multimodal Chat Translation (MCT), aiming to generate more accurate translations with the help of the associated dialogue history and visual context. To this end, we firstly construct a Multimodal Sentiment Chat Translation Dataset (MSCTD) containing 142,871 English-Chinese utterance pairs in 14,762 bilingual dialogues. Each utterance pair, corresponding to the visual context that reflects the current conversational scene, is annotated with a sentiment label. Then, we benchmark the task by establishing multiple baseline systems that incorporate multimodal and sentiment features for MCT. Preliminary experiments on two language directions (English-Chinese) verify the potential of contextual and multimodal information fusion and the positive impact of sentiment on the MCT task. Additionally, we provide a new benchmark on multimodal dialogue sentiment analysis with the constructed MSCTD. Our work can facilitate researches on both multimodal chat translation and multimodal dialogue sentiment analysis.
Neural-Progressive Hedging: Enforcing Constraints in Reinforcement Learning with Stochastic Programming

We propose a framework, called neural-progressive hedging (NP), that leverages stochastic programming during the online phase of executing a reinforcement learning (RL) policy. The goal is to ensure feasibility with respect to constraints and risk-based objectives such as conditional value-at-risk (CVaR) during the execution of the policy, using probabilistic models of the state transitions to guide policy adjustments. The framework is particularly amenable to the class of sequential resource allocation problems since feasibility with respect to typical resource constraints cannot be enforced in a scalable manner. The NP framework provides an alternative that adds modest overhead during the online phase. Experimental results demonstrate the efficacy of the NP framework on two continuous real-world tasks: (i) the portfolio optimization problem with liquidity constraints for financial planning, characterized by non-stationary state distributions; and (ii) the dynamic repositioning problem in bike sharing systems, that embodies the class of supply-demand matching problems. We show that the NP framework produces policies that are better than deep RL and other baseline approaches, adapting to non-stationarity, whilst satisfying structural constraints and accommodating risk measures in the resulting policies. Additional benefits of the NP framework are ease of implementation and better explainability of the policies.
Out of Hypervisor (OoH): When Nested Virtualization Becomes Practical

This paper introduces Out of Hypervisor (OoH), a new research axis close to nested virtualization. Instead of emulating a full virtual hardware inside a VM to support a hypervisor, the OoH principle is to individually expose current hypervisor-oriented hardware virtualization features to the guest OS so that its processes could also take benefit from those features. In fact, several hardware virtualization features such as Intel PML, SPP, CAT, and EPT which currently can only be used by the hypervisor also be beneficial for processes that run inside the VM. We illustrate OoH with Intel PML (Page Modification Logging), a feature which allows efficient dirty page tracking for improving VM live migration. According to the fact that dirty page tracking is at the heart of process checkpointing (CRIU) and concurrent garbage collection (Boehm), we present two OoH PML designs namely Shadow PML (SPML) and Extended PML (EPML). The former requires no hardware changes but incurs significant overhead, justifying EPML which extends PML. We evaluated and compared SPML and EPML with /proc and userfaultfd,t wo default solutions in Linux. We do this using a key-value store database as the benchmark. The results show that EPML reduces CRIU checkpointing time by about 14% while leading to a negligible overhead (of about 0.5%) compared to SPML, /proc, and userfaultfd.
Joint Camera Intrinsic and LiDAR-Camera Extrinsic Calibration

Sensor-based environmental perception is a crucial step for autonomous driving systems, for which an accurate calibration between multiple sensors plays a critical role. For the calibration of LiDAR and camera, the existing method is generally to calibrate the intrinsic of the camera first and then calibrate the extrinsic of the LiDAR and camera. If the camera's intrinsic is not calibrated correctly in the first stage, it is not easy to calibrate the LiDAR-camera extrinsic accurately. Due to the complex internal structure of the camera and the lack of an effective quantitative evaluation method for the camera's intrinsic calibration, in the actual calibration, the accuracy of extrinsic parameter calibration is often reduced due to the tiny error of the camera's intrinsic parameters. To this end, we propose a novel target-based joint calibration method of the camera intrinsic and LiDAR-camera extrinsic parameters. Firstly, we design a novel calibration board pattern, adding four circular holes around the checkerboard for locating the LiDAR pose. Subsequently, a cost function defined under the reprojection constraints of the checkerboard and circular holes features is designed to solve the camera's intrinsic parameters, distortion factor, and LiDAR-camera extrinsic parameter. In the end, quantitative and qualitative experiments are conducted in actual and simulated environments, and the result shows the proposed method can achieve accuracy and robust performance. The open-source code is available at https://github.com/OpenCalib/JointCalib.
Feasibility of Material Decomposition-based Photon-counting CT Thermometry

Over the past decades, thermal ablation procedures such as high intensity focused ultrasound (HIFU) have been developed vaporize cancerous tissues in a focal area. Thermal ablation has the potential to non-invasively eliminate tumors and treat other medical conditions with high efficacy and better patient outcomes. However, it is necessary for treatment to be coupled with real time temperature monitoring in order to deliver sufficient thermal dosage to the target while sparing surrounding, healthy tissues. To this end, computed tomography (CT) has been explored to offer fine spatial resolution at a rapid speed. However, current CT thermometry techniques are sensitive to heterogeneous tissue properties, imaging noise, and artifacts. To address this challenge, this paper utilizes the emerging photon-counting CT technology, performs tomographic thermometry based on material decomposition, and obtains excellent results in realistic numerical simulation. Specifically, three algorithms were designed to decouple material composition and thermal expansion from spectral CT reconstruction and compared in a comparative study. The best algorithm, referred to as the one-step algorithm, uniquely finds both material decomposition and the associated temperature at the same time in a closed form solution, which is robust to changes in tissue composition and generates temperature predictions with centigrade accuracy under realistic CT image noise levels.
On Inhibition of the Rayleigh-Taylor Instability by a Horizontal Magnetic Field in 2D Non-Resistive MHD Fluids: The Viscous Case

It is still open whether the phenomenon of inhibition of Rayleigh--Taylor (RT) instability by a horizontal magnetic field can be mathematically verified for a non-resistive \emph{viscous} magnetohydrodynamic (MHD) fluid in a two-dimensional (2D) horizontal slab domain, since it was roughly proved in the linearized case by Wang in \cite{WYC}. In this paper, we prove such inhibition phenomenon by the (nonlinear) inhomogeneous, incompressible, \emph{viscous case} with \emph{Navier (slip) boundary condition}. More precisely, we show that there is a critical number of field strength $m_{\mm{C}}$, such that if the strength $|m|$ of a horizontal magnetic field is bigger than $m_{\mm{C}}$, then the small perturbation solution around the magnetic RT equilibrium state is {algebraically} stable in time. In addition, we also provide a nonlinear instability result for the case $|m|\in[0, m_{\mm{C}})$. The instability result presents that a horizontal magnetic field can not inhibit the RT instability, if it's strength is too small.
Almost Tight Approximation Hardness for Single-Source Directed k-Edge-Connectivity

In the $k$-connected directed Steiner tree problem ($k$-DST), we are given an $n$-vertex directed graph $G=(V,E)$ with edge costs, a connectivity requirement $k$, a root $r\in V$ and a set of terminals $T\subseteq V$. The goal is to find a minimum-cost subgraph $H\subseteq G$ that has $k$ internally disjoint paths from the root vertex $r$ to every terminal $t\in T$. In this paper, we show the approximation hardness of $k$-DST for various parameters, which thus close some long-standing open problems. - $\Omega\left(|T|/\log |T|\right)$-approximation hardness, which holds under the standard assumption $\mathrm{NP}\neq \mathrm{ZPP}$. The inapproximability ratio is tightened to $\Omega\left(|T|\right)$ under the Strongish Planted Clique Hypothesis [Manurangsi, Rubinstein and Schramm, ITCS 2021]. The latter hardness result matches the approximation ratio of $|T|$ obtained by a trivial approximation algorithm, thus closing the long-standing open problem. - $\Omega\left(\sqrt{2}^k / k\right)$-approximation hardness for the general case of $k$-DST under the assumption $\mathrm{NP}\neq\mathrm{ZPP}$. This is the first hardness result known for survivable network design problems with an inapproximability ratio exponential in $k$. - $\Omega\left((k/L)^{L/4}\right)$-approximation hardness for $k$-DST on $L$-layered graphs for $L\le O\left(\log n\right)$. This almost matches the approximation ratio of $O(k^{L-1}\cdot L \cdot \log |T|)$ achieving in $O\left(n^L\right)$-time due to Laekhanukit [ICALP`16].
Split HE: Fast Secure Inference Combining Split Learning and Homomorphic Encryption

This work presents a novel protocol for fast secure inference of neural networks applied to computer vision applications. It focuses on improving the overall performance of the online execution by deploying a subset of the model weights in plaintext on the client's machine, in the fashion of SplitNNs. We evaluate our protocol on benchmark neural networks trained on the CIFAR-10 dataset using SEAL via TenSEAL and discuss runtime and security performances. Empirical security evaluation using Membership Inference and Model Extraction attacks showed that the protocol was more resilient under the same attacks than a similar approach also based on SplitNN. When compared to related work, we demonstrate improvements of 2.5x-10x for the inference time and 14x-290x in communication costs.
Entanglement spectrum and quantum phase diagram of the long-range XXZ chain

Entanglement is a central feature of many-body quantum systems and plays a unique role in quantum phase transitions. In many cases, the entanglement spectrum, which represents the spectrum of the density matrix of a bipartite system, contains valuable information beyond the sole entanglement entropy. Here we investigate the entanglement spectrum of the long-range XXZ model. We show that within the critical phase it exhibits a remarkable self-similarity. The breakdown of self-similarity and the transition away from a Luttinger liquid is consistent with renormalization group theory. Combining the two, we are able to determine the quantum phase diagram of the model and locate the corresponding phase transitions. Our results are confirmed by numerically-exact calculations using tensor-network techniques. Moreover, we show that the self-similar rescaling extends to the geometrical entanglement as well as the Luttinger parameter in the critical phase. Our results pave the way to further studies of entanglement properties in long-range quantum models.
A Computer Vision-assisted Approach to Automated Real-Time Road Infrastructure Management

Accurate automated detection of road pavement distresses is critical for the timely identification and repair of potentially accident-inducing road hazards such as potholes and other surface-level asphalt cracks. Deployment of such a system would be further advantageous in low-resource environments where lack of government funding for infrastructure maintenance typically entails heightened risks of potentially fatal vehicular road accidents as a result of inadequate and infrequent manual inspection of road systems for road hazards. To remedy this, a recent research initiative organized by the Institute of Electrical and Electronics Engineers ("IEEE") as part of their 2020 Global Road Damage Detection ("GRDC") Challenge published in May 2020 a novel 21,041 annotated image dataset of various road distresses calling upon academic and other researchers to submit innovative deep learning-based solutions to these road hazard detection problems. Making use of this dataset, we propose a supervised object detection approach leveraging You Only Look Once ("YOLO") and the Faster R-CNN frameworks to detect and classify road distresses in real-time via a vehicle dashboard-mounted smartphone camera, producing 0.68 F1-score experimental results ranking in the top 5 of 121 teams that entered this challenge as of December 2021.
Blind Image Super Resolution with Semantic-Aware Quantized Texture Prior

A key challenge of blind image super resolution is to recover realistic textures for low-resolution images with unknown degradations. Most recent works completely rely on the generative ability of GANs, which are difﬁcult to train. Other methods resort to high-resolution image references that are usually not available. In this work, we propose a novel framework, denoted as QuanTexSR , to restore realistic textures with the Quant ized Tex ture Priors encoded in Vector Quantized GAN. The QuanTexSR generates textures by aligning the textureless content features to the quantized feature vectors, i.e., a pretrained feature codebook. Speciﬁcally, QuanTexSR formulates the texture generation as a feature matching problem between textureless features and a pretrained feature codebook. The ﬁnal textures are then generated by the quantized features from the codebook. Since features in the codebook have shown the ability to generate natural textures in the pretrain stage, QuanTexSR can generate rich and realistic textures with the pretrained codebook as texture priors. Moreover, we propose a semantic regularization technique that regularizes the pre-training of the codebook using clusters of features extracted from the pretrained VGG19 network. This further improves texture generation with semantic context. Experiments demonstrate that the proposed QuanTexSR can generate competitive or better textures than previous approaches.
Large-Scale 3D Semantic Reconstruction for Automated Driving Vehicles with Adaptive Truncated Signed Distance Function

The Large-scale 3D reconstruction, texturing and semantic mapping are nowadays widely used for automated driving vehicles, virtual reality and automatic data generation. However, most approaches are developed for RGB-D cameras with colored dense point clouds and not suitable for large-scale outdoor environments using sparse LiDAR point clouds. Since a 3D surface can be usually observed from multiple camera images with different view poses, an optimal image patch selection for the texturing and an optimal semantic class estimation for the semantic mapping are still challenging.To address these problems, we propose a novel 3D reconstruction, texturing and semantic mapping system using LiDAR and camera sensors. An Adaptive Truncated Signed Distance Function is introduced to describe surfaces implicitly, which can deal with different LiDAR point sparsities and improve model quality. The from this implicit function extracted triangle mesh map is then textured from a series of registered camera images by applying an optimal image patch selection strategy. Besides that, a Markov Random Field-based data fusion approach is proposed to estimate the optimal semantic class for each triangle mesh. Our approach is evaluated on a synthetic dataset, the KITTI dataset and a dataset recorded with our experimental vehicle. The results show that the 3D models generated using our approach are more accurate in comparison to using other state-of-the-art approaches. The texturing and semantic mapping achieve also very promising results.
Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations

End-to-end learning of dynamical systems with black-box models, such as neural ordinary differential equations (ODEs), provides a flexible framework for learning dynamics from data without prescribing a mathematical model for the dynamics. Unfortunately, this flexibility comes at the cost of understanding the dynamical system, for which ODEs are used ubiquitously. Further, experimental data are collected under various conditions (inputs), such as treatments, or grouped in some way, such as part of sub-populations. Understanding the effects of these system inputs on system outputs is crucial to have any meaningful model of a dynamical system. To that end, we propose a structured latent ODE model that explicitly captures system input variations within its latent representation. Building on a static latent variable specification, our model learns (independent) stochastic factors of variation for each input to the system, thus separating the effects of the system inputs in the latent space. This approach provides actionable modeling through the controlled generation of time-series data for novel input combinations (or perturbations). Additionally, we propose a flexible approach for quantifying uncertainties, leveraging a quantile regression formulation. Results on challenging biological datasets show consistent improvements over competitive baselines in the controlled generation of observational data and inference of biologically meaningful system inputs.
DialFRED: Dialogue-Enabled Agents for Embodied Instruction Following

Language-guided Embodied AI benchmarks requiring an agent to navigate an environment and manipulate objects typically allow one-way communication: the human user gives a natural language command to the agent, and the agent can only follow the command passively. We present DialFRED, a dialogue-enabled embodied instruction following benchmark based on the ALFRED benchmark. DialFRED allows an agent to actively ask questions to the human user; the additional information in the user's response is used by the agent to better complete its task. We release a human-annotated dataset with 53 K task-relevant questions and answers and an oracle to answer questions. To tackle DialFRED, we propose a questioner-performer framework wherein the questioner is pre-trained with the human-annotated data and fine-tuned with reinforcement learning. Experimental results show that asking the right questions leads to significantly improved task performance. We make DialFRED publicly available and encourage researchers to propose and evaluate their solutions to building dialog-enabled embodied agents: https://github.com/xfgao/DialFRED.