OpenRaiser/Intern-Atlas huggingface dataset:
The...Tinderverse?: Opportunities and Challenges for User Safety in Extended Reality (XR) Dating Apps
Venue: arXiv cs.HC 2022
Year: 2022

Dating apps such as Tinder have announced plans for a dating metaverse: the incorporation of XR technologies into the online dating process to augment interactions between potential sexual partners across virtual and physical worlds. While the dating metaverse is still in conceptual stages we can forecast significant harms that it may expose daters to given prior research into the frequency and severity of sexual harms facilitated by dating apps as well as harms within social VR environments. In this workshop paper we envision how XR could enrich virtual-to-physical interaction between potential sexual partners and outline harms that it will likely perpetuate as well. We then introduce our ongoing research to preempt such harms: a participatory design study with sexual violence experts and demographics at disproportionate risk of sexual violence to produce mitigative solutions to sexual violence perpetuated by XR-enabled dating apps.
Tightly Seal Your Sensitive Pointers with PACTight
Venue: arXiv cs.CR 2022
Year: 2022

ARM is becoming more popular in desktops and data centers, opening a new realm in terms of security attacks against ARM. ARM has released Pointer Authentication, a new hardware security feature that is intended to ensure pointer integrity with cryptographic primitives. In this paper, we utilize Pointer Authentication (PA) to build a novel scheme to completely prevent any misuse of security-sensitive pointers. We propose PACTight to tightly seal these pointers. PACTight utilizes a strong and unique modifier that addresses the current issues with the state-of-the-art PA defense mechanisms. We implement four defenses based on the PACTight mechanism. Our security and performance evaluation results show that PACTight defenses are more efficient and secure. Using real PA instructions, we evaluated PACTight on 30 different applications, including NGINX web server, with an average performance overhead of 4.07% even when enforcing our strongest defense. PACTight demonstrates its effectiveness and efficiency with real PA instructions on real hardware.
A Study of Reinforcement Learning Algorithms for Aggregates of Minimalistic Robots
Venue: arXiv cs.RO 2022
Year: 2022

The aim of this paper is to study how to apply deep reinforcement learning for the control of aggregates of minimalistic robots. We define aggregates as groups of robots with a physical connection that compels them to form a specified shape. In our case, the robots are pre-attached to an object that must be collectively transported to a known location. Minimalism, in our setting, stems from the barebone capabilities we assume: The robots can sense the target location and the immediate obstacles, but lack the means to communicate explicitly through, e.g., message-passing. In our setting, communication is implicit, i.e., mediated by aggregated push-and-pull on the object exerted by each robot. We analyze the ability to reach coordinated behavior of four well-known algorithms for deep reinforcement learning (DQN, DDQN, DDPG, and TD3). Our experiments include robot failures and different types of environmental obstacles. We compare the performance of the best control strategies found, highlighting strengths and weaknesses of each of the considered training algorithms.
Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Support
Venue: arXiv cs.CL 2022
Year: 2022

Advances in artificial intelligence (AI) are enabling systems that augment and collaborate with humans to perform simple, mechanistic tasks like scheduling meetings and grammar-checking text. However, such Human-AI collaboration poses challenges for more complex, creative tasks, such as carrying out empathic conversations, due to difficulties of AI systems in understanding complex human emotions and the open-ended nature of these tasks. Here, we focus on peer-to-peer mental health support, a setting in which empathy is critical for success, and examine how AI can collaborate with humans to facilitate peer empathy during textual, online supportive conversations. We develop Hailey, an AI-in-the-loop agent that provides just-in-time feedback to help participants who provide support (peer supporters) respond more empathically to those seeking help (support seekers). We evaluate Hailey in a non-clinical randomized controlled trial with real-world peer supporters on TalkLife (N=300), a large online peer-to-peer support platform. We show that our Human-AI collaboration approach leads to a 19.60% increase in conversational empathy between peers overall. Furthermore, we find a larger 38.88% increase in empathy within the subsample of peer supporters who self-identify as experiencing difficulty providing support. We systematically analyze the Human-AI collaboration patterns and find that peer supporters are able to use the AI feedback both directly and indirectly without becoming overly reliant on AI while reporting improved self-efficacy post-feedback. Our findings demonstrate the potential of feedback-driven, AI-in-the-loop writing systems to empower humans in open-ended, social, creative tasks such as empathic conversations.
Shifted Chunk Encoder for Transformer Based Streaming End-to-End ASR
Venue: arXiv cs.SD 2022
Year: 2022

Currently, there are mainly three kinds of Transformer encoder based streaming End to End (E2E) Automatic Speech Recognition (ASR) approaches, namely time-restricted methods, chunk-wise methods, and memory-based methods. Generally, all of them have limitations in aspects of linear computational complexity, global context modeling, and parallel training. In this work, we aim to build a model to take all these three advantages for streaming Transformer ASR. Particularly, we propose a shifted chunk mechanism for the chunk-wise Transformer which provides cross-chunk connections between chunks. Therefore, the global context modeling ability of chunk-wise models can be significantly enhanced while all the original merits inherited. We integrate this scheme with the chunk-wise Transformer and Conformer, and identify them as SChunk-Transformer and SChunk-Conformer, respectively. Experiments on AISHELL-1 show that the SChunk-Transformer and SChunk-Conformer can respectively achieve CER 6.43% and 5.77%. And the linear complexity makes them possible to train with large batches and infer more efficiently. Our models can significantly outperform their conventional chunk-wise counterparts, while being competitive, with only 0.22 absolute CER drop, when compared with U2 which has quadratic complexity. A better CER can be achieved if compared with existing chunk-wise or memory-based methods, such as HS-DACS and MMA. Code is released.
Unified Performance Assessment of Optical Wireless Communication over Multi-Layer Underwater Channels
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we model the multi-layer vertical underwater link as a cascaded channel and unify the performance analysis for the underwater optical communication (UWOC) system using generalized Gamma (GG), exponential GG (EGG), exponentiated Weibull (EW), and Gamma-Gamma (ΓΓ) oceanic turbulence models. We derive unified analytical expressions for probability density function (PDF) and cumulative distribution function (CDF) for the signal-to-noise ratios (SNR) considering independent and non-identical (i.ni.d.) turbulent models and zero bore-sight model for pointing errors. We develop performance metrics of the considered UWOC system using outage probability, average bit error rate (BER), and ergodic capacity with asymptotic expressions for outage probability and average BER. We develop the diversity order of the proposed system to provide a better insight into the system performance at a high SNR. We also integrate a terrestrial OWC (TOWC) subjected to the combined effect of generalized Malaga atmospheric turbulence, fog-induced random path gain, and pointing errors to communicate with the UWOC link using the fixed-gain amplify-and-forward (AF) relaying. We analyze the performance of the mixed TWOC and multi-layer UWOC system by deriving PDF, CDF, outage probability, and average BER using the bivariate Fox H-function. We use Monte-Carlo simulation results to validate our exact and asymptotic expressions and demonstrate the performance of the considered underwater UWOC system using measurement-based parametric data available for turbulent oceanic channels.
MatchNorm: Learning-based Point Cloud Registration for 6D Object Pose Estimation in the Real World
Venue: arXiv cs.CV 2022
Year: 2022

In this work, we tackle the task of estimating the 6D pose of an object from point cloud data. While recent learning-based approaches to addressing this task have shown great success on synthetic datasets, we have observed them to fail in the presence of real-world data. We thus analyze the causes of these failures, which we trace back to the difference between the feature distributions of the source and target point clouds, and the sensitivity of the widely-used SVD-based loss function to the range of rotation between the two point clouds. We address the first challenge by introducing a new normalization strategy, Match Normalization, and the second via the use of a loss function based on the negative log likelihood of point correspondences. Our two contributions are general and can be applied to many existing learning-based 3D object registration frameworks, which we illustrate by implementing them in two of them, DCP and IDAM. Our experiments on the real-scene TUD-L, LINEMOD and Occluded-LINEMOD datasets evidence the benefits of our strategies. They allow for the first time learning-based 3D object registration methods to achieve meaningful results on real-world data. We therefore expect them to be key to the future development of point cloud registration methods.
In-N-Out Generative Learning for Dense Unsupervised Video Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we focus on unsupervised learning for Video Object Segmentation (VOS) which learns visual correspondence (i.e., the similarity between pixel-level features) from unlabeled videos. Previous methods are mainly based on the contrastive learning paradigm, which optimize either in image level or pixel level. Image-level optimization (e.g., the spatially pooled feature of ResNet) learns robust high-level semantics but is sub-optimal since the pixel-level features are optimized implicitly. By contrast, pixel-level optimization is more explicit, however, it is sensitive to the visual quality of training data and is not robust to object deformation. To complementarily perform these two levels of optimization in a unified framework, we propose the In-aNd-Out (INO) generative learning from a purely generative perspective with the help of naturally designed class tokens and patch tokens in Vision Transformer (ViT). Specifically, for image-level optimization, we force the out-view imagination from local to global views on class tokens, which helps capture high-level semantics, and we name it as out-generative learning. As to pixel-level optimization, we perform in-view masked image modeling on patch tokens, which recovers the corrupted parts of an image via inferring its fine-grained structure, and we term it as in-generative learning. To discover the temporal information better, we additionally force the inter-frame consistency from both feature and affinity matrix levels. Extensive experiments on DAVIS-2017 val and YouTube-VOS 2018 val show that our INO outperforms previous state-of-the-art methods by significant margins. Code is available: https://github.com/pansanity666/INO_VOS
Evolving Multi-Label Fuzzy Classifier
Venue: arXiv cs.LG 2022
Year: 2022

Multi-label classification has attracted much attention in the machine learning community to address the problem of assigning single samples to more than one class at the same time. We propose an evolving multi-label fuzzy classifier (EFC-ML) which is able to self-adapt and self-evolve its structure with new incoming multi-label samples in an incremental, single-pass manner. It is based on a multi-output Takagi-Sugeno type architecture, where for each class a separate consequent hyper-plane is defined. The learning procedure embeds a locally weighted incremental correlation-based algorithm combined with (conventional) recursive fuzzily weighted least squares and Lasso-based regularization. The correlation-based part ensures that the interrelations between class labels, a specific well-known property in multi-label classification for improved performance, are preserved properly; the Lasso-based regularization reduces the curse of dimensionality effects in the case of a higher number of inputs. Antecedent learning is achieved by product-space clustering and conducted for all class labels together, which yields a single rule base, allowing a compact knowledge view. Furthermore, our approach comes with an online active learning (AL) strategy for updating the classifier on just a number of selected samples, which in turn makes the approach applicable for scarcely labelled streams in applications, where the annotation effort is typically expensive. Our approach was evaluated on several data sets from the MULAN repository and showed significantly improved classification accuracy compared to (evolving) one-versus-rest or classifier chaining concepts. A significant result was that, due to the online AL method, a 90\% reduction in the number of samples used for classifier updates had little effect on the accumulated accuracy trend lines compared to a full update in most data set cases.
Dynamic-subarray with Fixed Phase Shifters for Energy-efficient Terahertz Hybrid Beamforming under Partial CSI
Venue: arXiv cs.IT 2022
Year: 2022

Terahertz (THz) communications are regarded as a pillar technology for the 6G systems, by offering multi-ten-GHz bandwidth. To overcome the huge propagation loss while reducing the hardware complexity, THz ultra-massive (UM) MIMO systems with hybrid beamforming are proposed to offer high array gain. Notably, the adjustable-phase-shifters considered in most existing hybrid beamforming studies are power-hungry and difficult to realize in the THz band. Moreover, due to the ultra-massive antennas, full channel-state-information (CSI) is challenging to obtain. To address these practical concerns, in this paper, an energy-efficient dynamic-subarray with fixed-phase-shifters (DS-FPS) architecture is proposed for THz hybrid beamforming. To compensate for the spectral efficiency loss caused by the fixed-phase of FPS, a switch network is inserted to enable dynamic connections. In addition, by considering the partial CSI, we propose a row-successive-decomposition (RSD) algorithm to design the hybrid beamforming matrices for DS-FPS. A row-by-row (RBR) algorithm is further proposed to reduce computational complexity. Extensive simulation results show that, the proposed DS-FPS architecture with the RSD and RBR algorithms achieves much higher energy efficiency than the existing architectures. Moreover, the DS-FPS architecture with partial CSI achieves 97% spectral efficiency of that with full CSI.
Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language Practice
Venue: arXiv cs.SE 2022
Year: 2022

Due to the migration megatrend, efficient and effective second-language acquisition is vital. One proposed solution involves AI-enabled conversational agents for person-centered interactive language practice. We present results from ongoing action research targeting quality assurance of proprietary generative dialog models trained for virtual job interviews. The action team elicited a set of 38 requirements for which we designed corresponding automated test cases for 15 of particular interest to the evolving solution. Our results show that six of the test case designs can detect meaningful differences between candidate models. While quality assurance of natural language processing applications is complex, we provide initial steps toward an automated framework for machine learning model selection in the context of an evolving conversational agent. Future work will focus on model selection in an MLOps setting.
Semantic properties of English nominal pluralization: Insights from word embeddings
Venue: arXiv cs.CL 2022
Year: 2022

Semantic differentiation of nominal pluralization is grammaticalized in many languages. For example, plural markers may only be relevant for human nouns. English does not appear to make such distinctions. Using distributional semantics, we show that English nominal pluralization exhibits semantic clusters. For instance, pluralization of fruit words is more similar to one another and less similar to pluralization of other semantic classes. Therefore, reduction of the meaning shift in plural formation to the addition of an abstract plural meaning is too simplistic. A semantically informed method, called CosClassAvg, is introduced that outperforms pluralization methods in distributional semantics which assume plural formation amounts to the addition of a fixed plural vector. In comparison with our approach, a method from compositional distributional semantics, called FRACSS, predicted plural vectors that were more similar to the corpus-extracted plural vectors in terms of direction but not vector length. A modeling study reveals that the observed difference between the two predicted semantic spaces by CosClassAvg and FRACSS carries over to how well a computational model of the listener can understand previously unencountered plural forms. Mappings from word forms, represented with triphone vectors, to predicted semantic vectors are more productive when CosClassAvg-generated semantic vectors are employed as gold standard vectors instead of FRACSS-generated vectors.
ZK-SecreC: a Domain-Specific Language for Zero Knowledge Proofs
Venue: arXiv cs.PL 2022
Year: 2022

We present ZK-SecreC, a domain-specific language for zero-knowledge proofs. We present the rationale for its design, its syntax and semantics, and demonstrate its usefulness on the basis of a number of non-trivial examples. The design features a type system, where each piece of data is assigned both a confidentiality and an integrity type, which are not orthogonal to each other. We perform an empiric evaluation of the statements produced by its compiler in terms of their size. We also show the integration of the compiler with the implementation of a zero-knowledge proof technique, and evaluate the running time of both Prover and Verifier.
Interactive Audio-text Representation for Automated Audio Captioning with Contrastive Learning
Venue: arXiv cs.SD 2022
Year: 2022

Automated Audio captioning (AAC) is a cross-modal task that generates natural language to describe the content of input audio. Most prior works usually extract single-modality acoustic features and are therefore sub-optimal for the cross-modal decoding task. In this work, we propose a novel AAC system called CLIP-AAC to learn interactive cross-modality representation with both acoustic and textual information. Specifically, the proposed CLIP-AAC introduces an audio-head and a text-head in the pre-trained encoder to extract audio-text information. Furthermore, we also apply contrastive learning to narrow the domain difference by learning the correspondence between the audio signal and its paired captions. Experimental results show that the proposed CLIP-AAC approach surpasses the best baseline by a significant margin on the Clotho dataset in terms of NLP evaluation metrics. The ablation study indicates that both the pre-trained model and contrastive learning contribute to the performance gain of the AAC model.
Integrating Lattice-Free MMI into End-to-End Speech Recognition
Venue: arXiv cs.CL 2022
Year: 2022

In automatic speech recognition (ASR) research, discriminative criteria have achieved superior performance in DNN-HMM systems. Given this success, the adoption of discriminative criteria is promising to boost the performance of end-to-end (E2E) ASR systems. With this motivation, previous works have introduced the minimum Bayesian risk (MBR, one of the discriminative criteria) into E2E ASR systems. However, the effectiveness and efficiency of the MBR-based methods are compromised: the MBR criterion is only used in system training, which creates a mismatch between training and decoding; the on-the-fly decoding process in MBR-based methods results in the need for pre-trained models and slow training speeds. To this end, novel algorithms are proposed in this work to integrate another widely used discriminative criterion, lattice-free maximum mutual information (LF-MMI), into E2E ASR systems not only in the training stage but also in the decoding process. The proposed LF-MMI training and decoding methods show their effectiveness on two widely used E2E frameworks: Attention-Based Encoder-Decoders (AEDs) and Neural Transducers (NTs). Compared with MBR-based methods, the proposed LF-MMI method: maintains the consistency between training and decoding; eschews the on-the-fly decoding process; trains from randomly initialized models with superior training efficiency. Experiments suggest that the LF-MMI method outperforms its MBR counterparts and consistently leads to statistically significant performance improvements on various frameworks and datasets from 30 hours to 14.3k hours. The proposed method achieves state-of-the-art (SOTA) results on Aishell-1 (CER 4.10%) and Aishell-2 (CER 5.02%) datasets. Code is released.
Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems
Venue: arXiv cs.LG 2022
Year: 2022

In data-driven modeling of spatiotemporal phenomena careful consideration often needs to be made in capturing the dynamics of the high wavenumbers. This problem becomes especially challenging when the system of interest exhibits shocks or chaotic dynamics. We present a data-driven modeling method that accurately captures shocks and chaotic dynamics by proposing a novel architecture, stabilized neural ordinary differential equation (ODE). In our proposed architecture, we learn the right-hand-side (RHS) of an ODE by adding the outputs of two NN together where one learns a linear term and the other a nonlinear term. Specifically, we implement this by training a sparse linear convolutional NN to learn the linear term and a dense fully-connected nonlinear NN to learn the nonlinear term. This is in contrast with the standard neural ODE which involves training only a single NN for learning the RHS. We apply this setup to the viscous Burgers equation, which exhibits shocked behavior, and show better short-time tracking and prediction of the energy spectrum at high wavenumbers than a standard neural ODE. We also find that the stabilized neural ODE models are much more robust to noisy initial conditions than the standard neural ODE approach. We also apply this method to chaotic trajectories of the Kuramoto-Sivashinsky equation. In this case, stabilized neural ODEs keep long-time trajectories on the attractor, and are highly robust to noisy initial conditions, while standard neural ODEs fail at achieving either of these results. We conclude by demonstrating how stabilizing neural ODEs provide a natural extension for use in reduced-order modeling by projecting the dynamics onto the eigenvectors of the learned linear term.