OpenRaiser/Intern-Atlas huggingface dataset:
Voice-Based Conversational Agents for self-reporting fluid consumption and sleep quality
Venue: arXiv cs.HC 2022
Year: 2022

Intelligent conversational agents and virtual assistants, such as chatbots and voice assistants, have the potential of augmenting health service capacity to screen symptoms and deliver healthcare interventions. In this paper, we developed voice-based conversational agents (VCAs) in the Google Actions Console to deliver periodic self-assessment health surveys. The focus of this paper is to accommodate self-monitoring for patients with specific fluid consumption requirements or sleep disorders. Our VCAs, named FluidMonitor and Sleepy, have been tested to integrate naturally into a patient's daily lifestyle for the purpose of providing useful interventions. We show the functionality of our Google Actions and discuss the considerations for using VCAs as an at-home self-reporting survey technique. User testing showed satisfaction with the ease of use, likeability, and burden level of the VCAs.
Video Violence Recognition and Localization Using a Semi-Supervised Hard Attention Model
Venue: arXiv cs.CV 2022
Year: 2022

The significant growth of surveillance camera networks necessitates scalable AI solutions to efficiently analyze the large amount of video data produced by these networks. As a typical analysis performed on surveillance footage, video violence detection has recently received considerable attention. The majority of research has focused on improving existing methods using supervised methods, with little, if any, attention to the semi-supervised learning approaches. In this study, a reinforcement learning model is introduced that can outperform existing models through a semi-supervised approach. The main novelty of the proposed method lies in the introduction of a semi-supervised hard attention mechanism. Using hard attention, the essential regions of videos are identified and separated from the non-informative parts of the data. A model's accuracy is improved by removing redundant data and focusing on useful visual information in a higher resolution. Implementing hard attention mechanisms using semi-supervised reinforcement learning algorithms eliminates the need for attention annotations in video violence datasets, thus making them readily applicable. The proposed model utilizes a pre-trained I3D backbone to accelerate and stabilize the training process. The proposed model achieved state-of-the-art accuracy of 90.4% and 98.7% on RWF and Hockey datasets, respectively.
Systems for Memory Disaggregation: Challenges & Opportunities
Venue: arXiv cs.DC 2022
Year: 2022

Memory disaggregation addresses memory imbalance in a cluster by decoupling CPU and memory allocations of applications while also increasing the effective memory capacity for (memory-intensive) applications beyond the local memory limit imposed by traditional fixed-capacity servers. As the network speeds in the tightly-knit environments like modern datacenters inch closer to the DRAM speeds, there has been a recent proliferation of work in this space ranging from software solutions that pool memory of traditional servers for the shared use of the cluster to systems targeting the memory disaggregation in the hardware. In this report, we look at some of these recent memory disaggregation systems and study the important factors that guide their design, such as the interface through which the memory is exposed to the application, their runtime design and relevant optimizations to retain the near-native application performance, various approaches they employ in managing cluster memory to maximize utilization, etc. and we analyze the associated trade-offs. We conclude with a discussion on some open questions and potential future directions that can render disaggregation more amenable for adoption.
Multi-task head pose estimation in-the-wild
Venue: arXiv cs.CV 2022
Year: 2022

We present a deep learning-based multi-task approach for head pose estimation in images. We contribute with a network architecture and training strategy that harness the strong dependencies among face pose, alignment and visibility, to produce a top performing model for all three tasks. Our architecture is an encoder-decoder CNN with residual blocks and lateral skip connections. We show that the combination of head pose estimation and landmark-based face alignment significantly improve the performance of the former task. Further, the location of the pose task at the bottleneck layer, at the end of the encoder, and that of tasks depending on spatial information, such as visibility and alignment, in the final decoder layer, also contribute to increase the final performance. In the experiments conducted the proposed model outperforms the state-of-the-art in the face pose and visibility tasks. By including a final landmark regression step it also produces face alignment results on par with the state-of-the-art.
An integrated heterogeneous computing framework for ensemble simulations of laser-induced ignition
Venue: arXiv cs.CE 2022
Year: 2022

An integrated computational framework is introduced to study complex engineering systems through physics-based ensemble simulations on heterogeneous supercomputers. The framework is primarily designed for the quantitative assessment of laser-induced ignition in rocket engines. We develop and combine an implicit programming system, a compressible reacting flow solver, and a data generation/management strategy on a robust and portable platform. We systematically present this framework using test problems on a hybrid CPU/GPU machine. Efficiency, scalability, and accuracy of the solver are comprehensively assessed with canonical unit problems. Ensemble data management and autoencoding are demonstrated using a canonical diffusion flame case. Sensitivity analysis of the ignition of a turbulent, gaseous fuel jet is performed using a simplified, three-dimensional model combustor. Our approach unifies computer science, physics and engineering, and data science to realize a cross-disciplinary workflow. The framework is exascale-oriented and can be considered a benchmark for future computational science studies of real-world systems.
Malleable Agents for Re-Configurable Robotic Manipulators
Venue: arXiv cs.RO 2022
Year: 2022

Re-configurable robots have more utility and flexibility for many real-world tasks. Designing a learning agent to operate such robots requires adapting to different configurations. Here, we focus on robotic arms with multiple rigid links connected by joints. We propose a deep reinforcement learning agent with sequence neural networks embedded in the agent to adapt to robotic arms that have a varying number of links. Further, with the additional tool of domain randomization, this agent adapts to different configurations. We perform simulations on a 2D N-link arm to show the ability of our network to transfer and generalize efficiently.
StandardSim: A Synthetic Dataset For Retail Environments
Venue: arXiv cs.CV 2022
Year: 2022

Autonomous checkout systems rely on visual and sensory inputs to carry out fine-grained scene understanding in retail environments. Retail environments present unique challenges compared to typical indoor scenes owing to the vast number of densely packed, unique yet similar objects. The problem becomes even more difficult when only RGB input is available, especially for data-hungry tasks such as instance segmentation. To address the lack of datasets for retail, we present StandardSim, a large-scale photorealistic synthetic dataset featuring annotations for semantic segmentation, instance segmentation, depth estimation, and object detection. Our dataset provides multiple views per scene, enabling multi-view representation learning. Further, we introduce a novel task central to autonomous checkout called change detection, requiring pixel-level classification of takes, puts and shifts in objects over time. We benchmark widely-used models for segmentation and depth estimation on our dataset, show that our test set constitutes a difficult benchmark compared to current smaller-scale datasets and that our training set provides models with crucial information for autonomous checkout tasks.
Transformers and the representation of biomedical background knowledge
Venue: arXiv cs.CL 2022
Year: 2022

Specialised transformers-based models (such as BioBERT and BioMegatron) are adapted for the biomedical domain based on publicly available biomedical corpora. As such, they have the potential to encode large-scale biological knowledge. We investigate the encoding and representation of biological knowledge in these models, and its potential utility to support inference in cancer precision medicine - namely, the interpretation of the clinical significance of genomic alterations. We compare the performance of different transformer baselines; we use probing to determine the consistency of encodings for distinct entities; and we use clustering methods to compare and contrast the internal properties of the embeddings for genes, variants, drugs and diseases. We show that these models do indeed encode biological knowledge, although some of this is lost in fine-tuning for specific tasks. Finally, we analyse how the models behave with regard to biases and imbalances in the dataset.
Neural Logic Analogy Learning
Venue: arXiv cs.CL 2022
Year: 2022

Letter-string analogy is an important analogy learning task which seems to be easy for humans but very challenging for machines. The main idea behind current approaches to solving letter-string analogies is to design heuristic rules for extracting analogy structures and constructing analogy mappings. However, one key problem is that it is difficult to build a comprehensive and exhaustive set of analogy structures which can fully describe the subtlety of analogies. This problem makes current approaches unable to handle complicated letter-string analogy problems. In this paper, we propose Neural logic analogy learning (Noan), which is a dynamic neural architecture driven by differentiable logic reasoning to solve analogy problems. Each analogy problem is converted into logical expressions consisting of logical variables and basic logical operations (AND, OR, and NOT). More specifically, Noan learns the logical variables as vector embeddings and learns each logical operation as a neural module. In this way, the model builds computational graph integrating neural network with logical reasoning to capture the internal logical structure of the input letter strings. The analogy learning problem then becomes a True/False evaluation problem of the logical expressions. Experiments show that our machine learning-based Noan approach outperforms state-of-the-art approaches on standard letter-string analogy benchmark datasets.
Linear Model Against Malicious Adversaries with Local Differential Privacy
Venue: arXiv cs.CR 2022
Year: 2022

Scientific collaborations benefit from collaborative learning of distributed sources, but remain difficult to achieve when data are sensitive. In recent years, privacy preserving techniques have been widely studied to analyze distributed data across different agencies while protecting sensitive information. Most existing privacy preserving techniques are designed to resist semi-honest adversaries and require intense computation to perform data analysis. Secure collaborative learning is significantly difficult with the presence of malicious adversaries who may deviates from the secure protocol. Another challenge is to maintain high computation efficiency with privacy protection. In this paper, matrix encryption is applied to encrypt data such that the secure schemes are against malicious adversaries, including chosen plaintext attack, known plaintext attack, and collusion attack. The encryption scheme also achieves local differential privacy. Moreover, cross validation is studied to prevent overfitting without additional communication cost. Empirical experiments on real-world datasets demonstrate that the proposed schemes are computationally efficient compared to existing techniques against malicious adversary and semi-honest model.
Path Planning for the Dynamic UAV-Aided Wireless Systems using Monte Carlo Tree Search
Venue: arXiv cs.NI 2022
Year: 2022

For UAV-aided wireless systems, online path planning attracts much attention recently. To better adapt to the real-time dynamic environment, we, for the first time, propose a Monte Carlo Tree Search (MCTS)-based path planning scheme. In details, we consider a single UAV acts as a mobile server to provide computation tasks offloading services for a set of mobile users on the ground, where the movement of ground users follows a Random Way Point model. Our model aims at maximizing the average throughput under energy consumption and user fairness constraints, and the proposed timesaving MCTS algorithm can further improve the performance. Simulation results show that the proposed algorithm achieves a larger average throughput and a faster convergence performance compared with the baseline algorithms of Q-learning and Deep Q-Network.
Few-shot Learning as Cluster-induced Voronoi Diagrams: A Geometric Approach
Venue: arXiv cs.LG 2022
Year: 2022

Few-shot learning (FSL) is the process of rapid generalization from abundant base samples to inadequate novel samples. Despite extensive research in recent years, FSL is still not yet able to generate satisfactory solutions for a wide range of real-world applications. To confront this challenge, we study the FSL problem from a geometric point of view in this paper. One observation is that the widely embraced ProtoNet model is essentially a Voronoi Diagram (VD) in the feature space. We retrofit it by making use of a recent advance in computational geometry called Cluster-induced Voronoi Diagram (CIVD). Starting from the simplest nearest neighbor model, CIVD gradually incorporates cluster-to-point and then cluster-to-cluster relationships for space subdivision, which is used to improve the accuracy and robustness at multiple stages of FSL. Specifically, we use CIVD (1) to integrate parametric and nonparametric few-shot classifiers; (2) to combine feature representation and surrogate representation; (3) and to leverage feature-level, transformation-level, and geometry-level heterogeneities for a better ensemble. Our CIVD-based workflow enables us to achieve new state-of-the-art results on mini-ImageNet, CUB, and tiered-ImagenNet datasets, with ${\sim}2\%{-}5\%$ improvements upon the next best. To summarize, CIVD provides a mathematically elegant and geometrically interpretable framework that compensates for extreme data insufficiency, prevents overfitting, and allows for fast geometric ensemble for thousands of individual VD. These together make FSL stronger.
A "Distance Matters" Paradox: Facilitating Intra-Team Collaboration Can Harm Inter-Team Collaboration
Venue: arXiv cs.HC 2022
Year: 2022

By identifying the socio-technical conditions required for teams to work effectively remotely, the Distance Matters framework has been influential in CSCW since its introduction in 2000. Advances in collaboration technology and practices have since brought teams increasingly closer to achieving these conditions. This paper presents a ten-month ethnography in a remote organization, where we observed that despite exhibiting excellent remote collaboration, teams paradoxically struggled to collaborate across team boundaries. We extend the Distance Matters framework to account for inter-team collaboration, arguing that challenges analogous to those in the original intra-team framework -- common ground, collaboration readiness, collaboration technology readiness, and coupling of work -- persist but are actualized differently at the inter-team scale. Finally, we identify a fundamental tension between the intra- and inter-team layers: the collaboration technology and practices that help individual teams thrive (e.g., adopting customized collaboration software) can also prompt collaboration challenges in the inter-team layer, and conversely the technology and practices that facilitate inter-team collaboration (e.g., strong centralized IT organizations) can harm practices at the intra-team layer. The addition of the inter-team layer to the Distance Matters framework opens new opportunities for CSCW, where balancing the tension between team and organizational collaboration needs will be a critical technological, operational, and organizational challenge for remote work in the coming decades.
Digital Twin of Wireless Systems: Overview, Taxonomy, Challenges, and Opportunities
Venue: arXiv cs.NI 2022
Year: 2022

Future wireless services must be focused on improving the quality of life by enabling various applications, such as extended reality, brain-computer interaction, and healthcare. These applications have diverse performance requirements (e.g., user-defined quality of experience metrics, latency, and reliability) that are challenging to be fulfilled by existing wireless systems. To meet the diverse requirements of the emerging applications, the concept of a digital twin has been recently proposed. A digital twin uses a virtual representation along with security-related technologies (e.g., blockchain), communication technologies (e.g., 6G), computing technologies (e.g., edge computing), and machine learning, so as to enable the smart applications. In this tutorial, we present a comprehensive overview on digital twins for wireless systems. First, we present an overview of fundamental concepts (i.e., design aspects, high-level architecture, and frameworks) of digital twin of wireless systems. Second, a comprehensive taxonomy is devised for both different aspects. These aspects are twins for wireless and wireless for twins. For the twins for wireless aspect, we consider parameters, such as twin objects design, prototyping, deployment trends, physical devices design, interface design, incentive mechanism, twins isolation, and decoupling. On the other hand, for wireless for twins, parameters such as, twin objects access aspects, security and privacy, and air interface design are considered. Finally, open research challenges and opportunities are presented along with causes and possible solutions.
Using SEQUAL for Identifying Requirements to Ecore Editors
Venue: arXiv cs.SE 2022
Year: 2022

Software engineers who use Model-Driven Development may be using Ecore for their work. Ecore is traditionally edited in Eclipse IDE, but a recent transition to Web tools allows for development of new Ecore editors. To investigate the needed functionality of such modeling tools, the model quality framework SEQUAL has been applied. The paper presents the current results of this task, producing requirements for tool functionality as quality improving means for the following quality aspects: physical, empirical, syntactic, semantic, pragmatic, social and deontic. The result is an extensive list of tool functionality that could be implemented by the Ecore editor developers. Although many requirements are identified, the framework should also help in making trade-offs in case not all requirements can be implemented. In this way the paper both contribute to identifying modeling tool functionality, and to have input to improve SEQUAL as a general model quality framework. Further work will need to be done on the implementation of the tools for properly evaluating this work.
GhostTalk: Interactive Attack on Smartphone Voice System Through Power Line
Venue: arXiv cs.CR 2022
Year: 2022

Inaudible voice command injection is one of the most threatening attacks towards voice assistants. Existing attacks aim at injecting the attack signals over the air, but they require the access to the authorized user's voice for activating the voice assistants. Moreover, the effectiveness of the attacks can be greatly deteriorated in a noisy environment. In this paper, we explore a new type of channel, the power line side-channel, to launch the inaudible voice command injection. By injecting the audio signals over the power line through a modified charging cable, the attack becomes more resilient against various environmental factors and liveness detection models. Meanwhile, the smartphone audio output can be eavesdropped through the modified cable, enabling a highly-interactive attack. To exploit the power line side-channel, we present GhostTalk, a new hidden voice attack that is capable of injecting and eavesdropping simultaneously. Via a quick modification of the power bank cables, the attackers could launch interactive attacks by remotely making a phone call or capturing private information from the voice assistants. GhostTalk overcomes the challenge of bypassing the speaker verification system by stealthily triggering a switch component to simulate the press button on the headphone. In case when the smartphones are charged by an unaltered standard cable, we discover that it is possible to recover the audio signal from smartphone loudspeakers by monitoring the charging current on the power line. To demonstrate the feasibility, we design GhostTalk-SC, an adaptive eavesdropper system targeting smartphones charged in the public USB ports. To correctly recognize the private information in the audio, GhostTalk-SC carefully extracts audio spectra and integrates a neural network model to classify spoken digits in the speech.
Exemplar-Based Contrastive Self-Supervised Learning with Few-Shot Class Incremental Learning
Venue: arXiv cs.LG 2022
Year: 2022

Humans are capable of learning new concepts from only a few (labeled) exemplars, incrementally and continually. This happens within the context that we can differentiate among the exemplars, and between the exemplars and large amounts of other data (unlabeled and labeled). This suggests, in human learning, supervised learning of concepts based on exemplars takes place within the larger context of contrastive self-supervised learning (CSSL) based on unlabeled and labeled data. We discuss extending CSSL (1) to be based mainly on exemplars and only secondly on data augmentation, and (2) to apply to both unlabeled data (a large amount is available in general) and labeled data (a few exemplars can be obtained with valuable supervised knowledge). A major benefit of the extensions is that exemplar-based CSSL, with supervised finetuning, supports few-shot class incremental learning (CIL). Specifically, we discuss exemplar-based CSSL including: nearest-neighbor CSSL, neighborhood CSSL with supervised pretraining, and exemplar CSSL with supervised finetuning. We further discuss using exemplar-based CSSL to facilitate few-shot learning and, in particular, few-shot CIL.
On the Complexity of Coordinated Table Selective Substitution Systems
Venue: arXiv cs.FL 2022
Year: 2022

We investigate computational resources used by Turing machines (TMs) and alternating Turing machines (ATMs) to accept languages generated by coordinated table selective substitution systems with two components. We prove that the class of languages generated by real-time (RL; 0S)-systems, an alternative device to generate lambda-free labeled marked Petri nets languages, can be accepted by nondeterministic TMs in O(log n) space and O(nlog n) time. Consequently, this proper sub-class of Petri nets languages (known also as L-languages) is included in NSPACE(log n). The class of languages generated by (RL; RB)-systems for which the nonterminal alphabet of the RL-grammar is composed of only one symbol and the nonterminal alphabet of the RB-grammar is composed of two symbols, can be accepted by ATMs in O(log n) time and space. Consequently, this proper subclass of one-counter languages generated by one-counter machines with only one control state is included in U_E*-uniform NC1, hence in SPACE(log n).