OpenRaiser/Intern-Atlas huggingface dataset:
Automated Detection of Patients in Hospital Video Recordings
Venue: arXiv cs.CV 2021
Year: 2021

In a clinical setting, epilepsy patients are monitored via video electroencephalogram (EEG) tests. A video EEG records what the patient experiences on videotape while an EEG device records their brainwaves. Currently, there are no existing automated methods for tracking the patient's location during a seizure, and video recordings of hospital patients are substantially different from publicly available video benchmark datasets. For example, the camera angle can be unusual, and patients can be partially covered with bedding sheets and electrode sets. Being able to track a patient in real-time with video EEG would be a promising innovation towards improving the quality of healthcare. Specifically, an automated patient detection system could supplement clinical oversight and reduce the resource-intensive efforts of nurses and doctors who need to continuously monitor patients. We evaluate an ImageNet pre-trained Mask R-CNN, a standard deep learning model for object detection, on the task of patient detection using our own curated dataset of 45 videos of hospital patients. The dataset was aggregated and curated for this work. We show that without fine-tuning, ImageNet pre-trained Mask R-CNN models perform poorly on such data. By fine-tuning the models with a subset of our dataset, we observe a substantial improvement in patient detection performance, with a mean average precision of 0.64. We show that the results vary substantially depending on the video clip.
Customer Sentiment Analysis using Weak Supervision for Customer-Agent Chat
Venue: arXiv cs.CL 2021
Year: 2021

Prior work on sentiment analysis using weak supervision primarily focuses on different reviews such as movies (IMDB), restaurants (Yelp), products (Amazon).~One under-explored field in this regard is customer chat data for a customer-agent chat in customer support due to the lack of availability of free public data. Here, we perform sentiment analysis on customer chat using weak supervision on our in-house dataset. We fine-tune the pre-trained language model (LM) RoBERTa as a sentiment classifier using weak supervision. Our contribution is as follows:1) We show that by using weak sentiment classifiers along with domain-specific lexicon-based rules as Labeling Functions (LF), we can train a fairly accurate customer chat sentiment classifier using weak supervision. 2) We compare the performance of our custom-trained model with off-the-shelf google cloud NLP API for sentiment analysis. We show that by injecting domain-specific knowledge using LFs, even with weak supervision, we can train a model to handle some domain-specific use cases better than off-the-shelf google cloud NLP API. 3) We also present an analysis of how customer sentiment in a chat relates to problem resolution.
The CSIRO Crown-of-Thorn Starfish Detection Dataset
Venue: arXiv cs.CV 2021
Year: 2021

Crown-of-Thorn Starfish (COTS) outbreaks are a major cause of coral loss on the Great Barrier Reef (GBR) and substantial surveillance and control programs are underway in an attempt to manage COTS populations to ecologically sustainable levels. We release a large-scale, annotated underwater image dataset from a COTS outbreak area on the GBR, to encourage research on Machine Learning and AI-driven technologies to improve the detection, monitoring, and management of COTS populations at reef scale. The dataset is released and hosted in a Kaggle competition that challenges the international Machine Learning community with the task of COTS detection from these underwater images.
Braking and Body Angles Control of an Insect-Computer Hybrid Robot by Electrical Stimulation of Beetle Flight Muscle in Free Flight
Venue: arXiv cs.RO 2021
Year: 2021

While engineers put lots of effort, resources, and time in building insect scale micro aerial vehicles (MAVs) that fly like insects, insects themselves are the real masters of flight. What if we would use living insect as platform for MAV instead? Here, we reported a flight control via electrical stimulation of a flight muscle of an insect-computer hybrid robot, which is the interface of a mountable wireless backpack controller and a living beetle. The beetle uses indirect flight muscles to drive wing flapping and three major direct flight muscles (basalar, subalar and third axilliary (3Ax) muscles) to control the kinematics of the wings for flight maneuver. While turning control was already achieved by stimulating basalar and 3Ax muscles, electrical stimulation of subalar muscles resulted in braking and elevation control in flight. We also demonstrated around 20 degrees of contralateral yaw and roll by stimulating individual subalar muscle. Stimulating both subalar muscles lead to an increase of 20 degrees in pitch and decelerate the flight by 1.5 m/s2 as well as an induce an elevation of 2 m/s2.
Responding to Challenge Call of Machine Learning Model Development in Diagnosing Respiratory Disease Sounds
Venue: arXiv cs.SD 2021
Year: 2021

In this study, a machine learning model was developed for automatically detecting respiratory system sounds such as sneezing and coughing in disease diagnosis. The automatic model and approach development of breath sounds, which carry valuable information, results in early diagnosis and treatment. A successful machine learning model was developed in this study, which was a strong response to the challenge called the "Pfizer digital medicine challenge" on the "OSFHOME" open access platform. "Environmental sound classification" called ESC-50 and AudioSet sound files were used to prepare the dataset. In this dataset, which consisted of three parts, features that effectively showed coughing and sneezing sound analysis were extracted from training, testing and validating samples. Based on the Mel frequency cepstral coefficients (MFCC) feature extraction method, mathematical and statistical features were prepared. Three different classification techniques were considered to perform successful respiratory sound classification in the dataset containing more than 3800 different sounds. Support vector machine (SVM) with radial basis function (RBF) kernels, ensemble aggregation and decision tree classification methods were used as classification techniques. In an attempt to classify coughing and sneezing sounds from other sounds, SVM with RBF kernels was achieved with 83% success.
Testing Robot System Safety by creating Hazardous Human Worker Behavior in Simulation
Venue: arXiv cs.RO 2021
Year: 2021

We introduce a novel simulation-based approach to identify hazards that result from unexpected worker behavior in human-robot collaboration. Simulation-based safety testing must take into account the fact that human behavior is variable and that human error can occur. When only the expected worker behavior is simulated, critical hazards can remain undiscovered. On the other hand, simulating all possible worker behaviors is computationally infeasible. This raises the problem of how to find interesting (i.e., potentially hazardous) worker behaviors given a limited number of simulation runs. We frame this as a search problem in the space of possible worker behaviors. Because this search space can get quite complex, we introduce the following measures: (1) Search space restriction based on workflow-constraints, (2) prioritization of behaviors based on how far they deviate from the nominal behavior, and (3) the use of a risk metric to guide the search towards high-risk behaviors which are more likely to expose hazards. We demonstrate the approach in a collaborative workflow scenario that involves a human worker, a robot arm, and a mobile robot.
Action based Network for Conversation Question Reformulation
Venue: arXiv cs.CL 2021
Year: 2021

Conversation question answering requires the ability to interpret a question correctly. Current models, however, are still unsatisfactory due to the difficulty of understanding the co-references and ellipsis in daily conversation. Even though generative approaches achieved remarkable progress, they are still trapped by semantic incompleteness. This paper presents an action-based approach to recover the complete expression of the question. Specifically, we first locate the positions of co-reference or ellipsis in the question while assigning the corresponding action to each candidate span. We then look for matching phrases related to the candidate clues in the conversation context. Finally, according to the predicted action, we decide whether to replace the co-reference or supplement the ellipsis with the matched information. We demonstrate the effectiveness of our method on both English and Chinese utterance rewrite tasks, improving the state-of-the-art EM (exact match) by 3.9\% and ROUGE-L by 1.0\% respectively on the Restoration-200K dataset.
CoNIC: Colon Nuclei Identification and Counting Challenge 2022
Venue: arXiv cs.CV 2021
Year: 2021

Nuclear segmentation, classification and quantification within Haematoxylin & Eosin stained histology images enables the extraction of interpretable cell-based features that can be used in downstream explainable models in computational pathology (CPath). However, automatic recognition of different nuclei is faced with a major challenge in that there are several different types of nuclei, some of them exhibiting large intra-class variability. To help drive forward research and innovation for automatic nuclei recognition in CPath, we organise the Colon Nuclei Identification and Counting (CoNIC) Challenge. The challenge encourages researchers to develop algorithms that perform segmentation, classification and counting of nuclei within the current largest known publicly available nuclei-level dataset in CPath, containing around half a million labelled nuclei. Therefore, the CoNIC challenge utilises over 10 times the number of nuclei as the previous largest challenge dataset for nuclei recognition. It is important for algorithms to be robust to input variation if we wish to deploy them in a clinical setting. Therefore, as part of this challenge we will also test the sensitivity of each submitted algorithm to certain input variations.
On the Effectiveness of Neural Ensembles for Image Classification with Small Datasets
Venue: arXiv cs.CV 2021
Year: 2021

Deep neural networks represent the gold standard for image classification. However, they usually need large amounts of data to reach superior performance. In this work, we focus on image classification problems with a few labeled examples per class and improve data efficiency by using an ensemble of relatively small networks. For the first time, our work broadly studies the existing concept of neural ensembling in domains with small data, through extensive validation using popular datasets and architectures. We compare ensembles of networks to their deeper or wider single competitors given a total fixed computational budget. We show that ensembling relatively shallow networks is a simple yet effective technique that is generally better than current state-of-the-art approaches for learning from small datasets. Finally, we present our interpretation according to which neural ensembles are more sample efficient because they learn simpler functions.
Spherical Matrix Factorization
Venue: arXiv cs.LG 2021
Year: 2021

Matrix Factorization plays an important role in machine learning such as Non-negative Matrix Factorization, Principal Component Analysis, Dictionary Learning, etc. However, most of the studies aim to minimize the loss by measuring the Euclidean distance, though in some fields, angle distance is known to be more important and critical for analysis. In this paper, we propose a method by adding constraints on factors to unify the Euclidean and angle distance. However, due to non-convexity of the objective and constraints, the optimized solution is not easy to obtain. In this paper we propose a general framework to systematically solve it with provable convergence guarantee with various constraints.
Searching to Learn with Instructional Scaffolding
Venue: arXiv cs.IR 2021
Year: 2021

Search engines are considered the primary tool to assist and empower learners in finding information relevant to their learning goals-be it learning something new, improving their existing skills, or just fulfilling a curiosity. While several approaches for improving search engines for the learning scenario have been proposed, instructional scaffolding has not been studied in the context of search as learning, despite being shown to be effective for improving learning in both digital and traditional learning contexts. When scaffolding is employed, instructors provide learners with support throughout their autonomous learning process. We hypothesize that the usage of scaffolding techniques within a search system can be an effective way to help learners achieve their learning objectives whilst searching. As such, this paper investigates the incorporation of scaffolding into a search system employing three different strategies (as well as a control condition): (I) AQE_{SC}, the automatic expansion of user queries with relevant subtopics; (ii) CURATED_{SC}, the presenting of a manually curated static list of relevant subtopics on the search engine result page; and (iii) FEEDBACK_{SC}, which projects real-time feedback about a user's exploration of the topic space on top of the CURATED_{SC} visualization. To investigate the effectiveness of these approaches with respect to human learning, we conduct a user study (N=126) where participants were tasked with searching and learning about topics such as `genetically modified organisms'. We find that (I) the introduction of the proposed scaffolding methods does not significantly improve learning gains. However, (ii) it does significantly impact search behavior. Furthermore, (iii) immediate feedback of the participants' learning leads to undesirable user behavior, with participants focusing on the feedback gauges instead of learning.
Weakly-supervised Generative Adversarial Networks for medical image classification
Venue: arXiv cs.CV 2021
Year: 2021

Weakly-supervised learning has become a popular technology in recent years. In this paper, we propose a novel medical image classification algorithm, called Weakly-Supervised Generative Adversarial Networks (WSGAN), which only uses a small number of real images without labels to generate fake images or mask images to enlarge the sample size of the training set. First, we combine with MixMatch to generate pseudo labels for the fake images and unlabeled images to do the classification. Second, contrastive learning and self-attention mechanism are introduced into the proposed problem to enhance the classification accuracy. Third, the problem of mode collapse is well addressed by cyclic consistency loss. Finally, we design global and local classifiers to complement each other with the key information needed for classification. The experimental results on four medical image datasets show that WSGAN can obtain relatively high learning performance by using few labeled and unlabeled data. For example, the classification accuracy of WSGAN is 11% higher than that of the second-ranked MIXMATCH with 100 labeled images and 1000 unlabeled images on the OCT dataset. In addition, we also conduct ablation experiments to verify the effectiveness of our algorithm.
WiFi-based Multi-task Sensing
Venue: arXiv cs.LG 2021
Year: 2021

WiFi-based sensing has aroused immense attention over recent years. The rationale is that the signal fluctuations caused by humans carry the information of human behavior which can be extracted from the channel state information of WiFi. Still, the prior studies mainly focus on single-task sensing (STS), e.g., gesture recognition, indoor localization, user identification. Since the fluctuations caused by gestures are highly coupling with body features and the user's location, we propose a WiFi-based multi-task sensing model (Wimuse) to perform gesture recognition, indoor localization, and user identification tasks simultaneously. However, these tasks have different difficulty levels (i.e., imbalance issue) and need task-specific information (i.e., discrepancy issue). To address these issues, the knowledge distillation technique and task-specific residual adaptor are adopted in Wimuse. We first train the STS model for each task. Then, for solving the imbalance issue, the extracted common feature in Wimuse is encouraged to get close to the counterpart features of the STS models. Further, for each task, a task-specific residual adaptor is applied to extract the task-specific compensation feature which is fused with the common feature to address the discrepancy issue. We conduct comprehensive experiments on three public datasets and evaluation suggests that Wimuse achieves state-of-the-art performance with the average accuracy of 85.20%, 98.39%, and 98.725% on the joint task of gesture recognition, indoor localization, and user identification, respectively.
Improving Zero-shot Generalization in Offline Reinforcement Learning using Generalized Similarity Functions
Venue: arXiv cs.LG 2021
Year: 2021

Reinforcement learning (RL) agents are widely used for solving complex sequential decision making tasks, but still exhibit difficulty in generalizing to scenarios not seen during training. While prior online approaches demonstrated that using additional signals beyond the reward function can lead to better generalization capabilities in RL agents, i.e. using self-supervised learning (SSL), they struggle in the offline RL setting, i.e. learning from a static dataset. We show that performance of online algorithms for generalization in RL can be hindered in the offline setting due to poor estimation of similarity between observations. We propose a new theoretically-motivated framework called Generalized Similarity Functions (GSF), which uses contrastive learning to train an offline RL agent to aggregate observations based on the similarity of their expected future behavior, where we quantify this similarity using \emph{generalized value functions}. We show that GSF is general enough to recover existing SSL objectives while also improving zero-shot generalization performance on a complex offline RL benchmark, offline Procgen.
Graph Embedding via High Dimensional Model Representation for Hyperspectral Images
Venue: arXiv cs.CV 2021
Year: 2021

Learning the manifold structure of remote sensing images is of paramount relevance for modeling and understanding processes, as well as to encapsulate the high dimensionality in a reduced set of informative features for subsequent classification, regression, or unmixing. Manifold learning methods have shown excellent performance to deal with hyperspectral image (HSI) analysis but, unless specifically designed, they cannot provide an explicit embedding map readily applicable to out-of-sample data. A common assumption to deal with the problem is that the transformation between the high-dimensional input space and the (typically low) latent space is linear. This is a particularly strong assumption, especially when dealing with hyperspectral images due to the well-known nonlinear nature of the data. To address this problem, a manifold learning method based on High Dimensional Model Representation (HDMR) is proposed, which enables to present a nonlinear embedding function to project out-of-sample samples into the latent space. The proposed method is compared to manifold learning methods along with its linear counterparts and achieves promising performance in terms of classification accuracy of a representative set of hyperspectral images.
Human-machine Symbiosis: A Multivariate Perspective for Physically Coupled Human-machine Systems
Venue: arXiv cs.HC 2021
Year: 2021

The notion of symbiosis has been increasingly mentioned in research on physically coupled human-machine systems. Yet, a uniform specification on which aspects constitute human-machine symbiosis is missing. By combining the expertise of different disciplines, we elaborate on a multivariate perspective of symbiosis as the highest form of physically coupled human-machine systems. Four dimensions are considered: Task, interaction, performance, and experience. First, human and machine work together to accomplish a common task conceptualized on both a decision and an action level (task dimension). Second, each partner possesses an internal representation of own as well as the other partner's intentions and influence on the environment. This alignment, which is the core of the interaction, constitutes the symbiotic understanding between both partners, being the basis of a joint, highly coordinated and effective action (interaction dimension). Third, the symbiotic interaction leads to synergetic effects regarding the intention recognition and complementary strengths of the partners, resulting in a higher overall performance (performance dimension). Fourth, symbiotic systems specifically change the user's experiences, like flow, acceptance, sense of agency, and embodiment (experience dimension). This multivariate perspective is flexible and generic and is also applicable in diverse human-machine scenarios, helping to bridge barriers between different disciplines.
A Case for a Programmable Edge Storage Middleware
Venue: arXiv cs.DC 2021
Year: 2021

Edge computing is a fast-growing computing paradigm where data is processed at the local site where it is generated, close to the end-devices. This can benefit a set of disruptive applications like autonomous driving, augmented reality, and collaborative machine learning, which produce incredible amounts of data that need to be shared, processed and stored at the edge to meet low latency requirements. However, edge storage poses new challenges due to the scarcity and heterogeneity of edge infrastructures and the diversity of edge applications. In particular, edge applications may impose conflicting constraints and optimizations that are hard to be reconciled on the limited, hard-to-scale edge resources. In this vision paper we argue that a new middleware for constrained edge resources is needed, providing a unified storage service for diverse edge applications. We identify programmability as a critical feature that should be leveraged to optimize the resource sharing while delivering the specialization needed for edge applications. Following this line, we make a case for eBPF and present the design for Griffin - a flexible, lightweight programmable edge storage middleware powered by eBPF.
Crime Prediction with Graph Neural Networks and Multivariate Normal Distributions
Venue: arXiv cs.LG 2021
Year: 2021

Existing approaches to the crime prediction problem are unsuccessful in expressing the details since they assign the probability values to large regions. This paper introduces a new architecture with the graph convolutional networks (GCN) and multivariate Gaussian distributions to perform high-resolution forecasting that applies to any spatiotemporal data. We tackle the sparsity problem in high resolution by leveraging the flexible structure of GCNs and providing a subdivision algorithm. We build our model with Graph Convolutional Gated Recurrent Units (Graph-ConvGRU) to learn spatial, temporal, and categorical relations. In each node of the graph, we learn a multivariate probability distribution from the extracted features of GCNs. We perform experiments on real-life and synthetic datasets, and our model obtains the best validation and the best test score among the baseline models with significant improvements. We show that our model is not only generative but also precise.