OpenRaiser/Intern-Atlas huggingface dataset:
Variational Topic Inference for Chest X-Ray Report Generation
Venue: arXiv cs.CV 2021
Year: 2021

Automating report generation for medical imaging promises to reduce workload and assist diagnosis in clinical practice. Recent work has shown that deep learning models can successfully caption natural images. However, learning from medical data is challenging due to the diversity and uncertainty inherent in the reports written by different radiologists with discrepant expertise and experience. To tackle these challenges, we propose variational topic inference for automatic report generation. Specifically, we introduce a set of topics as latent variables to guide sentence generation by aligning image and language modalities in a latent space. The topics are inferred in a conditional variational inference framework, with each topic governing the generation of a sentence in the report. Further, we adopt a visual attention module that enables the model to attend to different locations in the image and generate more informative descriptions. We conduct extensive experiments on two benchmarks, namely Indiana U. Chest X-rays and MIMIC-CXR. The results demonstrate that our proposed variational topic inference method can generate novel reports rather than mere copies of reports used in training, while still achieving comparable performance to state-of-the-art methods in terms of standard language generation criteria.
Tournesol: A quest for a large, secure and trustworthy database of reliable human judgments
Venue: arXiv cs.HC 2021
Year: 2021

Today's large-scale algorithms have become immensely influential, as they recommend and moderate the content that billions of humans are exposed to on a daily basis. They are the de-facto regulators of our societies' information diet, from shaping opinions on public health to organizing groups for social movements. This creates serious concerns, but also great opportunities to promote quality information. Addressing the concerns and seizing the opportunities is a challenging, enormous and fabulous endeavor, as intuitively appealing ideas often come with unwanted {\it side effects}, and as it requires us to think about what we deeply prefer. Understanding how today's large-scale algorithms are built is critical to determine what interventions will be most effective. Given that these algorithms rely heavily on {\it machine learning}, we make the following key observation: \emph{any algorithm trained on uncontrolled data must not be trusted}. Indeed, a malicious entity could take control over the data, poison it with dangerously manipulative fabricated inputs, and thereby make the trained algorithm extremely unsafe. We thus argue that the first step towards safe and ethical large-scale algorithms must be the collection of a large, secure and trustworthy dataset of reliable human judgments. To achieve this, we introduce \emph{Tournesol}, an open source platform available at \url{https://tournesol.app}. Tournesol aims to collect a large database of human judgments on what algorithms ought to widely recommend (and what they ought to stop widely recommending). We outline the structure of the Tournesol database, the key features of the Tournesol platform and the main hurdles that must be overcome to make it a successful project. Most importantly, we argue that, if successful, Tournesol may then serve as the essential foundation for any safe and ethical large-scale algorithm.
An Overview of Machine Learning-aided Optical Performance Monitoring Techniques
Venue: arXiv cs.LG 2021
Year: 2021

Future communication systems are faced with increased demand for high capacity, dynamic bandwidth, reliability and heterogeneous traffic. To meet these requirements, networks have become more complex and thus require new design methods and monitoring techniques, as they evolve towards becoming autonomous. Machine learning has come to the forefront in recent years as a promising technology to aid in this evolution. Optical fiber communications can already provide the high capacity required for most applications, however, there is a need for increased scalability and adaptability to changing user demands and link conditions. Accurate performance monitoring is an integral part of this transformation. In this paper we review optical performance monitoring techniques where machine learning algorithms have been applied. Moreover, since alot of OPM depends on knowledge of the signal type, we also review work for modulation format recognition and bitrate identification. We additionally briefly introduce a neuromorphic approach to OPM as an emerging technique that has only recently been applied to this domain.
Framework for A Personalized Intelligent Assistant to Elderly People for Activities of Daily Living
Venue: arXiv cs.HC 2021
Year: 2021

The increasing population of elderly people is associated with the need to meet their increasing requirements and to provide solutions that can improve their quality of life in a smart home. In addition to fear and anxiety towards interfacing with systems; cognitive disabilities, weakened memory, disorganized behavior and even physical limitations are some of the problems that elderly people tend to face with increasing age. The essence of providing technology-based solutions to address these needs of elderly people and to create smart and assisted living spaces for the elderly; lies in developing systems that can adapt by addressing their diversity and can augment their performances in the context of their day to day goals. Therefore, this work proposes a framework for development of a Personalized Intelligent Assistant to help elderly people perform Activities of Daily Living (ADLs) in a smart and connected Internet of Things (IoT) based environment. This Personalized Intelligent Assistant can analyze different tasks performed by the user and recommend activities by considering their daily routine, current affective state and the underlining user experience. To uphold the efficacy of this proposed framework, it has been tested on a couple of datasets for modelling an average user and a specific user respectively. The results presented show that the model achieves a performance accuracy of 73.12% when modelling a specific user, which is considerably higher than its performance while modelling an average user, this upholds the relevance for development and implementation of this proposed framework.
Computing Permanents on a Trellis
Venue: arXiv cs.IT 2021
Year: 2021

The problem of computing the permanent of a matrix has attracted interest since the work of Ryser(1963) and Valiant(1979). On the other hand, trellises were extensively studied in coding theory since the 1960s. In this work, we establish a connection between the two domains. We introduce the canonical trellis $T_n$ that represents all permutations, and show that the permanent of a $n$ by $n$ matrix $A$ can be computed as a flow on this trellis. Under certain normalization, the trellis-based method invokes slightly less operations than best known exact methods. Moreover, if $A$ has structure, then $T_n$ becomes amenable to vertex merging, thereby significantly reducing its complexity. - Repeated rows: Suppose $A$ has only $t<n$ distinct rows. The best known method to compute $per(A)$, due to Clifford and Clifford (2020), has complexity $O(n^{t+1})$. Merging vertices in $T_n$, we obtain a reduced trellis that has complexity $O(n^t)$. - Order statistics: Using trellises, we compute the joint distribution of $t$ order statistics of $n$ independent, but not identically distributed, random variables in time $O(n^{t+1})$. Previously, polynomial-time methods were known only when the variables are drawn from two non-identical distributions. - Sparse matrices: Suppose each entry in $A$ is nonzero with probability $d/n$ with $d$ is constant. We show that $T_n$ can be pruned to exponentially fewer vertices, resulting in complexity $O(ϕ^n)$ with $ϕ<2$. - TSP: Intersecting $T_n$ with another trellis that represents walks, we obtain a trellis that represents circular permutations. Using the latter trellis to solve the traveling salesperson problem recovers the well-known Held-Karp algorithm. Notably, in all cases, the reduced trellis are obtained using known techniques in trellis theory. We expect other trellis-theoretic results to apply to other structured matrices.
Hybrid Ant Swarm-Based Data Clustering
Venue: arXiv cs.NE 2021
Year: 2021

Biologically inspired computing techniques are very effective and useful in many areas of research including data clustering. Ant clustering algorithm is a nature-inspired clustering technique which is extensively studied for over two decades. In this study, we extend the ant clustering algorithm (ACA) to a hybrid ant clustering algorithm (hACA). Specifically, we include a genetic algorithm in standard ACA to extend the hybrid algorithm for better performance. We also introduced novel pick up and drop off rules to speed up the clustering performance. We study the performance of the hACA algorithm and compare with standard ACA as a benchmark.
Design and Implementation of an IoT Based LPG and CO Gases Monitoring System
Venue: arXiv cs.DC 2021
Year: 2021

Nowadays use of liquefied petroleum gas (LPG) has increased. LPG is an asphyxiating, volatile and highly flammable gas. In a LPG leak situation, potential health accidents are increased either by inhalation or by combustion of the gas. On the other hand, carbon monoxide (CO) is a toxic gas that comes mainly from combustion in car engines. Breathing CO-polluted air can cause dizziness, fainting, breathing problems, and sometimes death. To prevent health accidents, including explosions, in open or closed environments, remote and real-time monitoring of the concentration levels of CO and LPG gases has become a necessity. The aim of this work is to demonstrate the use of Internet of Things (IoT) techniques to design and build a telemetry system to monitor in real-time the concentration of GLP and CO gases in the surrounding air. To implement this work, as central hardware there is a microcontroller, CO and PLG sensors on the electronic station. Besides, Amazon Web Services (AWS) was used as an IoT platform and data storage in the cloud. The main result was a telematics system to monitor in real time the concentrations of both GLP and CO gases, whose data is accessible from any device with internet access through a website. Field tests have been successful and have shown that the proposed system is an efficient and low-cost option.
Reliability Analysis of Slotted Aloha with Capture for an OWC-based IoT system
Venue: arXiv cs.IT 2021
Year: 2021

In this article, we consider a random access scheme for an indoor Internet of Things (IoT) framework that uses optical wireless communication (OWC). We focus on a Slotted ALOHA (SA)-based solution where a number of OWC IoT users contend to send data to a central OWC receiver. In any given slot, and for a randomly selected active user, we consider the reliability of decoding the user's data packet at the receiver. This is done by deriving the signal-to-noise-and-interference-ratio (SINR) statistics from a randomly chosen user and evaluating the probability that the user's SINR is below a given threshold. By placing our analysis in the context of an indoor OWC IoT uplink setup, and employing the standard OWC channel model, we investigate the trade-offs between the reliability and the OWC system parameters such as the cell area or the transmitter's semi-angle. We obtain valuable insights into the design of an SA-based random access solution for a typical indoor OWC cell.
Personalizing User Engagement Dynamics in a Non-Verbal Communication Game for Cerebral Palsy
Venue: arXiv cs.RO 2021
Year: 2021

Children and adults with cerebral palsy (CP) can have involuntary upper limb movements as a consequence of the symptoms that characterize their motor disability, leading to difficulties in communicating with caretakers and peers. We describe how a socially assistive robot may help individuals with CP to practice non-verbal communicative gestures using an active orthosis in a one-on-one number-guessing game. We performed a user study and data collection with participants with CP; we found that participants preferred an embodied robot over a screen-based agent, and we used the participant data to train personalized models of participant engagement dynamics that can be used to select personalized robot actions. Our work highlights the benefit of personalized models in the engagement of users with CP with a socially assistive robot and offers design insights for future work in this area.
Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving
Venue: arXiv cs.CV 2021
Year: 2021

Deep neural networks (DNNs) have accomplished impressive success in various applications, including autonomous driving perception tasks, in recent years. On the other hand, current deep neural networks are easily fooled by adversarial attacks. This vulnerability raises significant concerns, particularly in safety-critical applications. As a result, research into attacking and defending DNNs has gained much coverage. In this work, detailed adversarial attacks are applied on a diverse multi-task visual perception deep network across distance estimation, semantic segmentation, motion detection, and object detection. The experiments consider both white and black box attacks for targeted and un-targeted cases, while attacking a task and inspecting the effect on all the others, in addition to inspecting the effect of applying a simple defense method. We conclude this paper by comparing and discussing the experimental results, proposing insights and future work. The visualizations of the attacks are available at https://youtu.be/6AixN90budY.
GI-NNet \& RGI-NNet: Development of Robotic Grasp Pose Models, Trainable with Large as well as Limited Labelled Training Datasets, under supervised and semi supervised paradigms
Venue: arXiv cs.RO 2021
Year: 2021

Our way of grasping objects is challenging for efficient, intelligent and optimal grasp by COBOTs. To streamline the process, here we use deep learning techniques to help robots learn to generate and execute appropriate grasps quickly. We developed a Generative Inception Neural Network (GI-NNet) model, capable of generating antipodal robotic grasps on seen as well as unseen objects. It is trained on Cornell Grasping Dataset (CGD) and attained 98.87% grasp pose accuracy for detecting both regular and irregular shaped objects from RGB-Depth (RGB-D) images while requiring only one third of the network trainable parameters as compared to the existing approaches. However, to attain this level of performance the model requires the entire 90% of the available labelled data of CGD keeping only 10% labelled data for testing which makes it vulnerable to poor generalization. Furthermore, getting sufficient and quality labelled dataset is becoming increasingly difficult keeping in pace with the requirement of gigantic networks. To address these issues, we attach our model as a decoder with a semi-supervised learning based architecture known as Vector Quantized Variational Auto Encoder (VQVAE), which works efficiently when trained both with the available labelled and unlabelled data. The proposed model, which we name as Representation based GI-NNet (RGI-NNet), has been trained with various splits of label data on CGD with as minimum as 10% labelled dataset together with latent embedding generated from VQVAE up to 50% labelled data with latent embedding obtained from VQVAE. The performance level, in terms of grasp pose accuracy of RGI-NNet, varies between 92.13% to 95.6% which is far better than several existing models trained with only labelled dataset. For the performance verification of both GI-NNet and RGI-NNet models, we use Anukul (Baxter) hardware cobot.
Next-item Recommendations in Short Sessions
Venue: arXiv cs.IR 2021
Year: 2021

The changing preferences of users towards items trigger the emergence of session-based recommender systems (SBRSs), which aim to model the dynamic preferences of users for next-item recommendations. However, most of the existing studies on SBRSs are based on long sessions only for recommendations, ignoring short sessions, though short sessions, in fact, account for a large proportion in most of the real-world datasets. As a result, the applicability of existing SBRSs solutions is greatly reduced. In a short session, quite limited contextual information is available, making the next-item recommendation very challenging. To this end, in this paper, inspired by the success of few-shot learning (FSL) in effectively learning a model with limited instances, we formulate the next-item recommendation as an FSL problem. Accordingly, following the basic idea of a representative approach for FSL, i.e., meta-learning, we devise an effective SBRS called INter-SEssion collaborative Recommender netTwork (INSERT) for next-item recommendations in short sessions. With the carefully devised local module and global module, INSERT is able to learn an optimal preference representation of the current user in a given short session. In particular, in the global module, a similar session retrieval network (SSRN) is designed to find out the sessions similar to the current short session from the historical sessions of both the current user and other users, respectively. The obtained similar sessions are then utilized to complement and optimize the preference representation learned from the current short session by the local module for more accurate next-item recommendations in this short session. Extensive experiments conducted on two real-world datasets demonstrate the superiority of our proposed INSERT over the state-of-the-art SBRSs when making next-item recommendations in short sessions.
Context-Conditional Adaptation for Recognizing Unseen Classes in Unseen Domains
Venue: arXiv cs.CV 2021
Year: 2021

Recent progress towards designing models that can generalize to unseen domains (i.e domain generalization) or unseen classes (i.e zero-shot learning) has embarked interest towards building models that can tackle both domain-shift and semantic shift simultaneously (i.e zero-shot domain generalization). For models to generalize to unseen classes in unseen domains, it is crucial to learn feature representation that preserves class-level (domain-invariant) as well as domain-specific information. Motivated from the success of generative zero-shot approaches, we propose a feature generative framework integrated with a COntext COnditional Adaptive (COCOA) Batch-Normalization to seamlessly integrate class-level semantic and domain-specific information. The generated visual features better capture the underlying data distribution enabling us to generalize to unseen classes and domains at test-time. We thoroughly evaluate and analyse our approach on established large-scale benchmark - DomainNet and demonstrate promising performance over baselines and state-of-art methods.
Internet-Augmented Dialogue Generation
Venue: arXiv cs.AI 2021
Year: 2021

The largest store of continually updating knowledge on our planet can be accessed via internet search. In this work we study giving access to this information to conversational agents. Large language models, even though they store an impressive amount of knowledge within their weights, are known to hallucinate facts when generating dialogue (Shuster et al., 2021); moreover, those facts are frozen in time at the point of model training. In contrast, we propose an approach that learns to generate an internet search query based on the context, and then conditions on the search results to finally generate a response, a method that can employ up-to-the-minute relevant information. We train and evaluate such models on a newly collected dataset of human-human conversations whereby one of the speakers is given access to internet search during knowledgedriven discussions in order to ground their responses. We find that search-query based access of the internet in conversation provides superior performance compared to existing approaches that either use no augmentation or FAISS-based retrieval (Lewis et al., 2020).
Real-Time Violence Detection Using CNN-LSTM
Venue: arXiv cs.CV 2021
Year: 2021

Violence rates however have been brought down about 57% during the span of the past 4 decades yet it doesn't change the way that the demonstration of violence actually happens, unseen by the law. Violence can be mass controlled sometimes by higher authorities, however, to hold everything in line one must "Microgovern" over each movement occurring in every road of each square. To address the butterfly effects impact in our setting, I made a unique model and a theorized system to handle the issue utilizing deep learning. The model takes the input of the CCTV video feeds and after drawing inference, recognizes if a violent movement is going on. And hypothesized architecture aims towards probability-driven computation of video feeds and reduces overhead from naively computing for every CCTV video feeds.
Partially Observable Markov Decision Processes (POMDPs) and Robotics
Venue: arXiv cs.RO 2021
Year: 2021

Planning under uncertainty is critical to robotics. The Partially Observable Markov Decision Process (POMDP) is a mathematical framework for such planning problems. It is powerful due to its careful quantification of the non-deterministic effects of actions and partial observability of the states. But precisely because of this, POMDP is notorious for its high computational complexity and deemed impractical for robotics. However, since early 2000, POMDPs solving capabilities have advanced tremendously, thanks to sampling-based approximate solvers. Although these solvers do not generate the optimal solution, they can compute good POMDP solutions that significantly improve the robustness of robotics systems within reasonable computational resources, thereby making POMDPs practical for many realistic robotics problems. This paper presents a review of POMDPs, emphasizing computational issues that have hindered its practicality in robotics and ideas in sampling-based solvers that have alleviated such difficulties, together with lessons learned from applying POMDPs to physical robots.
Geometric Value Iteration: Dynamic Error-Aware KL Regularization for Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

The recent boom in the literature on entropy-regularized reinforcement learning (RL) approaches reveals that Kullback-Leibler (KL) regularization brings advantages to RL algorithms by canceling out errors under mild assumptions. However, existing analyses focus on fixed regularization with a constant weighting coefficient and do not consider cases where the coefficient is allowed to change dynamically. In this paper, we study the dynamic coefficient scheme and present the first asymptotic error bound. Based on the dynamic coefficient error bound, we propose an effective scheme to tune the coefficient according to the magnitude of error in favor of more robust learning. Complementing this development, we propose a novel algorithm, Geometric Value Iteration (GVI), that features a dynamic error-aware KL coefficient design with the aim of mitigating the impact of errors on performance. Our experiments demonstrate that GVI can effectively exploit the trade-off between learning speed and robustness over uniform averaging of a constant KL coefficient. The combination of GVI and deep networks shows stable learning behavior even in the absence of a target network, where algorithms with a constant KL coefficient would greatly oscillate or even fail to converge.