OpenRaiser/Intern-Atlas huggingface dataset:
Removing Operational Friction Using Process Mining: Challenges Provided by the Internet of Production (IoP)
Venue: arXiv cs.DB 2021
Year: 2021

Operational processes in production, logistics, material handling, maintenance, etc., are supported by cyber-physical systems combining hardware and software components. As a result, the digital and the physical world are closely aligned, and it is possible to track operational processes in detail (e.g., using sensors). The abundance of event data generated by today's operational processes provides opportunities and challenges for process mining techniques supporting process discovery, performance analysis, and conformance checking. Using existing process mining tools, it is already possible to automatically discover process models and uncover performance and compliance problems. In the DFG-funded Cluster of Excellence "Internet of Production" (IoP), process mining is used to create "digital shadows" to improve a wide variety of operational processes. However, operational processes are dynamic, distributed, and complex. Driven by the challenges identified in the IoP cluster, we work on novel techniques for comparative process mining (comparing process variants for different products at different locations at different times), object-centric process mining (to handle processes involving different types of objects that interact), and forward-looking process mining (to explore "What if?" questions). By addressing these challenges, we aim to develop valuable "digital shadows" that can be used to remove operational friction.
A-star path planning simulation for UAS Traffic Management (UTM) application
Venue: arXiv cs.RO 2021
Year: 2021

This paper presents a Robot Operating System and Gazebo application to calculate and simulate an optimal route for a drone in an urban environment by developing new ROS packages and executing them along with open-source tools. Firstly, the current regulations about UAS are presented to guide the building of the simulated environment, and multiple path planning algorithms are reviewed to guide the search method selection. After selecting the A-star algorithm, both the 2D and 3D versions of them were implemented in this paper, with both Manhattan and Euclidean distances heuristics. The performance of these algorithms was evaluated considering the distance to be covered by the drone and the execution time of the route planning method, aiming to support algorithm's choice based on the environment in which it will be applied. The algorithm execution time was 3.2 and 17.2 higher when using the Euclidean distance for the 2D and 3D A-star algorithm, respectively. Along with the performance analysis of the algorithm, this paper is also the first step for building a complete UAS Traffic Management (UTM) system simulation using ROS and Gazebo.
Experimenting with Self-Supervision using Rotation Prediction for Image Captioning
Venue: arXiv cs.CV 2021
Year: 2021

Image captioning is a task in the field of Artificial Intelligence that merges between computer vision and natural language processing. It is responsible for generating legends that describe images, and has various applications like descriptions used by assistive technology or indexing images (for search engines for instance). This makes it a crucial topic in AI that is undergoing a lot of research. This task however, like many others, is trained on large images labeled via human annotation, which can be very cumbersome: it needs manual effort, both financial and temporal costs, it is error-prone and potentially difficult to execute in some cases (e.g. medical images). To mitigate the need for labels, we attempt to use self-supervised learning, a type of learning where models use the data contained within the images themselves as labels. It is challenging to accomplish though, since the task is two-fold: the images and captions come from two different modalities and usually handled by different types of networks. It is thus not obvious what a completely self-supervised solution would look like. How it would achieve captioning in a comparable way to how self-supervision is applied today on image recognition tasks is still an ongoing research topic. In this project, we are using an encoder-decoder architecture where the encoder is a convolutional neural network (CNN) trained on OpenImages dataset and learns image features in a self-supervised fashion using the rotation pretext task. The decoder is a Long Short-Term Memory (LSTM), and it is trained, along within the image captioning model, on MS COCO dataset and is responsible of generating captions. Our GitHub repository can be found: https://github.com/elhagry1/SSL_ImageCaptioning_RotationPrediction
An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregivers
Venue: arXiv cs.HC 2021
Year: 2021

With the advent of off-the-shelf intelligent home products and broader internet adoption, researchers increasingly explore smart computing applications that provide easier access to health and wellness resources. AI-based systems like chatbots have the potential to provide services that could provide mental health support. However, existing therapy chatbots are often retrieval-based, requiring users to respond with a constrained set of answers, which may not be appropriate given that such pre-determined inquiries may not reflect each patient's unique circumstances. Generative-based approaches, such as the OpenAI GPT models, could allow for more dynamic conversations in therapy chatbot contexts than previous approaches. To investigate the generative-based model's potential in therapy chatbot contexts, we built a chatbot using the GPT-2 model. We fine-tuned it with 306 therapy session transcripts between family caregivers of individuals with dementia and therapists conducting Problem Solving Therapy. We then evaluated the model's pre-trained and the fine-tuned model in terms of basic qualities using three meta-information measurements: the proportion of non-word outputs, the length of response, and sentiment components. Results showed that: (1) the fine-tuned model created more non-word outputs than the pre-trained model; (2) the fine-tuned model generated outputs whose length was more similar to that of the therapists compared to the pre-trained model; (3) both the pre-trained model and fine-tuned model were likely to generate more negative and fewer positive outputs than the therapists. We discuss potential reasons for the problem, the implications, and solutions for developing therapy chatbots and call for investigations of the AI-based system application.
Image color correction, enhancement, and editing
Venue: arXiv cs.CV 2021
Year: 2021

This thesis presents methods and approaches to image color correction, color enhancement, and color editing. To begin, we study the color correction problem from the standpoint of the camera's image signal processor (ISP). A camera's ISP is hardware that applies a series of in-camera image processing and color manipulation steps, many of which are nonlinear in nature, to render the initial sensor image to its final photo-finished representation saved in the 8-bit standard RGB (sRGB) color space. As white balance (WB) is one of the major procedures applied by the ISP for color correction, this thesis presents two different methods for ISP white balancing. Afterward, we discuss another scenario of correcting and editing image colors, where we present a set of methods to correct and edit WB settings for images that have been improperly white-balanced by the ISP. Then, we explore another factor that has a significant impact on the quality of camera-rendered colors, in which we outline two different methods to correct exposure errors in camera-rendered images. Lastly, we discuss post-capture auto color editing and manipulation. In particular, we propose auto image recoloring methods to generate different realistic versions of the same camera-rendered image with new colors. Through extensive evaluations, we demonstrate that our methods provide superior solutions compared to existing alternatives targeting color correction, color enhancement, and color editing.
Towards Emotion-Aware Agents For Negotiation Dialogues
Venue: arXiv cs.HC 2021
Year: 2021

Negotiation is a complex social interaction that encapsulates emotional encounters in human decision-making. Virtual agents that can negotiate with humans are useful in pedagogy and conversational AI. To advance the development of such agents, we explore the prediction of two important subjective goals in a negotiation - outcome satisfaction and partner perception. Specifically, we analyze the extent to which emotion attributes extracted from the negotiation help in the prediction, above and beyond the individual difference variables. We focus on a recent dataset in chat-based negotiations, grounded in a realistic camping scenario. We study three degrees of emotion dimensions - emoticons, lexical, and contextual by leveraging affective lexicons and a state-of-the-art deep learning architecture. Our insights will be helpful in designing adaptive negotiation agents that interact through realistic communication interfaces.
Packet Routing with Graph Attention Multi-agent Reinforcement Learning
Venue: arXiv cs.AI 2021
Year: 2021

Packet routing is a fundamental problem in communication networks that decides how the packets are directed from their source nodes to their destination nodes through some intermediate nodes. With the increasing complexity of network topology and highly dynamic traffic demand, conventional model-based and rule-based routing schemes show significant limitations, due to the simplified and unrealistic model assumptions, and lack of flexibility and adaption. Adding intelligence to the network control is becoming a trend and the key to achieving high-efficiency network operation. In this paper, we develop a model-free and data-driven routing strategy by leveraging reinforcement learning (RL), where routers interact with the network and learn from the experience to make some good routing configurations for the future. Considering the graph nature of the network topology, we design a multi-agent RL framework in combination with Graph Neural Network (GNN), tailored to the routing problem. Three deployment paradigms, centralized, federated, and cooperated learning, are explored respectively. Simulation results demonstrate that our algorithm outperforms some existing benchmark algorithms in terms of packet transmission delay and affordable load.
DeepTeeth: A Teeth-photo Based Human Authentication System for Mobile and Hand-held Devices
Venue: arXiv cs.CV 2021
Year: 2021

This paper proposes teeth-photo, a new biometric modality for human authentication on mobile and hand held devices. Biometrics samples are acquired using the camera mounted on mobile device with the help of a mobile application having specific markers to register the teeth area. Region of interest (RoI) is then extracted using the markers and the obtained sample is enhanced using contrast limited adaptive histogram equalization (CLAHE) for better visual clarity. We propose a deep learning architecture and novel regularization scheme to obtain highly discriminative embedding for small size RoI. Proposed custom loss function was able to achieve perfect classification for the tiny RoI of $75\times 75$ size. The model is end-to-end and few-shot and therefore is very efficient in terms of time and energy requirements. The system can be used in many ways including device unlocking and secure authentication. To the best of our understanding, this is the first work on teeth-photo based authentication for mobile device. Experiments have been conducted on an in-house teeth-photo database collected using our application. The database is made publicly available. Results have shown that the proposed system has perfect accuracy.
Library and Information Science Research in Indian Universities: Growth, Core Journals, Keywords and Collaboration Patterns
Venue: arXiv cs.DL 2021
Year: 2021

This article maps Library and Information Science (LIS) research in Indian universities. As the two prominent citation databases, Web of Science and Scopus have very limited coverage of Indian LIS journals, the publications generated by the library and science departments of about 114 selected Indian universities and the two national institutions of importance in LIS research were extracted from Library, Information Science & Technology Abstracts (LISTA). The relevant publication records were analyzed using scientometrics and Social Network Analysis (SNA) tools. The study traces the growth of publications, prominent keywords, leading journals where the articles are published and the institutional collaboration patterns of Indian university publications. The results show that there is a growth in scholarly publications from Indian universities in LIS. However, the numbers of publications are limited to only a few universities and national institutes of importance. The maximum LIS research outputs are published in Indian journals. Bibliometrics related investigations are the most important research areas. Located in major cities of India, the productive institutes show healthy collaboration. The study concludes with some observations which may be useful for formulating policies in LIS research in India.
Learning-Based Depth and Pose Estimation for Monocular Endoscope with Loss Generalization
Venue: arXiv cs.CV 2021
Year: 2021

Gastroendoscopy has been a clinical standard for diagnosing and treating conditions that affect a part of a patient's digestive system, such as the stomach. Despite the fact that gastroendoscopy has a lot of advantages for patients, there exist some challenges for practitioners, such as the lack of 3D perception, including the depth and the endoscope pose information. Such challenges make navigating the endoscope and localizing any found lesion in a digestive tract difficult. To tackle these problems, deep learning-based approaches have been proposed to provide monocular gastroendoscopy with additional yet important depth and pose information. In this paper, we propose a novel supervised approach to train depth and pose estimation networks using consecutive endoscopy images to assist the endoscope navigation in the stomach. We firstly generate real depth and pose training data using our previously proposed whole stomach 3D reconstruction pipeline to avoid poor generalization ability between computer-generated (CG) models and real data for the stomach. In addition, we propose a novel generalized photometric loss function to avoid the complicated process of finding proper weights for balancing the depth and the pose loss terms, which is required for existing direct depth and pose supervision approaches. We then experimentally show that our proposed generalized loss performs better than existing direct supervision losses.
Second-order phase-field formulations for anisotropic brittle fracture
Venue: arXiv cs.CE 2021
Year: 2021

We address brittle fracture in anisotropic materials featuring two-fold and four-fold symmetric fracture toughness. For these two classes, we develop two variational phase-field models based on the family of regularizations proposed by Focardi (Focardi, M. On the variational approximation of free-discontinuity problems in the vectorial case. Math. Models Methods App. Sci., 11:663{684, 2001), for which Gamma-convergence results hold. Since both models are of second order, as opposed to the previously available fourth-order models for four-fold symmetric fracture toughness, they do not require basis functions of C1-continuity nor mixed variational principles for finite element discretization. For the four-fold symmetric formulation we show that the standard quadratic degradation function is unsuitable and devise a procedure to derive a suitable one. The performance of the new models is assessed via several numerical examples that simulate anisotropic fracture under anti-plane shear loading. For both formulations at fixed displacements (i.e. within an alternate minimization procedure), we also provide some existence and uniqueness results for the phase-field solution.
Sorrel: an IDE Plugin for Managing Licenses and Detecting License Incompatibilities
Venue: arXiv cs.SE 2021
Year: 2021

Software development is a complex process that includes many different tasks besides just writing code. One of the aspects of software engineering is selecting and managing licenses for the given project. In this paper, we present Sorrel - a plugin for managing licenses and detecting potential incompatibilities for IntelliJ IDEA, a popular Java IDE. The plugin scans the project in search of information about the project license and the licenses of its libraries. If the project does not yet have a license, the plugin provides the developer with recommendations for choosing the most suitable open license, and if there is a license, it informs the programmer about potential licensing violations. The tool makes it easier for developers to choose a proper license for a project and avoid most of the licensing errors - all inside the familiar IDE editor. The plugin and its source code are available online on GitHub: https://github.com/JetBrains-Research/sorrel. A demonstration video can be found at https://youtu.be/doUeAwPjcPE.
Ranker-agnostic Contextual Position Bias Estimation
Venue: arXiv cs.IR 2021
Year: 2021

Learning-to-rank (LTR) algorithms are ubiquitous and necessary to explore the extensive catalogs of media providers. To avoid the user examining all the results, its preferences are used to provide a subset of relatively small size. The user preferences can be inferred from the interactions with the presented content if explicit ratings are unavailable. However, directly using implicit feedback can lead to learning wrong relevance models and is known as biased LTR. The mismatch between implicit feedback and true relevances is due to various nuisances, with position bias one of the most relevant. Position bias models consider that the lack of interaction with a presented item is not only attributed to the item being irrelevant but because the item was not examined. This paper introduces a method for modeling the probability of an item being seen in different contexts, e.g., for different users, with a single estimator. Our suggested method, denoted as contextual (EM)-based regression, is ranker-agnostic and able to correctly learn the latent examination probabilities while only using implicit feedback. Our empirical results indicate that the method introduced in this paper outperforms other existing position bias estimators in terms of relative error when the examination probability varies across queries. Moreover, the estimated values provide a ranking performance boost when used to debias the implicit ranking data even if there is no context dependency on the examination probabilities.
A Computer Vision-Based Approach for Driver Distraction Recognition using Deep Learning and Genetic Algorithm Based Ensemble
Venue: arXiv cs.CV 2021
Year: 2021

As the proportion of road accidents increases each year, driver distraction continues to be an important risk component in road traffic injuries and deaths. The distractions caused by the increasing use of mobile phones and other wireless devices pose a potential risk to road safety. Our current study aims to aid the already existing techniques in driver posture recognition by improving the performance in the driver distraction classification problem. We present an approach using a genetic algorithm-based ensemble of six independent deep neural architectures, namely, AlexNet, VGG-16, EfficientNet B0, Vanilla CNN, Modified DenseNet, and InceptionV3 + BiLSTM. We test it on two comprehensive datasets, the AUC Distracted Driver Dataset, on which our technique achieves an accuracy of 96.37%, surpassing the previously obtained 95.98%, and on the State Farm Driver Distraction Dataset, on which we attain an accuracy of 99.75%. The 6-Model Ensemble gave an inference time of 0.024 seconds as measured on our machine with Ubuntu 20.04(64-bit) and GPU as GeForce GTX 1080.
Value-Based Reinforcement Learning for Continuous Control Robotic Manipulation in Multi-Task Sparse Reward Settings
Venue: arXiv cs.RO 2021
Year: 2021

Learning continuous control in high-dimensional sparse reward settings, such as robotic manipulation, is a challenging problem due to the number of samples often required to obtain accurate optimal value and policy estimates. While many deep reinforcement learning methods have aimed at improving sample efficiency through replay or improved exploration techniques, state of the art actor-critic and policy gradient methods still suffer from the hard exploration problem in sparse reward settings. Motivated by recent successes of value-based methods for approximating state-action values, like RBF-DQN, we explore the potential of value-based reinforcement learning for learning continuous robotic manipulation tasks in multi-task sparse reward settings. On robotic manipulation tasks, we empirically show RBF-DQN converges faster than current state of the art algorithms such as TD3, SAC, and PPO. We also perform ablation studies with RBF-DQN and have shown that some enhancement techniques for vanilla Deep Q learning such as Hindsight Experience Replay (HER) and Prioritized Experience Replay (PER) can also be applied to RBF-DQN. Our experimental analysis suggests that value-based approaches may be more sensitive to data augmentation and replay buffer sample techniques than policy-gradient methods, and that the benefits of these methods for robot manipulation are heavily dependent on the transition dynamics of generated subgoal states.
Evaluating the Use of Reconstruction Error for Novelty Localization
Venue: arXiv cs.CV 2021
Year: 2021

The pixelwise reconstruction error of deep autoencoders is often utilized for image novelty detection and localization under the assumption that pixels with high error indicate which parts of the input image are unfamiliar and therefore likely to be novel. This assumed correlation between pixels with high reconstruction error and novel regions of input images has not been verified and may limit the accuracy of these methods. In this paper we utilize saliency maps to evaluate whether this correlation exists. Saliency maps reveal directly how much a change in each input pixel would affect reconstruction loss, while each pixel's reconstruction error may be attributed to many input pixels when layers are fully connected. We compare saliency maps to reconstruction error maps via qualitative visualizations as well as quantitative correspondence between the top K elements of the maps for both novel and normal images. Our results indicate that reconstruction error maps do not closely correlate with the importance of pixels in the input images, making them insufficient for novelty localization.