OpenRaiser/Intern-Atlas huggingface dataset:
Spoken Dialogue System Based on Attribute Vector for Travel Agent Robot
Venue: arXiv cs.HC 2022
Year: 2022

In this study, we develop a dialogue system for a dialogue robot competition. In the system, the characteristics of sightseeing spots are expressed as "attribute vectors" in advance, and the user is questioned on the different attributes of the two candidate spots. Consequently, the system can make recommendations based on user intentions. A dialogue experiment is conducted during a preliminary round of competition. The overall satisfaction score obtained is 40.1 out of 63 points, which is a reasonable result. Analysis of the relationship between the system behavior and satisfaction scores reveals that satisfaction increases when the system correctly understands the user intention and responds appropriately. However, a negative correlation is observed between the number of user utterances and the satisfaction score. This implies that inappropriate responses reduce the usefulness of the system as a consultation partner.
Row-wise LiDAR Lane Detection Network with Lane Correlation Refinement
Venue: arXiv cs.CV 2022
Year: 2022

Lane detection is one of the most important functions for autonomous driving. In recent years, deep learning-based lane detection networks with RGB camera images have shown promising performance. However, camera-based methods are inherently vulnerable to adverse lighting conditions such as poor or dazzling lighting. Unlike camera, LiDAR sensor is robust to the lighting conditions. In this work, we propose a novel two-stage LiDAR lane detection network with row-wise detection approach. The first-stage network produces lane proposals through a global feature correlator backbone and a row-wise detection head. Meanwhile, the second-stage network refines the feature map of the first-stage network via attention-based mechanism between the local features around the lane proposals, and outputs a set of new lane proposals. Experimental results on the K-Lane dataset show that the proposed network advances the state-of-the-art in terms of F1-score with 30% less GFLOPs. In addition, the second-stage network is found to be especially robust to lane occlusions, thus, demonstrating the robustness of the proposed network for driving in crowded environments.
Cross-layer Attention Network for Fine-grained Visual Categorization
Venue: arXiv cs.CV 2022
Year: 2022

Learning discriminative representations for subtle localized details plays a significant role in Fine-grained Visual Categorization (FGVC). Compared to previous attention-based works, our work does not explicitly define or localize the part regions of interest; instead, we leverage the complementary properties of different stages of the network, and build a mutual refinement mechanism between the mid-level feature maps and the top-level feature map by our proposed Cross-layer Attention Network (CLAN). Specifically, CLAN is composed of 1) the Cross-layer Context Attention (CLCA) module, which enhances the global context information in the intermediate feature maps with the help of the top-level feature map, thereby improving the expressive power of the middle layers, and 2) the Cross-layer Spatial Attention (CLSA) module, which takes advantage of the local attention in the mid-level feature maps to boost the feature extraction of local regions at the top-level feature maps. Experimental results show our approach achieves state-of-the-art on three publicly available fine-grained recognition datasets (CUB-200-2011, Stanford Cars and FGVC-Aircraft). Ablation studies and visualizations are provided to understand our approach. Experimental results show our approach achieves state-of-the-art on three publicly available fine-grained recognition datasets (CUB-200-2011, Stanford Cars and FGVC-Aircraft).
Disentangling Confidence Score Distribution for Out-of-Domain Intent Detection with Energy-Based Learning
Venue: arXiv cs.CL 2022
Year: 2022

Detecting Out-of-Domain (OOD) or unknown intents from user queries is essential in a task-oriented dialog system. Traditional softmax-based confidence scores are susceptible to the overconfidence issue. In this paper, we propose a simple but strong energy-based score function to detect OOD where the energy scores of OOD samples are higher than IND samples. Further, given a small set of labeled OOD samples, we introduce an energy-based margin objective for supervised OOD detection to explicitly distinguish OOD samples from INDs. Comprehensive experiments and analysis prove our method helps disentangle confidence score distributions of IND and OOD data.\footnote{Our code is available at \url{https://github.com/pris-nlp/EMNLP2022-energy_for_OOD/}.}
TIVE: A Toolbox for Identifying Video Instance Segmentation Errors
Venue: arXiv cs.CV 2022
Year: 2022

Since first proposed, Video Instance Segmentation(VIS) task has attracted vast researchers' focus on architecture modeling to boost performance. Though great advances achieved in online and offline paradigms, there are still insufficient means to identify model errors and distinguish discrepancies between methods, as well approaches that correctly reflect models' performance in recognizing object instances of various temporal lengths remain barely available. More importantly, as the fundamental model abilities demanded by the task, spatial segmentation and temporal association are still understudied in both evaluation and interaction mechanisms. In this paper, we introduce TIVE, a Toolbox for Identifying Video instance segmentation Errors. By directly operating output prediction files, TIVE defines isolated error types and weights each type's damage to mAP, for the purpose of distinguishing model characters. By decomposing localization quality in spatial-temporal dimensions, model's potential drawbacks on spatial segmentation and temporal association can be revealed. TIVE can also report mAP over instance temporal length for real applications. We conduct extensive experiments by the toolbox to further illustrate how spatial segmentation and temporal association affect each other. We expect the analysis of TIVE can give the researchers more insights, guiding the community to promote more meaningful explorations for video instance segmentation. The proposed toolbox is available at https://github.com/wenhe-jia/TIVE.
Automatic Analysis of Human Body Representations in Western Art
Venue: arXiv cs.CV 2022
Year: 2022

The way the human body is depicted in classical and modern paintings is relevant for art historical analyses. Each artist has certain themes and concerns, resulting in different poses being used more heavily than others. In this paper, we propose a computer vision pipeline to analyse human pose and representations in paintings, which can be used for specific artists or periods. Specifically, we combine two pose estimation approaches (OpenPose and DensePose, respectively) and introduce methods to deal with occlusion and perspective issues. For normalisation, we map the detected poses and contours to Leonardo da Vinci's Vitruvian Man, the classical depiction of body proportions. We propose a visualisation approach for illustrating the articulation of joints in a set of paintings. Combined with a hierarchical clustering of poses, our approach reveals common and uncommon poses used by artists. Our approach improves over purely skeleton based analyses of human body in paintings.
Industry-Scale Orchestrated Federated Learning for Drug Discovery
Venue: arXiv cs.LG 2022
Year: 2022

To apply federated learning to drug discovery we developed a novel platform in the context of European Innovative Medicines Initiative (IMI) project MELLODDY (grant n°831472), which was comprised of 10 pharmaceutical companies, academic research labs, large industrial companies and startups. The MELLODDY platform was the first industry-scale platform to enable the creation of a global federated model for drug discovery without sharing the confidential data sets of the individual partners. The federated model was trained on the platform by aggregating the gradients of all contributing partners in a cryptographic, secure way following each training iteration. The platform was deployed on an Amazon Web Services (AWS) multi-account architecture running Kubernetes clusters in private subnets. Organisationally, the roles of the different partners were codified as different rights and permissions on the platform and administrated in a decentralized way. The MELLODDY platform generated new scientific discoveries which are described in a companion paper.
Bridging the Gap between Local Semantic Concepts and Bag of Visual Words for Natural Scene Image Retrieval
Venue: arXiv cs.CV 2022
Year: 2022

This paper addresses the problem of semantic-based image retrieval of natural scenes. A typical content-based image retrieval system deals with the query image and images in the dataset as a collection of low-level features and retrieves a ranked list of images based on the similarities between features of the query image and features of images in the image dataset. However, top ranked images in the retrieved list, which have high similarities to the query image, may be different from the query image in terms of the semantic interpretation of the user which is known as the semantic gap. In order to reduce the semantic gap, this paper investigates how natural scene retrieval can be performed using the bag of visual word model and the distribution of local semantic concepts. The paper studies the efficiency of using different approaches for representing the semantic information, depicted in natural scene images, for image retrieval. An extensive experimental work has been conducted to study the efficiency of using semantic information as well as the bag of visual words model for natural and urban scene image retrieval.
Robust Planning for Human-Robot Joint Tasks with Explicit Reasoning on Human Mental State
Venue: arXiv cs.RO 2022
Year: 2022

We consider the human-aware task planning problem where a human-robot team is given a shared task with a known objective to achieve. Recent approaches tackle it by modeling it as a team of independent, rational agents, where the robot plans for both agents' (shared) tasks. However, the robot knows that humans cannot be administered like artificial agents, so it emulates and predicts the human's decisions, actions, and reactions. Based on earlier approaches, we describe a novel approach to solve such problems, which models and uses execution-time observability conventions. Abstractly, this modeling is based on situation assessment, which helps our approach capture the evolution of individual agents' beliefs and anticipate belief divergences that arise in practice. It decides if and when belief alignment is needed and achieves it with communication. These changes improve the solver's performance: (a) communication is effectively used, and (b) robust for more realistic and challenging problems.
The Role of User Reviews in App Updates: A Preliminary Investigation on App Release Notes
Venue: arXiv cs.SE 2022
Year: 2022

Release planning for mobile apps has recently become an area of active research. Prior research in this area concentrated on the analysis of release notes and on tracking user reviews to support app evolution with issue trackers. However, little is known about the impact of user reviews on the evolution of mobile apps. Our work explores the role of user reviews in app updates based on release notes. For this purpose, we collected user reviews and release notes of Spotify, the 'number one' app in the 'Music' category in Apple App Store, as the research data. Then, we manually removed non-informative parts of each release note, and manually determined the relevance of the app reviews with respect to the release notes. We did this by using Word2Vec calculation techniques based on the top 80 app release notes with the highest similarities. Our empirical results show that more than 60% of the matched reviews are actually irrelevant to the corresponding release notes. When zooming in at these relevant user reviews, we found that around half of them were posted before the new release and referred to requests, suggestions, and complaints. Whereas, the other half of the relevant user reviews were posted after updating the apps and concentrated more on bug reports and praise.
Flipped Classroom: Effective Teaching for Time Series Forecasting
Venue: arXiv cs.LG 2022
Year: 2022

Sequence-to-sequence models based on LSTM and GRU are a most popular choice for forecasting time series data reaching state-of-the-art performance. Training such models can be delicate though. The two most common training strategies within this context are teacher forcing (TF) and free running (FR). TF can be used to help the model to converge faster but may provoke an exposure bias issue due to a discrepancy between training and inference phase. FR helps to avoid this but does not necessarily lead to better results, since it tends to make the training slow and unstable instead. Scheduled sampling was the first approach tackling these issues by picking the best from both worlds and combining it into a curriculum learning (CL) strategy. Although scheduled sampling seems to be a convincing alternative to FR and TF, we found that, even if parametrized carefully, scheduled sampling may lead to premature termination of the training when applied for time series forecasting. To mitigate the problems of the above approaches we formalize CL strategies along the training as well as the training iteration scale. We propose several new curricula, and systematically evaluate their performance in two experimental sets. For our experiments, we utilize six datasets generated from prominent chaotic systems. We found that the newly proposed increasing training scale curricula with a probabilistic iteration scale curriculum consistently outperforms previous training strategies yielding an NRMSE improvement of up to 81% over FR or TF training. For some datasets we additionally observe a reduced number of training iterations. We observed that all models trained with the new curricula yield higher prediction stability allowing for longer prediction horizons.
Determinants Influencing Intention to Use Social Commerce for Shopping in developing countries: A Case Study of Oman
Venue: arXiv cs.CY 2022
Year: 2022

Social media has had a significant impact on our individual lives, including our behavior regarding the purchasing of daily products. This study investigates the factors influencing Omani nationals' intentions to obtain products via social commerce. The researcher surveyed 202 participants and utilized the Technology Acceptance Model to develop the theoretical framework. The data collection was analyzed statistically using an appropriate testing mechanism. Statistical methods, including Cronbach's alpha and multiple linear regression, were utilized for reliability and hypotheses testing. After analyzing the collected data and testing the hypotheses, the findings indicated that perceived usefulness, enjoyment, and ease of use of social commerce affect positively on Omani nationals' intentions to utilize social commerce for shopping. The independent variables had a statistically significant impact on the intention to use social commerce shopping for products; these explain 69.9% of the variation on customers intention to utilize social commerce for shopping.
A new nonparametric interpoint distance-based measure for assessment of clustering
Venue: arXiv cs.LG 2022
Year: 2022

A new interpoint distance-based measure is proposed to identify the optimal number of clusters present in a data set. Designed in nonparametric approach, it is independent of the distribution of given data. Interpoint distances between the data members make our cluster validity index applicable to univariate and multivariate data measured on arbitrary scales, or having observations in any dimensional space where the number of study variables can be even larger than the sample size. Our proposed criterion is compatible with any clustering algorithm, and can be used to determine the unknown number of clusters or to assess the quality of the resulting clusters for a data set. Demonstration through synthetic and real-life data establishes its superiority over the well-known clustering accuracy measures of the literature.
Towards sustainability assessment of artificial intelligence in artistic practices
Venue: arXiv cs.CY 2022
Year: 2022

An increasing number of artists use Ai in their creative practices (Creative-Ai) and their works have by now become visible at prominent art venues. The research community has, on the other hand, recognized that there are sustainability concerns of using Ai technologies related to, for instance, energy consumption and the increasing size and complexity of models. These two conflicting trajectories constitute the starting point of our research. Here, we discuss insights from our currently on-going fieldwork research and outline considerations for drawing various limitations in sustainability assessment studies of Ai art. We provide ground for further, more specific sustainability assessments in the domain, as well as knowledge on the state of sustainability assessments in this domain.
Temporal Analysis and Gender Bias in Computing
Venue: arXiv cs.CY 2022
Year: 2022

Recent studies of gender bias in computing use large datasets involving automatic predictions of gender to analyze computing publications, conferences, and other key populations. Gender bias is partly defined by software-driven algorithmic analysis, but widely used gender prediction tools can result in unacknowledged gender bias when used for historical research. Many names change ascribed gender over decades: the "Leslie problem." Systematic analysis of the Social Security Administration dataset -- each year, all given names, identified by ascribed gender and frequency of use -- in 1900, 1925, 1950, 1975, and 2000 permits a rigorous assessment of the "Leslie problem." This article identifies 300 given names with measurable "gender shifts" across 1925-1975, spotlighting the 50 given names with the largest such shifts. This article demonstrates, quantitatively, there is net "female shift" that likely results in the overcounting of women (and undercounting of men) in earlier decades, just as computer science was professionalizing. Some aspects of the widely accepted 'making programming masculine' perspective may need revision.
AI, Opacity, and Personal Autonomy
Venue: arXiv cs.CY 2022
Year: 2022

Advancements in machine learning have fuelled the popularity of using AI decision algorithms in procedures such as bail hearings (Feller et al. 2016), medical diagnoses (Rajkomar et al. 2018; Esteva et al. 2019) and recruitment (Heilweil 2019, Van Esch et al. 2019). Academic articles (Floridi et al. 2018), policy texts (HLEG 2019), and popularizing books (O'Neill 2016, Eubanks 2018) alike warn that such algorithms tend to be _opaque_: they do not provide explanations for their outcomes. Building on a causal account of transparency and opacity as well as recent work on the value of causal explanation (Lombrozo 2011, Hitchcock 2012), I formulate a moral concern for opaque algorithms that is yet to receive a systematic treatment in the literature: when such algorithms are used in life-changing decisions, they can obstruct us from effectively shaping our lives according to our goals and preferences, thus undermining our autonomy. I argue that this concern deserves closer attention as it furnishes the call for transparency in algorithmic decision-making with both new tools and new challenges.
Real-Time Automated Answer Scoring
Venue: arXiv cs.CY 2022
Year: 2022

In recent years, the role of big data analytics has exponentially grown and is now slowly making its way into the education industry. Several attempts are being made in this sphere in order to improve the quality of education being provided to students and while many collaborations have been carried out before, automated scoring of answers has been explored to a rather limited extent. One of the biggest hurdles to choosing constructed-response assessments over multiple-choice assessments is the effort and large cost that comes with their evaluation and this is precisely the issue that this project aims to solve. The aim is to accept raw-input from the student in the form of their answer, preprocess the answer, and automatically score the answer. In addition, we have made this a real-time system that captures "snapshots" of the writer's progress with respect to the answer, allowing us to unearth trends with respect to the way a student thinks, and how the student has arrived at their final answer.
ANFIS-based prediction of power generation for combined cycle power plant
Venue: arXiv cs.AI 2022
Year: 2022

This paper presents the application of an adaptive neuro-fuzzy inference system (ANFIS) to predict the generated electrical power in a combined cycle power plant. The ANFIS architecture is implemented in MATLAB through a code that utilizes a hybrid algorithm that combines gradient descent and the least square estimator to train the network. The Model is verified by applying it to approximate a nonlinear equation with three variables, the time series Mackey-Glass equation and the ANFIS toolbox in MATLAB. Once its validity is confirmed, ANFIS is implemented to forecast the generated electrical power by the power plant. The ANFIS has three inputs: temperature, pressure, and relative humidity. Each input is fuzzified by three Gaussian membership functions. The first-order Sugeno type defuzzification approach is utilized to evaluate a crisp output. Proposed ANFIS is cable of successfully predicting power generation with extremely high accuracy and being much faster than Toolbox, which makes it a promising tool for energy generation applications.
Machine Learning Technique Predicting Video Streaming Views to Reduce Cost of Cloud Services
Venue: arXiv cs.LG 2022
Year: 2022

Video streams tremendously occupied the highest portion of online traffic. Multiple versions of a video are created to fit the user's device specifications. In cloud storage, Keeping all versions of frequently accessed video streams in the repository for the long term imposes a significant cost paid by video streaming providers. Generally, the popularity of a video changes each period of time, which means the number of views received by a video could be dropped, thus, the video must be deleted from the repository. Therefore, in this paper, we develop a method that predicts the popularity of each video stream in the repository in the next period. On the other hand, we propose an algorithm that utilizes the predicted popularity of a video to compute the storage cost, and then it decides whether the video will be kept or deleted from the cloud repository. The experiment results show a cost reduction of the cloud services by 15% compared to keeping all video streams.
Modeling the Lighting in Scenes as Style for Auto White-Balance Correction
Venue: arXiv cs.CV 2022
Year: 2022

Style may refer to different concepts (e.g. painting style, hairstyle, texture, color, filter, etc.) depending on how the feature space is formed. In this work, we propose a novel idea of interpreting the lighting in the single- and multi-illuminant scenes as the concept of style. To verify this idea, we introduce an enhanced auto white-balance (AWB) method that models the lighting in single- and mixed-illuminant scenes as the style factor. Our AWB method does not require any illumination estimation step, yet contains a network learning to generate the weighting maps of the images with different WB settings. Proposed network utilizes the style information, extracted from the scene by a multi-head style extraction module. AWB correction is completed after blending these weighting maps and the scene. Experiments on single- and mixed-illuminant datasets demonstrate that our proposed method achieves promising correction results when compared to the recent works. This shows that the lighting in the scenes with multiple illuminations can be modeled by the concept of style. Source code and trained models are available on https://github.com/birdortyedi/lighting-as-style-awb-correction.