Reviewerly/exHarmony huggingface dataset:
Pooling Revisited: Your Receptive Field is Suboptimal

The size and shape of the receptive field determine how the network aggregates local features, and affect the overall performance of a model considerably. Many components in a neural network, such as depth, kernel sizes, and strides for convolution and pooling, influence the receptive field. However, they still rely on hyperparameters, and the receptive fields of existing models result in suboptimal shapes and sizes. Hence, we propose a simple yet effective Dynamically Optimized Pooling operation, referred to as DynOPool, which learns the optimized scale factors offeature maps end-to-end. Moreover, DynOPool determines the proper resolution of a feature map by learning the desirable size and shape of its receptive field, which allows an operator in a deeper layer to observe an input image in the optimal scale. Any kind of resizing modules in a deep neural network can be replaced by DynOPool with minimal cost. Also, DynOPool controls the complexity of the model by introducing an additional loss term that constrains computational cost. Our experiments show that the models equipped with the proposed learnable resizing module outperform the baseline algorithms on multiple datasets in image classification and semantic segmentation.
Human-Robot Interaction in Public Spaces

There has been a recent trend to test robots and intelligent virtual agents as social interaction partners in public domains. Commercial solutions such as Pepper or Cruz are increasingly being tested in scenarios outside the lab. Though at the same, time customer value and business models for social robots in public spaces are scarce, and with the recently halted production of the Pepper, it seems evident that there is no killer application for social robots in public spaces yet. This work wants to break the boundaries between academia and business and give both sides a venue to exchange lessons learned and develop a roadmap on the technical, legal, ethical, and business challenges for deploying social robots.
Consensus-Guided Keyword Targeting for Video Captioning

Mainstream video captioning models (VCMs) are trained under fully supervised learning that relies heavily on large-scaled high-quality video-caption pairs. Unfortunately, evaluating the corpora of benchmark datasets shows that there are many defects associated with humanly labeled annotations, such as variation of the caption length and quality for one video and word imbalance in captions. Such defects may pose a significant impact on model training. In this study, we propose to lower down the adverse impact of annotations and encourage VCMs to learn high-quality captions and more informative words via Consensus-Guided Keyword Targeting (CGKT) training strategy. Specifically, CGKT firstly aims at re-weighting each training caption using a consensus-based metric named CIDEr. Secondly, CGKT attaches more weights to those informative and uncommonly used words based on their frequency. Extensive experiments on MSVD and MSR-VTT show that the proposed CGKT can easily work with three VCMs to achieve significant CIDEr improvements. Moreover, compared with the conventional cross-entropy objective, our CGKT facilitates the generation of more comprehensive and better-quality captions.
Soft Error Reliability Assessment of Lightweight Cryptographic Algorithms for IoT Edge Devices

Security and reliability problems in edge devices can become the Achilles’ heel for their massive use in Internet of Things (IoT) systems. While most works address security by implementing lightweight cryptographic algorithms, this paper is the first to assess the soft error reliability of lightweight cryptographic algorithms targeting IoT edge devices. This paper performs soft error analysis for ten lightweight cryptographic algorithms considering two compilers and running on top of an IoT-ready commercial processor model (i.e. Arm Cortex-M7). From the ten lightweight cryptographic algorithms, IDEA shows the best trade-off between reliability and number of instructions. Index Terms-Lightweight Cryptographic (LWC) Algorithms, Soft Error, Reliability, Internet of Things (IoT), Edge Devices.
A Two-Phase Approach to Fast and Accurate Classification of Encrypted Traffic

Encryption technology has been widely used in today’s network communications. The early classification of encrypted flows is of great value to the control, allocation and management of resources in TCP/IP networks. In this paper, we propose TaTic, an early classification method for encrypted traffic, which aims to reduce the time spent observing the encrypted flows to be classified, and at the same time ensure the flow classification accuracy. TaTic is based on our key observation that the majority of encrypted flows can be classified accurately using only the first few packets, and we call such flows “easy flows”, whereas the rest of encrypted flows requires more packets for fine-grained analysis to achieve accurate traffic classification, and we call such flows “hard flows”. Given an encrypted flow, in the first phase, we use only the first few packets to quickly determine whether it is an easy flow or a hard flow; if it is an easy flow, we directly classify it in this phase; otherwise, we use more packets to perform traffic classification in the second phase. Therefore, we can greatly reduce the time spent in observing the flows without sacrificing the classification accuracy. Our experimental results show that TaTic can greatly reduce the unnecessary time spent in observing the flow to be classified, and at the same time ensure high classification accuracy. We compare our experimental results of TaTic with four existing methods. TaTic is superior to the existing methods in terms of both classification accuracy and average waiting time.
Turtle Heroes: Designing a Serious Game for a VR Interactive Tunnel

Immersive playful activities are regarded to be a promising way for increasing children's understanding of significant issues. This paper proposes an immersive interactive tunnel environment, for which we developed a serious game to raise awareness on children and young adults on environmental issues and endangered species, and specifically, about Caretta caretta. The paper first describes the design and implementation of the CAVE-tunnel system; then, it discusses the design and development of the serious game "Turtle Heroes". This is a work in progress while more games are in the design process for this VR interactive tunnel setup. Possible uses, opportunities and future works-games in this environment are also discussed.
StarSum: A Star Architecture Based Model for Extractive Summarization

Extractive summarization aims to produce a concise summary while retaining the key information through the way of selecting sentences from the original document. Under such background, learning inter-sentence relations has hitherto been the issue of most concern. In this study, we propose a Star architecture based model for extractive summarization (StarSum), that takes advantage of self-attention strategy based Transformer and star-shaped structure, models sentences within a document as satellite nodes and introduces a virtual star node, constructs a star model for each document to learn inter-sentence relations. Based on the constructed star-shaped model, we further develop two sentence representation learning algorithms, namely star guiding satellite (SGS) algorithm and star incorporating satellite (SIS) algorithm, in order to extract summary-worthy sentences. Experimental results on CNN/Daily Mail, New York Times (NYT) and XSum datasets prove that StarSum model achieves advanced performance for extractive summarization and has comparable performance to the state-of-the-art extractive summarization model. The results also demonstrate that the SIS algorithm is more effective than the SGS algorithm.
Methods for stabilizing models across large samples of projects (with case studies on predicting defect and project health)

Despite decades of research, Software Engineering (SE) lacks widely accepted models (that offer precise quantitative stable predictions) about what factors most influence software quality. This paper provides a promising result showing such stable models can be generated using a new transfer learning framework called "STABILIZER". Given a tree of recursively clustered projects (using project meta-data), STABILIZER promotes a model upwards if it performs best in the lower clusters (stopping when the promoted model performs worse than the models seen at a lower level).
Machine Learning Approach for Detection of Track Assets for Railroad Health Monitoring with Drone Images

With the advancements in technology made in the 21 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">st</sup> century, object detection has attracted a lot of attention in recent years. It is probably the most well-known term within the domain of computer vision and it encounters some really interesting problems. In order to gain complete image understanding, we classify different images along with also trying precise estimation of the objects and the locations of these objects contained in each of the images. Object detection is a collection of these related tasks for identifying objects in digital photographs. In this work, we perform detection of different track assets in drone images. For this purpose we have utilized a pretrained model as they are convenient for serving the purpose. The classes in which the images can be categorized into are listed as ‘Construction’, ‘Power Junction/Brick’, ‘Cement Slabs’, ‘Transformer wires’, ‘Garbage’, ‘Person’. We have used “YOLO”v3 (You Only Look Once) for track asset detection. We have initially given an image as input to YOLOv3 and the framework then divides the input image into grids. Image classification and localization are then applied onto each grid. When an image is provided as an input, YOLOv3 framework predicts the bounding boxes and their corresponding class probabilities for objects (if any are found).
Binarization of photographed documents image quality, processing time and size assessment

Today, over eighty percent of the world's population owns a smart-phone with an in-built camera, and they are very often used to photograph documents. Document binarization is a key process in many document processing platforms. This competition on binarizing photographed documents assessed the quality, time, space, and performance of five new algorithms and sixty-four "classical" and alternative algorithms. The evaluation dataset is composed of offset, laser, and deskjet printed documents, photographed using six widely-used mobile devices with the strobe flash on and off, under two different angles and places of capture.
Uncertainty-Based Spatial-Temporal Attention for Online Action Detection

Online action detection aims at detecting the ongoing action in a streaming video. In this paper, we proposed an uncertainty-based spatial-temporal attention for online action detection. By explicitly modeling the distribution of model parameters, we extend the baseline models in a probabilistic manner. Then we quantify the predictive uncertainty and use it to generate spatial-temporal attention that focus on large mutual information regions and frames. For inference, we introduce a two-stream framework that combines the baseline model and the probabilistic model based on the input uncertainty. We validate the effectiveness of our method on three benchmark datasets: THUMOS-14, TVSeries, and HDD. Furthermore, we demonstrate that our method generalizes better under different views and occlusions, and is more robust when training with small-scale data.
Binary Gravitational Subspace Search for Outlier Detection in High Dimensional Data Streams

In recent years, technology has continued to rapidly evolve, resulting in the generation of high-dimensional data streams. Combining the streaming scenario and high dimensionality is a particularly complex task specifically for outlier detection. This is due to the data stream’s unique properties, such as restricted space and time, and concept drift, in addition to the influence of the curse of dimensionality in high-dimensional space. Typically, interesting knowledge including outliers resides in low-dimensional subspaces of the full feature space. Finding these subspaces is considered an NP-Hard problem and requires careful attention, especially in the context of data streams. To address these issues, we proposed BGSSA (Binary Gravitational Subspace Search Algorithm), a novel metaheuristic-based subspace search method for outliers in high dimensional data streams. The idea behind is to adapt the binary GSA algorithm by producing the top best solutions instead of a single one in the original method to find, for each streaming window, relevant subspaces composed of independent features, where outlier detection will be performed. The relevance of a subspace is evaluated by the contrast measure. Experiments on real and synthetic datasets confirm the feasibility of our solution as well as its performance improvement in comparison with the main approaches studied in the literature.
Detecting interpersonal conflict in issues and code review

Interpersonal conflict in code review, such as toxic language or an unnecessary pushback, is associated with negative outcomes such as stress and turnover. Automatic detection is one approach to prevent and mitigate interpersonal conflict. Two recent automatic detection approaches were developed in different settings: a toxicity detector using text analytics for open source issue discussions and a pushback detector using logs-based metrics for corporate code reviews. This paper tests how the toxicity detector and the pushback detector can be generalized beyond their respective contexts and discussion types, and how the combination of the two can help improve interpersonal conflict detection. The results reveal connections between the two concepts.
Entry-Flipped Transformer for Inference and Prediction of Participant Behavior

Some group activities, such as team sports and choreographed dances, involve closely coupled interaction between participants. Here we investigate the tasks of inferring and predicting participant behavior, in terms of motion paths and actions, under such conditions. We narrow the problem to that of estimating how a set target participants react to the behavior of other observed participants. Our key idea is to model the spatio-temporal relations among participants in a manner that is robust to error accumulation during frame-wise inference and prediction. We propose a novel Entry-Flipped Transformer (EF-Transformer), which models the relations of participants by attention mechanisms on both spatial and temporal domains. Unlike typical transformers, we tackle the problem of error accumulation by flipping the order of query, key, and value entries, to increase the importance and fidelity of observed features in the current frame. Comparative experiments show that our EF-Transformer achieves the best performance on a newly-collected tennis doubles dataset, a Ceilidh dance dataset, and two pedestrian datasets. Furthermore, it is also demonstrated that our EF-Transformer is better at limiting accumulated errors and recovering from wrong estimations.
Multi-level Temporal Relation Graph for Continuous Sign Language Recognition

Temporal relation modeling is one of the key points to describe gesture changes in continuous sign language recognition. However, there are many similar gestures in sign language, therefore focusing only on global information can exacerbate the recognition ambiguity caused by various gesture combination. To alleviate this problem, we attempt to achieve the balance between the global information and the local information in gesture changes. Therefore, we construct a multi-level temporal relation graph (MLTRG). Specifically, the multi-level temporal relation graph of the video sequence is established according to different time spans, where the graph nodes are the corresponding visual features. Then the feature fusion and propagation of the multi-level temporal relation graph are performed by a graph convolutional network (GCN). Finally, we can reason and balance the global and the local temporal information of gesture changes in continuous sign language videos. We evaluate our method on the large-scale public datasets RWTH-PHOENIX-Weather-2014 and 2014T, the results prove the advantages and effectiveness of our method.
The combination of transformer and CNN in computer vision

CNN, which has been dominant in the field of computer vision, has performed well in various downstream tasks. CNN architecture improves the performance of CNN backbone network and promotes the rapid development of the whole industry by expanding the scale, increasing connections and finding more complex convolution relationships. On the other hand, in the evolution of network architecture, different strategies have been adopted in the field of natural language processing (NLP), and now transformer architecture is dominant. Its great success in the field of NLP has prompted researchers to study its adaptability to computer vision. This paper summarizes various forms of transformer in computer vision, and makes experiments and comparisons at the end. Finally, it summarizes the current situation of transformer in the field of computer vision and looks forward to its great potential.
Anomaly Detection on Health Data

The identification of anomalous records in medical data is an important problem with numerous applications such as detecting anomalous reading, anomalous patient health condition, health insurance fraud detection and fault detection in mechanical components. This paper compares the performances of seven state-of-the-art anomaly detection algorithms to do detect anomalies in healthcare data. Our experimental results in six datasets show that the state-of-the-art method of isolation based method iForest has a better performance overall in terms of AUC and runtime.
ANALYSIS OF THE FACTORS INFLUENCING THE FORMATION OF THE TRANSACTION PRICE IN THE BLOCKCHAIN

The blockchain as the tool was invented together with the Bitcoin and evolving. Despite the fact that new blockchains appeared, like Ethereum and other, the first one was not explored enough from the perspective of understanding dependencies between its elements. Mathematical and cryptographic dependencies are well described in the scientific literature, but dependencies exploration and description in relation to its economical usage of it is still missing. The blockchain structure and mechanics provide that transaction registered within the blockchain are not free for the final user. The transaction price in the Blockchain exists. This transaction price can be decisive for the implementation of a project using blockchain. Understanding factors, which affect this price forming helps to understand frames, when usage of the blockchain is effective. Taking into account all above mentioned, one of the targets of this article is to analyze the life cycle of transactions in the blockchain and determine the factors that have a significant impact on the formation of the transaction price in the Bitcoin Blockchain. From the other hand blockchain assumed as the competitor or even “killer” of the traditional payment systems. There is the list of elements, which should be taken in account comparing both approaches for the transaction execution. This list includes comparing deficits and advantages of both systems, like security issues, transaction speed issues and so on. From this perspective based on the comparing the transactional price forming in both systems, author assess the prospects for using transactions in the Blockchain in comparing with the traditional payment systems.
‘Cool glasses, where did you get them?” Generating Visually Grounded Conversation Starters for Human-Robot Dialogue

Visually situated language interaction is an important challenge in multi-modal Human-Robot Interaction (HRI).In this context we present a data-driven method to generate situated conversation starters based on visual context.We take visual data about the interactants and generate appropriate greetings for conversational agents in the context of HRI.For this, we constructed a novel open-source data set consisting of 4000 HRIoriented images of people facing the camera, each augmented by three conversation-starting questions.We compared a baseline retrieval-based model and a generative model.Human evaluation of the models using crowdsourcing shows that the generative model scores best, specifically at correctly referencing visual features.We also investigated how automated metrics can be used as a proxy for human evaluation and found that common automated metrics are a poor substitute for human judgement.Finally, we provide a proof-of-concept demonstrator through an interaction with a Furhat social robot.
Automated Knowledge Graph Construction for Healthcare Domain

This research seeks to optimize the process of identifying correlations in common and high severity diseases via the fusion of knowledge graphs and deep learning artificial intelligence. Knowledge graphs can be complicated to construct and resource-intensive, alternatively, knowledge graphs can be seen to legitimize correlation incidence and better explain AI outputs. We propose automation of knowledge graph construction from identifying significant text frequency relations within established knowledge base document structures to identifying inter-feature relations and creating a novel approach for artificial intelligence and machine-learning feature extraction and feature selection. Our knowledge graph construction exploits the structured World Health Organization (WHO) International Classification of Disease (ICD) code chapters, which are specific to a single organ system of the human body. A sorted vector of text-to-chapter frequencies enables Wilcox Rank significance tests to determine the most related features.
Telemedicine System Based on Mixed Reality and Cognitive Speech Service Technologies

Because of the prevalence of COVID-19, medical capacity reduction, and inconvenience due to epidemic prevention policies, the development of telemedicine has been emphasized. An AI sensing device (i.e., Azure Kinect DK) with a mixed reality device (i.e., HoloLens 2) and window application are combined to design a system that completes the consultation process from a distance. This registering system uses an SQL server database, which reads the data of registered patients, selects the registration time and department on its own, and converts the data by PC/SC with an ISO7816 database. For the window application, the patient side screen and audio content for the window screen are captured through the lens and microphone array of Azure Kinect DK. HoloLens 2 is used as the main device on the doctor's side to design the window application for dictating case conversion and transmission, whereas the Unity engine is used to create mixed-reality scenes for viewing medical records and photos of patients. In the consultation part, doctor and patient devices use Dynamic 365 Assist to conduct video consultations via Microsoft Teams, transforming the traditional consultation form into a remote consultation service via the Internet. The consultation process records the consultation data by voice and then the prescription by Speech to Text and Azure cognitive voice service. It is expected to develop a hybrid-reality telemedicine system and combine metaverse-related technologies for more online interactive service functions. The future development goal is to integrate the virtual reality rehabilitation training system further to make the application of telemedicine more complete.