Reviewerly/exHarmony huggingface dataset:
Robust Scene Parsing by Mining Supportive Knowledge From Dataset

Scene parsing, or semantic segmentation, aims at labeling all pixels in an image with the predefined categories of things and stuff. Learning a robust representation for each pixel is crucial for this task. Existing state-of-the-art (SOTA) algorithms employ deep neural networks to learn (discover) the representations needed for parsing from raw data. Nevertheless, these networks discover desired features or representations only from the given image (content), ignoring more generic knowledge contained in the dataset. To overcome this deficiency, we make the first attempt to explore the meaningful supportive knowledge, including general visual concepts (i.e., the generic representations for objects and stuff) and their relations from the whole dataset to enhance the underlying representations of a specific scene for better scene parsing. Specifically, we propose a novel supportive knowledge mining module (SKMM) and a knowledge augmentation operator (KAO), which can be easily plugged into modern scene parsing networks. By taking image-specific content and dataset-level supportive knowledge into full consideration, the resulting model, called knowledge augmented neural network (KANN), can better understand the given scene and provide greater representational power. Experiments are conducted on three challenging scene parsing and semantic segmentation datasets: Cityscapes, Pascal-Context, and ADE20K. The results show that our KANN is effective and achieves better results than all existing SOTA methods.
Exploring adaptive boosting (AdaBoost) as a platform for the predictive modeling of tangible collection usage

Low-use tangible print collections represent a long-standing problem for academic libraries. Expanding on the previous research aimed at leveraging machine learning (ML) toward predicting patterns of collection use, this study explores the potential for adaptive boosting (AdaBoost) as a foundation for developing actionable predictive models of print title use. This study deploys the AdaBoost algorithm, with random forests used as the base classifier, via the adabag package for R. Methodological considerations associated with dataset congruence, as well as sample-based modeling versus novel data modeling, are explored in relation to four AdaBoost models that are trained and tested. Results of this study show AdaBoost as a promising ML solution for predictive modeling of print collections, with the central model of interest able to accurately predict use in over 85% of cases. This research also explores peripheral questions of interest related to general considerations when evaluating ML models, as well as the compatibility of similar models trained with e-book versus print book usage data.
Efficient Adversarial Defense without Adversarial Training: A Batch Normalization Approach

Adversarial training is one of the most effective methods for defending against adversarial attacks, but it is computationally costly. In this paper, we propose Saliency Adversarial Defense (SAD), an efficient defense algorithm that avoids adversarial training by minor modification of a deployed model. The saliency map is added to the input to enhance those pixels that are important for making decisions. This process causes a distribution shift to the original data. Interestingly, we find that this shift can be effectively fixed by only updating the statistics of batch normalization with the processed data without further training. We verify that SAD enlarges the average distance between processed data and the updated decision boundary, and significantly smooths the landscape. Hence the model can defend against stronger attacks and improve robustness. We show in experiments that the results hold for complex models and datasets. Our results demonstrate that SAD is superior in defending various attacks, including both white-box and black-box ones.
Identity Authentication Based on Dynamic Touch Behavior on Smartphone

Nowadays, smartphones are widely used in people's daily lives, which provides a new way for consumers to study, work, entertain, and socialize anytime and anywhere. Smartphones are commonly used to store users' private data and identity information, which is vulnerable to malicious attacks. Therefore, the identity authentication of smartphone users is crucial. In recent years, authentication using dynamic touch behaviors, based on the smartphone's touch screens and built-in sensors, has received wide attention. In this paper, we investigate user authentication methods based on dynamic touch behavior in recent years, including keystroke-based methods, tap-based methods, and swiping operation-based strategies. This survey summarizes the basic principle, system structure, feature parameters, classification, and evaluation of identity authentication based on touch dynamics. In addition, the research work of identity authentication based on different dynamic touch behaviors is described and compared in detail. Finally, this paper discusses the limitations, future work, and challenges of identity authentication using touch dynamics.
Mobile Interface Personalization During the Application Usage Based on Patterns Ontology Model for People with Special Needs

The paper is devoted to the problem of interface adaptation for people with special needs during interface usage. After creating the interface according to the user’s health problems and device characteristics the interface can be improved by applying interface patterns. This work evolves our previous researches on the development of adaptive user interfaces and presents the second part of the interface adaptation algorithm. Much attention in this work was given to the analysis of existing pattern-based adaptation methods, patterns classification and recommendations to their application. The second part of the algorithm was developed and described based on the ontological representation of the interface patterns and the information about user’s device and user’s behaviour and preferences. The set of adaptation rules was presented, an example of the adaptation mechanism reaction on user’s behaviour was described. In the paper, we implemented the set of patterns to be applicable and the corresponding ontology set of rules.
A real-time deep learning system for the translation of mexican signal language into text

The ability to communicate is one of the most important rights of a healthy individual. Hearing-impaired citizens face several challenges: There is not a common sign language and the percentage of the population that understands a given sign language is negligible. The benefits of a system capable of translating from a sign language, in particular, the Mexican Sign language, to text, is outstanding. In this contribution, we describe such system. It is based on convolutional neural networks, which are particularly useful tool to extract relevant features from images. The sign language is captured in hundreds of images, and the system is able to translate them to text via several convolutional stages and transfer learning with 91% accuracy. We report the architecture and performance of the system.
Delivering User Experience over Networks: Towards a Quality of Experience Centered Design Cycle for Improved Design of Networked Applications

Abstract To deliver the best user experience (UX), the human-centered design cycle (HCDC) serves as a well-established guideline to application developers. However, it does not yet cover network-specific requirements, which become increasingly crucial, as most applications deliver experience over the Internet. The missing network-centric view is provided by Quality of Experience (QoE), which could team up with UX towards an improved overall experience. By considering QoE aspects during the development process, it can be achieved that applications become network-aware by design. In this paper, the Quality of Experience Centered Design Cycle (QoE-CDC) is proposed, which provides guidelines on how to design applications with respect to network-specific requirements and QoE. Its practical value is showcased for popular application types and validated by outlining the design of a new smartphone application. We show that combining HCDC and QoE-CDC will result in an application design, which reaches a high UX and avoids QoE degradation.
TRACE: A Differentiable Approach to Line-Level Stroke Recovery for Offline Handwritten Text

Stroke order and velocity are helpful features in the fields of signature verification, handwriting recognition, and handwriting synthesis. Recovering these features from offline handwritten text is a challenging and well-studied problem. We propose a new model called TRACE (Trajectory Recovery by an Adaptively-trained Convolutional Encoder). TRACE is a differentiable approach that uses a convolutional recurrent neural network (CRNN) to infer temporal stroke information from long lines of offline handwritten text with many characters and dynamic time warping (DTW) to align predictions and ground truth points. TRACE is perhaps the first system to be trained end-to-end on entire lines of text of arbitrary width and does not require the use of dynamic exemplars. Moreover, the system does not require images to undergo any pre-processing, nor do the predictions require any post-processing. Consequently, the recovered trajectory is differentiable and can be used as a loss function for other tasks, including synthesizing offline handwritten text.
Integrating Deep Reinforcement and Supervised Learning to Expedite Indoor Mapping

The challenge of mapping indoor environments is addressed. Typical heuristic algorithms for solving the motion planning problem are frontier-based methods, that are especially effective when the environment is completely unknown. However, in cases where prior statistical data on the environment's architectonic features is available, such algorithms can be far from optimal. Furthermore, their calculation time may increase substantially as more areas are exposed. In this paper we propose two means by which to overcome these shortcomings. One is the use of deep reinforcement learning to train the motion planner. The second is the inclusion of a pre-trained generative deep neural network, acting as a map predictor. Each one helps to improve the decision making through use of the learned structural statistics of the environment, and both, being realized as neural networks, ensure a constant calculation time. We show that combining the two methods can shorten the duration of the mapping process by up to 4 times, compared to frontier-based motion planning.
Multi-Task Learning Framework for Motion Estimation and Dynamic Scene Deblurring

Motion blur, which disturbs human and machine perceptions of a scene, has been considered an unnecessary artifact that should be removed. However, the blur can be a useful clue to understanding the dynamic scene, since various sources of motion generate different types of artifacts. Motivated by the relationship between motion and blur, we propose a motion-aware feature learning framework for dynamic scene deblurring through multi-task learning. Our multi-task framework simultaneously estimates a deblurred image and a motion field from a blurred image. We design the encoder-decoder architectures for two tasks, and the encoder part is shared between them. Our motion estimation network could effectively distinguish between different types of blur, which facilitates image deblurring. Understanding implicit motion information through image deblurring could improve the performance of motion estimation. In addition to sharing the network between two tasks, we propose a reblurring loss function to optimize the overall parameters in our multi-task architecture. We provide an intensive analysis of complementary tasks to show the effectiveness of our multi-task framework. Furthermore, the experimental results demonstrate that the proposed method outperforms the state-of-the-art deblurring methods with respect to both qualitative and quantitative evaluations.
Disturbance-immune weight sharing for neural architecture search

Neural architecture search (NAS) has gained increasing attention in the community of architecture design. One of the key factors behind the success lies in the training efficiency brought by the weight sharing (WS) technique. However, WS-based NAS methods often suffer from a performance disturbance (PD) issue. That is, the training of subsequent architectures inevitably disturbs the performance of previously trained architectures due to the partially shared weights. This leads to inaccurate performance estimation for the previous architectures, which makes it hard to learn a good search strategy. To alleviate the performance disturbance issue, we propose a new disturbance-immune update strategy for model updating. Specifically, to preserve the knowledge learned by previous architectures, we constrain the training of subsequent architectures in an orthogonal space via orthogonal gradient descent. Equipped with this strategy, we propose a novel disturbance-immune training scheme for NAS. We theoretically analyze the effectiveness of our strategy in alleviating the PD risk. Extensive experiments on CIFAR-10 and ImageNet verify the superiority of our method.
Adversarial Variational Knowledge Distillation

Knowledge Distillation (KD) is one of the most popular and effective techniques for model compression and knowledge transfer. However, most existing KD approaches are heavily relying on the labeled training data, which is usually unavailable due to privacy concerns. Thus, data-free KD focus on restoring the training data with Generative Adversarial Networks (GANs) by either catering the pre-trained teacher or fooling the student. In this paper we introduce Adversarial Variational Knowledge Distillation (AVKD), a framework that formulates the restoring process as Variational Autoencoders (VAEs). Different from vanilla VAEs, AVKD is specified by a pre-trained teacher model \(p(y|x)\) of the visible labels \(y\) given the latent \(x\), a prior \(p(x)\) over the latent variables and an approximate generative model \(q(x|y)\). In practice, we refer the prior \(p(x)\) as an alternate unlabeled data distribution from other related domains. Similar to Adversarial Variational Bayes (AVB), we estimate the KL-divergence term between \(p(x)\) and \(q(x|y)\) by introducing a discriminator network. Although the original training data are unavailable, we argue that the prior data drawn from other related domains can be easily obtained to learn the knowledge distillation efficiently. Extensive experiments testify that our method outperforms the state-of-the-art algorithms in the absence of the original training data, with performance approaching the case where the original training data are provided.
Security-Based Safety Hazard Analysis Using FMEA: A DAM Case Study

Safety and security emerge to be the most significant features of a Cyber-Physical System (CPS). Safety and security of a system are interlaced concepts and have mutual impact on each other. In the last decade, there are many cases where security breach resulted in safety hazards. There have been very few studies in the literature that address the integrated safety security risk assessment. Since, the need of the time is to consider both safety and security concurrently not even consequently. To close this gap, we aim to: (i) perform hazard analysis using Failure Mode Effect Analysis (FMEA) of a cyber physical system case i.e., Dam case study, and (ii) perform risk identification, risk analysis and mitigation for the said case. As a result, we extracted the potential failure modes, failure causes, failure effects, and the risk priority number. In addition, we also identified the safety requirements for the modes of the subject.
A Span Extraction Approach for Information Extraction on Visually-Rich Documents

Information extraction (IE) for visually-rich documents (VRDs) has achieved SOTA performance recently thanks to the adaptation of Transformer-based language models, which shows the great potential of pre-training methods. In this paper, we present a new approach to improve the capability of language model pre-training on VRDs. Firstly, we introduce a new query-based IE model that employs span extraction instead of using the common sequence labeling approach. Secondly, to extend the span extraction formulation, we propose a new training task focusing on modelling the relationships among semantic entities within a document. This task enables target spans to be extracted recursively and can be used to pre-train the model or as an IE downstream task. Evaluation on three datasets of popular business documents (invoices, receipts) shows that our proposed method achieves significant improvements compared to existing models. The method also provides a mechanism for knowledge accumulation from multiple downstream IE tasks.
Fuzzy Logic Implementation on Motion of Tennis Ball Picker Robot

The role of the tennis ball picker robot is to work to pick up tennis balls that are on the court after they are not used in a match. The tennis ball picker robot is an implementation of a vision-based robot designed to use object tracking techniques in tracking an object by utilizing digital image processing. This process gives the robot the ability to know the motion of the tennis ball. In this research, the robot will track the position of a tennis ball which has the characteristics of color, shape and size. Pixy2 CMUcam5 image sensor use to detect the tennis ball, while to control the position of the robot uses the fuzzy logic method. Input from fuzzy logic is the x and y coordinate points of the tennis ball's position towards the robot. The output of the fuzzy logic is the speed of the motor actuator on the robot wheel to move the robot to detected tennis ball position. Infrared sensor can detect tennis ball as far as 100 cm and tennis ball position point is detected by Pixy2 CMUcam5 camera sensor at coordinates (169, 170). By using the fuzzy logic method the robot can pick up a tennis ball at the 7th second.
Tracing COVID-19 - Older Adults’ Attitudes Toward Digital Contact Tracing and How to Increase Their Participation

The COVID-19 pandemic poses major challenges for health care systems. Contact tracing apps are being used around the world to help track and break the chain of infection. We conducted a qualitative study with eight older adults (65+) to find out how their lives have been affected by the pandemic. One of the topics covered was the "Stopp Corona" app, a contact tracing app by the national Red Cross and participants' attitude towards it. Despite the fact that most participants did not use the app, they expressed little concerns about the misuse of data as they have high trust in the Red Cross and the national government. Highlighting the societal benefit of contact tracing seems to be a major factor for uptake. Based on our results in comparison with recent studies on digital contact tracing, we suggest four recommendations that may support adoption of contact tracing apps by older adults.
A Multi-Task MRC Framework for Chinese Emotion Cause and Experiencer Extraction

Extracting emotion cause and experiencer from text can help people better understand users’ behavior patterns behind expressed emotions. Machine reading comprehension framework explicitly introduces a task-oriented query to boost the extraction task. In practice, how to learn a good task-oriented representation, accurately locate the boundary, and extract multiple causes and experiencers are the key technical challenges. To solve the above problems, this paper proposes BERT-based Machine Reading Comprehension Extraction Model with Multi-Task Learning (BERT-MRC-MTL). It first introduces query as prior knowledge and obtains text representation via BERT. Then, boundary-based and tag-based strategies are designed to select characters to be extracted, so as to extract multiple causes or experiencers simultaneously. Finally, hierarchical multi-task learning structure with residual connection is adopted to combine the answer extraction strategies. We conduct experiments on two public Chinese emotion datasets, and the results demonstrate the efficacy of our proposed model.
Learning a Voice-based Conversational Recommender using Offline Policy Optimization

Voice-based conversational recommenders offer a natural way to improve recommendation quality by asking the user for missing information. This talk details how we use offline policy optimization to learn a dialog manager that determines what items to present and what clarifying questions to ask, in order to maximize the success of the conversation. Counter-factual learning allows us to compare various modeling techniques using only logged conversational data. Our approach is applied to Amazon Music's first voice browsing experience (Alexa, help me find music), which interleaves disambiguation questions and music sample suggestions. Offline policy evaluation results show that an XGBoost reward regressor outperforms linear and neural policies on held out data. A first user-facing A/B test confirms our offline results, by increasing our task completion rate by 8% relative compared to our production rule-based conversational recommender, while reducing the number of turns to complete the task by 20%. A second A/B test shows that extending the set of candidate items to present and adding an embedding-based user-item affinity action feature improves task success rate further by 4% relative, while reducing the number of turns further by 13%. These results suggest that offline policy optimization from conversation logs is a viable way to foster conversational recommender research, while minimizing the number of user-facing experiments needed to determine the optimal dialog policy.
Blind video quality assessment based on multilevel video perception

Blind video quality assessment (VQA) metrics predict the quality of videos without the presence of reference videos. This paper proposes a new blind VQA model based on multilevel video perception, abbreviated as MVP. The model fuses three levels of video features occurring in natural video scenes to predict video quality: natural video statistics (NVS) features, global motion features and motion temporal correlation features. They represent video scene characteristics, video motion types, and video temporal correlation variations. In the process of motion feature extraction, motion compensation filtering video enhancement is adopted to highlight the motion characteristics of videos so as to improve the perceptual correlations of the video features. The experimental results on the LIVE and CSIQ video databases show that the predicted video scores of the new model are highly correlated with human perception and have low root mean square errors. MVP obviously outperforms state-of-art blind VQA metrics, and particularly demonstrates competitive performance even compared against top-performing full reference VQA metrics.