Reviewerly/exHarmony huggingface dataset:
Simulating quantized inference on convolutional neural networks

Mobile and embedded applications of convolutional neural networks (CNNs) use quantization to reduce model size and increase computational efficiency. However, working with quantized networks often implies using non-standard training and execution methods, as modern frameworks offer limited support to fixed-point operations. We propose a quantization approach simulating the effects of quantization in CNN inference without needing to be re-implemented using fixed-point arithmetic, reducing overhead and complexity in evaluating existing networks’ responses to quantization. The proposed method provides a fast way of performing post-training quantization with different bit widths in activations and weights. Our experimental results on ImageNet CNNs show a model size reduction of more than 50%, while maintaining classification accuracy without a need for retraining. We also measured the relationship between classification complexity and tolerance to quantization, finding an inverse correlation between quantization level and dataset complexity.
A Dual-Questioning Attention Network for Emotion-Cause Pair Extraction with Context Awareness

Emotion-cause pair extraction (ECPE), an emerging task in sentiment analysis, aims at extracting pairs of emotions and their corresponding causes in documents. This is a more challenging problem than emotion cause extraction (ECE), since it requires no emotion signals which are demonstrated as an important role in the ECE task. Existing work follows a two-stage pipeline which identifies emotions and causes at the first step and pairs them at the second step. However, error propagation across steps and pair combining without contextual information limits the effectiveness. Therefore, we propose a Dual-Questioning Attention Network to alleviate these limitations. Specifically, we question candidate emotions and causes to the context independently through attention networks for a contextual and semantical answer. Also, we explore how weighted loss functions in controlling error propagation between steps. Empirical results show that our method performs better than baselines in terms of multiple evaluation metrics. The source code can be obtained at https://github.com/QixuanSun/DQAN.
Fast Deep Neural Architecture Search for Wearable Activity Recognition by Early Prediction of Converged Performance

Neural Architecture Search (NAS) has the potential to uncover more performant networks for wearable activity recognition, but a naive evaluation of the search space is computationally expensive. We introduce neural regression methods for predicting the converged performance of a Deep Neural Network (DNN) using validation performance in early epochs and topological and computational statistics. Our approach shows a significant improvement in predicting converged testing performance. We apply this to the optimisation of the convolutional feature extractor of an LSTM recurrent network using NAS with deep Q-learning, optimising the kernel size, number of kernels, number of layers and the connections between layers, allowing for arbitrary skip connections and dimensionality reduction with pooling layers. We find architectures which achieve up to 4% better F1 score on the recognition of gestures in the Opportunity dataset than our implementation of the state of the art model DeepConvLSTM, while reducing the search time by >90% over a random search. This opens the way to rapidly search for well performing dataset-specific architectures.
Ubiquitous Interactions for Heads-Up Computing: Understanding Users’ Preferences for Subtle Interaction Techniques in Everyday Settings

In order to satisfy users’ information needs while incurring minimum interference to their ongoing activities, previous studies have proposed using Optical Head-mounted Displays (OHMDs) with different input techniques. However, it is unclear how these techniques compare against one another in terms of being comfortable and non-intrusive to a user’s everyday tasks. Through a wizard-of-oz study, we thus compared four subtle interaction techniques (feet, arms, thumb-index-fingers, and teeth) in three daily hands-busy tasks under different settings (giving a presentation–sitting, carrying bags–walking, and folding clothes–standing). We found that while each interaction technique has its niche, thumb-index-finger interaction has the best overall balance and is most preferred as a cross-scenario subtle interaction technique for smart glasses. We provide further evaluation of thumb-index-finger interaction with an in-the-wild study with 8 users. Our results contribute to an enhanced understanding of user preferences for subtle interaction techniques with smart glasses for everyday use.
CtnR: Compress-then-Reconstruct Approach for Multimodal Abstractive Summarization

With the rapid growth of multimodal data in social medias and the huge requirement of short but abundant information. Multimodal summarization has drawn much attention in both industry and academia. It usually obtains textual summary from multiple sources by computer vision or nature language processing technologies. However, there are also two challenges in modeling such task: 1) The feature representation is limited by the non-alignment among multimodal data; 2) Massive parallel data is required during training, which is time-consuming and laborious. In this paper, we introduce an unsupervised architecture (Compress-then-Reconstruct, CtnR) to generate the summary in an end-to-end manner and a Cross-Modal Transformer module (CMTrans) to fuse the multimodal non-alignment information. Comprehensive experiments show that the proposed CtnR framework with CMTrans outperforms mainstream unsupervised approaches in terms of BLEU, ROUGE and relevance scores on MSMO and Youtube News dataset, which increase 8.82% and 11.01% on average respectively.
Performance Evaluation of the Designed Two-Axis Solar Tracking System for Trabzon

In this paper, a photovoltaic system, which can move both east–west directions horizontally and in a north–south vertically, is designed according to the sun’s angle of incidence. Experimental studies have been carried out on this system, which can follow the change of the sun’s daily angle depending on the local time throughout the day. This study was evaluated for one of the cloudy provinces of Turkey: Trabzon. Optimum motor steps of the two-axis solar tracking system were determined, according to the open-loop control system in order that solar rays reach a perpendicular (90°) to the system. The power generation performance of the two-axis solar tracking system prototype was compared with that of a PV panel, which has a fixed position facing the south. Experimental studies were carried out for three cloudy days. The average power obtained from the fixed system is 12.5, 11.15, and 12.77 W, and the designed system generated 15.67, 14.05, and 15.79 W power, respectively. The results show that the designed system generates approximately 24.7% more power than the 45° fixed-tilted system with the proposed method. In Trabzon, that the number of sunny days is low encourages the widespread use of the two-axis solar tracking system design. The prototype will be an inspiration for researchers who want to generate maximum energy from solar tracking systems in cloudy places or any locations.
PlaTe: Visually-Grounded Planning With Transformers in Procedural Tasks

In this work, we study the problem of how to leverage instructional videos to facilitate the understanding of human decision-making processes, focusing on training a model with the ability to plan a goal-directed procedure from real-world videos. Learning structured and plannable state and action spaces directly from unstructured videos is the key technical challenge of our task. There are two problems: first, the appearance gap between the training and validation datasets could be large for unstructured videos; second, these gaps lead to decision errors that compound over the steps. We address these limitations with Planning Transformer (PlaTe), which has the advantage of circumventing the compounding prediction errors that occur with single-step models during long model-based rollouts. Our method simultaneously learns the latent state and action information of assigned tasks and the representations of the decision-making process from human demonstrations. Experiments conducted on real-world instructional videos and an interactive environment show that our method can achieve a better performance in reaching the indicated goal than previous algorithms. We also validated the possibility of applying procedural tasks on a UR-5 platform. We make our code publicly available and support academic research purposes.
How Templated Requirements Specifications Inhibit Creativity in Software Engineering

Desiderata is a general term for stakeholder needs, desires or preferences. Recent experiments demonstrate that presenting desiderata as templated requirements specifications leads to less creative solutions. However, these experiments do not establish how the presentation of desiderata affects design creativity. This study, therefore, aims to explore the cognitive mechanisms by which presenting desiderata as templated requirements specifications reduces creativity during software design. Forty-two software designers, organized into 21 pairs, participated in a dialog-based protocol study. Their interactions were transcribed and the transcripts were analyzed in two ways: (1) using inductive process coding and (2) using an a-priori coding scheme focusing on fixation and critical thinking. Process coding shows that participants exhibited seven categories of behavior: making design moves, uncritically accepting, rejecting, grouping, questioning, assuming and considering quality criteria. Closed coding shows that participants tend to accept given requirements and priority levels while rejecting newer, more innovative design ideas. Overall, the results suggest that designers fixate on desiderata presented as templated requirements specifications, hindering critical thinking. More precisely, requirements fixation mediates the negative relationship between specification formality and creativity.
Fast Pedestrian Detection for Real-World Crowded Scenarios on Embedded GPU

The behavior of individuals in crowds in public places has gained enormously in importance last year, for example through distancing requirements. However, automatically detecting pedestrians in real-world uncooperative scenarios remains a very challenging task. Especially crowded areas in surveillance footage are not only challenging for automatic vision systems, but also for human operators. Furthermore, complex detection models do not scale easily and are not traditionally designed for on-device processing in resource-constrained smart cameras, which become more and more popular due to technical and privacy issues at large events. In this work, we propose a new Fast Pedestrian Detector (FPD) based on RetinaNet which is a fast and efficient architecture for embedded platforms. The proposed FPD provides near real-time and real-time detection of hundreds of pedestrians on embedded platforms, outperforming popular YOLO-based approaches traditionally tuned for speed. Furthermore, by evaluating our approach on several different Jetson platforms in terms of speed and energy profiles, we highlight the challenges related to the deployment of a deep learning based pedestrian detector on embedded platforms for smart surveillance cameras.
Fine-Tuning Self-Supervised Multilingual Sequence-To-Sequence Models for Extremely Low-Resource NMT

Neural Machine Translation (NMT) tends to perform poorly in low-resource language settings due to the scarcity of parallel data. Instead of relying on inadequate parallel corpora, we can take advantage of monolingual data available in abundance. Training a denoising self-supervised multilingual sequence-to-sequence model by noising the available large scale monolingual corpora is one way to utilize monolingual data. For a pair of languages for which monolingual data is available in such a pre-trained multilingual denoising model, the model can be fine-tuned with a smaller amount of parallel data from this language pair. This paper presents fine-tuning self-supervised multilingual sequence-to-sequence pre-trained models for extremely low-resource domain-specific NMT settings. We choose one such pre-trained model: mBART. We are the first to implement and demonstrate the viability of non-English centric complete fine-tuning on multilingual sequence-to-sequence pre-trained models. We select Sinhala, Tamil and English languages to demonstrate fine-tuning on extremely low-resource settings in the domain of official government documents. Experiments show that our fine-tuned mBART model significantly outperforms state-of-the-art Transformer based NMT models in all pairs in all six bilingual directions, where we report a 4.41 BLEU score increase on Tamil→Sinhala and a 2.85 BLUE increase on Sinhala→ Tamil translation.
Arabic Speech Recognition Independent of Vocabulary for Isolated Words

This paper presents a vocabulary-independent speech recognition model of isolated Arabic words. This approach uses a restricted dictionary of syllables and phonemes, and it is based on the hidden Markov model (HMM). The main objective of this contribution is to remedy the problem of insufficient vocabularies in automatic speech recognition systems. This new model has made it possible to recognize any given word even if it has never been pronounced. The model gives a very considerable recognition rate comparable to the limited vocabulary speech recognition system of isolated words.
A new similarity-based classifier with Dombi aggregative operators

In this paper we extend the similarity classifier to cover Dombi aggregation operators. The similarity classifier was earlier studied with the ordered weighted averaging (OWA) operator, the generalized mean, and other operators. We concentrate on the use of Dombi operators during aggregation of similarities within the classifier. Four Dombi aggregation operators applied here include: conjunctive, disjunctive, weighted conjunctive, and the product operator. From each of these operators, we form a variant of the similarity classifier. The proposed methods were tested on four real world medical datasets which include: fertility, lung cancer, Haberman’s survival, and liver disorder. The new classifiers achieved improved classification accuracies compared with earlier methods. Compared with the classifier using the generalized mean, the proposed method achieved an improvement of 17.80% on fertility, 5.88% on lung cancer, 0.65% on Haberman’s survival, and 0.29% on liver disorder datasets.
Adaptive clustering-based approach for forgery detection in images containing similar appearing but authentic objects

Copy-move forgery is one of the well-known image forgery technique which exploits regions of the same image to create forged image by replicating or hiding authentic content of the original image. Original images can also contain similar looking but authentic objects. In such cases, identification of authentic and tampered images is a complicated task. To tackle this problem, we propose a method in which Stationary Wavelet Transform (SWT) and spatial-constrained edge preserving watershed segmentation are applied over input image in preprocessing step. Keypoint extraction and descriptor computation are performed using Cascaded Features from Accelerated Segment Test (Cascaded FAST) and Binary Robust Invariant Scalable Keypoint (BRISK) descriptor, respectively. Approximate nearest neighbor search is performed using Random Binary Search Tree (RBST) method. For keypoint clustering, Adaptive Density Peak Clustering (ADPC) technique is employed. Outlier removal is performed using Random Sample Consensus (RANSAC) technique. Further, forged regions are localized using correlation map generation. Experimental results display that the proposed approach can effectively distinguish between forged and original images containing similar appearing but authentic objects. It is also able to detect forged images sustaining different post-processing attacks. For COVERAGE dataset, proposed technique achieves high F-Measure = 86.901% and low False Positive Rate (FPR) = 15.241% in comparison to state-of-the-art techniques.
Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Medical image analysis using deep learning has recently been prevalent, showing great performance for various downstream tasks including medical image segmentation and its sibling, volumetric image segmentation. Particularly, a typical volumetric segmentation network strongly relies on a voxel grid representation which treats volumetric data as a stack of individual voxel ‘slices’, which allows learning to segment a voxel grid to be as straightforward as extending existing image-based segmentation networks to the 3D domain. However, using a voxel grid representation requires a large memory footprint, expensive test-time and limiting the scalability of the solutions. In this paper, we propose Point-Unet, a novel method that incorporates the efficiency of deep learning with 3D point clouds into volumetric segmentation. Our key idea is to first predict the regions of interest in the volume by learning an attentional probability map, which is then used for sampling the volume into a sparse point cloud that is subsequently segmented using a point-based neural network. We have conducted the experiments on the medical volumetric segmentation task with both a small-scale dataset Pancreas and large-scale datasets BraTS18, BraTS19, and BraTS20 challenges. A comprehensive benchmark on different metrics has shown that our context-aware Point-Unet robustly outperforms the SOTA voxel-based networks at both accuracies, memory usage during training, and time consumption during testing. Our code is available at https://github.com/VinAIResearch/Point-Unet.
Harmonizing Case Retrieval and Adaptation with Alternating Optimization

Case-based reasoning (CBR) research has developed numerous methods for learning to improve case retrieval and adaptation knowledge. Learning for each type of knowledge is usually pursued independently. However, it is well known that the knowledge containers of CBR are tightly coupled, in that changes in one can affect requirements for another, which suggests potential benefit for coupling learning across knowledge containers. This paper proposes applying alternative optimization to learn retrieval and adaptation knowledge together, in order to harmonize their behaviors. For a testbed system using neural network based similarity and adaptation, this study compares alternative optimization, independent learning, and learning by prioritizing adaptation for adaptation-guided retrieval. Results support that alternative optimization can help to balance both components and achieve good performance.
Signal Recognition for English Speech Translation Based on Improved Wavelet Denoising Method

The signal corresponding to English speech contains a lot of redundant information and environmental interference information, which will produce a lot of distortion in the process of English speech translation signal recognition. Based on this, a large number of studies focus on encoding and processing English speech, so as to achieve high-precision speech recognition. The traditional wavelet denoising algorithm plays an obvious role in the recognition of English speech translation signals, which mainly depends on the excellent local time-frequency domain characteristics of the wavelet signal algorithm, but the traditional wavelet signal algorithm is still difficult to select the recognition threshold, and the recognition accuracy is easy to be affected. Based on this, this paper will improve the traditional wavelet denoising algorithm, abandon the single-threshold judgment of the original traditional algorithm, innovatively adopt the combination of soft threshold and hard threshold, further solve the distortion problem of the denoising algorithm in the process of English speech translation signal recognition, improve the signal-to-noise ratio of English speech recognition, and further reduce the root mean square error of the signal. Good noise reduction effect is realized, and the accuracy of speech recognition is improved. In the experiment, the algorithm is compared with the traditional algorithm based on MATLAB simulation software. The simulation results are consistent with the actual theoretical results. At the same time, the algorithm proposed in this paper has obvious advantages in the recognition accuracy of English speech translation signals, which reflects the superiority and practical value of the algorithm.
Improving test case selection by handling class and attribute noise

Big data and machine learning models have been increasingly used to support software engineering processes and practices. One example is the use of machine learning models to improve test case selection in continuous integration. However, one of the challenges in building such models is the large volume of noise that comes in data, which impedes their predictive performance. In this paper, we address this issue by studying the effect of two types of noise, called class and attribute, on the predictive performance of a test selection model. For this purpose, we analyze the effect of class noise by using an approach that relies on domain knowledge for relabeling contradictory entries and removing duplicate ones. Thereafter, an existing approach from the literature is used to experimentally study the effect of attribute noise removal on learning. The analysis results show that the best learning is achieved when training a model on class-noise cleaned data only — irrespective of attribute noise. Specifically, the learning performance of the model reported 81% precision, 87% recall, and 84% f-score compared with 44% precision, 17% recall, and 25% f-score for a model built on uncleaned data. Finally, no causality relationship between attribute noise removal and the learning of a model for test case selection was drawn.
Complementary strengths? Evaluation of a hybrid human-machine scoring approach for a test of oral academic English

(2021). Complementary strengths? Evaluation of a hybrid human-machine scoring approach for a test of oral academic English. Assessment in Education: Principles, Policy & Practice: Vol. 28, Use of Innovative Technology in Oral Language Assessment, pp. 437-455.