Reviewerly/exHarmony huggingface dataset:
Analyse digital forensic evidences through a semantic-based methodology and NLP techniques

The increasing adoption of digital technologies to manage and process information used in everyday life, results in an increase in the demand for digital data analysis for investigative purposes. In fact, the reconstruction of computer and telematic crimes, or, in general, of crimes committed with computer systems, require the adoption of Computer Forensics best practices in order to extract relevant evidences from electronic devices, guaranteeing the integrity of data and their admissibility during a trial. The process of extraction, conservation, analysis and documentation of a forensic investigation can be enhanced by a framework that support investigators during their work, correlating evidences collected by different forensic tools. So, in this work we propose a semantic methodology and a system architecture for evidences correlation aiming to provide enhanced retrieval and reasoning capabilities.
Veritatem Dies Aperit - Temporally Consistent Depth Prediction Enabled by a Multi-Task Geometric and Semantic Scene Understanding Approach

Robust geometric and semantic scene understanding is ever more important in many real-world applications such as autonomous driving and robotic navigation. In this paper, we propose a multi-task learning-based approach capable of jointly performing geometric and semantic scene understanding, namely depth prediction (monocular depth estimation and depth completion) and semantic scene segmentation. Within a single temporally constrained recurrent network, our approach uniquely takes advantage of a complex series of skip connections, adversarial training and the temporal constraint of sequential frame recurrence to produce consistent depth and semantic class labels simultaneously. Extensive experimental evaluation demonstrates the efficacy of our approach compared to other contemporary state-of-the-art techniques.
An ultra-low voltage chaos-based true random number generator for IoT applications

Low-power consumption and low-voltage operation are critical enabling features for Internet of Things (IoT) devices that are powered from energy harvesting. This paper presents a chaos-based true random number generator (TRNG) that can operate at an ultra-low voltage (ULV) and be integrated into energy-constrained IoT devices for secure communications. Folded Bernoulli maps are adopted for random number generation. Switched-capacitor chaotic circuits utilize bulk-driven amplifiers to mitigate gate leakage issue, two-stage comparators to increase voltage headroom, and low-complexity calibration schemes to ensure robustness. This system covers a wide range of process and temperature variations while consuming 142 nW from a 0.4 V supply at a bit rate of 10 kb/s. The generated bits pass National Institute of Standards and Technology (NIST) Pub-800.22 randomness tests successfully.
Hierarchical multi-modal fusion FCN with attention model for RGB-D tracking

In this paper, we propose a RGB-D tracking algorithm, built upon hierarchical multi-modal fusion fully convolutional neural network (FCN) with attention model. First, we encode the depth images into three channels using the HHA representation to obtain the similar structure to the RGB images. Second, a multi-modal fusing features learning FCN with attention model is constructed, which can extract hierarchical multi-modal fusing features of the samples in RGB-D data. The attention model is adopted to exploit the importance weight of RGB and depth to fuse the features of the two modalities at multiple layers effectively, rather than concatenating the feature vectors of the two channels simply. Finally, the hierarchical multi-modal fusing features of the samples are input to the Efficient Convolution Operators (ECO) tracker, and the update strategy is improved by the occlusion detection in the depth images. Experimental results datasets demonstrate that the proposed RGB-D tracker achieves new state-of-art performance on the large-scale Princeton RGB-D Tracking Benchmark (PTB) dataset and the University of Birmingham RGB-D Tracking Benchmark (BTB).
Towards Visually Grounded Sub-word Speech Unit Discovery

In this paper, we investigate the manner in which interpretable sub-word speech units emerge within a convolutional neural network model trained to associate raw speech waveforms with semantically related natural image scenes. We show how diphone boundaries can be superficially extracted from the activation patterns of intermediate layers of the model, suggesting that the model may be leveraging these events for the purpose of word recognition. We present a series of experiments investigating the information encoded by these events.
Text Input in Virtual Reality: A Preliminary Evaluation of the Drum-Like VR Keyboard

The drum-like virtual reality (VR) keyboard is a contemporary, controller-based interface for text input in VR that uses a drum set metaphor. The controllers are used as sticks which, through downward movements, “press” the keys of the virtual keyboard. In this work, a preliminary feasibility study of the drum-like VR keyboard is described, focusing on the text entry rate and accuracy as well as its usability and the user experience it offers. Seventeen participants evaluated the drum-like VR keyboard by having a typing session and completing a usability and a user experience questionnaire. The interface achieved a good usability score, positive experiential feedback around its entertaining and immersive qualities, a satisfying text entry rate (24.61 words-per-minute), as well as moderate-to-high total error rate (7.2%) that can probably be further improved in future studies. The work provides strong indications that the drum-like VR keyboard can be an effective and entertaining way to type in VR.
Assessing the Practices and Products of Darkweb Firearm Vendors

The development of the Darknet as a parallel network to the Web in the 21st century has facilitated illegal trafficking in small arms, as defined by the United Nations. The authors have used investigative research methodologies to observe six weapon sale sites on the Darknet over a six-month period to identify sellers of firearms, the type and caliber of weapons for sale, manufacturer, price in Bitcoin, and the principle national origins of the firearms. This is the first study of its type to explore the illegal sale of firearms on the Darknet. This evidence can be used by law enforcement to intercept and shut down said sites and provide insight to the nature of the illegal arms trade on the Darknet.
Example‐Based Colourization Via Dense Encoding Pyramids

Abstract We propose a novel deep example‐based image colourization method called dense encoding pyramid network. In our study, we define the colourization as a multinomial classification problem. Given a greyscale image and a reference image, the proposed network leverages large‐scale data and then predicts colours by analysing the colour distribution of the reference image. We design the network as a pyramid structure in order to exploit the inherent multi‐scale, pyramidal hierarchy of colour representations. Between two adjacent levels, we propose a hierarchical decoder–encoder filter to pass the colour distributions from the lower level to higher level in order to take both semantic information and fine details into account during the colourization process. Within the network, a novel parallel residual dense block is proposed to effectively extract the local–global context of the colour representations by widening the network. Several experiments, as well as a user study, are conducted to evaluate the performance of our network against state‐of‐the‐art colourization methods. Experimental results show that our network is able to generate colourful, semantically correct and visually pleasant colour images. In addition, unlike fully automatic colourization that produces fixed colour images, the reference image of our network is flexible; both natural images and simple colour palettes can be used to guide the colourization.
Acoustically Grounded Word Embeddings for Improved Acoustics-to-word Speech Recognition

Direct acoustics-to-word (A2W) systems for end-to-end automatic speech recognition are simpler to train, and more efficient to decode with, than sub-word systems. However, A2W systems can have difficulties at training time when data is limited, and at decoding time when recognizing words outside the training vocabulary. To address these shortcomings, we investigate the use of recently proposed acoustic and acoustically grounded word embedding techniques in A2W systems. The idea is based on treating the final pre-softmax weight matrix of an AWE recognizer as a matrix of word embedding vectors, and using an externally trained set of word embeddings to improve the quality of this matrix. In particular we introduce two ideas: (1) Enforcing similarity at training time between the external embeddings and the recognizer weights, and (2) using the word embeddings at test time for predicting out-of-vocabulary words. Our word embedding model is acoustically grounded, that is it is learned jointly with acoustic embeddings so as to encode the words' acoustic-phonetic content; and it is parametric, so that it can embed any arbitrary (potentially out-of-vocabulary) sequence of characters. We find that both techniques improve the performance of an A2W recognizer on conversational telephone speech.
Machine Learning in Forensic Applications

The 2009 National Academy of Sciences report found pattern-evidence disciplines to be rife with subjectivity. In the decade since, machine learning methods have been developed to try to address that issue. By Alicia Carriquiry, Heike Hofmann, Xiao Hui Tai and Susan VanderPlas.
Designing to Distract

Many public buildings are entered through reception areas, intended for visitors to sit and wait in to be met. A concern is how to make visitors feel welcome while in transit. Hospitals, medical centres and other healthcare organisations are a special case where the challenge is to enable patients and families feel less anxious when waiting. One approach has been to design for distraction – where displays, surfaces, and interactive installations are created to draw visitor's attention away from their immediate thoughts. However, little is known as to how people respond to such interventions. We present the findings of an ethnographic study that examined the social and psychological effects of using distraction as a design principle in a children's hospital reception area. We discuss the challenges of designing to distract, in relation to how it can be combined with other architectural and HCI ones, when developing new human–building interfaces.
A New Coverage Flight Path Planning Algorithm Based on Footprint Sweep Fitting for Unmanned Aerial Vehicle Navigation in Urban Environments

This paper presents a new coverage flight path planning algorithm that finds collision-free, minimum length and flyable paths for unmanned aerial vehicle (UAV) navigation in three-dimensional (3D) urban environments with fixed obstacles for coverage missions. The proposed algorithm significantly reduces computational time, number of turns, and path overlapping while finding a path that passes over all reachable points of an area or volume of interest by using sensor footprints’ sweeps fitting and a sparse waypoint graph in the pathfinding process. We devise a novel footprints’ sweep fitting method considering UAV sensor footprint as coverage unit in the free spaces to achieve maximal coverage with fewer and longer footprints’ sweeps. After footprints’ sweeps fitting, the proposed algorithm determines the visiting sequence of footprints’ sweeps by formulating it as travelling salesman problem (TSP), and ant colony optimization (ACO) algorithm is employed to solve the TSP. Furthermore, we generate a sparse waypoint graph by connecting footprints’ sweeps’ endpoints to obtain a complete coverage flight path. The simulation results obtained from various scenarios fortify the effectiveness of the proposed algorithm and verify the aforementioned claims.
Track deformable objects from point clouds with structure preserved registration

Manipulating deformable objects is a challenging task for robots. The major difficulty lies in how to track these objects accurately, robustly, and efficiently, considering they have infinite-dimensional configuration space. To deal with these problems, this paper proposes a novel state estimator to track deformable objects from point clouds. A non-rigid registration method, named structure preserved registration (SPR), is developed to update the estimation by registering the object model towards the current point cloud measurement. Both the local structure and the global topology of the deformable object are considered during registration, which improves the estimation robustness under noise, outliers, and occlusions. The tracking result is further refined by running a dynamic simulation in parallel, which enforces the estimates to satisfy the physical constraints of the object. A series of real-time tracking experiments on 1D objects (ropes) and 2D objects (clothes) are performed to evaluate the proposed state estimator. A wire harness manipulation platform is also introduced where robots can manipulate soft wires to desired shapes and autonomously evaluate the manipulation quality through visual feedback.
Device Identification Based on Communication Analysis for the Internet of Things

As the Internet of Things (IoT) is rapidly expanding, a huge variety of devices is being connected to the Internet. Device management is becoming an important topic for IoT. Especially for using devices properly and securely, it is necessary to visualize what types of devices are in the network. However, most conventional device identification methods are not suitable for resource-constrained IoT devices. Therefore, we have developed a method of device identification that identifies the type and model of devices on the basis of general communication information. It determines the type and model of devices by calculating the similarity of features extracted from their network packets. The great merit of our proposed device identifier is that it can be applied to various IoT devices without special equipment. We conducted three experiments to evaluate its effectiveness. In the experiments, we focused on devices specialized for specific functions such as network cameras and factory-used devices because they are effective targets of our device identifier. In addition, we tried identifying models from devices of the same type. The first experiment revealed the relationship between the packet header information used for identification and the success of identification. The second experiment with 11 types of network cameras showed that the device identifier correctly identified nine of them. In addition, the third experiment in a simulated factory environment showed the device identifier correctly identified six types of factory-used devices. Thus, we have demonstrated the feasibility of the proposed device identifier in a real environment.
newNECTAR: Collaborative active learning for knowledge-based probabilistic activity recognition

The increasing popularity of ambient assisted living solutions is claiming adaptive and scalable tools to monitor activities of daily living. Currently, most sensor-based activity recognition techniques rely on supervised learning algorithms. However, the acquisition of comprehensive training sets of activities in smart homes is expensive and violates the individual's privacy. In this work, we address this problem by proposing a novel hybrid approach that couples collaborative active learning with probabilistic and knowledge-based reasoning. The rationale of our approach is that a generic, and possibly incomplete, knowledge-based model of activities can be refined to target specific individuals and environments by collaboratively acquiring feedback from inhabitants. Specifically, we propose a collaborative active learning method exploiting users' feedback to (i) refine correlations among sensor events and activity types that are initially extracted from a high-level ontology, and (ii) mine temporal patterns of sensor events that are frequently generated by the execution of specific activities. A Markov Logic Network is used to recognize activities with probabilistic rules that capture both the ontological knowledge and the information obtained by active learning. We experimented our solution with a real-world dataset of activities carried out by several individuals in an interleaved fashion. Experimental results show that our collaborative and personalized active learning solution significantly improves recognition rates, while triggering a small number of feedback requests. Moreover, the overall recognition rates compare favorably with existing supervised and unsupervised activity recognition methods.
Robust adaptive unscented Kalman filter for bearings-only tracking in three dimensional case

This paper proposes an improved version of Unscented Kalman Filter (UKF), namely Robust Adaptive UKF (RAUKF), with a special focus on Bearings-Only Target Tracking for three-dimensional case (3DBOT). The automatic tuning of the noise covariance matrices and the robust estimation of the target states form a critical point for the performance of the Kalman-type filtering algorithms, especially in the variable environmental conditions exposed in underwater. The key idea of the proposed filter is to combine robust aspects of UKF and adaption of the process and measurement noise covariance matrices with low computational complexity. The main contribution of this paper is to adjust these matrices by means of the steepest descent algorithm, and the H∞ technique is embedded to achieve superior performance in terms of accuracy and robustness against initial conditions and model uncertainties. Different experiments are performed to evaluate the performance of the proposed algorithm in the 3DBOT problem with a single moving observer. Simulations demonstrate that the proposed filter produce more accurate results with satisfactory computational burden in comparison with other methods.
DataSelfie

Many personal informatics systems allow people to collect and manage personal data and reflect more deeply about themselves. However, these tools rarely offer ways to customize how the data is visualized. In this work, we investigate the question of how to enable people to determine the representation of their data. We analyzed the Dear Data project to gain insights into the design elements of personal visualizations. We developed DataSelfie, a novel system that allows individuals to gather personal data and design custom visuals to represent the collected data. We conducted a user study to evaluate the usability of the system as well as its potential for individual and collaborative sensemaking of the data.
"Everyone Brings Their Grain of Salt"

Significant research has demonstrated the crucial role that parents play in supporting the development of children's literacy, but in contexts where adults may lack sufficient literacy in the target language, it is not clear how to most effectively scaffold parental support for children's literacy. Prior work has designed technologies to teach children literacy directly, but this work has not focused on designing for low-literate parents, particularly for multilingual and developing contexts. In this paper, we describe findings from a qualitative study conducted in several regions of rural Côte d'Ivoire to understand Ivorian parents' beliefs, desires, and preferences for French literacy. We discuss themes that emerged from these interviews, surrounding ideas of trust, collaboration, and culturally-responsive design, and we highlight implications for the design of technology to scaffold low-literate parental support for children's literacy.
Bidirectional deep architecture for Arabic speech recognition

Abstract Nowadays, the real life constraints necessitates controlling modern machines using human intervention by means of sensorial organs. The voice is one of the human senses that can control/monitor modern interfaces. In this context, Automatic Speech Recognition is principally used to convert natural voice into computer text as well as to perform an action based on the instructions given by the human. In this paper, we propose a general framework for Arabic speech recognition that uses Long Short-Term Memory (LSTM) and Neural Network (Multi-Layer Perceptron: MLP) classifier to cope with the nonuniform sequence length of the speech utterances issued fromboth feature extraction techniques, (1)Mel Frequency Cepstral Coefficients MFCC (static and dynamic features), (2) the Filter Banks (FB) coefficients. The neural architecture can recognize the isolated Arabic speech via classification technique. The proposed system involves, first, extracting pertinent features from the natural speech signal using MFCC (static and dynamic features) and FB. Next, the extracted features are padded in order to deal with the non-uniformity of the sequences length. Then, a deep architecture represented by a recurrent LSTM or GRU (Gated Recurrent Unit) architectures are used to encode the sequences of MFCC/FB features as a fixed size vector that will be introduced to a Multi-Layer Perceptron network (MLP) to perform the classification (recognition). The proposed system is assessed using two different databases, the first one concerns the spoken digit recognition where a comparison with other related works in the literature is performed, whereas the second one contains the spoken TV commands. The obtained results show the superiority of the proposed approach.
Speech interface reformulations and voice assistant personification preferences of children and parents

The pervasive availability of voice assistants may support children in finding answers to informational queries by removing the literacy requirements of text search (e.g., typing, spelling). However, most such systems are not designed for the specific needs and preferences of children and may struggle with understanding the intent of their questions. In our investigation, we observed 87 children and 27 adults interacting with three Wizard-of-Oz speech interfaces to arrive at answers to questions that required reformulation. We found that many children and some adults required help to reach an effective question reformulation. We report the common types of reformulations (both effective and ineffective ones) and the role of age in these. We also compared three versions of speech interfaces with different approaches to referring to itself (personification) and to the participant (naming personalization). We found that generally children preferred personified interfaces, but naming personalization was only preferred by younger children. We provide implications for design of speech systems for families.
Automated Usability Evaluation of Virtual Reality Applications

Virtual reality (VR) and VR applications have reached the end-user and, hence, the demands on usability, also for novel applications, have increased. This situation requires VR usability evaluation methods that can be applied quickly, even after a first release of an application. In this article, we describe such an approach, which is fully automated and does not ask users to perform predefined tasks in a fixed test setting. Instead, it works on recordings of the actual usage of a VR application from which it generates task trees. Afterwards, it analyzes these task trees to search for usability smells, i.e., user behavior indicating usability issues. Our approach provides detailed descriptions of the usability issues that have been found and how they can be solved. We performed a large case study to evaluate our approach and show that it is capable of correctly identifying usability issues. Although our approach is applicable for different VR interaction modalities, such as gaze, controller, or hand interaction, it also has limitations. For example, it can detect diverse issues related to user efficiency, but specific misunderstandings of users cannot be uncovered.