Reviewerly/exHarmony huggingface dataset:
Image segmentation using dense and sparse hierarchies of superpixels

We investigate the intersection between hierarchical and superpixel image segmentation. Two strategies are considered: (i) the classical region merging, that creates a dense hierarchy with a higher number of levels, and (ii) the recursive execution of some superpixel algorithm, which generates a sparse hierarchy with fewer levels. We show that, while dense methods can capture more intermediate or higher-level object information, sparse methods are considerably faster and usually with higher boundary adherence at finer levels. We first formalize the two strategies and present a sparse method, which is faster than its superpixel algorithm and with similar boundary adherence. We then propose a new dense method to be used as post-processing from the intermediate level, as obtained by our sparse method, upwards. This combination results in a unique strategy and the most effective hierarchical segmentation method among the compared state-of-the-art approaches, with efficiency comparable to the fastest superpixel algorithms.
Disrupting (More-than-) Human-Food Interaction

Digital technology has become a frequent companion of daily food practices, shaping the ways we produce, consume, and interact with food. Smart kitchenware, diet tracking apps, and other techno-solutions carry promise for healthy and sustainable food futures but are often problematic in their impact on food cultures. We conducted four Human-Food Interaction (HFI) workshops to reflect on and anticipate food-tech issues, using experimental food design co-creation as our primary method. At the workshops, food and food practices served as the central research theme and accessible starting point to engage stakeholders and explore values, desires, and imaginaries associated with food-tech. Drawing on these explorations, we discuss diverse roles that experimental design co-creation, performed with and around food, can play in supporting critical, interdisciplinary HFI inquiries. Our findings will appeal to design researchers interested in food as a research theme or as a tangible (and compostable!) design material affording diverse co-creative engagements.
Development of a Distortion-Induced Fatigue Crack Characterization Methodology Using Digital Image Correlation

Abstract Distortion-induced fatigue cracking is a primary maintenance and structural safety concern in steel bridges built prior to the 1980s in the United States. Manual, hands-on inspections are ...
Unsupervised Domain Adaptation Through Inter-Modal Rotation for RGB-D Object Recognition

Unsupervised Domain Adaptation (DA) exploits the supervision of a label-rich source dataset to make predictions on an unlabeled target dataset by aligning the two data distributions. In robotics, DA is used to take advantage of automatically generated synthetic data, that come with “free” annotation, to make effective predictions on real data. However, existing DA methods are not designed to cope with the multi-modal nature of RGB-D data, which are widely used in robotic vision. We propose a novel RGB-D DA method that reduces the synthetic-to-real domain shift by exploiting the inter-modal relation between the RGB and depth image. Our method consists of training a convolutional neural network to solve, in addition to the main recognition task, the pretext task of predicting the relative rotation between the RGB and depth image. To evaluate our method and encourage further research in this area, we define two benchmark datasets for object categorization and instance recognition. With extensive experiments, we show the benefits of leveraging the inter-modal relations for RGB-D DA. The code is available at: “https://github.com/MRLoghmani/relative-rotation”.
Tree Edit Distance Cannot be Computed in Strongly Subcubic Time (Unless APSP Can)

The edit distance between two rooted ordered trees with n nodes labeled from an alphabet Ʃ is the minimum cost of transforming one tree into the other by a sequence of elementary operations consisting of deleting and relabeling existing nodes, as well as inserting new nodes. Tree edit distance is a well-known generalization of string edit distance. The fastest known algorithm for tree edit distance runs in cubic O ( n 3 ) time and is based on a similar dynamic programming solution as string edit distance. In this article, we show that a truly subcubic O ( n 3-ε ) time algorithm for tree edit distance is unlikely: For |Ʃ| = Ω ( n ), a truly subcubic algorithm for tree edit distance implies a truly subcubic algorithm for the all pairs shortest paths problem. For |Ʃ| = O (1), a truly subcubic algorithm for tree edit distance implies an O ( n k-ε ) algorithm for finding a maximum weight k -clique. Thus, while in terms of upper bounds string edit distance and tree edit distance are highly related, in terms of lower bounds string edit distance exhibits the hardness of the strong exponential time hypothesis (Backurs, Indyk STOC’15) whereas tree edit distance exhibits the hardness of all pairs shortest paths. Our result provides a matching conditional lower bound for one of the last remaining classic dynamic programming problems.
Efficient Parameter-Free Adaptive Multi-Modal Hashing

Unsupervised multi-modal hashing has recently attracted broad attention in research area of large-scale multimedia retrieval for its low storage cost, high retrieval speed, and independence on semantic labels. However, the model learning process of existing methods still suffer from the problem of low efficiency: 1) Many existing methods measure the contributions of different modalities using fixed modality weights. In order to avoid over-fitting, they need an inefficient hyper-parameter adjustment process. 2) Most existing methods adopt inefficient optimization strategies to learn hash codes. In this letter, we propose an unsupervised Efficient Parameter-free Adaptive Multi-modal Hashing (EPAMH) model to adaptively capture the modality variations and preserve the discriminative semantics of multi-modal features into the binary hash codes. Moreover, we directly learn the binary codes with simple and efficient operations, which prevents the relaxing quantization errors and improves the model learning efficiency. Experiments prove the superior performance of EPAMH on three public multimedia retrieval datasets. Our source codes and testing datasets can be obtained at https://github.com/ChaoqunZheng/EPAMH.
Woman-Centered Design through Humanity, Activism, and Inclusion

Women account for over half of the global population, however, continue to be subject to systematic and systemic disadvantage, particularly in terms of access to health and education. At every intersection, where systemic inequality accounts for greater loss of life or limitations on full and healthy living, women are more greatly impacted by those inequalities. The design of technologies is no different, the very definition of technology is historically cast in terms of male activities, and advancements in the field are critical to improve women's quality of life. This article views HCI, a relatively new field, as well positioned to act critically in the ways that technology serve, refigure, and redefine women's bodies. Indeed, the female body remains a contested topic, a restriction to the development of women's health. On one hand, the field of women's health has attended to the medicalization of the body and therefore is to be understood through medical language and knowledge. On the other hand, the framing of issues associated with women's health and people's experiences of and within such system(s) remain problematic for many. This is visible today in, e.g., socio-cultural practices in disparate geographies or medical devices within a clinic or the home. Moreover, the biological body is part of a great unmentionable, i.e., the perils of essentialism. We contend that it is necessary, pragmatically and ethically, for HCI to turn its attention toward a woman-centered design approach. While previous research has argued for the dangers of gender-demarcated design work, we advance that designing for and with women should not be regarded as ghettoizing, but instead as critical to improving women's experiences in bodily transactions, choices, rights, and access to and in health and care. In this article, we consider how and why designing with and for woman matters. We use our design-led research as a way to speak to and illustrate alternatives to designing for and with women within HCI.
Towards Explanatory Interactive Image Captioning Using Top-Down and Bottom-Up Features, Beam Search and Re-ranking

Abstract Image captioning is a challenging multimodal task. Significant improvements could be obtained by deep learning. Yet, captions generated by humans are still considered better, which makes it an interesting application for interactive machine learning and explainable artificial intelligence methods. In this work, we aim at improving the performance and explainability of the state-of-the-art method Show, Attend and Tell by augmenting their attention mechanism using additional bottom-up features. We compute visual attention on the joint embedding space formed by the union of high-level features and the low-level features obtained from the object specific salient regions of the input image. We embed the content of bounding boxes from a pre-trained Mask R-CNN model. This delivers state-of-the-art performance, while it provides explanatory features. Further, we discuss how interactive model improvement can be realized through re-ranking caption candidates using beam search decoders and explanatory features. We show that interactive re-ranking of beam search candidates has the potential to outperform the state-of-the-art in image captioning.
A Novel Hammerstein Model for Nonlinear Networked Systems Based on an Interval Type-2 Fuzzy Takagi–Sugeno–Kang System

In this article, a novel Hammerstein structure is proposed for nonlinear networked systems based on an interval type-2 Takagi–Sugeno–Kang (IT2TSK) fuzzy system. The proposed approach is designed based on the general Hammerstein form, where an autoregressive moving average and an IT2TSK structures are designed as the linear energetic and nonlinear static components, respectively. The consequents of the nonlinear subsystem are characterized by a TSK-type system while the antecedents are characterized by interval type-2 fuzzy sets. The structure of the nonlinear subsystem is learned online based on the type-2 fuzzy clustering. Two new updating algorithms are introduced based on the Lyapunov theorem for learning the proposed model parameters and adaptive learning rates to assure the model stability and the parameter fast convergence. To illustrate the proposed model robustness, a test is performed under networked environment with time-varying delay and packet losses. The simulation results prove a higher performance for the proposed model than that of compared models.
Filtering for systems subject to unknown inputs without a priori initial information

The last few decades have witnessed much development in filtering of systems with Gaussian noises and arbitrary unknown inputs. Nonetheless, there are still some important design questions that warrant thorough discussions. Especially, the existing literature has shown that for unbiased and minimum variance estimation of the state and the unknown input, the initial guess of the state has to be unbiased. This clearly raises the question of whether and under what conditions one can design an unbiased and minimum variance filter, without making such a stringent assumption. The above-mentioned question will be investigated systematically in this paper, i.e., design of the filter is sought to be independent of a priori information about the initial conditions. In particular, for both cases with and without direct feedthrough, we establish necessary and sufficient conditions for unbiased and minimum variance estimation of the state/unknown input, independently of a priori initial conditions, respectively. When the former conditions do not hold, we carry out a thorough analysis of all possible scenarios. For each scenario, we present detailed discussions regarding whether and what can be achieved in terms of unbiased estimation, independently of a priori initial conditions. Extensions to the case with time-delays, conceptually like Kalman smoothing where future measurements are allowed in estimation, will also be presented, amongst others.
Anomaly detection in Network Traffic Using Unsupervised Machine learning Approach

The advent of IoT technology and the increase in wireless networking devices has led to an enormous increase in network attacks from different sources. To maintain networks as safe and secure, the Intrusion Detection System (IDS) have become very critical. Intrusion Detection Systems (IDS) are designed to protect the network by identifying anomaly behaviors or improper uses. Intrusion Detection systems provide more meticulous security functionality than access control barriers by detecting attempted and successful attacks at the end-point or within the network. Intrusion prevention systems are the next logical step to this approach as they can take real-time action against breaches. To have an accurate IDS, detailed visibility is required into the network traffic. The intrusion detection system should be able to detect inside the network threats as well as access control breaches. IDS has been around for a very long time now. These traditional IDS were rules and signature-based. Though they were able to reduce false positives they were not able to detect new attacks. In today's world due to the growth of connectivity, attacks have increased at an exponential rate and it has become essential to use a data-driven approach to tackle these issues. In this paper, the KDD data set was used to train the unsupervised machine learning algorithm called Isolation Forest. The data set is highly imbalanced and contains various attacks such as DOS, Probe, U2R, R2L. Since this data set suffers from a redundancy of values and class imbalance, the data preprocessing will be performed first and also used unsupervised learning. For this network traffic based anomaly detection model isolation forest was used to detect outliers and probable attacks the results were evaluated using the anomaly score.
Pedestrian attribute recognition based on multiple time steps attention

Pedestrian Attribute Recognition (PAR) plays an important role in intelligent video surveillance. This paper tackles two severe challenges in it i.e., complex relations between images and attributes, and imbalanced distribution of pedestrian attributes. Specifically, a new multiple time steps attention mechanism is proposed to boost the modeling of the relations. Different from existing attention approaches that only focus on the current and previous time steps, it also exploits the knowledge of next time step. By adaptively capturing the knowledge of multiple time steps, more contextual knowledge is exploited. Meanwhile, to alleviate the challenge of imbalanced distribution of pedestrian attributes, a focal balance loss function is developed by increasing the cost of those attributes difficult to recognize. The proposed framework is dubbed as MTA-Net, which is demonstrated to be effective on two benchmark datasets, i.e., PETA and RAP.
A Survey on Representation Learning for User Modeling

Artificial intelligent systems are changing every aspect of our daily life. In the past decades, numerous approaches have been developed to characterize user behavior, in order to deliver personalized experience to users in scenarios like online shopping or movie recommendation. This paper presents a comprehensive survey of recent advances in user modeling from the perspective of representation learning. In particular, we formulate user modeling as a process of learning latent representations for users. We discuss both the static and sequential representation learning methods for the purpose of user modeling, and review representative approaches in each category, such as matrix factorization, deep collaborative filtering, and recurrent neural networks. Both shallow and deep learning methods are reviewed and discussed. Finally, we conclude this survey and discuss a number of open research problems that would inspire further research in this field.
Floating-point discrete wavelet transform-based image compression on FPGA

In the era of data transmission through internet, image compression is considered an active research topic, decreasing the amount of data storage for faster data transfer. In this paper, the hardware implementation of an image compression system using Discrete Wavelet Transform (DWT) is presented. The transposed form Finite Impulse Response (FIR) filter is employed for performing the convolution process, on which the DWT is based. The design is generic to fit for different wavelet types and symmetric to expand for filters of multiple taps. The architecture is implemented on FPGA using IEEE-754 single precision. Floating-Point representation offered higher precision and better accuracy compared to scaled integer values. The proposed hardware design is implemented on Virtex 5 FPGA achieving 243.6 MHz clock frequency.
A RF Fingerprint Recognition Method Based on Deeply Convolutional Neural Network

Radio frequency (RF) fingerprinting is a process of identifying a device or annunciator. Because of the specificity of the wireless transmitting circuit and the imperfections of the components that make up the transmitting circuit, it is possible to extract the RF fingerprint used to identify the wireless transmitter from the wireless signal. At present, the traditional RF fingerprint identification method is performed based on a preset determination formula, and there are problems such as high prior information requirement and limited application range. To solve these problems, this paper proposes a RF fingerprint recognition method based on convolutional neural network (CNN). The research focuses on three aspects: RF fingerprint extraction, convolutional neural network design and wireless transmitter identification and verification. The result proves that this method can complete the wireless transmitter identification well.
Space-Time Memory Networks for Video Object Segmentation With User Guidance

We propose a novel and unified solution for user-guided video object segmentation tasks. In this work, we consider two scenarios of user-guided segmentation: semi-supervised and interactive segmentation. Due to the nature of the problem, available cues - video frame(s) with object masks (or scribbles) - become richer with the intermediate predictions (or additional user inputs). However, the existing methods make it impossible to fully exploit this rich source of information. We resolve the issue by leveraging memory networks and learning to read relevant information from all available sources. In the semi-supervised scenario, the previous frames with object masks form an external memory, and the current frame as the query is segmented using the information in the memory. Similarly, to work with user interactions, the frames that are given user inputs form the memory that guides segmentation. Internally, the query and the memory are densely matched in the feature space, covering all the space-time pixel locations in a feed-forward fashion. The abundant use of the guidance information allows us to better handle challenges such as appearance changes and occlusions. We validate our method on the latest benchmark sets and achieve state-of-the-art performance along with a fast runtime.
Development of improved whale optimization-based FCM clustering for image watermarking

With the growing interest in copyright protection of multimedia, digital watermarking has been introduced and widely researched. The digital image watermarking approaches are generally adopted in the spatial or transform domain. Most of the transform-domain watermarking approaches are based on Discrete Cosine Transforms (DCT) and robust to JPEG lossy compression. Few contributions have been done using the clustering technique using for robust watermarking, and it still seeks for further improvement. Hence, this paper plans to develop an effective digital watermarking framework using an optimized clustering approach. The proposed model consists of several phases like image scaling, block separation, computation of feature vectors, spotting of regions for watermarking, message transformation, watermark embedding, IDCT, and message restoration. After image scaling, the block separation is done by DCT, and further, the feature vectors correspond to the pixel values are extracted. The optimized FCM clustering is adopted to categorize the blocks into suitable and unsuitable watermarking regions. The optimized FCM with Least Favorable-based Whale Optimization Algorithm (LF-WOA) enabled initial centroid selection takes the decision regarding the regions where the watermark can be inserted. After the watermarking embedding, the message is restored by the reverse process. Finally, the experimental results will achieve a higher demand for watermarking in terms of robustness and sensitivity.
Enhanced Multi-Channel Feature Synthesis for Hand Gesture Recognition Based on CNN With a Channel and Spatial Attention Mechanism

Millimeter-wave (MMW) radar hand gesture recognition technology is becoming important in many electronic device control applications.Currently, most existing approaches utilize the radical and micro-Doppler features from single-channel MMW radar, which ignores the different importance of the information contained in the micro-Doppler feature background or target areas.In this paper, we propose an algorithm for hand gesture recognition jointly using multi-channel signatures.The algorithm blends the information of both micro-Doppler features and instantaneous angles (azimuth and elevation) to accomplish hand gesture recognition performed with the convolutional neural network (CNN).To have a better features fusion and make CNN focus on the most important target signal regions and suppress the unnecessary noise areas, we apply the channel and spatial attention-based feature refinement modules.We also employ gesture movement mechanism-based data augmentation for more effective training to alleviate potential overfitting.Extensive experiments demonstrate the effectiveness and superiorities of the proposed algorithm.This method achieves a correct classification rate of 96.61%, approximately 5% higher than that of the singlechannel-based recognition strategy as measured based on MMW radar datasets.
Speech diversity and speech interfaces

The number of speech interfaces and services made available through them continue to grow. This has opened up interactions to people who rely on speech as a critical modality for interacting with systems. However, people with diverse speech patterns such as those who stammer are at risk of being negatively affected or excluded from speech interface interaction. In this paper, we consider what an inclusive speech interface future may look like for people who stammer. In doing so, we identify three key challenges: (1) developing effective speech recognition, (2) understanding the user experiences of people who stammer and (3) supporting speech interfaces designers through appropriate heuristics. We believe the interdisciplinary and cross-community strengths of venues like CUI are well positioned to address these challenges going forward.