Reviewerly/exHarmony huggingface dataset:
Explore User Neighborhood for Real-time E-commerce Recommendation

Recommender systems play a vital role in modern online services, such as Amazon and Taobao. Traditional personalized methods, which focus on user-item (UI) relations, have been widely applied in industrial settings, owing to their efficiency and effectiveness. Despite their success, we argue that these approaches ignore local information hidden in similar users. To tackle this problem, user-based methods exploit similar user relations to make recommendations in a local perspective. Nevertheless, traditional user-based methods, like userKNN and matrix factorization, are intractable to be deployed in the real-time applications since such transductive models have to be recomputed or retrained with any new interaction. To overcome this challenge, we propose a framework called self-complementary collaborative filtering (SCCF) which can make recommendations with both global and local information in real time. On the one hand, it utilizes UI relations and user neighborhood to capture both global and local information. On the other hand, it can identify similar users for each user in real time by inferring user representations on the fly with an inductive model. The proposed framework can be seamlessly incorporated into existing inductive UI approach and benefit from user neighborhood with little additional computation. It is also the first attempt to apply user-based methods in real-time settings. The effectiveness and efficiency of SCCF are demonstrated through extensive offline experiments on four public datasets, as well as a large scale online A/B test in Taobao.
Privacy-Preserving Federated Learning Framework with General Aggregation and Multiparty Entity Matching

The requirement for data sharing and privacy has brought increasing attention to federated learning. However, the existing aggregation models are too specialized and deal less with users’ withdrawal issue. Moreover, protocols for multiparty entity matching are rarely covered. Thus, there is no systematic framework to perform federated learning tasks. In this paper, we systematically propose a privacy-preserving federated learning framework (PFLF) where we first construct a general secure aggregation model in federated learning scenarios by combining the Shamir secret sharing with homomorphic cryptography to ensure that the aggregated value can be decrypted correctly only when the number of participants is greater than <math xmlns="http://www.w3.org/1998/Math/MathML" id="M1"> <mi>t</mi> </math> . Furthermore, we propose a multiparty entity matching protocol by employing secure multiparty computing to solve the entity alignment problems and a logistic regression algorithm to achieve privacy-preserving model training and support the withdrawal of users in vertical federated learning (VFL) scenarios. Finally, the security analyses prove that PFLF preserves the data privacy in the honest-but-curious model, and the experimental evaluations show PFLF attains consistent accuracy with the original model and demonstrates the practical feasibility.
Head Posture Estimation by Deep Learning Using 3-D Point Cloud Data From a Depth Sensor

Head posture estimation is performed by capturing characteristic areas of the face, such as the eyes and nose, in images acquired from a camera installed in front of the subject. However, with this method, parts of the eyes and nose are hidden when the subject turns away and faces the side, making estimation difficult. In this letter, we aim to realize a more effective head estimation method than previous research using 3-D point cloud data from a depth sensor. We pursued the estimation of five head posture classes. In the proposed method, first, the 3-D point cloud data of the postures in the five classes are learned by a deep learning model. Next, the posture of the head is estimated using the model. In this letter, many verification experiments confirmed that the proposed method is very effective for head posture estimation with five posture classes.
INFOSHIELD: Generalizable Information-Theoretic Human-Trafficking Detection

Given a million escort advertisements, how can we spot near-duplicates? Such micro-clusters of ads are usually signals of human trafficking. How can we summarize them, visually, to convince law enforcement to act? Can we build a general tool that works for different languages? Spotting micro-clusters of near-duplicate documents is useful in multiple, additional settings, including spam-bot detection in Twitter ads, plagiarism, and more.We present INFOSHIELD, which makes the following contributions: (a) Practical, being scalable and effective on real data, (b) Parameter-free and Principled, requiring no user-defined parameters, (c) Interpretable, finding a document to be the cluster representative, highlighting all the common phrases, and automatically detecting "slots", i.e. phrases that differ in every document; and (d) Generalizable, beating or matching domain-specific methods in Twitter bot detection and human trafficking detection respectively, as well as being language-independent finding clusters in Spanish, Italian, and Japanese. Interpretability is particularly important for the anti human-trafficking domain, where law enforcement must visually inspect ads.Our experiments on real data show that INFOSHIELD correctly identifies Twitter bots with an F1 score over 90% and detects human-trafficking ads with 84% precision. Moreover, it is scalable, requiring about 8 hours for 4 million documents on a stock laptop.
Active Object Detection Based on a Novel Deep Q-Learning Network and Long-Term Learning Strategy for the Service Robot

This article focuses on active object detection (AOD), one of the greatest challenges in the robotics field. A novel deep-Q-learning-network-based approach is proposed to utilize more useful status information for enhancing the training efficiency and testing accuracy of AOD by adding the cropped target object (TGOJ) from the current state as a new input. Different from the existing researches, a novel reward function, combing the area factor and distance factor of the bounding box, is designed to make the robot not only get closer to the TGOJ but also obtain a better observation viewpoint. Moreover, to overcome the differences between the training dataset and new environments as well as improving the adaptation of the AOD model, a reward-based long-term learning strategy including a novel training strategy is presented. The comparable experiments and the ablation study have been implemented in an AOD dataset, proving that our method owns better performance and efficiency than the comparable methods. Meanwhile, the experiments in the real-world scenario with a robot indicate the validity of the proposed method.
Deep Field Relation Neural Network for click-through rate prediction

Click-Through Rate (CTR) prediction is crucial in calculating advertisements and recommendation systems. To effectively predict CTR, it is important to properly model the interaction among features of data. This work tends to fully utilise the interaction information among features while employing deep neural networks for CTR prediction. To this end, we propose a Deep Field Relation Neural Network (DFRNN), which models feature interaction via a 3-dimensional relation tensor. The proposed method is evaluated on real data sets and compared with related methods. The results demonstrate that our method could be used to derive significant information contained in feature interaction and achieve an accurate CTR prediction.
A Densely Connected GRU Neural Network Based on Coattention Mechanism for Chinese Rice-Related Question Similarity Matching

In the question-and-answer (Q&amp;A) communities of the “China Agricultural Technology Extension Information Platform”, thousands of rice-related Chinese questions are newly added every day. The rapid detection of the same semantic question is the key to the success of a rice-related intelligent Q&amp;A system. To allow the fast and automatic detection of the same semantic rice-related questions, we propose a new method based on the Coattention-DenseGRU (Gated Recurrent Unit). According to the rice-related question characteristics, we applied word2vec with the TF-IDF (Term Frequency–Inverse Document Frequency) method to process and analyze the text data and compare it with the Word2vec, GloVe, and TF-IDF methods. Combined with the agricultural word segmentation dictionary, we applied Word2vec with the TF-IDF method, effectively solving the problem of high dimension and sparse data in the rice-related text. Each network layer employed the connection information of features and all previous recursive layers’ hidden features. To alleviate the problem of feature vector size increasing due to dense splicing, an autoencoder was used after dense concatenation. The experimental results show that rice-related question similarity matching based on Coattention-DenseGRU can improve the utilization of text features, reduce the loss of features, and achieve fast and accurate similarity matching of the rice-related question dataset. The precision and F1 values of the proposed model were 96.3% and 96.9%, respectively. Compared with seven other kinds of question similarity matching models, we present a new state-of-the-art method with our rice-related question dataset.
Recent trends in pedestrian detection for robotic vision using deep learning techniques

Pedestrian detection has become more and more important in the field of robotic vision, having applications in autonomous driving, automated surveillance, smart homes, as well as mobile robots (or mobot). With the help of smart cameras, mobile robots have been able to detect, localize, and recognize pedestrians in a scene. In recent years, researchers from all around the world have developed robust deep learning-based systems for detecting pedestrians with subpar results. In this chapter, we have presented a review of the most superior deep learning techniques used by robotic vision systems in recent years and how well they have performed on different benchmark datasets available all around the world. All the techniques differ in two major respects, firstly the architecture of the system, and secondly the preprocessing, where input data is used in different capacities. The field of robotic vision is still under constant development and the day isn't far when full automation will be a reality.
Multi-Level Memory Compensation Network for Rain Removal via Divide-and-Conquer Strategy

Recently, an increasing number of algorithms have been proposed for rain streak removal. However, most existing methods ignore the discrepancy in removing rain streaks from background contents with different texture richness (frequencies). They adopt a unified scheme to learn the final distribution of rain streaks directly, thus sacrificing the representative accuracy of rain information. To this end, this paper leverages the divide-and-conquer strategy for rain streak removal by decomposing the learning task into several subproblems according to the levels (frequencies) of texture richness in background contents. Particularly, we construct a novel multi-level memory compensation network (MLMCN) for rain streak removal. It achieves a promising solution by individually handling these subproblems under the specific texture richness via several parallel subnetworks. Each subnetwork takes as input a specific sub-sampled image, sampled from the original rain ones via the Gaussian kernel, to individually learn one of the sub-distributions of the rain information. We thus produce a high-quality rain-free image by subtracting the predicted rain information from multiple subnetworks in turn. We experimentally show that our proposed MLMCN outperforms the existing deraining methods in terms of quantitative indicators, visual effects on several benchmark datasets, and the high-level object detection task.
TrainAR: A Scalable Interaction Concept and Didactic Framework for Procedural Trainings Using Handheld Augmented Reality

The potential of Augmented Reality (AR) for educational and training purposes is well known. While large-scale deployments of head-mounted AR headsets remain challenging due to technical limitations and cost factors, advances in mobile devices and tracking solutions introduce handheld AR devices as a powerful, broadly available alternative, yet with some restrictions. One of the current limitations of AR training applications on handheld AR devices is that most offer rather static experiences, only providing descriptive knowledge with little interactivity. Holistic concepts for the coverage of procedural knowledge are largely missing. The contribution of this paper is twofold. We propose a scalabe interaction concept for handheld AR devices with an accompanied didactic framework for procedural training tasks called TrainAR. Then, we implement TrainAR for a training scenario in academics for the context of midwifery and explain the educational theories behind our framework and how to apply it for procedural training tasks. We evaluate and subsequently improve the concept based on three formative usability studies (n = 24), where explicitness, redundant feedback mechanisms and onboarding were identified as major success factors. Finally, we conclude by discussing derived implications for improvements and ongoing and future work.
Practices and A Strong Baseline for Traffic Anomaly Detection

The detection of traffic anomalies is a critical component of the intelligent city transportation management sys-tem. Previous works have proposed a variety of notable in-sights and taken a step forward in this field, however, dealing with the complex traffic environment remains a challenge. Moreover, the lack of high-quality data and the complexity of the traffic scene, motivate us to study this problem from a hand-crafted perspective. In this paper, we propose a straightforward and efficient framework that includes pre-processing, a dynamic track module, and post-processing. With video stabilization, background modeling, and vehicle detection, the pro-processing phase aims to generate candidate anomalies. The dynamic tracking module seeks and locates the start time of anomalies by utilizing vehicle motion patterns and spatiotemporal status. Finally, we use the post-processing to fine-tune the temporal boundary of anomalies. Not surprisingly, our proposed framework was ranked 1 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">st</sup> in the NVIDIA AI CITY 2021 leaderboard for traffic anomaly detection. Codes will be available.
Investigation and evaluation of algorithms for unmanned aerial vehicle multispectral image registration

Automatically registration of unmanned aerial vehicle (UAV) multispectral images is fundamental for subsequent applications. Although many studies exist for visible camera images and satellite multispectral image registration, studies for UAV are still rare. Under this context, this study firstly evaluates the performance of several widely used traditional methods (i.e., SIFT, SURF, ORB, and CFOG) for visible camera and satellite images in UAV multispectral image registration. This study further proposes an unsupervised and end-to-end deep learning network (i.e., DSIM) for multispectral image registration. An evident feature of DSIM is to regress the homography parameters with convolutional neural networks and to use the pyramid structural similarity loss to optimize the network. 1200 groups of UAV multispectral images acquired over three different sites in four months are used to comprehensively test the aforementioned five methods. Results show that CFOG achieves the highest correct matching rate in the test set, followed by DSIM and SIFT. Nevertheless, DSIM is more robust in images with weak or repeated texture than CFOG and SIFT. In addition, performance of CFOG and SIFT is closely related to the number of the found matching points. Based on the findings, we propose a multi-method ensemble strategy to combine CFOG, DSIM, and SIFT according to the number of the found matching points. This strategy outperforms the individual methods with a correct matching rate of 96.2%. Lower correct matching rate of CFOG + SIFT confirms that DSIM and traditional methods are very complementary in UAV multispectral image registrations.
Into the Mine

Our global livelihoods are intrinsically tied to mining. The technologies we use, as currently designed, are not possible without the minerals and metals that are an essential part of several of their components. As a result, HCI research and applications are tightly dependent on mining, including the negative environmental and social impacts resulting from it. This paper aims to describe and reflect on this problematic entanglement as a ”wicked cycle.” We present a dilemma faced by communities living near mining sites in the Amazon, which are affected by the ecological impacts of mining and rely on digital technologies made with such mines’ products, including telecommunication technologies, to effectively and successfully advocate for and realise their own local visions of development. We promote a discussion built on concepts from decolonial thinking and critical sustainability. With this paper, we want to create space and necessity to acknowledge our complicity as HCI researchers in this dilemma and propose a series of questions to reflect on our part in these specific, and other, wicked cycles.
Hyperspectral Image Mixed Denoising Using Difference Continuity-Regularized Nonlocal Tensor Subspace Low-Rank Learning

With the rapid advancement of spectrometers, the imaging range of the electromagnetic spectrum starts growing narrower. The reduction of electromagnetic wave energy received in a single wavelength range leads more complex noise into the generated hyperspectral image (HSI), thus causing a severe cripple in the accuracy of subsequent applications. The requirement for the HSI mixed denoising algorithm’s accuracy is further lifted. To address this challenge, in this letter, we propose a novel difference continuity-regularized nonlocal tensor subspace low-rank learning (named DNTSLR) method for HSI mixed denoising. Technically, the original high-dimensional HSI data was first projected into a low-dimensional subspace spanned by a spectral difference continuous basis instead of an orthogonal basis, so the data continuity of the restored HSI spectrum and tensor low-rankness was guaranteed. Then, a cube matching strategy was employed to stack the nonlocal tensor patches from the projected coefficient tensor, and a shrinkage algorithm was used to approximate the low-rank coefficient tensor. Eventually, the subspace low-rank learning algorithm was designed to alternately separate the noise tensor and restore the latent clean low-rank HSI tensor. Extensive experiments on multiple open datasets validate that the proposed method realizes the state-of-the-art denoising accuracy for HSI.
I/O Bits: User-Driven, Situated, and Dedicated Self-Tracking

We present I/O Bits, a prototype personal informatics system that explores the potential for user-driven and situated self-tracking. With simple tactile inputs and small e-paper visualizations, I/O Bits are dedicated physical devices that allow individuals to track and visualize different kinds of personal activities in-situ. This is in contrast to most self-tracking systems, which automate data collection, centralize information displays, or integrate into multi-purpose devices like smartwatches or mobile phones. We report findings from an e-paper visualization workshop and a prototype deployment where participants constructed their own I/O Bits and used them to track a range of personal data. Based on these experiences, we contribute insights and opportunities for situated and user-driven personal informatics.
Black box algorithms and the rights of individuals: no easy solution to the “explainability” problem

Over the last few years, the interpretability of classification models has been a very active area of research.Recently, the concept of interpretability was given a more specific legal context.In 2016, the EU adopted the General Data Protection Regulation (GDPR), containing the right to explanation for people subjected to automated decision-making (ADM).The regulation itself is very reticent about what such a right might imply.As a result, since the introduction of the GDPR there has been an ongoing discussion about not only the need to introduce such a right, but also about its scope and practical consequences in the digital world.While there is no doubt that the right to explanation may be very difficult to implement due to technical challenges, any difficulty in explaining how algorithms work cannot be considered a sufficient reason to completely abandon this legal safeguard.The aim of this article is twofold.First, to demonstrate that the interpretability of "black box" machine learning algorithms is a challenging technical problem for which no solutions have been found.Second, to demonstrate how the explanation task should instead be completed using well-known and well-trialled IT solutions, such as event logging or statistical analysis of the algorithm.Based on the evidence exposed in this paper, the authors find that the most effective solution would be to benchmark the automated decision-making algorithms using certification frameworks, thus balancing the need to ensure adequate protection of individuals' rights with the understandable expectations of AI technology providers to have their intellectual property rights protected.
Learning Contextualized User Preferences for Co‐Adaptive Guidance in Mixed‐Initiative Topic Model Refinement

Abstract Mixed‐initiative visual analytics systems support collaborative human‐machine decision‐making processes. However, many multi‐objective optimization tasks, such as topic model refinement, are highly subjective and context‐dependent. Hence, systems need to adapt their optimization suggestions throughout the interactive refinement process to provide efficient guidance. To tackle this challenge, we present a technique for learning context‐dependent user preferences and demonstrate its applicability to topic model refinement. We deploy agents with distinct associated optimization strategies that compete for the user's acceptance of their suggestions. To decide when to provide guidance, each agent maintains an intelligible, rule‐based classifier over context vectorizations that captures the development of quality metrics between distinct analysis states. By observing implicit and explicit user feedback, agents learn in which contexts to provide their specific guidance operation. An agent in topic model refinement might, for example, learn to react to declining model coherence by suggesting to split a topic. Our results confirm that the rules learned by agents capture contextual user preferences. Further, we show that the learned rules are transferable between similar datasets, avoiding common cold‐start problems and enabling a continuous refinement of agents across corpora.
The decentralization of Social Media through the blockchain technology

Online Social Networks (OSNs) have become one of the most popular applications of the daily life of users in the worldwide. Today, the number of Social Media users is more than 4 billion, and this trend increases year after year with a high impact on the privacy issue. During the last years, decentralization of social services has been considered as a big opportunity to overcome the main privacy issues in OSNs, and not only (fake news, censorship, etc.). Blockchain technology represents today the most well-known decentralized technique, which has been taken into account to develop the new generation of decentralized social platforms. Blockchain-based Online Social Media (BOSMs) are decentralized Social Media platforms that use the blockchain technology as the underlying technology or as a tool in order to provide rewarding strategies. In this tutorial, we will highlight the BOSMs scenario by presenting their main characteristics and how data could be collected and analysed.