Reviewerly/exHarmony huggingface dataset:
User-specific Adaptive Fine-tuning for Cross-domain Recommendations

Making accurate recommendations for cold-start users has been a longstanding and critical challenge for recommender systems (RS). Cross-domain recommendations (CDR) offer a solution to tackle such a cold-start problem when there is no sufficient data for the users who have rarely used the system. An effective approach in CDR is to leverage the knowledge (e.g., user representations) learned from a related but different domain and transfer it to the target domain. Fine-tuning works as an effective transfer learning technique for this objective, which adapts the parameters of a pre-trained model from the source domain to the target domain. However, current methods are mainly based on the global fine-tuning strategy: the decision of which layers of the pre-trained model to freeze or fine-tune is taken for all users in the target domain. In this paper, we argue that users in RS are personalized and should have their own fine-tuning policies for better preference transfer learning. As such, we propose a novel User-specific Adaptive Fine-tuning method (UAF), selecting which layers of the pre-trained network to fine-tune, on a per-user basis. Specifically, we devise a policy network with three alternative strategies to automatically decide which layers to be fine-tuned and which layers to have their parameters frozen for each user. Extensive experiments show that the proposed UAF exhibits significantly better and more robust performance for user cold-start recommendation.
Improving Deep Segmentation of Abdominal Organs MRI by Post-Processing

Today Deep Learning (DL) is state-of-the-art in medical imaging segmentation tasks, including accurate localization of abdominal organs in MRI images. But segmentation still exhibits inaccuracies, which may be due to texture similarities, proximity or confusion between organs, morphology variations, acquisition conditions or other parameters. Examples include regions classified as the wrong organ, some noisy regions and inaccuracies near borders. To improve robustness, the DL output can be supplemented by more traditional image postprocessing operations that enforce simple semantic invariants. In this paper we define and apply totally automatic post-processing operations applying semantic invariants to correct segmentation mistakes. Organs are assigned relative spatial location restrictions (atlas fencing), 3D organ continuity requirements (envelop continuity), and smoothness constraints. A reclassification is done within organ envelopes to correct classification mistakes, and noise is removed (fencing, enveloping, noise removal, re-classifying and smoothing). Our experimental evaluation quantifies the improvement and compares the resulting quality with prior work on DL-based organ segmentation. Based on the experiments, we conclude post-processing improved the Jaccard index over independent test MRI sequences by a sum of 12 to 25 percentage points over the four segmented organs. This work has an important impact on research and practical application of DL because it describes how to post-process, quantifies the advantages, and can be applied to any DL approach.
Priority Patrolling using Multiple Agents

The Patrolling Problem is a crucial feature of the surveillance task in defense and other establishments. Most of the works in the literature concentrate on reducing the Idleness value at each location in the environment. However, there are often a few prioritized locations that cannot be left unvisited beyond a certain Time Period. In this paper, we study the problem of Prioritized patrolling - the task of patrolling the given environment using multiple agents while ensuring the prioritized locations are visited within the pre-specified Time Period. We present a novel algorithm, namely, Time Period Based Patrolling (TPBP) algorithm, to solve the prioritized patrolling problem. It determines a sequence of walks for each agent online that complies with the Time Period requirement of the Priority nodes while reducing the Idleness of all the other nodes. We have tested and validated the algorithm using SUMO - a realistic simulator developed for traffic management. Since the existing strategies are not designed for Prioritized Patrolling, we show through comparison that proposed algorithm is required to solve the problem.
DC-GNet: Deep Mesh Relation Capturing Graph Convolution Network for 3D Human Shape Reconstruction

In this paper, we aim to reconstruct a full 3D human shape from a single image. Previous vertex-level and parameter regression approaches reconstruct 3D human shape based on a pre-defined adjacency matrix to encode positive relations between nodes. The deep topological relations for the surface of the 3D human body are not carefully exploited. Moreover, the performance of most existing approaches often suffer from domain gap when handling more occlusion cases in real-world scenes. In this work, we propose a Deep Mesh Relation Capturing Graph Convolution Network, DC-GNet, with a shape completion task for 3D human shape reconstruction. Firstly, we propose to capture deep relations within mesh vertices, where an adaptive matrix encoding both positive and negative relations is introduced. Secondly, we propose a shape completion task to learn prior about various kinds of occlusion cases. Our approach encodes mesh structure from more subtle relations between nodes in a more distant region. Furthermore, our shape completion module alleviates the performance degradation issue in the outdoor scene. Extensive experiments on several benchmarks show that our approach outperforms the previous 3D human pose and shape estimation approaches.
Learning Efficient Constraint Graph Sampling for Robotic Sequential Manipulation

Efficient sampling from constraint manifolds, and thereby generating a diverse set of solutions for feasibility problems, is a fundamental challenge. We consider the case where a problem is factored, that is, the underlying nonlinear program is decomposed into differentiable equality and inequality constraints, each of which depends only on some variables. Such problems are at the core of efficient and robust sequential robot manipulation planning. Naive sequential conditional sampling of individual variables, as well as fully joint sampling of all variables at once (e.g., leveraging optimization methods), can be highly inefficient and non-robust. We propose a novel framework to learn how to break the overall problem into smaller sequential sampling problems. Specifically, we leverage Monte-Carlo Tree Search to learn assignment orders for the variable-subsets, in order to minimize the computation time to generate feasible full samples. This strategy allows us to efficiently compute a set of diverse valid robot configurations for mode-switches within sequential manipulation tasks, which are waypoints for subsequent trajectory optimization or sampling-based motion planning algorithms. We show that the learning method quickly converges to the best sampling strategy for a given problem, and outperforms user-defined orderings or fully joint optimization, while providing a higher sample diversity. Video: https://youtu.be/mCNdvjTbHNI
An Efficient Channel-Aware Sparse Binarized Neural Networks Inference Accelerator

The binarized neural network (BNN) inference accelerators show great promise in cost- and power-restricted domains. However, the performances of these accelerators are still severely limited by the significant redundancies in BNNs inference. In this brief, we introduce channel-aware sparse accelerator (CAA) to alleviate the performance degradations induced by the redundancies in BNNs while maintaining original accuracies. First, motivated by the observation that the convolution processes of our rebuilt rectangle kernels contain many redundant operations which can be skipped by exploiting the BNN-specific property, we convert the entire original XNOR-popcount convolutions of each neuron into channel-aware-popcount (CAP) operations for all binarized convolutional and fully-connected layers in CAA by employing rectangle kernel simplification strategy and eliminate the unnecessary operations. Meanwhile, these CAP operations can be implemented to directly gain the final output without any extra steps. Furthermore, inspired by our new observations on two specific kinds of properties of the CAP operations, we adopt group pruning approach to save the remaining redundant CAP operations. Experimental results show that our design evaluated on an embedded FPGA achieves 4.2- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$6.6{\times }$ </tex-math></inline-formula> inference-speedup, 3.6- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5.5{\times }$ </tex-math></inline-formula> energy-efficiency enhancement, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.35{\times }$ </tex-math></inline-formula> resource-efficiency improvement compared with state-of-the-art works.
Neural representations for modeling variation in speech

Variation in speech is often quantified by comparing phonetic transcriptions of the same utterance. However, manually transcribing speech is time-consuming and error prone. As an alternative, therefore, we investigate the extraction of acoustic embeddings from several self-supervised neural models. We use these representations to compute word-based pronunciation differences between non-native and native speakers of English, and between Norwegian dialect speakers. For comparison with several earlier studies, we evaluate how well these differences match human perception by comparing them with available human judgements of similarity. We show that speech representations extracted from a specific type of neural model (i.e. Transformers) lead to a better match with human perception than two earlier approaches on the basis of phonetic transcriptions and MFCC-based acoustic features. We furthermore find that features from the neural models can generally best be extracted from one of the middle hidden layers than from the final layer. We also demonstrate that neural speech representations not only capture segmental differences, but also intonational and durational differences that cannot adequately be represented by a set of discrete symbols used in phonetic transcriptions.
WeClick: Weakly-Supervised Video Semantic Segmentation with Click Annotations

Compared with tedious per-pixel mask annotating, it is much easier to annotate data by clicks, which costs only several seconds for an image. However, applying clicks to learn video semantic segmentation model has not been explored before. In this work, we propose an effective weakly-supervised video semantic segmentation pipeline with click annotations, called WeClick, for saving laborious annotating effort by segmenting an instance of the semantic class with only a single click. Since detailed semantic information is not captured by clicks, directly training with click labels leads to poor segmentation predictions. To mitigate this problem, we design a novel memory flow knowledge distillation strategy to exploit temporal information (named memory flow) in abundant unlabeled video frames, by distilling the neighboring predictions to the target frame via estimated motion. Moreover, we adopt vanilla knowledge distillation for model compression. In this case, WeClick learns compact video semantic segmentation models with the low-cost click annotations during the training phase yet achieves real-time and accurate models during the inference period. Experimental results on Cityscapes and Camvid show that WeClick outperforms the state-of-the-art methods, increases performance by 10.24% mIoU than baseline, and achieves real-time execution.
Evaluating Dropout Placements in Bayesian Regression Resnet

Abstract Deep Neural Networks (DNNs) have shown great success in many fields. Various network architectures have been developed for different applications. Regardless of the complexities of the networks, DNNs do not provide model uncertainty. Bayesian Neural Networks (BNNs), on the other hand, is able to make probabilistic inference. Among various types of BNNs, Dropout as a Bayesian Approximation converts a Neural Network (NN) to a BNN by adding a dropout layer after each weight layer in the NN. This technique provides a simple transformation from a NN to a BNN. However, for DNNs, adding a dropout layer to each weight layer would lead to a strong regularization due to the deep architecture. Previous researches [1, 2, 3] have shown that adding a dropout layer after each weight layer in a DNN is unnecessary. However, how to place dropout layers in a ResNet for regression tasks are less explored. In this work, we perform an empirical study on how different dropout placements would affect the performance of a Bayesian DNN. We use a regression model modified from ResNet as the DNN and place the dropout layers at different places in the regression ResNet. Our experimental results show that it is not necessary to add a dropout layer after every weight layer in the Regression ResNet to let it be able to make Bayesian Inference. Placing Dropout layers between the stacked blocks i.e. Dense+Identity+Identity blocks has the best performance in Predictive Interval Coverage Probability (PICP). Placing a dropout layer after each stacked block has the best performance in Root Mean Square Error (RMSE).
Sweep-to-Unlock: Fingerprinting Smartphones Based on Loudspeaker Roll-Off Characteristics

Fingerprinting smartphones based on acoustic characteristics of their loudspeaker may have a number of applications in device-to-device authentication as well as in forensic investigations. In this work we propose an efficient fingerprinting methodology by using the roll-off characteristics of the device speaker, i.e., the transition between the low and high stopbands to the passband segment of the speaker. We extract roll-off characteristics from sweep signals, also know as chirps, that are commonly used in practice to test speaker response. This procedure appears to be more stable against variations of the volume level and allows the use of simple linear approximations, which are intuitive and easy to compute, in order to extract the fingerprint. To increase detection accuracy, on the basis of the proven performance of deep learning techniques, a convolutional and a bi-directional long short term memory neural network are further proposed and their performance demonstrated for authentication purposes. While numerous applications may be envisioned, we specifically focus on the use of speaker characteristics in relation to in-vehicle infotainment units, checking if recordings from these units can be used to fingerprint a specific phone.
A Non-Hierarchical Attention Network with Modality Dropout for Textual Response Generation in Multimodal Dialogue Systems

Existing text- and image-based multimodal dialogue systems use the traditional Hierarchical Recurrent Encoder-Decoder (HRED) framework, which has an utterance-level encoder to model utterance representation and a context-level encoder to model context representation. Although pioneer efforts have shown promising performances, they still suffer from the following challenges: (1) the interaction between textual features and visual features is not fine-grained enough. (2) the context representation can not provide a complete representation for the context. To address the issues mentioned above, we propose a non-hierarchical attention network with modality dropout, which abandons the HRED framework and utilizes attention modules to encode each utterance and model the context representation. To evaluate our proposed model, we conduct comprehensive experiments on a public multimodal dialogue dataset. Automatic and human evaluation demonstrate that our proposed model outperforms the existing methods and achieves state-of-the-art performance.
Non-Monotone Energy-Aware Information Gathering for Heterogeneous Robot Teams

This paper considers the problem of planning trajectories for a team of sensor-equipped robots to reduce uncertainty about a dynamical process. Optimizing the trade-off between information gain and energy cost (e.g., control effort, distance travelled) is desirable but leads to a non-monotone objective function in the set of robot trajectories. Therefore, common multi-robot planning algorithms based on techniques such as coordinate descent lose their performance guarantees. Methods based on local search provide performance guarantees for optimizing a non-monotone submodular function, but require access to all robots&#x2019; trajectories, making it not suitable for distributed execution. This work proposes a distributed planning approach based on local search and shows how lazy/greedy methods can be adopted to reduce the computation and communication of the approach. We demonstrate the efficacy of the proposed method by coordinating robot teams composed of both ground and aerial vehicles with different sensing/control profiles and evaluate the algorithm&#x2019;s performance in two target tracking scenarios. Compared to the naive distributed execution of local search, our approach saves up to 60% communication and 80&#x2013;92% computation on average when coordinating up to 10 robots, while outperforming the coordinate descent based algorithm in achieving a desirable trade-off between sensing and energy cost.
Evidential Weighted Multi-view Clustering

Generally, the data to be clustered are from one single view. In real clustering applications, sometimes the data are insufficient so that it is difficult to learn an ideal cluster model. In such cases, multi-view data can be taken into consideration in the clustering task. However, the inconsistency cross views may increase the cluster uncertainty. In this research, a new clustering method for multi-view object data, called MvWECM (Multi-view Weighted Evidential C-Means) is introduced in the framework of belief functions. The proposed method can take consistency and diversity cross each view into account by incorporating the concept of view weights to measure the importance of each view. An objective function is defined to look for the best credal partitions over the different views. Experimental results on generated and UCI data sets show the advantage of the proposed method.
Revisiting Radial Distortion Rectification in Polar-Coordinates: A New and Efficient Learning Perspective

Fisheye cameras can capture a large field-of-view (Fov) scene but it introduces severe radial distortion in images. Thus, distortion rectification is a crucial step for subsequent computer vision tasks using fisheye cameras. A prevalent type of method predicts the displacement field between the input and output to rectify the distorted images. However, it is challenging to estimate the accurate flow in Cartesian coordinates (both <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$x$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$y$ </tex-math></inline-formula> directions), in which the sampling strategy of the convolution kernel ignores the radial symmetry of distortion. In general, the pixel’s distortion at the same radius from the center is the same, while the radius corresponds to one parameter in polar coordinates. Motivated by this fact, we exploit the radial symmetry of distortion to predict a more straightforward one-dimensional flow, transforming the distorted image into the polar coordinates domain instead of predicting two-dimensional flow in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$x$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$y$ </tex-math></inline-formula> directions. Specifically, we propose a Polar coordinates Distortion Rectification Network (PCDRN), whose sampling strategy corresponds to the radial distortion characteristic so that a more accurate flow can be predicted. To eliminate the blurs and ring artifacts induced by the coordinates transformation, a Polar-To-Cartesian Appearance Enhancement Network is designed to enhance the local appearance of rectified images. Experimental results on the synthesized dataset and real-world dataset demonstrate the superiority of our approach in both quantitative and qualitative evaluations.
Session Recommendation Model Based on Context-Aware and Gated Graph Neural Networks

The graph neural network (GNN) based approach has been successfully applied to session-based recommendation tasks. However, in the face of complex and changing real-world situations, the existing session recommendation algorithms do not fully consider the context information in user decision-making; furthermore, the importance of context information for the behavior model has been widely recognized. Based on this, this paper presents a session recommendation model based on context-aware and gated graph neural networks (CA-GGNNs). First, this paper presents the session sequence as data of graph structure. Second, the embedding vector representation of each item in the session graph is obtained by using the gated graph neural network (GGNN). In this paper, the GRU in GGNN is expanded to replace the input matrix and the state matrix in the conventional GRU with input context captured in the session (e.g., time, location, and holiday) and interval context (representing the proportion of the total session time of each item in the session). Finally, a soft attention mechanism is used to capture users' interests and preferences, and a recommendation list is given. The CA-GGNN model combines session sequence information with context information at each time. The results on the open Yoochoose and Diginetica datasets show that the model has significantly improved compared with the latest session recommendation methods.
UprightRL: Upright Orientation Estimation of 3D Shapes via Reinforcement Learning

Abstract In this paper, we study the problem of 3D shape upright orientation estimation from the perspective of reinforcement learning, i.e. we teach a machine (agent) to orientate 3D shapes step by step to upright given its current observation. Unlike previous methods, we take this problem as a sequential decision‐making process instead of a strong supervised learning problem. To achieve this, we propose UprightRL, a deep network architecture designed for upright orientation estimation. UprightRL mainly consists of two submodules: an Actor module and a Critic module which can be learned with a reinforcement learning manner. Specifically, the Actor module selects an action from the action space to perform a point cloud transformation and obtain the new point cloud for the next environment state, while the Critic module evaluates the strategy and guides the Actor to choose the next stage action. Moreover, we design a reward function that encourages the agent to select action which is conducive to orient model towards upright orientation with a positive reward and negative otherwise. We conducted extensive experiments to demonstrate the effectiveness of the proposed model, and experimental results show that our network outperforms the state‐of‐the‐art. We also apply our method to the robot grasping‐and‐placing experiment, to reveal the practicability of our method.