Reviewerly/exHarmony huggingface dataset:
Contrast-enhanced CT Image Synthesis of Thyroid Based on Transfomer and Texture Branching

Thyroid cancer is one of the most common cancers in young women, but due to the noise and artifacts of ultrasound images, there is still a certain misdiagnosis rate in clinical practice, and it is often combined with plain and Contrast-enhanced CT for further diagnosis. Compared with plain CT, Contrast-enhanced CT has better contrast with reflecting the erosion of organ margins, which is an important symptom for diagnosing thyroid cancer. Contrast-enhanced CT, however, relies on the patient being injected with a contrast agent and exposed to ionizing radiation. Our work proposes an improved Unet architecture. To generate enhanced CT images with clear texture and higher quality, we use the convolutional Transformer module to learn the global information of high-dimensional features, and then fuse the texture feature module to extract the local texture information of plain CT and the edge information extracted by superpixels as a priori knowledge to restore texture details. Experimental results show that our framework outperforms state-of-the-art generative networks and can generate higher-quality Contrast-enhanced CT.
The Influence of Embedding Size and Hidden Units Parameters Changes on the Sequence2Sequence Model

The parameters of the neural network are very important to the model, and the changes of the parameters have a greater impact on the accuracy of the neural network. This paper takes the Sequence2Sequence (Seq2Seq) model as an example, in which Encoder and Decoder both use basic RNN models to research the impact of parameter changes on neural network performance. The EmbeddingSize and Hidden Units are two important parameters of the Seq2Seq model. This paper uses parameter combinations to build different models based on the two important parameters. And using self-made Sequential-Reverse data sets to experiment on these models. Then visualize the experimental results in three dimensions and analyze the influence of these two parameters on the Seq2Seq model. The results of this paper can provide researchers with some suggestions when they build neural networks.
Single Image Reflection Removal Based on Reflection Region

Images taken through transparent glass usually contain reflections, which will produce visual noise and reduce image quality. Aiming at the problem of reflection affecting image quality, we propose a method based on reflection region to remove reflections. Firstly, according to the feature of small gradient in the reflection region, the initial reflection region marker map is obtained by gradient image; Then, the more accurate reflection region marker map is obtained by using both the background region marker map and the maximum chromaticity difference binary map; Finally, according to the reflection region marker map to suppress the reflection in the reflection region. The comparison experiments with objective and subjective evaluations show that our method is better at removing reflection and retaining background details than several state-of-the-art methods.
A Recommendation Model with Attention-Aware Graph Matching

Under the background of information overload, recommendation system can provide decision-making tools for information producers to promote information and information consumers to obtain personalized preference information. Use user-item attribute information for interaction can effectively improve the performance of the recommendation model. Existing work models and aggregates the attribute interaction of users and items in the same way, without effectively distinguishing them. However, this can't express the feature interaction between attributes well. In this work, we propose Attention-aware Graph Matching Recommendation model (AGMRec), which convert the recommendation process into a graph matching problem. In particular, we design internal-node message passing and external-node matching to model different orders feature interactions between attributes. Then the proposed model uses an attention approach to fuse different aspects of the nodes. Experimental results on two real-world datasets show its superiority over the state-of-the-arts.
APRSR: Adversarial Personalized Ranking Modeling for Sequential Recommendation

In recent years, the sequential recommendation achieves excellent results. But it also meets many challenges. For example, when we add adversarial perturbations to the input, the model’s performance might be weakened. To solve this problem, we propose a novel model named APRSR, Adversarial Personalized Ranking Modeling for Sequential Recommendation in short. APRSR model is designed based on a self-attention sequential recommendation model by adding adversarial perturbations. First, APRSR can learn local representation and the global representation respectively, then it can get the final representation by balancing the local representation and the global representation. In the meantime, it considers the influence of candidate items on user’s intent. To enhance the robustness and learn more expressive features, APRSR utilizes the idea of the adversarial matrix factorization which can generate the adversarial perturbations. Extensive experiments on five public real-world datasets demonstrate the effectiveness of APRSR and outperform those of other state-of-the-art models.
NEW GENERATION UNIVERSITIES: TYPES, CHARACTERISTICS AND MODERN CHALLENGES

Purpose of the research is to analyze the essence, role and models of universities of the new generation and to identify their modern challenges. Methodology of the research comprises of the critical review of fundamental and empirical research on entrepreneurial and engaged universities from the Web of Science database, processing and systematization of sources, comparative analysis of university models. Originality / value of the research. The results of the research can be used by the Ministry of Education and Science of the Republic of Kazakhstan, higher educational institutions, as well as other stakeholders and the general public. Findings – the article analyzes the most popular sources from the Web of Science database, dedicated to entrepreneurial and engaged universities in the framework of the Triple helix. The author identified the factors that affect their formation and development, consequently the comparative analysis of these two models was conducted.
A Continuous Learning Approach for Probabilistic Human Motion Prediction

Human Motion Prediction (HMP) plays a crucial role in safe Human-Robot-Interaction (HRI). Currently, the majority of HMP algorithms are trained by massive pre-collected data. As the training data only contains a few pre-defined motion patterns, these methods cannot handle the unfamiliar motion patterns. Moreover, the pre-collected data are usually non-interactive, which does not consider the real-time responses of collaborators. As a result, these methods usually perform unsatisfactorily in real HRI scenarios. To solve this problem, in this paper, we propose a novel Continual Learning (CL) approach for probabilistic HMP which makes the robot continually learns during its interaction with collaborators. The proposed approach consists of two steps. First, we leverage a Bayesian Neural Network to model diverse uncertainties of observed human motions for collecting online interactive data safely. Then we take Experience Replay and Knowledge Distillation to elevate the model with new experiences while maintaining the knowledge learned before. We first evaluate our approach on a large-scale benchmark dataset Human3.6m. The experimental results show that our approach achieves a lower prediction error compared with the baselines methods. Moreover, our approach could continually learn new motion patterns without forgetting the learned knowledge. We further conduct real-scene experiments using Kinect DK. The results show that our approach can learn the human kinematic model from scratch, which effectively secures the interaction.
Perception-Friendly Video Enhancement for Autonomous Driving under Adverse Weather Conditions

Visual perception of an autonomous vehicle is a crucial component of autonomous driving technologies. While visual perception research has achieved promising performance in recent years, modern methods are mostly trained, applied, and tested on single clean images. Recently, deep learning-based perception methods have addressed multiple degrading effects to reflect real-world bad weather cases, but have achieved only limited success, mainly due to 1) less or no temporal information across adjacent frames and 2) poor correlation between image enhancement and visual perception performance. To solve these issues, in this paper we propose a simple and effective video enhancement network incorporating the perception module, one of the high-level vision tasks, which takes video degraded by adverse weather conditions as an input, and produces an enhanced image and a recognition result as output. Besides, this allows us to leverage temporal information across consecutive frames without flow estimation. We also introduce a new training strategy to robustly guide the high-level task model suitable for both high-quality restoration of images and highly accurate perception. Further, considering sequence-level discrimination, we propose a Sequential Contrastive Loss, called SCL, to maximize apparent discrimination over sequential input. By doing this, our algorithm achieves full interaction showing mutual influence between the enhancement and perception tasks. Further, we introduce a novel low memory network dropping out most of the layer connections of dense blocks to reduce memory usage and computational cost while maintaining high performance. Experiment results demonstrate that the proposed method significantly improves the performance on object detection, distance estimation, and memory usage under adverse weather.
Prediction of Depth Camera Missing Measurements Using Deep Learning for Next Best View Planning

Depth images usually contain pixels with invalid measurements. This paper presents a deep learning approach that receives as input a partially-known volumetric model of the environment and a camera pose, and it predicts the probability that a pixel would contain a valid depth measurement if a camera was placed at the given pose. The proposed network architecture consists of a 3D Convolutional Neural Network (CNN) module and a 2D CNN module, connected by a deep learning attention-based projection module. The method was integrated into a CNN-based probabilistic Next Best View plan-ner, resulting in a more realistic prediction of the information gain for each possible viewpoint with respect to state of the art approaches. Experiments were carried out in tabletop scenarios using a robot manipulator with an eye-in-hand depth camera.
A Model Predictive-based Motion Planning Method for Safe and Agile Traversal of Unknown and Occluding Environments

Agile navigation through uncertain and obstacle-rich environments remains a challenging task for autonomous mobile robots (AMR). For most AMR, obstacles are identified using onboard sensors, e.g., lidar or cameras. The effectiveness of these sensors may be severely limited, however, by occlusions introduced from the presence of other obstacles. The occluded area may contain obstacles, static or dynamic, not included into the motion planning of the robot and could cause potential collisions if they suddenly appear in the field of view of the robot. This paper proposes a general Model Predictive Control (MPC)-based framework for handling occlusions in structured or unstructured environments, that contain known or unknown static or dynamic obstacles. Safety is promoted by commanding velocities that consider surrounding obstacle uncertainty, while perception is promoted through a specially designed objective that can reduce the occluded area created by obstacles. The effectiveness of this framework is validated through simulations that show swift and safe motion in a variety of different environments. Similarly, experimental validation is achieved with a Boston Dynamics' Spot quadruped robot operating in an occluding environment.
SiamX: An Efficient Long-term Tracker Using Cross-level Feature Correlation and Adaptive Tracking Scheme

Siamese network based trackers have achieved significant progress in visual object tracking. For the sake of speed, they mainly rely on offline training to learn a mono-level feature correlation between a target template and a search region. During the tracking period, they use a fixed strategy to infer target positions over sequences regardless of target states. However, such approaches are vulnerable in case of long-term challenges e.g. large variance, presence of distractors, fast motion, or target disappearing and the like. In this paper, we propose a new tracking framework, referred to as SiamX, by exploiting cross-level Siamese features to learn robust correlations between the target template and search regions, and also adaptive inference strategies to prevent tracking loss and realize fast target re-localization. Extensive experiments on four benchmarks including VOT-2019, LaSOT, GOT-10k, and TrackingNet show our method significantly enhances the tracker's ability to resist variance and interference, and achieve state-of-the-art results at around 50 FPS.
Few-Shot Keypoint Detection as Task Adaptation via Latent Embeddings

Dense object tracking, the ability to localize specific object points with pixel-level accuracy, is an important computer vision task with numerous downstream applications in robotics. Existing approaches either compute dense keypoint embeddings in a single forward pass, meaning the model is trained to track everything at once, or allocate their full capacity to a sparse predefined set of points, trading generality for accuracy. In this paper we explore a middle ground based on the observation that the number of relevant points at a given time are typically relatively few, e.g. grasp points on a target object. Our main contribution is a novel architecture, inspired by few-shot task adaptation, which allows a sparse-style network to condition on a keypoint embedding that indicates which point to track. Our central finding is that this approach provides the generality of dense-embedding models, while offering accuracy significantly closer to sparse-keypoint approaches. We present results illustrating this capacity vs. accuracy trade-off, and demonstrate the ability to zero-shot transfer to new object instances (within-class) using a real-robot pick-and-place task.
Flow Supervised Neural Radiance Fields for Static-Dynamic Decomposition

We present an approach to synthesize novel views from dynamics scenes captured by multi-view videos of cameras mounted on a driving vehicle. We unify existing methods and propose a new training loss to explicitly disentangle the static background from the dynamic foreground objects using scene flow's magnitude, learnt only from proxy 2D optical flow supervision. We obtain high quality static and dynamic contents separately, which allow us to combine them freely for novel view and time syntheses. We establish a dataset consisting of 5 dynamic scenes with varying difficulties on which we conduct experiments, and show that our method is able to handle challenging scenarios in real-world traffics and create high quality novel view and time syntheses.
Dynamic Mirror Descent based Model Predictive Control for Accelerating Robot Learning

Recent works in Reinforcement Learning (RL) combine model-free (Mf)-RL algorithms with model-based (Mb)-RL approaches to get the best from both: asymptotic performance of Mf-RL and high sample-efficiency of Mb-RL. Inspired by these works, we propose a hierarchical framework that integrates online learning for the Mb-trajectory optimization with off-policy methods for the Mf-RL. In particular, two loops are proposed, where the Dynamic Mirror Descent based Model Predictive Control (DMD-MPC) is used as the inner loop Mb-RL to obtain an optimal sequence of actions. These actions are in turn used to significantly accelerate the outer loop Mf-RL. We show that our formulation is generic for a broad class of MPC based policies and objectives, and includes some of the well-known Mb-Mf approaches. We finally introduce a new algorithm: Mirror-Descent Model Predictive RL (M-DeMoRL), which uses Cross-Entropy Method (CEM) with elite fractions for the inner loop. Our experiments show faster convergence of the proposed hierarchical approach on benchmark MuJoCo tasks. We also demonstrate hardware training for trajectory tracking in a 2R leg, and hardware transfer for robust walking in a quadruped. We show that the inner-loop Mb-RL significantly decreases the number of training iterations required in the hardware setting, thereby validating the proposed approach.
Star-Convolution for Image-Based 3D Object Detection

3D object detection with only image inputs is an interesting and important problem in computer vision and autonomous driving. Nowadays, most existing monocular 3D object detection algorithms rely solely on the approximation power of convolutional neural networks to learn a mapping from pixels to 3D predictions without knowing the projection matrix of the camera. To endow the networks with camera projection knowledge, we propose the Star-Convolution module for application to image-based 3D detection. The introduced module increases the receptive field of the detector and embeds the camera's projection geometry inside the network while keeping the network end-to-end trainable. We test the module with different baselines in both monocular and stereo 3D object detection, and we achieve significant improvements on both tasks. The code will be published at https://github.com/Owen-Liuyuxuan/visualDet3D.
Promoting Quality and Diversity in Population-based Reinforcement Learning via Hierarchical Trajectory Space Exploration

Quality Diversity (QD) algorithms in population-based reinforcement learning aim to optimize agents' returns and diversity among the population simultaneously. It is conducive to solving exploration problems in reinforcement learning and potentially getting multiple good and diverse strategies. However, previous methods typically define behavioral embedding in action space or outcome space, which neglect trajectory characteristics during the execution process. In this paper, we introduce a trajectory embedding model trained by Variational Autoencoder with similarity constraint to characterize trajectory features. Based on that, we propose a hierarchical trajectory-space exploration (HTSE) framework using Determinantal Point Processes (DPP) to generate high-quality and diverse solutions in the selection and mutation process. The experimental results show that our HTSE method effectively completes several simulated tasks, outperforming other Quality-Diversity Reinforcement Learning algorithms.
Using Eye Gaze to Forecast Human Pose in Everyday Pick and Place Actions

Collaborative robots that operate alongside humans require the ability to understand their intent and forecast their pose. Among the various indicators of intent, the eye gaze is particularly important as it signals action towards the gazed object. By observing a person's gaze, one can effectively predict the object of interest and subsequently, forecast the person's pose. We leverage this and present a method that forecasts the human pose using gaze information for everyday pick and place actions in a home environment. Our method first attends to fixations to locate the coordinates of the object of interest before inputting said coordinates to a pose forecasting network. Experiments on the MoGaze dataset show that our gaze network lowers the errors of existing pose forecasting methods and that incorporating prior in the form of textual instructions further lowers the errors by a significant amount. Furthermore, the use of eye gaze now allows a simple multilayer perceptron network to directly forecast the keypose. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</sup> Code available at www.imperial.ac.uk/personal-robotics/software
Composable Causality in Semantic Robot Programming

Assembly tasks are challenging for robot manipulation because the robot must reason over the composed effects of actions and execute multi-objective behaviors. Robots typically use predefined priorities provided by users to determine how to compose controller behaviors, but we want the robot to autonomously select these compositions based on their composed effects within the task. We present Composable Causality in Semantic Robot Programming to allow robots to reason over the composed effects of controllers when executing multi-objective actions and autonomously compose controllers without predefined priorities. Our proposed causal control basis combines controller behaviors with causal information about how the behaviors can be used to execute high-level symbolic actions. The robot uses the causal control basis to predict the transition probability of achieving the composed effects of a multi-objective action. The composed causality estimates are used to select which action to execute within the context of a furniture assembly task. We evaluate the robot's transition probability estimates in different furniture assembly trials in simulation on the Baxter robot. The robot's ability to assemble furniture using different multi-objective connection actions demonstrates the usefulness of the composed causality estimates from our causal control basis.
A Double Branch Next-Best-View Network and Novel Robot System for Active Object Reconstruction

Next best view (NBV) is a technology that finds the best view sequence for sensor to perform scanning based on partial information, which is the core part for robot active reconstruction. Traditional works are mostly based on the evaluation of candidate views through time-consuming volu-metric transformation and ray casting, which heavily limits the applications of NBV. Recent deep learning based NBV methods aim to approximately learn the evaluation function by large-scale training, and improve both the effectiveness and efficiency of NBV. However, these methods force the network to regress the exact groundtruth value of each candidate view, which is much harder than simply ranking all the candidate views. Besides, most previous NBV works assume perfect sensing and perform in simulation environments, lacking real application abilities. In this paper, we propose a novel double branch NBV network, DB-NBV, to utilize the ranking process together with the evaluation process. We further design a real NBV robot and a pipeline to conduct real active reconstruction. Experiments on both simulation and real robot show that our method achieves the best performance and can be applied to real application with high accuracy and speed.
Visually Grounding Language Instruction for History-Dependent Manipulation

This paper emphasizes the importance of a robot's ability to refer to its task history, especially when it exe-cutes a series of pick-and-place manipulations by following language instructions given one by one. The advantage of referring to the manipulation history can be categorized into two folds: (1) the language instructions omitting details but using expressions referring to the past can be interpreted, and (2) the visual information of objects occluded by previous manipulations can be inferred. For this, we introduce a history-dependent manipulation task which objective is to visually ground a series of language instructions for proper pick-and-place manipulations by referring to the past. We also suggest a relevant dataset and model which can be a baseline, and show that our model trained with the proposed dataset can also be applied to the real world based on the CycleGAN. Our dataset and code are publicly available on the project website: https://sites.google.com/view/history-dependent-manipulation.