Reviewerly/exHarmony huggingface dataset:
The P-DESTRE: A Fully Annotated Dataset for Pedestrian Detection, Tracking, and Short/Long-Term Re-Identification From Aerial Devices

Over the years, unmanned aerial vehicles (UAVs) have been regarded as a potential solution to surveil public spaces, providing a cheap way for data collection, while covering large and difficult-to-reach areas. This kind of solutions can be particularly useful to detect, track and identify subjects of interest in crowds, for security/safety purposes. In this context, various datasets are publicly available, yet most of them are only suitable for evaluating detection, tracking and short-term re-identification techniques. This paper announces the free availability of the P-DESTRE dataset, the first of its kind to provide video/UAV-based data for pedestrian long-term re-identification research, with ID annotations consistent across data collected in different days. As a secondary contribution, we provide the results attained by the state-of-the-art pedestrian detection, tracking, short/long term re-identification techniques in well-known surveillance datasets, used as baselines for the corresponding effectiveness observed in the P-DESTRE data. This comparison highlights the discriminating characteristics of P-DESTRE with respect to similar sets. Finally, we identify the most problematic data degradation factors and co-variates for UAV-based automated data analysis, which should be considered in subsequent technologic/conceptual advances in this field. The dataset and the full specification of the empirical evaluation carried out are freely available at http://p-destre.di.ubi.pt/.
Data-Efficient Hierarchical Reinforcement Learning for Robotic Assembly Control Applications

Hierarchical reinforcement learning (HRL) can learn the decomposed subpolicies corresponding to the local state-space; therefore, it is a promising solution to complex robotic assembly control tasks with fewer interactions with environments. Most existing HRL algorithms often require on-policy learning, where resampling is necessary for every training step. In this article, we propose a data-efficient HRL via off-policy learning with three main contributions. First, two augmented MDPs (Markov decision processes) are reformulated to learn the higher level policy and lower level policy from the same samples. Second, to learn higher level policy that leads to efficient exploration, a softmax gating policy is derived to determine the lower level policy for interacting with the environment. Third, to learn the lower level policies via off-policy samples from one lower level replay buffer, the higher level policy derived by the option-value network is adopted to select the appropriate option for learning the corresponding lower level policy. The data-efficiency performance of our algorithm is validated on two simulations and real-world robotic dual peg-in-hole assembly tasks.
Novel Coverless Steganography Method Based on Image Selection and StarGAN

With the development of steganalysis technology keeps progressing, the traditional image steganography has confronted some security bottleneck, as the secret embedding method destroys the naturality of cover image. To overcome this difficulty, cover less image steganography (CIS) has been established for selecting images which contains secrets, and it has received widespread attention due to its merits of persevering the cover image which contains secret information. However, the existing CIS methods have the limitation of being restricted by the size of image database. Up to now, the maximum hidden capacity by CIS is only 18 bits. In order to breakthrough this limitation, this paper proposes a new CIS method based on image selection and Star Generative Adversarial Network (StarGAN). In our scheme, a traditional CIS method is used to select a natural image from database to represent the first part of the secret information, and then an image mapping is established between the rest of the secret information and the face attributes. Finally, the StarGAN is used to construct a high-quality stego image with the mapping relationship. Compared with the existing CIS methods, our method can increase the hidden capacity and maintain a better image quality, while being stronger in terms of robustness and security performance.
MAD‐UNet: A deep U‐shaped network combined with an attention mechanism for pancreas segmentation in CT images

Purpose Pancreas segmentation is a difficult task because of the high intrapatient variability in the shape, size, and location of the organ, as well as the low contrast and small footprint of the CT scan. At present, the U‐Net model is likely to lead to the problems of intraclass inconsistency and interclass indistinction in pancreas segmentation. To solve this problem, we improved the contextual and semantic feature information acquisition method of the biomedical image segmentation model (U‐Net) based on a convolutional network and proposed an improved segmentation model called the multiscale attention dense residual U‐shaped network (MAD‐UNet). Methods There are two aspects considered in this method. First, we adopted dense residual blocks and weighted binary cross‐entropy to enhance the semantic features to learn the details of the pancreas. Using such an approach can reduce the effects of intraclass inconsistency. Second, we used an attention mechanism and multiscale convolution to enrich the contextual information and suppress learning in unrelated areas. We let the model be more sensitive to pancreatic marginal information and reduced the impact of interclass indistinction. Results We evaluated our model using fourfold cross‐validation on 82 abdominal enhanced three‐dimensional (3D) CT scans from the National Institutes of Health (NIH‐82) and 281 3D CT scans from the 2018 MICCAI segmentation decathlon challenge (MSD). The experimental results showed that our method achieved state‐of‐the‐art performance on the two pancreatic datasets. The mean Dice coefficients were 86.10% ± 3.52% and 88.50% ± 3.70%. Conclusions Our model can effectively solve the problems of intraclass inconsistency and interclass indistinction in the segmentation of the pancreas, and it has value in clinical application. Code is available at https://github.com/Mrqins/pancreas‐segmentation .
Interval Model of the Efficiency of the Functioning of Information Web Resources for Services on Ecological Expertise

Mathematical models of the efficiency dynamics of information web resources are considered in this paper. The application of interval discrete models in the form of difference equations is substantiated and the approach to estimation of the model parameters is proposed. The proposed approach is based on the artificial bee colony algorithm (ABCA). A number of experimental studies have been carried out based on data on the functioning of web resources related to environmental monitoring services. The indicator of an information web resource user’s activity has been investigated. Three cases of model building in the form of difference equations as interval discrete models (IDM) have been considered. They vary in the general kind of expression. As a result of the computational experiments, it is shown that the adequacy of a model depends on the expression of the difference equation. In the case of its incorrect choice, the proposed method of parameters’ identification may be ineffective. The obtained interval discrete model in the difference equation form, which describes the efficiency of a web resource, makes it possible to optimize business processes in an organization that uses this web resource, as well as optimally allocate organizational resources and the workload of employees of the administrative service center. Based on the conducted experiments, the efficiency of the proposed model’s application is confirmed.
Improvement of End-to-End Offline Handwritten Mathematical Expression Recognition by Weakly Supervised Learning

This paper presents an improvement in recognizing offline handwritten mathematical expressions (HMEs) by deep neural networks. We train it end-to-end using weakly supervised learning. The network has three parts: an encoder using a Convolutional Neural Network to encode high-level features from an input HME image; a decoder using gated recurrent units with attention to parse the high-level features and generate an output expression in the LaTeX format; and a symbol classifier to improve the localization and classification of the high-level features. Besides, we use the model ensemble method for the beam search process to average the probabilities from multiple models. For the dataset of the Competition on Recognition of Online Handwritten Mathematical Expressions (CROHME) 2014 and 2016, we have achieved an expression recognition rate of 53.65% and 51.96% correspondingly, which is 6 points better than without weakly supervised learning. Furthermore, when ensembling several models, the recognition rate of our method is increased to 55.68% for the CROHME 2014 testing set.
Joint Learning of Social Groups, Individuals Action and Sub-group Activities in Videos

The state-of-the art solutions for human activity understanding from a video stream formulate the task as a spatio-temporal problem which requires joint localization of all individuals in the scene and classification of their actions or group activity over time. Who is interacting with whom, e.g. not everyone in a queue is interacting with each other, is often not predicted. There are scenarios where people are best to be split into sub-groups, which we call social groups, and each social group may be engaged in a different social activity. In this paper, we solve the problem of simultaneously grouping people by their social interactions, predicting their individual actions and the social activity of each social group, which we call the social task. Our main contributions are: i) we propose an end-to-end trainable framework for the social task; ii) our proposed method also sets the state-of-the-art results on two widely adopted benchmarks for the traditional group activity recognition task (assuming individuals of the scene form a single group and predicting a single group activity label for the scene); iii) we introduce new annotations on an existing group activity dataset, re-purposing it for the social task. The data and code for our method is publicly available ( https://github.com/mahsaep/Social-human-activity-understanding-and-grouping ).
Cross-Domain Image Captioning via Cross-Modal Retrieval and Model Adaptation

In recent years, large scale datasets of paired images and sentences have enabled the remarkable success in automatically generating descriptions for images, namely image captioning. However, it is labour-intensive and time-consuming to collect a sufficient number of paired images and sentences in each domain. It may be beneficial to transfer the image captioning model trained in an existing domain with pairs of images and sentences (i.e., source domain) to a new domain with only unpaired data (i.e., target domain). In this paper, we propose a cross-modal retrieval aided approach to cross-domain image captioning that leverages a cross-modal retrieval model to generate pseudo pairs of images and sentences in the target domain to facilitate the adaptation of the captioning model. To learn the correlation between images and sentences in the target domain, we propose an iterative cross-modal retrieval process where a cross-modal retrieval model is first pre-trained using the source domain data and then applied to the target domain data to acquire an initial set of pseudo image-sentence pairs. The pseudo image-sentence pairs are further refined by iteratively fine-tuning the retrieval model with the pseudo image-sentence pairs and updating the pseudo image-sentence pairs using the retrieval model. To make the linguistic patterns of the sentences learned in the source domain adapt well to the target domain, we propose an adaptive image captioning model with a self-attention mechanism fine-tuned using the refined pseudo image-sentence pairs. Experimental results on several settings where MSCOCO is used as the source domain and five different datasets (Flickr30k, TGIF, CUB-200, Oxford-102 and Conceptual) are used as the target domains demonstrate that our method achieves mostly better or comparable performance against the state-of-the-art methods. We also extend our method to cross-domain video captioning where MSR-VTT is used as the source domain and two other datasets (MSVD and Charades Captions) are used as the target domains to further demonstrate the effectiveness of our method.
Modular deep reinforcement learning from reward and punishment for robot navigation

Modular Reinforcement Learning decomposes a monolithic task into several tasks with sub-goals and learns each one in parallel to solve the original problem. Such learning patterns can be traced in the brains of animals. Recent evidence in neuroscience shows that animals utilize separate systems for processing rewards and punishments, illuminating a different perspective for modularizing Reinforcement Learning tasks. MaxPain and its deep variant, Deep MaxPain, showed the advances of such dichotomy-based decomposing architecture over conventional Q-learning in terms of safety and learning efficiency. These two methods differ in policy derivation. MaxPain linearly unified the reward and punishment value functions and generated a joint policy based on unified values; Deep MaxPain tackled scaling problems in high-dimensional cases by linearly forming a joint policy from two sub-policies obtained from their value functions. However, the mixing weights in both methods were determined manually, causing inadequate use of the learned modules. In this work, we discuss the signal scaling of reward and punishment related to discounting factor γ, and propose a weak constraint for signaling design. To further exploit the learning models, we propose a state-value dependent weighting scheme that automatically tunes the mixing weights: hard-max and softmax based on a case analysis of Boltzmann distribution. We focus on maze-solving navigation tasks and investigate how two metrics (pain-avoiding and goal-reaching) influence each other’s behaviors during learning. We propose a sensor fusion network structure that utilizes lidar and images captured by a monocular camera instead of lidar-only and image-only sensing. Our results, both in the simulation of three types of mazes with different complexities and a real robot experiment of an L-maze on Turtlebot3 Waffle Pi, showed the improvements of our methods.
Physics-informed reinforcement learning optimization of nuclear assembly design

Optimization of nuclear fuel assemblies if performed effectively, will lead to fuel efficiency improvement, cost reduction, and safety assurance. However, assembly optimization involves solving high-dimensional and computationally expensive combinatorial problems. As such, fuel designers’ expert judgement has commonly prevailed over the use of stochastic optimization (SO) algorithms such as genetic algorithms and simulated annealing. To improve the state-of-art, we explore a class of artificial intelligence (AI) algorithms, namely, reinforcement learning (RL) in this work. We propose a physics-informed AI optimization methodology by establishing a connection through reward shaping between RL and the tactics fuel designers follow in practice by moving fuel rods in the assembly to meet specific constraints and objectives. The methodology utilizes RL algorithms, deep Q learning and proximal policy optimization, and compares their performance to SO algorithms. The methodology is applied on two boiling water reactor assemblies of low-dimensional (∼2×106 combinations) and high-dimensional (∼1031 combinations) natures. The results demonstrate that RL is more effective than SO in solving high dimensional problems, i.e., 10 × 10 assembly, through embedding expert knowledge in form of game rules and effectively exploring the search space. For a given computational resources and timeframe relevant to fuel designers, RL algorithms outperformed SO through finding more feasible patterns, 4–5 times more than SO, and through increasing search speed, as indicated by the RL outstanding computational efficiency. The results of this work clearly demonstrate RL effectiveness as another decision support tool for nuclear fuel assembly optimization.
Injecting Reliable Radio Frequency Fingerprints Using Metasurface for the Internet of Things

In Internet of Things, where billions of devices with limited resources are communicating with each other, security has become a major stumbling block affecting the progress of this technology. Existing authentication schemes based on digital signatures have overhead costs associated with them in terms of computation time, battery power, bandwidth, memory, and related hardware costs. Radio frequency fingerprint (RFF), utilizing the unique device-based information, can be a promising solution for IoT. However, traditional RFFs have become obsolete because of low reliability and reduced user capability. Our proposed solution, Metasurface RF-Fingerprinting Injection (MeRFFI), is to inject a carefully-designed radio frequency fingerprint into the wireless physical layer that can increase the security of a stationary IoT device with minimal overhead. The injection of fingerprint is implemented using a low cost metasurface developed and fabricated in our lab, which is designed to make small but detectable perturbations in the specific frequency band in which the IoT devices are communicating. We have conducted comprehensive system evaluations including distance, orientation, multiple channels where the feasibility, effectiveness, and reliability of these fingerprints are validated. The proposed MeRFFI system can be easily integrated into the existing authentication schemes. The security vulnerabilities are analyzed for some of the most threatening wireless physical layer-based attacks.
Adaptive Potential Field-Based Path Planning for Complex Autonomous Driving Scenarios

An adaptive potential field is designed to adapt the acceleration/deceleration and mass of the obstacle. The potential fields are established in a transformed road coordinate system to improve the feasibility and robustness. A path planning method is proposed based on the designed adaptive potential field to improve the driving safety and the ride comfort of autonomous vehicles in complex driving scenarios, which including the cut-in, emergency braking, obstacle sudden accelerating during overtaking and the curve road driving scenarios. The effectiveness of the proposed method is validated by simulations with constructed and real data, respectively. The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$TTC\text{s}$ </tex-math></inline-formula> (Time-to-Collision) and the maximum lateral accelerations are used to evaluate the improvements on safety and ride comfort. The results show that both the driving safety and ride comfort are efficiently improved by using the proposed approach in emergency braking and accelerating scenarios. Meanwhile, the proposed method can be well applied in a curve road driving environment.
An efficient copy move forgery detection using adaptive watershed segmentation with AGSO and hybrid feature extraction

Copy-move forgery detection (CMFD) is the process of determining the presence of copied areas in an image. CMFD approaches are mainly classified into two groups: keypoint-based and block-based techniques. In this paper, a new CMFD approach is proposed on the basis of both block and keypoint based approaches. Initially, the forged image is partitioned into non overlapped segments utilizing adaptive watershed segmentation, wherein adaptive H-minima transform is used for extracting the markers. Also, an Adaptive Galactic Swarm Optimization (AGSO) algorithm is used to select optimal gap parameter while selecting the markers for reducing the undesired regional minima, which can increase the segmentation performance. After that, the features from every segment are extracted as segment features (SF) using Hybrid Wavelet Hadamard Transform (HWHT). Then, feature matching is performed using adaptive thresholding. The false matches or outliers can be removed with the help of Random Sample Consensus (RANSAC) algorithm. Finally, the Forgery Region Extraction Algorithm (FREA) is utilized for detecting the copied portion from the host image. Experimental results indicate that the proposed scheme find out image forgery region with Precision = 92.45%; Recall = 93.67% and F1 = 92.75% on MICC-F600 dataset and Precision = 94.52%; Recall = 95.32% and F1 = 93.56% on Bench mark dataset at pixel level. Also, it outperforms the existing approaches when the image undergone certain geometrical transformation and image degradation.
High-resolution 3D abdominal segmentation with random patch network fusion

Deep learning for three dimensional (3D) abdominal organ segmentation on high-resolution computed tomography (CT) is a challenging topic, in part due to the limited memory provide by graphics processing units (GPU) and large number of parameters and in 3D fully convolutional networks (FCN). Two prevalent strategies, lower resolution with wider field of view and higher resolution with limited field of view, have been explored but have been presented with varying degrees of success. In this paper, we propose a novel patch-based network with random spatial initialization and statistical fusion on overlapping regions of interest (ROIs). We evaluate the proposed approach using three datasets consisting of 260 subjects with varying numbers of manual labels. Compared with the canonical "coarse-to-fine" baseline methods, the proposed method increases the performance on multi-organ segmentation from 0.799 to 0.856 in terms of mean DSC score (p-value < 0.01 with paired t-test). The effect of different numbers of patches is evaluated by increasing the depth of coverage (expected number of patches evaluated per voxel). In addition, our method outperforms other state-of-the-art methods in abdominal organ segmentation. In conclusion, the approach provides a memory-conservative framework to enable 3D segmentation on high-resolution CT. The approach is compatible with many base network structures, without substantially increasing the complexity during inference. Given a CT scan with at high resolution, a low-res section (left panel) is trained with multi-channel segmentation. The low-res part contains down-sampling and normalization in order to preserve the complete spatial information. Interpolation and random patch sampling (mid panel) is employed to collect patches. The high-dimensional probability maps are acquired (right panel) from integration of all patches on field of views.