Reviewerly/exHarmony huggingface dataset:
Correlation Filtering-Based Hashing for Fine-Grained Image Retrieval

The low storage and strong representation capabilities of hash codes for image retrievalhas made hashing technologies very popular. Several existing deep hashing methods focuson the task of general image retrieval, while neglecting the task of fine-grained image retrieval. Recently, some fine-grained hashing methods have been proposed to capture the subtle differences, which mainly utilize the single-modality visual features to solve the discriminative region localization while ignoring the semantic information. In this letter, we propose a correlation filtering hashing (CFH) method to learn discrete binary codes, which can adequately take advantage of the cross-modal correlation between the semantic information and the visual features for discriminative region localization. Specifically, we utilize a feature pyramid network to learn multi-level visual features. Subsequently, the label vector is embedded into the visual space, which can be used as a correlation filter on the feature maps to capture the latent location of objects. Finally, weperform global average pooling over the output maps and concatenate the features of different levels to produce the hash codes of query images. Extensive experiments on two fine-grained datasets show that the proposed CFH outperforms the state-of-the-art hashing methods.
Monocular Differentiable Rendering for Self-supervised 3D Object Detection

3D object detection from monocular images is an ill-posed problem due to the projective entanglement of depth and scale. To overcome this ambiguity, we present a novel self-supervised method for textured 3D shape reconstruction and pose estimation of rigid objects with the help of strong shape priors and 2D instance masks. Our method predicts the 3D location and meshes of each object in an image using differentiable rendering and a self-supervised objective derived from a pretrained monocular depth estimation network. We use the KITTI 3D object detection dataset to evaluate the accuracy of the method. Experiments demonstrate that we can effectively use noisy monocular depth and differentiable rendering as an alternative to expensive 3D ground-truth labels or LiDAR information.
Dimension by dimension dynamic sine cosine algorithm for global optimization problems

To solve global optimization problems, this paper proposed a novel improved version of sine cosine algorithm — the dimension by dimension dynamic sine cosine algorithm (DDSCA). In the update equation of sine cosine algorithm (SCA), the dimension by dimension strategy evaluates the solutions in each dimension, and the greedy strategy is used to form new solutions after combined them with other dimensions. Moreover, in order to balance the exploration and exploitation of SCA, a dynamic control parameter is designed to modify the position equation of this algorithm. To evaluate the effectiveness of DDSCA in solving global optimization problems, it is compared with state-of-art algorithms and modified SCA on 23 benchmark functions. The experimental results reveal the DDSCA has better robustness and efficiency. The IEEE CEC2010 large-scale functions are selected to solve high-dimensional optimization problem, the results show that the performance of the DDSCA is better than other algorithms. In addition, five engineering optimization problems are also verified the effectiveness of the DDSCA. The results of accuracy and speed show that the improved sine cosine algorithm (DDSCA) is competitive in solving global optimization problems.
CSART: Channel and spatial attention-guided residual learning for real-time object tracking

Siamese networks have achieved great success in object tracking due to the balance of precision and speed. However, Siamese trackers usually utilize the local feature of the last layer, which may degrade tracking performance in some difficult scenarios. In this paper, we propose a novel Channel and Spatial Attention-guided Residual learning framework for Tracking, referred to as CSART, which can improve feature representation of Siamese networks by exploiting self-attention mechanism to capture powerful contextual information. Specifically, to be efficient and seamless integration, different kinds of self-attention are appended on the template and search branches of Siamese networks respectively, that model global semantic inter-dependencies in channel and spatial dimensions. To avoid representation degradation, we consider to adaptively aggregate basic feature and its attention-weighted features with residual learning. Furthermore, a joint loss consisting of classic logistic loss as well as focal softmax loss is designed to emphasize difficult samples and guide the learning process of the whole model. Benefiting from the above scheme, CSART alleviates the over-fitting problem to some extent and enhances the discriminability. Extensive experiments on six popular tracking datasets indicate that the proposed tracker achieves better performance with a speed of 65 fps than other state-of-the-art trackers.
AttentionNAS: Spatiotemporal Attention Cell Search for Video Classification

Convolutional operations have two limitations: (1) do not explicitly model where to focus as the same filter is applied to all the positions, and (2) are unsuitable for modeling long-range dependencies as they only operate on a small neighborhood. While both limitations can be alleviated by attention operations, many design choices remain to be determined to use attention, especially when applying attention to videos. Towards a principled way of applying attention to videos, we address the task of spatiotemporal attention cell search. We propose a novel search space for spatiotemporal attention cells, which allows the search algorithm to flexibly explore various design choices in the cell. The discovered attention cells can be seamlessly inserted into existing backbone networks, e.g., I3D or S3D, and improve video classification accuracy by more than 2% on both Kinetics-600 and MiT datasets. The discovered attention cells outperform non-local blocks on both datasets, and demonstrate strong generalization across different modalities, backbones, and datasets. Inserting our attention cells into I3D-R50 yields state-of-the-art performance on both datasets.
HARD-Net: Hardness-AwaRe Discrimination Network for 3D Early Activity Prediction

Predicting the class label from the partially observed activity sequence is a very hard task, as the observed early segments of different activities can be very similar. In this paper, we propose a novel Hardness-AwaRe Discrimination Network (HARD-Net) to specifically investigate the relationships between the similar activity pairs that are hard to be discriminated. Specifically, a Hard Instance-Interference Class (HI-IC) bank is designed, which dynamically records the hard similar pairs. Based on the HI-IC bank, a novel adversarial learning scheme is proposed to train our HARD-Net, which thus grants our network with the strong capability in mining subtle discrimination information for 3D early activity prediction. We evaluate our proposed HARD-Net on two public activity datasets and achieve state-of-the-art performance.
Wavelet-Based Dual-Branch Network for Image Demoiréing

When smartphone cameras are used to take photos of digital screens, usually moiré patterns result, severely degrading photo quality. In this paper, we design a wavelet-based dual-branch network (WDNet) with a spatial attention mechanism for image demoiréing. Existing image restoration methods working in the RGB domain have difficulty in distinguishing moiré patterns from true scene texture. Unlike these methods, our network removes moiré patterns in the wavelet domain to separate the frequencies of moiré patterns from the image content. The network combines dense convolution modules and dilated convolution modules supporting large receptive fields. Extensive experiments demonstrate the effectiveness of our method, and we further show that WDNet generalizes to removing moiré artifacts on non-screen images. Although designed for image demoiréing, WDNet has been applied to two other low-level vision tasks, outperforming state-of-the-art image deraining and deraindrop methods on the Rain100h and Raindrop800 data sets, respectively.
Frequency Domain Image Translation: More Photo-realistic, Better Identity-preserving

Image-to-image translation has been revolutionized with GAN-based methods. However, existing methods lack the ability to preserve the identity of the source domain. As a result, synthesized images can often over-adapt to the reference domain, losing important structural characteristics and suffering from suboptimal visual quality. To solve these challenges, we propose a novel frequency domain image translation (FDIT) framework, exploiting frequency information for enhancing the image generation process. Our key idea is to decompose the image into low-frequency and high-frequency components, where the high-frequency feature captures object structure akin to the identity. Our training objective facilitates the preservation of frequency information in both pixel space and Fourier spectral space. We broadly evaluate FDIT across five large-scale datasets and multiple tasks including image translation and GAN inversion. Extensive experiments and ablations show that FDIT effectively preserves the identity of the source image, and produces photo-realistic images. FDIT establishes state-of-the-art performance, reducing the average FID score by 5.6% compared to the previous best method.
Still-camera multiview Spectral Optical Flow Imaging for 3D operating-deflection-shape identification

To measure high-frequency 3D vibrations, multi-camera, high-speed imaging hardware is normally required. An alternative using still-frame cameras was recently introduced with the Spectral Optical Flow Imaging (SOFI) method. In this research, the SOFI method is extended to multiview measurements of spatial operating deflection shapes. This is achieved by utilizing harmonically controlled illumination to perform an analogue Fourier transform on image-intensity data in multiple camera views. The obtained multiview displacement spectra are combined with geometrical data to perform frequency-domain triangulation and reconstruct spatial deflection shapes. By introducing additional camera views into the image-based measurement, its field of view is extended and the signal-to-noise ratio of the final result is increased. For linear, time-invariant mechanical structures under stationary excitation, full-field 3D measurements of high-frequency vibrations can be performed using a single still-frame monochrome camera. The proposed method identifies displacements in the frequency domain directly on the camera sensor, resulting in orders-of-magnitude smaller data sizes and post-processing times compared with conventional multiview image-based methods.
DATA BREACH MANAGEMENT: AN INTEGRATED RISK MODEL

In response to organizations’ increasing vulnerability to data breaches, we present an integrated risk model for data breach management based on a systematic review of the literature. Theoretically, the study extends the body of knowledge on data breach management by identifying and updating conceptualizations of data breach risks (items) and resolutions (actions) and by providing a foundation for organizational responses to emerging data breach incidents (heuristics). Practically, the study provides key insights that practitioners can use to organize and orchestrate effective data breach management based on comprehensive profiles of risk items and resolution techniques.
Person Retrieval in Surveillance Videos Via Deep Attribute Mining and Reasoning

Person retrieval largely relies on the appearance features of pedestrians. This task is rather more difficult in surveillance videos due to the limitations of extracting robust appearance features brought by the cross-view and cross-camera data with lower image resolution, motion blur, occlusion and other kinds of image degradation. To build up a more reliable person retrieval system, recent works introduced appearance attribute models to describe and distinguish different persons with high-level semantic concepts. Despite the progress of previous works, the value of utilizing appearance attributes is still under-explored. On one hand, existing methods lack for concise and precise attribute representations that are specific for each attribute category and, in the meantime, are able to filter noisy information in irrelevant spatial locations and useless patterns. On the other hand, correlation and reasoning between different attributes are neglected, which could generate more useful information and add more robustness to the retrieval system. In this paper, we propose an Attribute Mining and Reasoning (AMR) framework which is capable to handle the issues in question. The AMR makes better use of appearance attributes with two main components. First, the AMR disentangles the representations of different attributes by localizing their spatial positions and identifying their effective patterns in a weakly supervised manner. To achieve more reliable localization, we propose the Attribute Localization Ensemble (ALE) module that is consisted of multiple localization heads and a voting mechanism. Second, we introduce the Attribute Reasoning (AR) module to correlate different attributes together with the global appearance features and discover their latent relations to generate more comprehensive descriptions of pedestrians. Extensive experiments on DukeMTMC-ReID and Market-1501 datasets demonstrate the effectiveness of the proposed AMR framework as well as its superiority over the existing state-of-the-art methods. The AMR model also shows great generalization ability on the unseen CUHK03 dataset when it is only trained on Market-1501 dataset.
DOPE: Distillation of Part Experts for Whole-Body 3D Pose Estimation in the Wild

We introduce DOPE, the first method to detect and estimate whole-body 3D human poses, including bodies, hands and faces, in the wild. Achieving this level of details is key for a number of applications that require understanding the interactions of the people with each other or with the environment. The main challenge is the lack of in-the-wild data with labeled whole-body 3D poses. In previous work, training data has been annotated or generated for simpler tasks focusing on bodies, hands or faces separately. In this work, we propose to take advantage of these datasets to train independent experts for each part, namely a body, a hand and a face expert, and distill their knowledge into a single deep network designed for whole-body 2D-3D pose detection. In practice, given a training image with partial or no annotation, each part expert detects its subset of keypoints in 2D and 3D and the resulting estimations are combined to obtain whole-body pseudo ground-truth poses. A distillation loss encourages the whole-body predictions to mimic the experts’ outputs. Our results show that this approach significantly outperforms the same whole-body model trained without distillation while staying close to the performance of the experts. Importantly, DOPE is computationally less demanding than the ensemble of experts and can achieve real-time performance. Test code and models are available at https://europe.naverlabs.com/research/computer-vision/dope .
Motion Capture from Internet Videos

Recent advances in image-based human pose estimation make it possible to capture 3D human motion from a single RGB video. However, the inherent depth ambiguity and self-occlusion in a single view prohibit the recovery of as high-quality motion as multi-view reconstruction. While multi-view videos are not common, the videos of a celebrity performing a specific action are usually abundant on the Internet. Even if these videos were recorded at different time instances, they would encode the same motion characteristics of the person. Therefore, we propose to capture human motion by jointly analyzing these Internet videos instead of using single videos separately. However, this new task poses many new challenges that cannot be addressed by existing methods, as the videos are unsynchronized, the camera viewpoints are unknown, the background scenes are different, and the human motions are not exactly the same among videos. To address these challenges, we propose a novel optimization-based framework and experimentally demonstrate its ability to recover much more precise and detailed motion from multiple videos, compared against monocular motion capture methods.
CLAVER: An integrated framework of convolutional layer, bidirectional LSTM with attention mechanism based scholarly venue recommendation

Scholarly venue recommendation is an emerging field due to a rapid surge in the number of scholarly venues concomitant with exponential growth in interdisciplinary research and cross collaboration among researchers. Finding appropriate publication venues is confronted as one of the most challenging aspects in paper publication as a larger proportion of manuscripts face rejection due to a disjunction between the scope of the venue and the field of research pursued by the research article. We present CLAVER--an integrated framework of Convolutional Layer, bi-directional LSTM with an Attention mechanism-based scholarly VEnue Recommender system. The system is the first of its kind to integrate multiple deep learning-based concepts, that requires only the abstract and the title of a manuscript to identify academic venues. An extensive and exhaustive set of experiments conducted on the DBLP dataset certify that the postulated model CLAVER performs better than most of the modern techniques as entrenched by standard metrics such as stability, accuracy, MRR, average venue quality, [email protected], [email protected] and diversity.
Seamless, Bi-directional Transitions along the Reality-Virtuality Continuum: A Conceptualization and Prototype Exploration

With head mounted displays, consumers are able to transition from the real world to virtual realities. However, this requires frequent transitions between the two realities to maintain their physical integrity and awareness of the real world while in the virtual space. We completed two consecutive studies to investigate the dimensions of a system that supports seamless transition between realities without requiring the user to remove the headset. Our results are twofold: First, based on the the analysis of structured interviews (n=20), we present a conceptualization of existing solutions (n=37) and novel ideas (n=9) in the form of a design space. Second, we present the results of a user study (n=36) in which we tested two exemplary prototypes that evolved from the design space, called “Sky Portal” and “Virtual Phone.” Our exploration shows that our “Virtual Phone” metaphor has the potential to support HMD users in completing bidirectional transitions along Milgram's reality-virtuality continuum. Users are also enabled to complete micro-interactions across the realities, even without performance loss.
A systematic mapping review of context-aware analysis and its approach to mobile learning and ubiquitous learning processes

In the last ten years, it was observed that the continuous upgrades in mobile device’s technology have increased and demonstrated their great potential in various learning environments. Besides, it has motivated researchers to apply more innovative computational techniques regarding context-aware and a set of variables that have been used at different virtual learning proposals. This paper presents a Systematic Mapping Review that focuses on context-aware analysis and its approach to learning processes in mobile learning (m-learning) and ubiquitous learning (u-learning). Furthermore, the study identifies variables that have been used in the past decade for context-awareness analytics and that have been applied to those learning processes. Especially at systems’ adaptations to learning styles and student-specific characteristics. A methodological process was applied to address problems through identification, critical evaluation, and integration of the most relevant works, where high-quality individual studies address one or several research questions. Results show that external variables, including location, time, and software, are the most used variables in the context-aware analysis, with 52.25% prevalence in research papers. Internal variables, including personal information, learning styles, and teaching styles, had a prevalence of 33.33%; variables less frequently used in the research included socioeconomic information and emotional content. The remaining 14.41% represents academic activities. These findings provide a basis for future research and development in the field of m-learning.
Deep convolutional neural network in medical image processing

Researchers have started constructing systems that could automatically analyze the medical images. In the initial phase (starting from 1970 to 1990), image processing was carried out using a sequential application of pixel analysis and mathematical model. By the end of 1990, supervised learning became popular in medical image processing, in which the trained images were used to design the models. But the conventional machine learning techniques require a crucial process of feature extraction to be carried out by the researchers. The next level of development is to construct the system, which can do feature learning automatically and can represent the images in an optimal way. Here lies the basis of many deep learning (DL) techniques in which the network comprises multiple layers that process the input image to output result by doing the feature extraction by itself. The most popular and efficient DL model for medical image processing is the convolutional neural network (CNN). There are various medical imaging modalities with the help of which critical images of the patient's body structures are captured and are fed to the CNN model for the analysis. In this chapter, authors have done a detailed discussion on different CNN architectures and their applications in the medical imaging domain. Moreover, a state-of-the-art comparison has been carried out between several existing works inside medical imaging based on CNN. Lastly, this chapter concludes with several critical remarks highlighting future challenges and their solutions.
Artificial Intelligence Aided Electronic Warfare Systems- Recent Trends and Evolving Applications

Electronic warfare (EW) is one of the most important characteristics of modern battles. EW can affect a military force's use of the electromagnetic spectrum to detect targets or to provide information. Recent developments in artificial intelligence (AI) suggest that this emerging technology will have a deterministic and potentially transformative influence on military power. AI driven algorithms can be very effective in diverse domain of EW like processing of radar signals for efficient recognition and classification of emitters, detection of jammer and its characteristics and for developing efficient anti-jamming algorithms. AI techniques can also enable an EW system to operate autonomously. This paper provides a description of various branches of EW, the role of AI in EW systems and different AI techniques that have been deployed in EW systems.