Miracle-dz/UniFAR-FTUs huggingface dataset:
Training Object Detectors if Only Large Objects are Labeled

Conventional methods for object detection typically rely on large, well-annotated datasets, which are in short supply due to the high costs of labeling. In this paper, we propose to label only large, easy-to-spot objects. We argue that these contain more pixels and therefore usually more information about the underlying object class than small ones. At the same time, they are easier to spot and hence cheaper to label. Unfortunately, standard supervised learning algorithms do not learn to detect small objects if only large ones are labeled. Instead, they erroneously take up unlabeled objects as negative examples and their accuracy consequently deteriorates. To address that, we propose PCIS, a novel combination of P seudo-labels, output C onsistency across scales, and an anchor scale-dependent I gnore S trategy. In experiments on CityPersons, EuroCityPersons, and MS COCO, we show that our approach outperforms existing pseudo-label generation methods as well as an oracle which ensures that anchors overlapping missing annotations are ignored during training. We demonstrate that using our method it is possible to approach the performance of a fully labeled dataset with only a subset of the labels and also to train detectors on extremely sparsely labeled images, e.g. if only 1 out of 200 objects is annotated.
Intrinsic Examples: Robust Fingerprinting of Deep Neural Networks

This paper proposes to use intrinsic examples as a DNN ﬁngerprinting technique for the functionality veriﬁcation of DNN models implemented on edge devices. The proposed intrinsic examples do not affect the normal DNN training and can enable the black-box testing capability for DNN models packaged into edge device applications. We provide three algorithms for deriving intrinsic examples of the pre-trained model (the model before the DNN system design and implementation procedure) to retrieve the knowledge learnt from the training dataset for the detection of adversarial third-party attacks such as transfer learning and fault injection attack that may happen during the system implementation procedure. Besides, they can accommodate the model transformations due to various DNN model compression methods used by the system designer. quantization reduces redundancy in bit representation of weights [22, 26, 33]. With a k -bit weight representation, quantization maps weights into a total of 2 k quantized levels.
Reference Guided Image Inpainting using Facial Attributes

Image inpainting is a technique of completing missing pixels such as occluded region restoration, distracting objects removal, and facial completion. Among these inpainting tasks, facial completion algorithm performs face inpainting according to the user direction. Existing approaches require delicate and well controlled input by the user, thus it is difficult for an average user to provide the guidance sufficiently accurate for the algorithm to generate desired results. To overcome this limitation, we propose an alternative user-guided inpainting architecture that manipulates facial attributes using a single reference image as the guide. Our end-to-end model consists of attribute extractors for accurate reference image attribute transfer and an inpainting model to map the attributes realistically and accurately to generated images. We customize MS-SSIM loss and learnable bidirectional attention maps in which importance structures remain intact even with irregular shaped masks. Based on our evaluation using the publicly available dataset CelebA-HQ, we demonstrate that the proposed method delivers superior performance compared to some state-of-the-art methods specialized in inpainting tasks.
3D Flow Field Estimation around a Vehicle Using Convolutional Neural Networks

Flow fields, including velocity and pressure fields, are typically used as references for vehicle shape design in the automotive industry to ensure steering stability and energy conservation. Generally, flow fields are calculated using computational fluid dynamics (CFD) simulations which is time-consuming and expensive. Therefore, a more efficient and interactive method is desired by designers for advanced shape discussion and design. To this end, we propose a fast estimation model using 3D convolutional neural networks. We employ a style extractor to obtain sufficient deep features of each vehicle shape and apply them using adaptive instance normalisation to improve the estimation performance. In addition, a proposed loss function which mainly includes a slice-weighted loss function is used to train the estimation model. The findings show that our proposed method outperforms previous studies, especially on flow field estimation in wake regions and regions near the vehicle surface. Therefore, the proposed method allows designing vehicle shapes while ensuring desirable aerodynamic performance within a much shorter period than extended CFD simulations.
Learning Attention Map for 3D Human Recovery from a Single RGB Image

3D human recovery from a single RGB image is a promising topic in computer vision, virtual reality, and image processing. It focuses on estimating 3D pose and shape of human from a 2D image. Due to the lack of depth and local information, the task remains challenging. Targeting to solve these problems, this work proposes a LAMNet with three branches that learning attention map from depth and parsing features for 3D human recovery. The first branch explicitly leverages the depth and pose cues to learn an adjusted depth map, which alleviates the recovery error between 3D space and 2D plane. The second branch explicitly leverages human parsing to provide local information, which supplements the shape or edge details of 3D recovery. The last branch is the main branch which is responsible for learning 2D global features to estimate 3D pose and shape of human. Inspired by attention mechanism, an attention aware fusion is designed to integrating three branches. It can achieve an attention map containing depth and local cues, which effectively improves the precision of 3D recovery, especially in details and different perspectives. Extensive experimental results demonstrate that our proposed approach significantly outperforms most state-of-the-art methods on the popular Human3.6m, UP-3D, and 3DPW datasets.
Absolute Scale from Varifocal Monocular Camera through SfM and Defocus Combined

The absolute scale estimation of monocular structure from motion (SfM) is still under-explored even though it is essential for robotic tasks or real-world interaction. Typically, the use of physical scale cues requires a calibration process while context scale cues introduce geometric assumptions. In this paper, we propose a novel method to obtain absolute scales of the scene and camera motion by combining monocular SfM and uncalibrated depth from defocus (DfD) which is free for zooming and focusing on each shot independently. Speciﬁcally, we exploit that the scene structure and ﬁeld of view (FoV) of each camera estimated by SfM are tightly coupled to the focal length and focused distance of DfD, and the radius of the effective aperture of the lens constrains the absolute scale of the entire estimation. The effectiveness of the proposed method is veriﬁed by using a commercially available camera with a varifocal lens through various experiments.
Updated Paired Regions for Shadow Detection from Single Image

In recent years, deep neural network based shadow detection approaches achieve high performance on benchmarks, but they require large amounts of labeled data for training to learn the statistical attributes of the shadowed and unshadowed regions. The physical relationship between non-grid regions is ignored due to the character of deep convolution network. In this paper, we seek to analyze the physical principle of the shadowed region and present a paired region based algorithm for shadow detection without extra model training. To be speciﬁc, we ﬁrst segment the image via a region growing based approach to maintain the character of shadowed region. Then the penumbra is detected and used as an important cue. After that we adopt three physics based conﬁdence coefﬁcients when comparing the color of two regions. At last, we design a novel objective function to ﬁnd the best paired strategy and detect the shadowed region. The proposed approach is tested on three public datasets. The comparative results show that the proposed non-learning approach performs favorably against the state-of-the-art approaches.
Faster-FCoViAR: Faster Frequency-Domain Compressed Video Action Recognition

Human action recognition (HAR) is an essential task in computer vision, which still faces the critical challenge of reducing the data redundancy of decompressed video frames and extracting identiﬁcation information. To address this challenge, we pro-pose a novel faster frequency-domain compressed video action recognition framework (termed Faster-FCoViAR), which consists of a frequency-domain partial decompression method (FPDec), a frequency-domain channel selection strategy (FCS), and a spatial-to-frequency domain student-teacher network (S2FNet). The FPDec obtains frequency-domain DCT coefﬁcients of compressed videos directly without inverse discrete cosine transform (IDCT) for decompression. The FCS down-samples frequency-domain data to enhance the saliency of input. The S2FNet transfers spatial semantic knowledge from a spatial teacher network to a light-weight student network in the frequency domain, and it thus improves the spatial feature extraction ability of the frequency-domain network. Experiments on datasets UCF-101, HMDB-51, and Kinetics-400 show that our Faster-FCoViAR is 12.3 times faster than the frame-based methods and 6.7 times faster than other compressed domain methods based on competitive recognition accuracy compared with the state-of-the-art action recognition methods.
Bird's Eye View Segmentation Using Lifted 2D Semantic Features

We consider the problem of Bird’s Eye View (BEV) segmentation with perspective monocular camera view as input. An effective solution to this problem is important in many autonomous navigation tasks such as behavior prediction and planning, being that the BEV segmented image provides an explainable intermediate representation that captures both the geometry and layout of the surrounding scene. Our approach to this problem involves a novel BEV feature transformation layer that effectively exploits depth maps to transform 2D image features to the BEV space. The framework includes the de-sign of a neural network architecture to produce BEV segmentation maps using the proposed transformation layer. Of particular interest is evaluation of the proposed method in complex scenarios involving highly unstructured scenes that are not represented in static maps. In the absence of an appropriate dataset for this task, we introduce the EPOSH road-scene dataset that consists of 560 video-clips of highly unstructured construction scenes, annotated with unique labels in both perspective and BEV. For evaluation, we compare our approach with several competitive baselines and recently published works and show improvement over state of the art on the Nuscenes and on our EPOSH dataset. We plan to release the dataset, code and trained models at https://usa.honda-ri.com/eposh.
Multi-bit Adaptive Distillation for Binary Neural Networks

Binary neural networks (BNNs) represent weights and activations using 1-bit values, which has extremely lower memory costs and computational complexities, but usually suffer from severe accuracy degradation. Knowledge distillation is an effective way to improve the performance of BNN by inheriting the knowledge from higher-bit network. However, faced with the accuracy gap and bit gap between 1-bit network and different higher-bit networks, it is uncertain which higher-bit network is more suitable to be the teacher of a certain BNN. Therefore, we propose a novel multi-bit adaptive distillation(MAD) method for maximally integrating the advantages of various bit-width teacher networks( e.g . 2-bit, 4-bit, 8-bit and 32-bit). In practice, intermediate features and output logits of teachers will be simultaneously utilized for improving the performance of BNN. Moreover, an adaptive knowledge adjusting scheme is explored to dynamically adjust the contribution of different teachers in the distillation process. Comprehensive experiments conducted on CIFAR-10/100 and ImageNet datasets with various network architectures demonstrate the superiorities of MAD over many state-of-the-arts binarization methods. For instance, without introducing any extra inference calculations, our binarized ResNet-18 achieves 1.5% improvement for BirealNet binarization method on ImageNet.
Twitter-STMHD: An Extensive User-Level Database of Multiple Mental Health Disorders

Social Media is equipped with the ability to track and quantify user behavior, establishing it as an appropriate resource for mental health studies. However, previous efforts in the area have been limited by the lack of data and contextually relevant information. There is a need for large-scale, well-labeled mental health datasets with fast reproducible methods to facilitate their heuristic growth. In this paper, we cater to this need by building the Twitter - Self-Reported Temporally-Contextual Mental Health Diagnosis Dataset (Twitter-STMHD), a large scale, user-level dataset grouped into 8 disorder categories and a companion class of control users. The dataset is 60% hand-annotated, which lead to the creation of high-precision self-reported diagnosis report patterns, used for the construction of the rest of the dataset. The dataset, instead of being a corpus of tweets, is a collection of user-profiles of those suffering from mental health disorders to provide a holistic view of the problem statement. By leveraging temporal information, the data for a given profile in the dataset has been collected for disease prevalence periods: onset of disorder, diagnosis and progression, along with a fourth period: COVID-19. This is the only and the largest dataset that captures the tweeting activity of users suffering from mental health disorders during the COVID-19 period.
IET Renewable Power Generation Special Issue: Coordinated Control and Protection of Offshore Wind Power and Combined AC/DC Grid Introducing low-order system frequency response modelling of a future power system with high penetration of wind power plants with frequency support capabilities

Wind power generation has reached a significant share in power systems worldwide and will continue to increase. As the converter-connected generation reduces the grid inertia, more and more interest has been given to exploiting the kinetic energy and controllability of variable-speed wind turbine generators (VSWTGs) for frequency support. Consequently, the grid frequency dynamics are changing. Thus, it is necessary to include the frequency response of wind power plants in the system frequency response (SFR) model. A novel approach to low-order SFR modelling of a future power system with a high share of frequency-support-capable VSWTGs has been presented. Low-order model of VSWTGs with primary frequency response and natural inertial response has been developed considering different wind turbine operating regimes and compared to the nonlinear model for validation. Low-order model has been presented in a symbolic transfer function form. Model accuracy has been discussed and the impact of VSWTG parameters on frequency response has been analysed. The developed model facilitates studying power system frequency dynamics by avoiding the need for modelling complex VSWTG systems, while retaining a satisfying level of accuracy.
Knowing What, Where and When to Look: Video Action modelling with Attention

Attentive video modelling is essential for action recognition in unconstrained videos due to their rich yet redundant information over space and time. However, introducing attention in a deep neural network for action recognition is challenging for two reasons. First, an effective attention module needs to learn what (objects and their local motion patterns), where (spatially), and when (temporally) to focus on. Second, a video attention module must be efﬁcient because existing action recognition models already suffer from high computational cost. To address both challenges, a novel What-Where-When (W3) video attention module is proposed. Departing from existing alternatives, our W3 module models all three facets of video attention jointly. Crucially, it is extremely efﬁcient by factorising the high-dimensional video feature data into low-dimensional meaningful spaces (1D channel vector for ‘what’ and 2D spatial tensors for ‘where’), followed by a lightweight temporal attention reasoning. Extensive experiments show that our attention model brings signiﬁcant improvements to existing action recognition models, achieving a new state-of-the-art performance on a number of benchmarks.
Planar Shape Based Registration for Multi-modal Geometry

We present a global registration algorithm for multi-modal geometric data, typically 3D point clouds and meshes. Existing feature-based methods and recent deep learning based approaches typically rely upon point-to-point matching strategies that often fail to deliver accurate results from defect-laden data. In contrast, we reason at the scale of planar shapes whose detection from input data offers robustness on a range of defects, from noise to outliers through heterogeneous sampling. The detected planar shapes are projected into an accumulation space from which a rotational alignment is operated. A second step then reﬁnes the result with a local continuous optimization which also estimates the scale. We demonstrate the robustness and efﬁcacy of our algorithm on challenging real-world data. In particular, we show that our algorithm competes well against state-of-the-art methods, especially on piece-wise planar objects and scenes.
A new way of controlling NesCOPOs (Nested Cavity doubly resonant OPO) for faster and more efficient high resolution spectrum measurement

Optical Parametric Oscillator (OPO) is a well-known solution when wide tunability in the mid-infrared is needed. A specific design called NesCOPO (Nested Cavity doubly resonant OPO) is currently integrated in the X-FLR8 portable gas analyzer from Blue Industry and Science. Thanks to its low threshold this OPO can be pumped by a micro-chip nanosecond YAG (4 kHz repetition rate and a 30 GHz bandwidth). To achieve very high resolution spectra (10 pm of resolution or better), the emitted wavelength has to be finely controlled. Commercial Wavemeter do not meet price and compactness required in the context of an affordable and portable gas analyzer. To overcome this issue, Blue first integrated an active wavelength controller using multiple tunable Fabry-Perot (FP) interferometers. The required resolution was achieved at a 10 Hz measurement rate. We now present an enhanced Wavemeter architecture, based on fixed FP etalons, that is 100 times faster and 2 times smaller. We avoid having FP ‘blind zones’ thanks to one source characteristic: the knowledge of the FSR (Free Spectral Range) of the OPO source and thus, the fact that only discrete wavelengths can be emitted. First results are displayed showing faster measurement for spectroscopic application, and potential future improvement of the device are discussed.
Enhancing Human Motion Assessment by Self-supervised Representation Learning

The space of human motions is vast, ranging from daily behaviors of healthy adults to the slow and stiff motions of Parkinson’s patients, or to infant motions. This poses significant challenges when the task is focused on a relatively niche motion subspace such as physical rehabilitation: often the target datasets are limited and less-annotated; meanwhile, there exist large-scale, well-annotated benchmarks, typically consisting of daily activities from healthy adults. This observation inspires us to propose a two-stage pipeline that takes advantage of the best of both worlds: a non-expert network starts to learn the representation of normal motions from source datasets, by estimating the pace and a set of manually inpainted joints of the pose sequence; this is followed by an expert network that takes as input these representations as well as the appearance features of the dedicated motions from the target dataset, to assess the quality of the specific actions. Empirical experiments on two very different motion assessment applications (physical rehabilitation of Parkinson’s & stroke patients, and neuromotor behaviors of infants) demonstrate the superior performance of our approach.
DU-DARTS: Decreasing the Uncertainty of Differentiable Architecture Search

Differentiable Neural Architecture Search (DARTS) recently attracts a lot of research attention because of its high efﬁciency. However, the competition of candidate operations in DARTS introduces high uncertainty for selecting the truly important operation, thus leading to serious performance collapse. In this work, we decrease the uncertainty of differentiable architecture search (DU-DARTS) by enforcing the distribution of architecture parameters to approach the one-hot categorical distribution and by replacing the zero operation with a gate switch. Without any extra search cost, our method achieves state-of-the-art performance with 2.32%, 16.74%, and 24.1% test error on CIFAR-10, CIFAR-100, and ImageNet datasets, respectively. Moreover, DU-DARTS can robustly ﬁnd an excellent architecture on NAS-Bench-1Shot1 and NAS-Bench-201, which further demonstrates the effectiveness of our method. The source code is available at https://github.com/ShunLu91/DU-DARTS.
Robust Crowd Counting via Image Enhancement and Dynamic Feature Selection

In spite of their remarkable success in many vision tasks, convolutional neural networks (CNNs) often has trouble counting people in crowded scenes due to the following reasons. First, ordinary CNNs with ﬁxed receptive ﬁelds are inadequate to handle diverse sizes and densities of people. Second, CNNs for counting are sensitive to brightness and contrast changes of input image. This paper proposes a new CNN for crowd counting that resolves these two issues. First, we develop a new counting network called pyramid feature selection network (PFSNet) that adapts its receptive ﬁelds dynamically to local crowd densities of the input image. Second, we introduce a light-weight and effective image enhancement network, which manipulates input image to normalize its condition and make it more counting-friendly, leading to robust and improved crowd counting. The concatenation of the two networks, dubbed E-PFSNet, achieves the state of the art on three public benchmarks for crowd counting. Also, it outperforms previous arts in terms of robustness against changes in image conditions as well as counting accuracy.