Reviewerly/exHarmony huggingface dataset:
An Empirical Study on the Usage of Automated Machine Learning Tools

The popularity of automated machine learning (AutoML) tools in different domains has increased over the past few years. Machine learning (ML) practitioners use AutoML tools to automate and optimize the process of feature engineering, model training, and hyperparameter optimization and so on. Recent work performed qualitative studies on practitioners’ experiences of using AutoML tools and compared different AutoML tools based on their performance and provided features, but none of the existing work studied the practices of using AutoML tools in real-world projects at a large scale. Therefore, we conducted an empirical study to understand how ML practitioners use AutoML tools in their projects. To this end, we examined the top 10 most used AutoML tools and their respective usages in a large number of open-source project repositories hosted on GitHub. The results of our study show 1) which AutoML tools are mostly used by ML practitioners and 2) the characteristics of the repositories that use these AutoML tools. Also, we identified the purpose of using AutoML tools (e.g. model parameter sampling, search space management, model evaluation/error-analysis, Data/ feature transformation, and data labeling) and the stages of the ML pipeline (e.g. feature engineering) where AutoML tools are used. Finally, we report how often AutoML tools are used together in the same source code files. We hope our results can help ML practitioners learn about different AutoML tools and their usages, so that they can pick the right tool for their purposes. Besides, AutoML tool developers can benefit from our findings to gain insight into the usages of their tools and improve their tools to better fit the users’ usages and needs.
BS-SiamRPN: Hyperspectral Video Tracking based on Band Selection and the Siamese Region Proposal Network

The spectral information of hyperspectral data is helpful to improve the performance of target tracker. Limited by the number of training samples, most video tracking methods based on deep learning cannot effectively utilize the semantic information of hyperspectral data. Therefore, this paper proposes a hyperspectral video target tracking method based on Band Selection and the Siamese Region Proposal Network (BS-SiamRPN). Firstly, an intelligent optimization algorithm is utilized to determine three bands with the largest joint entropy. Based on limited hyperspectral information, Transfer Learning (TL) is performed on the pre-trained siamese networks to obtain a good parameter. Here, index. Then, the dataset in Generic Object Tracking (GOT) benchmark is utilized to retrain the SiamRPN method. Finally, the information of three specific bands referring to hyperspectral videos is regarded as the input of the migration network, which can obtain matching results. Experimental results on public hyperspectral data show that the proposed siamese network based tracking method effectively utilize spectral features, and do well in visual effects and objective evaluation.
3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal

Estimating 3D interacting hand pose from a single RGB image is essential for understanding human actions. Unlike most previous works that directly predict the 3D poses of two interacting hands simultaneously, we propose to decompose the challenging interacting hand pose estimation task and estimate the pose of each hand separately. In this way, it is straightforward to take advantage of the latest research progress on the single-hand pose estimation system. However, hand pose estimation in interacting scenarios is very challenging, due to (1) severe hand-hand occlusion and (2) ambiguity caused by the homogeneous appearance of hands. To tackle these two challenges, we propose a novel Hand De-occlusion and Removal (HDR) framework to perform hand de-occlusion and distractor removal. We also propose the first large-scale synthetic amodal hand dataset, termed Amodal InterHand Dataset (AIH), to facilitate model training and promote the development of the related research. Experiments show that the proposed method significantly outperforms previous state-of-the-art interacting hand pose estimation approaches. Codes and data are available at https://github.com/MengHao666/HDR .
S3C: Self-Supervised Stochastic Classifiers for Few-Shot Class-Incremental Learning

Few-shot class-incremental learning (FSCIL) aims to learn progressively about new classes with very few labeled samples, without forgetting the knowledge of already learnt classes. FSCIL suffers from two major challenges: (i) over-fitting on the new classes due to limited amount of data, (ii) catastrophically forgetting about the old classes due to unavailability of data from these classes in the incremental stages. In this work, we propose a self-supervised stochastic classifier (S3C) (code: https://github.com/JAYATEJAK/S3C ) to counter both these challenges in FSCIL. The stochasticity of the classifier weights (or class prototypes) not only mitigates the adverse effect of absence of large number of samples of the new classes, but also the absence of samples from previously learnt classes during the incremental steps. This is complemented by the self-supervision component, which helps to learn features from the base classes which generalize well to unseen classes that are encountered in future, thus reducing catastrophic forgetting. Extensive evaluation on three benchmark datasets using multiple evaluation metrics show the effectiveness of the proposed framework. We also experiment on two additional realistic scenarios of FSCIL, namely where the number of annotated data available for each of the new classes can be different, and also where the number of base classes is much lesser, and show that the proposed S3C performs significantly better than the state-of-the-art for all these challenging scenarios .
IDa-Det: An Information Discrepancy-Aware Distillation for 1-Bit Detectors

Knowledge distillation (KD) has been proven to be useful for training compact object detection models. However, we observe that KD is often effective when the teacher model and student counterpart share similar proposal information. This explains why existing KD methods are less effective for 1-bit detectors, caused by a significant information discrepancy between the real-valued teacher and the 1-bit student. This paper presents an Information Discrepancy-aware strategy (IDa-Det) to distill 1-bit detectors that can effectively eliminate information discrepancies and significantly reduce the performance gap between a 1-bit detector and its real-valued counterpart. We formulate the distillation process as a bi-level optimization formulation. At the inner level, we select the representative proposals with maximum information discrepancy. We then introduce a novel entropy distillation loss to reduce the disparity based on the selected proposals. Extensive experiments demonstrate IDa-Det’s superiority over state-of-the-art 1-bit detectors and KD methods on both PASCAL VOC and COCO datasets. IDa-Det achieves a 76.9% mAP for a 1-bit Faster-RCNN with ResNet-18 backbone. Our code is open-sourced on https://github.com/SteveTsui/IDa-Det .
A Survey of Ad Hoc Teamwork Research

Ad hoc teamwork is the research problem of designing agents that can collaborate with new teammates without prior coordination. This survey makes a two-fold contribution: First, it provides a structured description of the different facets of the ad hoc teamwork problem. Second, it discusses the progress that has been made in the field so far, and identifies the immediate and long-term open problems that need to be addressed in ad hoc teamwork.
Image Super-Resolution with Deep Dictionary

Since the first success of Dong et al., the deep-learning-based approach has become dominant in the field of single-image super-resolution. This replaces all the handcrafted image processing steps of traditional sparse-coding-based methods with a deep neural network. In contrast to sparse-coding-based methods, which explicitly create high/low-resolution dictionaries, the dictionaries in deep-learning-based methods are implicitly acquired as a nonlinear combination of multiple convolutions. One disadvantage of deep-learning-based methods is that their performance is degraded for images created differently from the training dataset (out-of-domain images). We propose an end-to-end super-resolution network with a deep dictionary (SRDD), where a high-resolution dictionary is explicitly learned without sacrificing the advantages of deep learning. Extensive experiments show that explicit learning of high-resolution dictionary makes the network more robust for out-of-domain test images while maintaining the performance of the in-domain test images. Code is available at https://github.com/shuntama/srdd .
ResDeepD: A residual super-resolution network for deep downscaling of daily precipitation over India

Abstract In the twenty-first century, machine learning and deep learning have been successfully used to find hidden information from coarse-grained data in various domains. In Computer Vision, scientists have used neural networks to identify hidden pixel-level information from low-resolution (LR) image data. This approach of estimating high-resolution (HR) information from LR data is called the super-resolution (SR) approach. This approach has been borrowed by climate scientists to downscale coarse-level measurements of climate variables to obtain their local-scale projections. Climate variables are spatial in nature and can be represented as images where each pixel denotes a grid point where the variables can be measured. We can apply the deep learning-based SR techniques on such “images” for statistical downscaling of such variables. This approach of downscaling can be termed as deep downscaling. In this work, we have tried to make HR projection of the Indian summer monsoon rainfall by using a novel deep residual network called ResDeepD. The aim is to downscale the 1 0 × 1 0 low LR precipitation data to get the values at 0.25 0 × 0.25 0 resolution. The proposed model uses a series of skip connections across residual blocks to give better results as compared to the existing models like super-resolution convolutional neural network, DeepSD, and Nest-UNet that have been used previously for this task. We have also examined the model’s performance for downscaling rainfall during some extreme climatic events like cyclonic storms and deep depression and found that the model performs better than the existing models.
Image Operation Chain Detection with Machine Translation Framework

The aim of operation chain detection for a given manipulated image is to reveal the operations involved and the order in which they were applied, which is significant for image processing and multimedia forensics. Currently, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">all</i> existing approaches simply treat image operation chain detection as a classification problem and consider only chains of at most two operations. Considering the complex interplay between operations and the exponentially increasing solution space, detecting longer operation chains is extremely challenging. To address this issue, in this work, we devise a new methodology for image operation chain detection. Different from existing approaches based on classification modeling, we strategically conduct operation chain detection within a machine translation framework. Specifically, the chain in our work is modeled as a sentence in a target language, with each possible operation represented by a word in that language. When executing chain detection, we propose first transforming the input image into a sentence in a latent source language from the learned deep features. Then, we propose translating the latent language into the target language within a machine translation framework and finally decoding all operations, arranged in order. Besides, a chain inversion strategy and a bi-directional modeling mechanism are developed to improve the detection performance. We further design a weighted cross-entropy loss to alleviate the problems presented by imbalance among chain lengths and chain categories. Our method can detect operation chains containing up to seven operations and obtains very promising results in various scenarios for the detection of both short and long chains.
Fairkit-learn

Advances in how we build and use software, specifically the integration of machine learning for decision making, have led to widespread concern around model and software fairness. We present fairkit-learn, an interactive Python toolkit designed to support data scientists' ability to reason about and understand model fairness. We outline how fairkit-learn can support model training, evaluation, and comparison and describe the potential benefit that comes with using fairkit-learn in comparison to the state-of-the-art. Fairkit-learn is open source at https://go.gmu.edu/fairkit-learn/.
TinyHAR: A Lightweight Deep Learning Model Designed for Human Activity Recognition

Deep learning models have shown excellent performance in human activity recognition tasks. However, these models typically require large amounts of computational resources, which makes them inefficient to deploy on edge devices. Furthermore, the superior performance of deep learning models relies heavily on the availability of large datasets to avoid over-fitting. However, the expensive efforts for labeling limits the amount of datasets. We address both challenges by designing a more lightweight model, called TinyHAR. TinyHAR is designed specifically for human activity recognition employing different saliency of multi modalities, multimodal collaboration, and temporal information extraction. Initial experimental results show that TinyHAR is several times smaller and often meets or even surpasses the performance of DeepConvLSTM, a state-of-the-art human activity recognition model.
Blind Image Quality Assessment via Cross-View Consistency

Image quality assessment (IQA) is very important for both end-users and service-providers since a high-quality image can significantly improve the user's quality of experience (QoE). Most existing blind image quality assessment (BIQA) models were developed for synthetically distorted images, however, they perform poorly on in-the-wild images, which are widely existed in various practical applications. In this paper, a BIQA model is proposed that consists of a desirable self-supervised feature learning approach to mitigate the data shortage problem and learn comprehensive feature representations, and a self-attention-based feature fusion module to introduce self-attention mechanism. We develop the image quality assessment model under the framework of contrastive learning with multi views. Since human visual system perceives signals through multiple channels, the most important visual information should exist among all views of the channels. So we design the cross-view consistent information mining (CVC-IM) module to extract compact mutual information between different views. Color information and pseudo-reference image (PRI) of different distortion types are employed to formulate rich feature embeddings and preserve the quality-aware fidelity of learned representations. We employ the Transformer as the self-attention-based architecture to integrate feature embeddings. Extensive experiments show that our model achieves remarkable image quality assessment results on in-the-wild IQA datasets.
SAPM: Self-Adaptive Parallel Manipulator With Pose and Force Adjustment for Robotic Ultrasonography

Robotic ultrasonography potentially acts as an essential aid to medical diagnosis. To overcome the limitations in robotic ultrasonography, in this paper, we proposed a novel self-adaptive parallel manipulator (SAPM) that can automatically adjust the ultrasound (US) probe pose to adapt to various contours of scanned areas, provide approximate constant operating forces/torques, achieve mechanical measurement, and cushion undesired produced forces. A novel parallel adjustment mechanism is proposed to attain automatic pose adjustment with 3 degrees of freedom (DOFs). This mechanism enables the US probe to adapt to different scanned areas and to perform the scanning with approximate constant forces and torques. Besides, we present a mechanical measurement and safety protection method that can be integrated into the SAPM and used as operation status monitoring and early warning during scanning procedures by capturing operating forces and torques. Experiments were carried out to calibrate the measurement and buffer units and evaluate the performance of the SAPM. Experimental results show the ability of the SAPM to provide 3-DoFs motion and operating force/torque measurement and automatically adjust the US probe pose to capture US images of equally good quality compared to a manual sonographer scan. Moreover, it has characteristics similar to soft robots that could significantly improve operation safety, and could be extended to some other engineering or medical applications.
Collaborative Transformers for Grounded Situation Recognition

Grounded situation recognition is the task of predicting the main activity, entities playing certain roles within the activity, and bounding-box groundings of the entities in the given image. To effectively deal with this challenging task, we introduce a novel approach where the two processes for activity classification and entity estimation are interactive and complementary. To implement this idea, we propose Collaborative Glance-Gaze TransFormer (CoFormer) that consists of two modules: Glance transformer for activity classification and Gaze transformer for entity estimation. Glance transformer predicts the main activity with the help of Gaze transformer that analyzes entities and their relations, while Gaze transformer estimates the grounded entities by focusing only on the entities relevant to the activity predicted by Glance transformer. Our CoFormer achieves the state of the art in all evaluation metrics on the SWiG dataset. Training code and model weights are available at https://github.com/jhcho99/CoFormer.
Applying Features Based on Word Embedding Techniques to 1D CNN for Natural Disaster Messages Classification

Messages of natural disasters on social media can be used as an early warning and mitigation of natural disasters. Researchers developed a machine learning-based classification model to identify natural disaster messages automatically. In comparison, the Previous research used shallow learning classification algorithms and feature extraction techniques with vector space representation. This feature extraction technique produces high-dimensional data. This technique eliminates word order information also so that it will lose the meaning of the sentence. Word embedding is a method for transforming the word into vectors with numeric values that capture a word's semantic and syntactic information. We use this method to generate structured data that keep word order, semantic, and syntactic information. The generated data are processed using deep learning, which is 1D CNN. This learning method is generally applied in signal classification, so we must study how to determine the input for 1D CNN to get the best accuracy. We use several techniques to resize the number of words and three-word embedding techniques, i.e. word2vec, Glove and fastText. We find that mean and word2vec are the resized number of word and word embedding techniques that can give the best accuracy to classify natural disaster messages.
LocalBins: Improving Depth Estimation by Learning Local Distributions

We propose a novel architecture for depth estimation from a single image. The architecture itself is based on the popular encoder-decoder architecture that is frequently used as a starting point for all dense regression tasks. We build on AdaBins which estimates a global distribution of depth values for the input image and evolve the architecture in two ways. First, instead of predicting global depth distributions, we predict depth distributions of local neighborhoods at every pixel. Second, instead of predicting depth distributions only towards the end of the decoder, we involve all layers of the decoder. We call this new architecture LocalBins. Our results demonstrate a clear improvement over the state-of-the-art in all metrics on the NYU-Depth V2 dataset. Code and pretrained models will be made publicly available ( https://github.com/shariqfarooq123/LocalBins ).
The spherical aberration correction by using bending and lens splitting: A compression

The spherical and chromatic are the primary aberration type that degrades the performance of optical instruments used in human eye vision correction. This paper demonstrates and compares two methods for spherical aberration correction. One of these methods is lens bending and how can an aspheric surface contributed to minimizing the spherical aberration. This is done by adjusting the conic constant according to the surface type. The second method is lens splitting. In this part, the ZEMAX program is used to illustrate the lens's splitting effect on the results. However, it is found that if a double (two elements) used, spherical aberration would reduce by a factor (4.7). If a triple (three elements) is used, spherical aberration will reduce by a (18.8). For more aberration reduction, compensation for the two methods is used. The results show that the triplet lens made a great reduction in aberration. This triplet is used to improve the human eye vision by using Liou-Brennan model based on zemax program. The final result is that a noticeable improvement in the vision occurs since the aberration is reduced.
Pairwise Contrastive Learning Network for Action Quality Assessment

Considering the complexity of modeling diverse actions of athletes, action quality assessment (AQA) in sports is a challenging task. A common solution is to tackle this problem as a regression task that map the input video to the final score provided by referees. However, it ignores the subtle and critical difference between videos. To address this problem, a new pairwise contrastive learning network (PCLN) is proposed to concern these differences and form an end-to-end AQA model with basic regression network. Specifically, the PCLN encodes video pairs to learn relative scores between videos to improve the performance of basic regression network. Furthermore, a new consistency constraint is defined to guide the training of the proposed AQA model. In the testing phase, only the basic regression network is employed, which makes the proposed method simple but high accuracy. The proposed method is verified on the AQA-7 and MTL-AQA datasets. Several ablation studies are built to verify the effectiveness of each component in the proposed method. The experimental results show that the proposed method achieves the state-of-the-art performance.