Reviewerly/exHarmony huggingface dataset:
Time Series Prediction Based on T-S Fuzzy Deep Network

Time series are widely existed in various fields such as industry, agriculture and finance, and time series prediction has been a classical problem for research. In order to improve prediction accuracy and model interpretability, this paper proposes a time series prediction model based on T-S fuzzy deep network. The fuzzy deep network model is based on the principle of stack generalization and is composed of several simple fuzzy model in series by referring to the layer connection structure of neural network. Each fuzzy subsystem is, each model contains multiple fuzzy if-then rules, and each fuzzy rule describes a subsystem. The proposed T-S fuzzy model compares five prediction models such as support vector machine regression, random forest regression, ridge regression, linear regression and T-S regression using open data sets. The results show that the prediction performance based on fuzzy deep network model is better in multiple data sets and has stronger prediction stability.
AREA: Adaptive Reweighting via Effective Area for Long-Tailed Classification

Large-scale data from the real-world usually follow a long-tailed distribution (i.e., a few majority classes occupy plentiful training data, while most minority classes have few samples), making the hyperplanes heavily skewed to the minority classes. Traditionally, reweighting is adopted to make the hyperplanes fairly split the feature space, where the weights are designed according to the number of samples. However, we find that the number of samples in a class can not accurately measure the size of its spanned space, especially for the majority class, where the size of its spanned space is usually larger than the samples’ number because of the high diversity. Therefore, weights designed based on the samples’ number will still compress the space of minority classes. In this paper, we reconsider reweighting from a totally new perspective of analyzing the spanned space of each class. We argue that, besides statistical numbers, relations between samples are also significant for sufficiently depicting the spanned space. Consequently, we estimate the size of the spanned space for each category, namely effective area, by detailedly analyzing its samples’ distribution. By treating samples of a class as identically distributed random variables and analyzing their correlations, a simple and non-parametric formula is derived to estimate the effective area. Then, the weight simply calculated inversely proportional to the effective area of each class is adopted to achieve fairer training. Note that our weights are more flexible as they can be adaptively adjusted along with the optimizing features during training. Experiments on four long-tailed datasets show that the proposed weights outperform the state-of-the-art reweighting methods. Moreover, our method can also achieve better results on statistically balanced CIFAR-10/100. Code is available at https://github.com/xiaohua-chen/AREA.
Adaptive Testing of Computer Vision Models

Vision models often fail systematically on groups of data that share common semantic characteristics (e.g., rare objects or unusual scenes), but identifying these failure modes is a challenge. We introduce AdaVision, an interactive process for testing vision models which helps users identify and fix coherent failure modes. Given a natural language description of a coherent group, AdaVision retrieves relevant images from LAION-5B with CLIP. The user then labels a small amount of data for model correctness, which is used in successive retrieval rounds to hill-climb towards high-error regions, refining the group definition. Once a group is saturated, AdaVision uses GPT-3 to suggest new group descriptions for the user to explore. We demonstrate the usefulness and generality of AdaVision in user studies, where users find major bugs in state-of-the-art classification, object detection, and image captioning models. These user-discovered groups have failure rates 2-3x higher than those surfaced by automatic error clustering methods. Finally, finetuning on examples found with AdaVision fixes the discovered bugs when evaluated on unseen examples, without degrading in-distribution accuracy, and while also improving performance on out-of-distribution datasets.
Not Every Side Is Equal: Localization Uncertainty Estimation for Semi-Supervised 3D Object Detection

Semi-supervised 3D object detection from point cloud aims to train a detector with a small number of labeled data and a large number of unlabeled data. The core of existing methods lies in how to select high-quality pseudo-labels using the designed quality evaluation criterion. However, these methods treat each pseudo bounding box as a whole and assign equal importance to each side during training, which is detrimental to model performance due to many sides having poor localization quality. Besides, existing methods filter out a large number of low-quality pseudo-labels, which also contain some correct regression values that can help with model training. To address the above issues, we propose a side-aware framework for semi-supervised 3D object detection consisting of three key designs: a 3D bounding box parameterization method, an uncertainty estimation module, and a pseudo-label selection strategy. These modules work together to explicitly estimate the localization quality of each side and assign different levels of importance during the training phase. Extensive experiment results demonstrate that the proposed method can consistently outperform baseline models under different scenes and evaluation criteria. Moreover, our method achieves state-of-the-art performance on three datasets with different labeled ratios.
Cross-view Semantic Alignment for Livestreaming Product Recognition

Live commerce is the act of selling products online through live streaming. The customer’s diverse demands for online products introduce more challenges to Livestreaming Product Recognition. Previous works have primarily focused on fashion clothing data or utilize single-modal input, which does not reflect the real-world scenario where multimodal data from various categories are present. In this paper, we present LPR4M, a large-scale multimodal dataset that covers 34 categories, comprises 3 modalities (image, video, and text), and is 50× larger than the largest publicly available dataset. LPR4M contains diverse videos and noise modality pairs while exhibiting a long-tailed distribution, resembling real-world problems. Moreover, a cRoss-vIew semantiC alignmEnt (RICE) model is proposed to learn discriminative instance features from the image and video views of the products. This is achieved through instance-level contrastive learning and cross-view patch-level feature propagation. A novel Patch Feature Reconstruction loss is proposed to penalize the semantic misalignment between cross-view patches. Extensive experiments demonstrate the effectiveness of RICE and provide insights into the importance of dataset diversity and expressivity. The dataset and code are available at https://github.com/adxcreative/RICE.
Fine-grained Affective Processing Capabilities Emerging from Large Language Models

Large language models, in particular generative pre-trained transformers (GPTs), show impressive results on a wide variety of language-related tasks. In this paper, we explore ChatGPT’s zero-shot ability to perform affective computing tasks using prompting alone. We show that ChatGPT a) performs meaningful sentiment analysis in the Valence, Arousal and Dominance dimensions, b) has meaningful emotion representations in terms of emotion categories and these affective dimensions, and c) can perform basic appraisal-based emotion elicitation of situations based on a prompt-based computational implementation of the OCC appraisal model. These findings are highly relevant: First, they show that the ability to solve complex affect processing tasks emerges from language-based token prediction trained on extensive data sets. Second, they show the potential of large language models for simulating, processing and analyzing human emotions, which has important implications for various applications such as sentiment analysis, socially interactive agents, and social robotics.
Anchor-Intermediate Detector: Decoupling and Coupling Bounding Boxes for Accurate Object Detection

Anchor-based detectors have been continuously developed for object detection. However, the individual anchor box makes it difficult to predict the boundary’s offset accurately. Instead of taking each bounding box as a closed individual, we consider using multiple boxes together to get prediction boxes. To this end, this paper proposes the Box Decouple-Couple(BDC) strategy in the inference, which no longer discards the overlapping boxes, but decouples the corner points of these boxes. Then, according to each corner’s score, we couple the corner points to select the most accurate corner pairs. To meet the BDC strategy, a simple but novel model is designed named the Anchor-Intermediate Detector(AID), which contains two head networks, i.e., an anchor-based head and an anchor-free Corner-aware head. The corner-aware head is able to score the corners of each bounding box to facilitate the coupling between corner points. Extensive experiments on MS COCO show that the proposed anchor-intermediate detector respectively outperforms their baseline RetinaNet and GFL method by ∼2.4 and ∼1.2 AP on the MS COCO test-dev dataset without any bells and whistles.
End-to-End Diffusion Latent Optimization Improves Classifier Guidance

Classifier guidance—using the gradients of an image classifier to steer the generations of a diffusion model—has the potential to dramatically expand the creative control over image generation and editing. However, currently classifier guidance requires either training new noise-aware models to obtain accurate gradients or using a one-step denoising approximation of the final generation, which leads to misaligned gradients and sub-optimal control. We highlight this approximation’s shortcomings and propose a novel guidance method: Direct Optimization of Diffusion Latents (DOODL), which enables plug-and-play guidance by optimizing diffusion latents w.r.t. the gradients of a pre-trained classifier on the true generated pixels, using an invertible diffusion process to achieve memory-efficient backpropagation. Showcasing the potential of more precise guidance, DOODL outperforms one-step classifier guidance on computational and human evaluation metrics across different forms of guidance: using CLIP guidance to improve generations of complex prompts from DrawBench, using fine-grained visual classifiers to expand the vocabulary of Stable Diffusion, enabling image-conditioned generation with a CLIP visual encoder, and improving image aesthetics using an aesthetic scoring network.
Kick Back &amp; Relax: Learning to Reconstruct the World by Watching SlowTV

Self-supervised monocular depth estimation (SS-MDE) has the potential to scale to vast quantities of data. Unfortunately, existing approaches limit themselves to the automotive domain, resulting in models incapable of generalizing to complex environments such as natural or indoor settings.To address this, we propose a large-scale SlowTV dataset curated from YouTube, containing an order of magnitude more data than existing automotive datasets. SlowTV contains 1.7M images from a rich diversity of environments, such as worldwide seasonal hiking, scenic driving and scuba diving. Using this dataset, we train an SS-MDE model that provides zero-shot generalization to a large collection of indoor/outdoor datasets. The resulting model outperforms all existing SSL approaches and closes the gap on supervised SoTA, despite using a more efficient architecture.We additionally introduce a collection of best-practices to further maximize performance and zero-shot generalization. This includes 1) aspect ratio augmentation, 2) camera intrinsic estimation, 3) support frame randomization and 4) flexible motion estimation. Code is available at https://github.com/jspenmar/slowtv_monodepth.
A New Task for Predicting Emotions and Dialogue Strategies in Task-Oriented Dialogue

With a focus on task-oriented conversational agents, we introduce a new approach to make the generated response more relevant to the emotional context of the interlocutor. This approach aims to predict the labels of the agent’s next speaker turn, in order to condition the generated response and ensure its consistency to the user’s socio-emotional context. First, we propose a new formulation of this prediction task, based on the joint prediction of dialogue strategies and emotional labels associated with the next speaker turn. To handle this new task, we describe a new annotation protocol for task-oriented dialogue systems, that we implement on real customer-relationship interactions provided by a company. Lastly, we conduct an experiment using two approaches: a classification and a generation model. We evaluate them on the new prediction task using the annotated data, before discussing the results. As expected, we see that our classifier tends to predict safe, similar labels where the generator has a more diverse output in spite of its lower performance on traditional metrics.
StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized Tokenizer of a Large-Scale Generative Model

Despite the progress made in the style transfer task, most previous work focus on transferring only relatively simple features like color or texture, while missing more abstract concepts such as overall art expression or painter-specific traits. However, these abstract semantics can be captured by models like DALL-E or CLIP, which have been trained using huge datasets of images and textual documents. In this paper, we propose StylerDALLE, a style transfer method that exploits both of these models and uses natural language to describe abstract art styles. Specifically, we formulate the language-guided style transfer task as a non-autoregressive token sequence translation, i.e., from input content image to output stylized image, in the discrete latent space of a large-scale pretrained vector-quantized tokenizer, e.g., the discrete variational auto-encoder (dVAE) of DALL-E. To incorporate style information, we propose a Reinforcement Learning strategy with CLIP-based language supervision that ensures stylization and content preservation simultaneously. Experimental results demonstrate the superiority of our method, which can effectively transfer art styles using language instructions at different granularities. Code is available at https://github.com/zipengxuc/StylerDALLE.
Minimum Latency Deep Online Video Stabilization

We present a novel camera path optimization framework for the task of online video stabilization. Typically, a stabilization pipeline consists of three steps: motion estimating, path smoothing, and novel view rendering. Most previous methods concentrate on motion estimation, proposing various global or local motion models. In contrast, path optimization receives relatively less attention, especially in the important online setting, where no future frames are available. In this work, we adopt recent off-the-shelf high-quality deep motion models for motion estimation to recover the camera trajectory and focus on the latter two steps. Our network takes a short 2D camera path in a sliding window as input and outputs the stabilizing warp field of the last frame in the window, which warps the coming frame to its stabilized position. A hybrid loss is well-defined to constrain the spatial and temporal consistency. In addition, we build a motion dataset that contains stable and unstable motion pairs for the training. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art online methods both qualitatively and quantitatively and achieves comparable performance to offline methods. Our code and dataset are available at https://github.com/liuzhen03/NNDVS.
Single Object Tracking Model Based on Shot Boundary Detection for Target Re-identification in Classroom Videos

Objects’ mobile behaviors in classroom videos are of great significance to education research. However, the frequent moving in and out of objects, namely the object out-of-view, are to blame for the unsatisfactory tracking results in classroom videos. To address this issue, we proposed a single object tracking model for target re-identification called TeaTrack. Initially, we introduced the Hopcroft Karp algorithm to NeighborTrack to determine if the target is out of the camera. Then, TeaTrack will detect the shot boundary objects through YOLOv5-DeepSORT optimized by alpha-IoU. Eventually, the target will be re-identified by a retrospective recognition module. The experimental results demonstrate that our model outperforms most state-of-the-art methods and can achieve efficient target re-identification.
A survey on Machine Learning for IoT Trust Management

In this paper, we offer an overview of research on machine learning for managing trust in IoT. Our focus is on providing background information on trust management and machine learning, exploring various ML methods used for trust management in IoT, trust expression, and performance evaluation techniques. We also highlight the challenges involved in deploying ML-based trust management solutions in IoT environments.
Machine Learning of k-Anonymity Data by using Feature Importance and Margin Preservation

Quality of data is the key for machine learning. In the case of sensitive data, de-identification must be adopted to guarantee data privacy. Supervisory organizations have enacted regulations, e.g., EU GDPR, to govern these behaviors. It makes the reliability of machine learning become a critical challenge as de-identification must lead to the information loss. Previous algorithms (such as k anonymity) only focused on optimizing metrics of anonymity. They do not consider requirements of machine learning and may degrade performance from conducted results. This paper adjusts k-anonymity algorithms by feature importance (FI) and margin preservation (MP) of data classes to resolve this drawback. FI was explored to have a high-degree correlation to machine learning performance. MP not only anonymizes data but also tries to preserve original class margins of data. FI and MP schemes are tested on 4 different k-anonymity algorithms with different values of k. Four different datasets are applied to evaluate the performance for three machine learning models. These comprehensive experiments demonstrate that our schemes can improve machine learning of k-anonymity data. Comparing to the original k-anonymity algorithms, the best improvement of our improved k-anonymity algorithms with FI and MP can lead to over 10% and 17% in f1score, respectively.
Towards Smart Interaction: Hand Gesture Recognition Using Machine Learning in IoT Scenarios

Hand gesture recognition is becoming popular in computer vision applications like sign language interpretation, IoT robotic controls, and home automation. In this paper, we propose a gesture recognition system using two machine learning models, VGG16 and MobileNet, and compare their performance in detecting hand gestures. Our dataset consists of over 20,000 images of ten different hand gestures that allow us to implement transfer learning and adapt the pre-trained models by customizing the fully connected layers. Our findings indicate that the MobileNet model outperforms VGG16 model in recognizing hand gestures. The paper focuses on creating a baseline that can combine different models to produce a more efficient result.
A Memcomputing Approach to Prime Factorization

We report preliminary results on using the MEMCPU <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">™</sup> Platform to compute the prime factorization of large biprimes. The approach described here uses a congruence model that returns smooth congruences to address the bottleneck of standard sieve methods. The model has size-dependent structure, and the MEMCPU Platform requires structure-dependent tuning for optimal performance. Therefore, we tuned the platform on sample problems up to a given size according to available resources. Then we generated RSA-like benchmark biprimes to perform rigorous scaling analysis. The MEMCPU timings over the tuned range followed low degree polynomials in the number of bits, markedly different from other tested methods including general number field sieve. MEMCPU’s model was scaled up to 300-bit factorization problems while following a 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nd</sup> degree polynomial fit. We also discuss the approach to tuning the MEMCPU Platform for problems beyond the reach of today’s most advanced methods. Finally, basic analysis of the acceleration expected from an ASIC implementation is provided and suggests the possibility of real time factorization of large biprimes.
MiniROAD: Minimal RNN Framework for Online Action Detection

Online Action Detection (OAD) is the task of identifying actions in streaming videos without access to future frames. Much effort has been devoted to effectively capturing long-range dependencies, with transformers receiving the spotlight for their ability to capture long-range temporal structures. In contrast, RNNs have received less attention lately, due to their lower performance compared to recent methods that utilize transformers. In this paper, we investigate the underlying reasons for the inferior performance of RNNs compared to transformer-based algorithms. Our findings indicate that the discrepancy between training and inference is the primary hindrance to the effective training of RNNs. To address this, we propose applying non-uniform weights to the loss computed at each time step, which allows the RNN model to learn from the predictions made in an environment that better resembles the inference stage. Extensive experiments on three benchmark datasets, THUMOS, TVSeries, and FineAction demonstrate that a minimal RNN-based model trained with the proposed methodology performs equally or better than the existing best methods with a significant increase in efficiency. The code is available at https://github.com/jbistanbul/MiniROAD.
Prototype-based Dataset Comparison

Dataset summarisation is a fruitful approach to dataset inspection. However, when applied to a single dataset the discovery of visual concepts is restricted to those most prominent. We argue that a comparative approach can expand upon this paradigm to enable richer forms of dataset inspection that go beyond the most prominent concepts.To enable dataset comparison we present a module that learns concept-level prototypes across datasets. We leverage self-supervised learning to discover these prototypes without supervision, and we demonstrate the benefits of our approach in two case-studies. Our findings show that dataset comparison extends dataset inspection and we hope to encourage more works in this direction. Code and usage instructions available at https://github.com/Nanne/ProtoSim