Reviewerly/exHarmony huggingface dataset:
On Fuzzy Method of “Deep Knowledge” and “Deep Data” in Project Quantitative Management

At present, the conventional database (DB) and knowledge base (KB) can’t fully meet the needs of decision-making for management problems. Because only the Shallow Data and Shallow Knowledge provided by conventional decision support system (DSS) usually cannot satisfy the requirement of reflecting the deep relationship of various management problems.
Differentially Private Probabilistic Social Choice in the Shuffle Model

Given a profile of ranking lists over a finite set of alternatives, probabilistic social choice seeks to select a probability function over the alternatives on the basis of the pairwise comparison voting data. In this paper, we establish a differentially private formalism for probabilistic social choice in the shuffle model. In the shuffle model, individual voters submit their locally randomized data anonymously through a trusted shuffler which randomly permutes these individual data. Anonymity here plays a double role as a fairness condition and privacy amplification. The crucial step in our construction is to employ spectral clustering to find data-independent cluster centers and then to approximately round each input ranking order to these centers. We proceed to define a local differentially private randomizer plus the shuffler and then implement standard probabilistic social choice protocols such as maximal lottery and random dictatorship. Moreover, we analyze both privacy and utility of the proposed shuffle model and run some experiments to show the effectiveness of our formalism. In the last section, we discuss some related works and future directions.
Mitigating Motion Blur for Robust 3D Baseball Player Pose Modeling for Pitch Analysis

Using videos to analyze pitchers in baseball can play a vital role in strategizing and injury prevention. Computer vision-based pose analysis offers a time-eficient and cost-effective approach. However, the use of accessible broadcast videos, with a 30fps framerate, often results in partial body motion blur during fast actions, limiting the performance of existing pose keypoint estimation models. Previous works have primarily relied on fixed backgrounds, assuming minimal motion differences between frames, or utilized multiview data to address this problem. To this end, we propose a synthetic data augmentation pipeline to enhance the model's capability to deal with the pitcher's blurry actions. In addition, we leverage in-the-wild videos to make our model robust under different real-world conditions and camera positions. By carefully optimizing the augmentation parameters, we observed a notable reduction in the loss by 54.2% and 36.2% on the test dataset for 2D and 3D pose estimation respectively. By applying our approach to existing state-of-the-art pose estimators, we demonstrate an average improvement of 29.2%. The findings highlight the effectiveness of our method in mitigating the challenges posed by motion blur, thereby enhancing the overall quality of pose estimation.
HowToEat: Exploring Human Object Interaction and Eating Action in Eating Scenarios

Recently, the analysis of multimedia of eating and diet has become a new trend in research. Detecting eating activities in videos and images is a basic requirement for further analysis. However, existing human-centric action detection tasks, such as human-object interaction detection and hand-object interaction detection lack the data in eating scenarios and annotations of eating actions. To fill this gap in research, we introduce a new large-scale dataset, HowToEat, which contains 66 days of videos in 12 eating scenarios, and 95k images with automatic annotations of hand-object interactions and eating actions. Based on the dataset, we propose an eating analysis system, which uses a single model to detect hand-object interaction and eating action at the same time.
Image- and Instance-Level Data Augmentation for Occluded Instance Segmentation

Instance segmentation is a fundamental computer vision task with widespread applications. Numerous novel methods have been proposed to address this task. However, limited data and occlusion are common issues that hinder the practical application of instance segmentation. In this paper, we address limited data issue by employing image-level data augmentation. Additionally, to address the occlusion issue, we propose Balanced Occlusion Aware Copy-Paste (BOACP), a method that can not only increase the number of instances in images but also balance occluded instances at the image level. This method can enhance the performance of model on occluded instances. For the model, we utilize the Hybrid Task Cascade (HTC) based on CBSwin-Base and CBFPN. Moreover, we conduct additional experiments to explore the Occlusion Metric (OM). Experimental results demonstrate the effectiveness of our proposed approach, and we achieve the first place in the first phase of DeepSportRadar Instance Segmentation Challenge in ACM MMSports 2023 Workshop.
Rink-Agnostic Hockey Rink Registration

Hockey rink registration is a useful tool for aiding and automating sports analysis. When combined with player tracking, it can provide location information of players on the rink by estimating a homography matrix that can warp broadcast video frames onto an overhead template of the rink, or vice versa. However, most existing techniques require accurate ground truth information, which can take many hours to annotate, and only work on the trained rink types. In this paper, we propose a generalized rink registration pipeline that, once trained, can be applied to both seen and unseen rink types with only an overhead rink template and the video frame as inputs. Our pipeline uses domain adaptation techniques, semi-supervised learning, and synthetic data during training to achieve this ability and overcome the lack of non-NHL training data. The proposed method is evaluated on both NHL (source) and non-NHL (target) rink data and the results demonstrate that our approach can generalize to non-NHL rinks, while maintaining competitive performance on NHL rinks.
Philosophical Consideration of Quantitative Management

In the management discipline, there are two different managerial modes, namely: qualitative (empirical) and quantitative (digital) managements. It is ever increasing recognition that under volatility, uncertainty, complexity and ambiguity (VUCA) era, the empirical mode of management, which is based on the heuristic knowledge of single manager (decision maker), is far from enough to cover such a sophisticated environmental changes, under such situation, the digital mode of management through computer will doubtlessly reveal in the stage of management and play its colorful and important role in the development of managerial science.
Jersey Number Recognition using Keyframe Identification from Low-Resolution Broadcast Videos

Player identification is a crucial component in vision-driven soccer analytics, enabling various downstream tasks such as player assess- ment, in-game analysis, and broadcast production. However, auto- matically detecting jersey numbers from player tracklets in videos presents challenges due to motion blur, low resolution, distortions, and occlusions. Existing methods, utilizing Spatial Transformer Networks, CNNs, and Vision Transformers, have shown success in image data but struggle with real-world video data, where jersey numbers are not visible in most of the frames. Hence, identifying frames that contain the jersey number is a key sub-problem to tackle. To address these issues, we propose a robust keyframe identification module that extracts frames containing essential high-level infor- mation about the jersey number. A spatio-temporal network is then employed to model spatial and temporal context and predict the probabilities of jersey numbers in the video. Additionally, we adopt a multi-task loss function to predict the probability distribution of each digit separately. Extensive evaluations on the SoccerNet dataset demonstrate that incorporating our proposed keyframe identification module results in a significant 37.81% and 37.70% increase in the accuracies of 2 different test sets with domain gaps. These results highlight the effectiveness and importance of our approach in tackling the challenges of automatic jersey number detection in sports videos.
Generating Factually Consistent Sport Highlights Narrations

Sports highlights are an important form of media for fans worldwide, as they provide short videos that capture key moments from games, often accompanied by the original commentaries of the game's announcers. However, traditional forms of presenting sports highlights have limitations in conveying the complexity and nuance of the game. In recent years, the use of Large Language Models (LLMs) for natural language generation has emerged and is a promising approach for generating narratives that can provide a more compelling and accessible viewing experience. In this paper, we propose an end-to-end solution to enhance the experience of watching sports highlights by automatically generating factually consistent narrations using LLMs and crowd noise extraction. Our solution involves several steps, including extracting the source of information from the live broadcast using a transcription model, prompt engineering, and comparing out-of-the-box models for consistency evaluation. We also propose a new dataset annotated on generated narratives from 143 Premier League plays and fine-tune a Natural Language Inference (NLI) model on it, achieving 92% precision. Furthermore, we extract crowd noise from the original video to create a more immersive and realistic viewing experience for sports fans by adapting speech enhancement SOTA models on a brand new dataset created from 155 Ligue 1 games.
ASTRA: An Action Spotting TRAnsformer for Soccer Videos

In this paper, we introduce ASTRA, a Transformer-based model designed for the task of Action Spotting in soccer matches. ASTRA addresses several challenges inherent in the task and dataset, including the requirement for precise action localization, the presence of a long-tail data distribution, non-visibility in certain actions, and inherent label noise. To do so, ASTRA incorporates (a) a Transformer encoder-decoder architecture to achieve the desired output temporal resolution and to produce precise predictions, (b) a balanced mixup strategy to handle the long-tail distribution of the data, (c) an uncertainty-aware displacement head to capture the label variability, and (d) input audio signal to enhance detection of non-visible actions. Results demonstrate the effectiveness of ASTRA, achieving a tight Average-mAP of 66.82 on the test set. Moreover, in the SoccerNet 2023 Action Spotting challenge, we secure the 3rd position with an Average-mAP of 70.21 on the challenge set.
Memory-Efficient High-Accuracy Food Intake Activity Recognition with 3D mmWave Radars

Non-invasive and privacy-preserving recognition of food intake activities has applications in diet management, telecare, and smarthome data monetization. In this paper, we tackle the challenging problem of recognizing food intake activity using sparse point clouds captured by a single mmWave radar, which is non-invasive and privacy-preserving. We propose: (i) an enhanced Skeletal Pose Estimator (SPE) capable of generating more precise skeletons for food intake activity recognition, outperforming the state-of-the-art MARS with an average reduction of 45.16% in the error of the estimated joints, (ii) a Dynamic Point Cloud Recognizer (DPR), which adapts SPE to directly process dynamic point clouds for food intake activity recognition, outperforming the state-of-the-art FIA with a 4.10% enhancement in classification accuracy and a 78.29% reduction in memory consumption, and (iii) a Lightweight Dynamic Point Cloud Recognizer (LDPR), which eliminates the need for CNNs, hence reducing model complexity and outperforming DPR by 0.15% in classification accuracy and a 42.80% reduction in memory consumption. In addition to food intake activity recognition, the skeletons generated by our SPE can also be used to recognize other fine- and coarse-grained activities for applications like rehabilitation, driver monitoring, and fitness assistance.
A Sparse Attention Pipeline for DeepSportRadar Basketball Player Instance Segmentation Challenge

The ACM MMSports2023 DeepSportRadar Basketball Player Instance Segmentation Challenge was focused on addressing the issue of occlusion. The dataset's primary characteristics include vast background areas, a high degree of occlusion between athletes, and limited data volume. To tackle the challenge of severe occlusion among athletes, we developed a sparse attention pipeline. Firstly, we introduced the InternImage backbone network and Sparse Multi-Head Self-Attention Module with a sparse Transformer. This allowed the segmentation pipeline to prioritize critical regions, effectively dealing with occlusion and class imbalance issues. Secondly, we adopted a multi-scale processing strategy using the Simple Dual Refinement Feature Pyramid Networks (SDRFPN) to fuse features of different scales. This approach improved the ability to handle athletes' features with different scales and fine details. Lastly, during the training phase, we employed random flipping data augmentation, which assisted the segmentation pipeline in recognizing targets from various angles and orientations. On the DeepSportRadar Basketball Player Instance Segmentation Challenge dataset, the pipeline achieved an impressive Occlusion Metric (OM) score of 0.316.
Hybrid graph-clothoid based path planning for a fixed wing aircraft

Abstract. Planning of safe and efficient trajectories is a critical task in the operation of unmanned aerial vehicles (UAVs), especially in urban or complex environments. With the increasing use of UAVs for various applications, such as surveillance, delivery, and inspection, it is becoming more important to automatically generate collision-free paths that also consider aircraft dynamics. This paper proposes an algorithmic approach based on the Rapidly exploring random tree (RRT) algorithm combined with a clothoid-based smoothing procedure to account for aircraft performance.
Hardware-in-the-loop validation of a sense and avoid system leveraging data fusion between radar and optical sensors for a mini UAV

Abstract. The present work illustrates the results obtained at the conclusion of the three-year project TERSA (Tecnologie Elettriche e Radar per Sapr Autonomi), involving the aerospace section of the Dept. of Civil and Industrial Engineering (DICI) of the University of Pisa and its industrial partners. The project aimed at the design and development of a fully autonomous Sense and Avoid (SAA) prototype system, based on data fusion between optical and radar sensors data, for a tactical lightweight surveillance UAV (MTOW&lt;25Kg). The problem of non-cooperative collision avoidance is well known in literature and is currently a central theme of investigation within the aeronautical industry, considering the growing UAV traffic and the consequent need to employ autonomous self-separation technologies in the market. Several past works have investigated the most varied solutions for the Sense problem utilizing optical, acoustic, electro-magnetic signals or a combination of the previous. Likewise, the Avoidance problem has been successfully tackled in literature by means of a wide variety of different approaches ranging from rule-based methods, strategies based on game theory, force field methods, optimization frameworks leveraging genetic algorithms and nonlinear programming techniques and geometric methods. Yet, to the knowledge of the authors, no previous work found in literature has successfully demonstrated and validated the real-time simultaneous interaction of both sense and avoid functionalities within a highly integrated simulation environment. The present work describes the implementation of a complex, nonlinear, simulation environment conceived in order to perform real-time, Hardware-in-the-Loop (HIL), testing of the effective cooperation between sense and the avoid algorithms constituting the core of the SAA system developed within the context of the project. The system effectiveness has been validated by means of complex dynamic simulations, comprising an accurate, fully nonlinear, flight mechanic model of the aircraft, a graphic rendering engine of the scene, proper video capture and transmission pipelines, computer vision algorithms and collision avoidance logics running on the target hardware (Nvidia Jetson Nano) and tailored noise resilient data fusion algorithms. Results show the effectiveness of the system in detecting impending collisions and performing last-resort resolution manoeuvres with high computational efficiency and update frequencies compatible with real world applications in Unmanned Aircraft Systems (UAS).
The Effect of Digital Learning Media Based on Wordwall Games on Students’ Learning Outcomes and Learning Motivation at SDN 17 Prabumulih

Wordwall game digital media is an interesting application, which increases motivation and learning outcomes with an active learning atmosphere in the classroom and learning while playing. The research uses the experimental method with the Posttest-Only Control Design. The research population indicates 334 people with purposive sampling, which consists of 28 students of class IIIA as the experimental class and 28 students of class IIIB as the control class. Data collection techniques used are tests and questionnaires. Based on the research results, it is concluded that there is an effect of Wordwall game-based digital learning media on students’ learning outcomes of fractional material with a sig of 0.000. There are differences in learning outcomes based on students’ learning motivation of fractional material with a sig of 0.000. There is no interaction of Wordwall game-based digital learning media on students’ learning motivation and students’ learning outcomes of fractional material with a sig of 0.788 at SDN 17 Prabumulih.
Fast Adversarial Training With Adaptive Step Size

While adversarial training and its variants have shown to be the most effective algorithms to defend against adversarial attacks, their extremely slow training process makes it hard to scale to large datasets like ImageNet. The key idea of recent works to accelerate adversarial training is to substitute multi-step attacks (e.g., PGD) with single-step attacks (e.g., FGSM). However, these single-step methods suffer from catastrophic overfitting, where the accuracy against PGD attack suddenly drops to nearly 0% during training, and the network totally loses its robustness. In this work, we study the phenomenon from the perspective of training instances. We show that catastrophic overfitting is instance-dependent, and fitting instances with larger input gradient norm is more likely to cause catastrophic overfitting. Based on our findings, we propose a simple but effective method, Adversarial Training with Adaptive Step size (ATAS). ATAS learns an instance-wise adaptive step size that is inversely proportional to its gradient norm. Our theoretical analysis shows that ATAS converges faster than the commonly adopted non-adaptive counterparts. Empirically, ATAS consistently mitigates catastrophic overfitting and achieves higher robust accuracy on CIFAR10, CIFAR100, and ImageNet when evaluated on various adversarial budgets. Our code is released at https://github.com/HuangZhiChao95/ATAS.
Attention and feature transfer based knowledge distillation

Existing knowledge distillation (KD) methods are mainly based on features, logic, or attention, where features and logic represent the results of reasoning at different stages of a convolutional neural network, and attention maps symbolize the reasoning process. Because of the continuity of the two in time, transferring only one of them to the student network will lead to unsatisfactory results. We study the knowledge transfer between the teacher-student network to different degrees, revealing the importance of simultaneously transferring knowledge related to the reasoning process and reasoning results to the student network, providing a new perspective for the study of KD. On this basis, we proposed the knowledge distillation method based on attention and feature transfer (AFT-KD). First, we use transformation structures to transform intermediate features into attentional and feature block (AFB) that contain both inference process information and inference outcome information, and force students to learn the knowledge in AFBs. To save computation in the learning process, we use block operations to align the teacher-student network. In addition, in order to balance the attenuation ratio between different losses, we design an adaptive loss function based on the loss optimization rate. Experiments have shown that AFT-KD achieves state-of-the-art performance in multiple benchmark tests.
DO-SA&amp;R: Distant Object Augmented Set Abstraction and Regression for Point-Based 3D Object Detection

Point-based 3D detection approaches usually suffer from the severe point sampling imbalance problem between foreground and background. We observe that prior works have attempted to alleviate this imbalance by emphasizing foreground sampling. However, even adequate foreground sampling may be extremely unbalanced between nearby and distant objects, yielding unsatisfactory performance in detecting distant objects. To tackle this issue, this paper first proposes a novel method named Distant Object Augmented Set Abstraction and Regression (DO-SA&R) to enhance distant object detection, which is vital for the timely response of decision-making systems like autonomous driving. Technically, our approach first designs DO-SA with novel distant object augmented farthest point sampling (DO-FPS) to emphasize sampling on distant objects by leveraging both object-dependent and depth-dependent information. Then, we propose distant object augmented regression to reweight all the instance boxes for strengthening regression training on distant objects. In practice, the proposed DO-SA&R can be easily embedded into the existing modules, yielding consistent performance improvements, especially on detecting distant objects. Extensive experiments are conducted on the popular KITTI, nuScenes and Waymo datasets, and DO-SA&R demonstrates superior performance, especially for distant object detection. Our code is available at https://github.com/mikasa3lili/DO-SAR.
A Unified Interactive Model Evaluation for Classification, Object Detection, and Instance Segmentation in Computer Vision

Existing model evaluation tools mainly focus on evaluating classification models, leaving a gap in evaluating more complex models, such as object detection. In this paper, we develop an open-source visual analysis tool, Uni-Evaluator, to support a unified model evaluation for classification, object detection, and instance segmentation in computer vision. The key idea behind our method is to formulate both discrete and continuous predictions in different tasks as unified probability distributions. Based on these distributions, we develop 1) a matrix-based visualization to provide an overview of model performance; 2) a table visualization to identify the problematic data subsets where the model performs poorly; 3) a grid visualization to display the samples of interest. These visualizations work together to facilitate the model evaluation from a global overview to individual samples. Two case studies demonstrate the effectiveness of Uni-Evaluator in evaluating model performance and making informed improvements