pwc-archive/papers-with-abstracts huggingface dataset:
A new solution and concrete implementation steps for Artificial General Intelligence
Authors: Yongcong Chen, Ting Zeng, Xingyue Chen
Date: 2023-08-12 00:00:00
Tasks: Decision Making, Large Language Model, reinforcement-learning, Reinforcement Learning

In this paper, we propose a new approach to building a artificial general intelligence with self awareness, which includes: (1) a new method to implement attention mechanisms; (2) a way to give machines self-demands; (3) how to form a value evaluation system compatible with the network; (4) a way to create the world models; (5) how to realize a top-down, hierarchical thinking decision-making chain; (6) a way to achieve general decision-making and response capabilities; (7) a way for a machine to directly obtain human experience through language. In the paper, we first analyze some of the shortcomings of current LLMs (Large Language Model) and propose ideas for improvement. Then we analyze why our scheme can solve the above problems and provide detailed steps for implementing our scheme. In chapter 4, we have presented a step-by-step mplementation roadmap. And in chapter 5, we have presented a specific implementation demonstration. In chapter 6, we analyze the advantages and disadvantages of our scheme and propose further research directions. In this article, we have put forward how to create genuine artificial general intelligence step by step. It can handle data of all modalities in a unified form and can directly understand the experience that humans already possess through language, thus avoiding the problem that reinforcement learning is required for every decision-making process.
Interpretable Ensemble Learning for Materials Property Prediction with Classical Interatomic Potentials: Carbon as an Example
Authors: Xinyu Jiang, Haofan Sun, Kamal Choudhary, Houlong Zhuang, Qiong Nian
Date: 2023-07-24 00:00:00
Tasks: Ensemble Learning, Formation Energy, Property Prediction, regression

Machine learning (ML) is widely used to explore crystal materials and predict their properties. However, the training is time-consuming for deep-learning models, and the regression process is a black box that is hard to interpret. Also, the preprocess to transfer a crystal structure into the input of ML, called descriptor, needs to be designed carefully. To efficiently predict important properties of materials, we propose an approach based on ensemble learning consisting of regression trees to predict formation energy and elastic constants based on small-size datasets of carbon allotropes as an example. Without using any descriptor, the inputs are the properties calculated by molecular dynamics with 9 different classical interatomic potentials. Overall, the results from ensemble learning are more accurate than those from classical interatomic potentials, and ensemble learning can capture the relatively accurate properties from the 9 classical potentials as criteria for predicting the final properties.
MTN: Forensic Analysis of MP4 Video Files Using Graph Neural Networks
Authors: Z Xiang, AKS Yadav, P Bestagini, S Tubaro, EJ Delp
Date: 2023-06-17 00:00:00
Proceeding: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2023 6
Tasks: Data Augmentation, Self-Supervised Learning, Video Forensics

MP4 video files are stored using a tree data structure. These trees contain rich information that can be used for forensic analysis. In this paper, we propose MP4 Tree Network (MTN), an approach based on an end-to-end Graph Neural Networks (GNNs) that is used for forensic analysis of MP4 trees. MTN does not use any video pixel data. MTN is trained using Self-Supervised Learning (SSL), which generates semantic-preserving node embeddings for the nodes in an MP4 tree. We also propose a data augmentation technique for MP4 trees, which helps train MTN in data-scarce scenarios. MTN achieves good performance across 3 video forensics tasks on the EVA-7K dataset. We show that MTN can gain more comprehensive understanding about the MP4 trees and is more robust to potential attacks compared to existing methods.
Less is more: a new machine-learning methodology for spatiotemporal systems
Authors: Sihan Feng, Kang Wang, Fuming Wang, Yong Zhang and Hong Zhao
Date: 2022-03-22 00:00:00
Proceeding: Communications in Theoretical Physics Commnu. 2022 3
Tasks: Time Series, Weather Forecasting

Machine learning provides a way to use only portions of the variables of a spatiotemporal system to predict its subsequent evolution and consequently avoids the curse of dimensionality. The learning machines employed for this purpose, in essence, are time-delayed recurrent neural networks with multiple input neurons and multiple output neurons. We show in this paper that such kinds of learning machines have a poor generalization ability to variables that have not been trained with. We then present a one-dimensional time-delayed recurrent neural network for the same aim of model-free prediction. It can be trained on different spatial variables in the training stage but initiated by the time series of only one spatial variable, and consequently possess an excellent generalization ability to new variables that have not been trained on. This network presents a new methodology to achieve fine-grained predictions from a learning machine trained on coarse-grained data, and thus provides a new strategy for certain applications such as weather forecasting. Numerical verifications are performed on the Kuramoto coupled oscillators and the Barrio- Varea-Aragon-Maini model.
TOPIC: A Parallel Association Paradigm for Multi-Object Tracking under Complex Motions and Diverse Scenes
Authors: Xiaoyan Cao, Yiyao Zheng, Yao Yao, Huapeng Qin, Xiaoyu Cao, Shihui Guo
Date: 2023-08-22 00:00:00
Tasks: Multi-Object Tracking, Object Tracking

Video data and algorithms have been driving advances in multi-object tracking (MOT). While existing MOT datasets focus on occlusion and appearance similarity, complex motion patterns are widespread yet overlooked. To address this issue, we introduce a new dataset called BEE24 to highlight complex motions. Identity association algorithms have long been the focus of MOT research. Existing trackers can be categorized into two association paradigms: single-feature paradigm (based on either motion or appearance feature) and serial paradigm (one feature serves as secondary while the other is primary). However, these paradigms are incapable of fully utilizing different features. In this paper, we propose a parallel paradigm and present the Two rOund Parallel matchIng meChanism (TOPIC) to implement it. The TOPIC leverages both motion and appearance features and can adaptively select the preferable one as the assignment metric based on motion level. Moreover, we provide an Attention-based Appearance Reconstruction Module (AARM) to reconstruct appearance feature embeddings, thus enhancing the representation of appearance features. Comprehensive experiments show that our approach achieves state-of-the-art performance on four public datasets and BEE24. Moreover, BEE24 challenges existing trackers to track multiple similar-appearing small objects with complex motions over long periods, which is critical in real-world applications such as beekeeping and drone swarm surveillance. Notably, our proposed parallel paradigm surpasses the performance of existing association paradigms by a large margin, e.g., reducing false negatives by 6% to 81% compared to the single-feature association paradigm. The introduced dataset and association paradigm in this work offer a fresh perspective for advancing the MOT field. The source code and dataset are available at https://github.com/holmescao/TOPICTrack.
Federated Learning on Patient Data for Privacy-Protecting Polycystic Ovary Syndrome Treatment
Authors: Lucia Morris, Tori Qiu, Nikhil Raghuraman
Date: 2023-08-22 00:00:00
Tasks: Federated Learning

The field of women's endocrinology has trailed behind data-driven medical solutions, largely due to concerns over the privacy of patient data. Valuable datapoints about hormone levels or menstrual cycling could expose patients who suffer from comorbidities or terminate a pregnancy, violating their privacy. We explore the application of Federated Learning (FL) to predict the optimal drug for patients with polycystic ovary syndrome (PCOS). PCOS is a serious hormonal disorder impacting millions of women worldwide, yet it's poorly understood and its research is stunted by a lack of patient data. We demonstrate that a variety of FL approaches succeed on a synthetic PCOS patient dataset. Our proposed FL models are a tool to access massive quantities of diverse data and identify the most effective treatment option while providing PCOS patients with privacy guarantees.
Decoupled Contrastive Multi-View Clustering with High-Order Random Walks
Authors: Yiding Lu, Yijie Lin, Mouxing Yang, Dezhong Peng, Peng Hu, Xi Peng
Date: 2023-08-22 00:00:00
Tasks: Clustering, Contrastive Learning

In recent, some robust contrastive multi-view clustering (MvC) methods have been proposed, which construct data pairs from neighborhoods to alleviate the false negative issue, i.e., some intra-cluster samples are wrongly treated as negative pairs. Although promising performance has been achieved by these methods, the false negative issue is still far from addressed and the false positive issue emerges because all in- and out-of-neighborhood samples are simply treated as positive and negative, respectively. To address the issues, we propose a novel robust method, dubbed decoupled contrastive multi-view clustering with high-order random walks (DIVIDE). In brief, DIVIDE leverages random walks to progressively identify data pairs in a global instead of local manner. As a result, DIVIDE could identify in-neighborhood negatives and out-of-neighborhood positives. Moreover, DIVIDE embraces a novel MvC architecture to perform inter- and intra-view contrastive learning in different embedding spaces, thus boosting clustering performance and embracing the robustness against missing views. To verify the efficacy of DIVIDE, we carry out extensive experiments on four benchmark datasets comparing with nine state-of-the-art MvC methods in both complete and incomplete MvC settings.
Tryage: Real-time, intelligent Routing of User Prompts to Large Language Models
Authors: Surya Narayanan Hari, Matt Thomson
Date: 2023-08-22 00:00:00
Tasks: Language Modeling, Language Modelling, Model Selection

The introduction of the transformer architecture and the self-attention mechanism has led to an explosive production of language models trained on specific downstream tasks and data domains. With over 200, 000 models in the Hugging Face ecosystem, users grapple with selecting and optimizing models to suit multifaceted workflows and data domains while addressing computational, security, and recency concerns. There is an urgent need for machine learning frameworks that can eliminate the burden of model selection and customization and unleash the incredible power of the vast emerging model library for end users. Here, we propose a context-aware routing system, Tryage, that leverages a language model router for optimal selection of expert models from a model library based on analysis of individual input prompts. Inspired by the thalamic router in the brain, Tryage employs a perceptive router to predict down-stream model performance on prompts and, then, makes a routing decision using an objective function that integrates performance predictions with user goals and constraints that are incorporated through flags (e.g., model size, model recency). Tryage allows users to explore a Pareto front and automatically trade-off between task accuracy and secondary goals including minimization of model size, recency, security, verbosity, and readability. Across heterogeneous data sets that include code, text, clinical data, and patents, the Tryage framework surpasses Gorilla and GPT3.5 turbo in dynamic model selection identifying the optimal model with an accuracy of 50.9% , compared to 23.6% by GPT 3.5 Turbo and 10.8% by Gorilla. Conceptually, Tryage demonstrates how routing models can be applied to program and control the behavior of multi-model LLM systems to maximize efficient use of the expanding and evolving language model ecosystem.
Leveraging Language Model Capabilities for Sound Event Detection
Authors: Hualei Wang, Jianguo Mao, Zhifang Guo, Jiarui Wan, Hong Liu, Xiangdong Wang
Date: 2023-08-22 00:00:00
Tasks: Decoder, Event Detection, Language Modeling, Language Modelling, model, Sound Event Detection, Text Generation

Large language models reveal deep comprehension and fluent generation in the field of multi-modality. Although significant advancements have been achieved in audio multi-modality, existing methods are rarely leverage language model for sound event detection (SED). In this work, we propose an end-to-end framework for understanding audio features while simultaneously generating sound event and their temporal location. Specifically, we employ pretrained acoustic models to capture discriminative features across different categories and language models for autoregressive text generation. Conventional methods generally struggle to obtain features in pure audio domain for classification. In contrast, our framework utilizes the language model to flexibly understand abundant semantic context aligned with the acoustic representation. The experimental results showcase the effectiveness of proposed method in enhancing timestamps precision and event classification.
TrackFlow: Multi-Object Tracking with Normalizing Flows
Authors: Gianluca Mancusi, Aniello Panariello, Angelo Porrello, Matteo Fabbri, Simone Calderara, Rita Cucchiara
Date: 2023-08-22 00:00:00
Proceeding: ICCV 2023 1
Tasks: Multi-Object Tracking, Object, Object Tracking

The field of multi-object tracking has recently seen a renewed interest in the good old schema of tracking-by-detection, as its simplicity and strong priors spare it from the complex design and painful babysitting of tracking-by-attention approaches. In view of this, we aim at extending tracking-by-detection to multi-modal settings, where a comprehensive cost has to be computed from heterogeneous information e.g., 2D motion cues, visual appearance, and pose estimates. More precisely, we follow a case study where a rough estimate of 3D information is also available and must be merged with other traditional metrics (e.g., the IoU). To achieve that, recent approaches resort to either simple rules or complex heuristics to balance the contribution of each cost. However, i) they require careful tuning of tailored hyperparameters on a hold-out set, and ii) they imply these costs to be independent, which does not hold in reality. We address these issues by building upon an elegant probabilistic formulation, which considers the cost of a candidate association as the negative log-likelihood yielded by a deep density estimator, trained to model the conditional joint probability distribution of correct associations. Our experiments, conducted on both simulated and real benchmarks, show that our approach consistently enhances the performance of several tracking-by-detection algorithms.
Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions
Authors: Pouya Pezeshkpour, Estevam Hruschka
Date: 2023-08-22 00:00:00
Tasks: Multiple-choice, Sensitivity

Large Language Models (LLMs) have demonstrated remarkable capabilities in various NLP tasks. However, previous works have shown these models are sensitive towards prompt wording, and few-shot demonstrations and their order, posing challenges to fair assessment of these models. As these models become more powerful, it becomes imperative to understand and address these limitations. In this paper, we focus on LLMs robustness on the task of multiple-choice questions -- commonly adopted task to study reasoning and fact-retrieving capability of LLMs. Investigating the sensitivity of LLMs towards the order of options in multiple-choice questions, we demonstrate a considerable performance gap of approximately 13% to 75% in LLMs on different benchmarks, when answer options are reordered, even when using demonstrations in a few-shot setting. Through a detailed analysis, we conjecture that this sensitivity arises when LLMs are uncertain about the prediction between the top-2/3 choices, and specific options placements may favor certain prediction between those top choices depending on the question caused by positional bias. We also identify patterns in top-2 choices that amplify or mitigate the model's bias toward option placement. We found that for amplifying bias, the optimal strategy involves positioning the top two choices as the first and last options. Conversely, to mitigate bias, we recommend placing these choices among the adjacent options. To validate our conjecture, we conduct various experiments and adopt two approaches to calibrate LLMs' predictions, leading to up to 8 percentage points improvement across different models and benchmarks.
Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection
Authors: Charles Guille-Escuret, Pierre-André Noël, Ioannis Mitliagkas, David Vazquez, Joao Monteiro
Date: 2023-08-22 00:00:00
Tasks: Benchmarking, Out-of-Distribution Detection

Improving the reliability of deployed machine learning systems often involves developing methods to detect out-of-distribution (OOD) inputs. However, existing research often narrowly focuses on samples from classes that are absent from the training set, neglecting other types of plausible distribution shifts. This limitation reduces the applicability of these methods in real-world scenarios, where systems encounter a wide variety of anomalous inputs. In this study, we categorize five distinct types of distribution shifts and critically evaluate the performance of recent OOD detection methods on each of them. We publicly release our benchmark under the name BROAD (Benchmarking Resilience Over Anomaly Diversity). Our findings reveal that while these methods excel in detecting unknown classes, their performance is inconsistent when encountering other types of distribution shifts. In other words, they only reliably detect unexpected inputs that they have been specifically designed to expect. As a first step toward broad OOD detection, we learn a generative model of existing detection scores with a Gaussian mixture. By doing so, we present an ensemble approach that offers a more consistent and comprehensive solution for broad OOD detection, demonstrating superior performance compared to existing methods. Our code to download BROAD and reproduce our experiments is publicly available.
An improved column-generation-based matheuristic for learning classification trees
Authors: Krunal Kishor Patel, Guy Desaulniers, Andrea Lodi
Date: 2023-08-22 00:00:00
Tasks: Classification

Decision trees are highly interpretable models for solving classification problems in machine learning (ML). The standard ML algorithms for training decision trees are fast but generate suboptimal trees in terms of accuracy. Other discrete optimization models in the literature address the optimality problem but only work well on relatively small datasets. \cite{firat2020column} proposed a column-generation-based heuristic approach for learning decision trees. This approach improves scalability and can work with large datasets. In this paper, we describe improvements to this column generation approach. First, we modify the subproblem model to significantly reduce the number of subproblems in multiclass classification instances. Next, we show that the data-dependent constraints in the master problem are implied, and use them as cutting planes. Furthermore, we describe a separation model to generate data points for which the linear programming relaxation solution violates their corresponding constraints. We conclude by presenting computational results that show that these modifications result in better scalability.
IT3D: Improved Text-to-3D Generation with Explicit View Synthesis
Authors: YiWen Chen, Chi Zhang, Xiaofeng Yang, Zhongang Cai, Gang Yu, Lei Yang, Guosheng Lin
Date: 2023-08-22 00:00:00
Tasks: 3D Generation, Text to 3D

Recent strides in Text-to-3D techniques have been propelled by distilling knowledge from powerful large text-to-image diffusion models (LDMs). Nonetheless, existing Text-to-3D approaches often grapple with challenges such as over-saturation, inadequate detailing, and unrealistic outputs. This study presents a novel strategy that leverages explicitly synthesized multi-view images to address these issues. Our approach involves the utilization of image-to-image pipelines, empowered by LDMs, to generate posed high-quality images based on the renderings of coarse 3D models. Although the generated images mostly alleviate the aforementioned issues, challenges such as view inconsistency and significant content variance persist due to the inherent generative nature of large diffusion models, posing extensive difficulties in leveraging these images effectively. To overcome this hurdle, we advocate integrating a discriminator alongside a novel Diffusion-GAN dual training strategy to guide the training of 3D models. For the incorporated discriminator, the synthesized multi-view images are considered real data, while the renderings of the optimized 3D models function as fake data. We conduct a comprehensive set of experiments that demonstrate the effectiveness of our method over baseline approaches.
Dynamic Open Vocabulary Enhanced Safe-landing with Intelligence (DOVESEI)
Authors: Haechan Mark Bong, Rongge Zhang, Ricardo de Azambuja, Giovanni Beltrame
Date: 2023-08-22 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

This work targets what we consider to be the foundational step for urban airborne robots, a safe landing. Our attention is directed toward what we deem the most crucial aspect of the safe landing perception stack: segmentation. We present a streamlined reactive UAV system that employs visual servoing by harnessing the capabilities of open vocabulary image segmentation. This approach can adapt to various scenarios with minimal adjustments, bypassing the necessity for extensive data accumulation for refining internal models, thanks to its open vocabulary methodology. Given the limitations imposed by local authorities, our primary focus centers on operations originating from altitudes of 100 meters. This choice is deliberate, as numerous preceding works have dealt with altitudes up to 30 meters, aligning with the capabilities of small stereo cameras. Consequently, we leave the remaining 20m to be navigated using conventional 3D path planning methods. Utilizing monocular cameras and image segmentation, our findings demonstrate the system's capability to successfully execute landing maneuvers at altitudes as low as 20 meters. However, this approach is vulnerable to intermittent and occasionally abrupt fluctuations in the segmentation between frames in a video stream. To address this challenge, we enhance the image segmentation output by introducing what we call a dynamic focus: a masking mechanism that self adjusts according to the current landing stage. This dynamic focus guides the control system to avoid regions beyond the drone's safety radius projected onto the ground, thus mitigating the problems with fluctuations. Through the implementation of this supplementary layer, our experiments have reached improvements in the landing success rate of almost tenfold when compared to global segmentation. All the source code is open source and available online (github.com/MISTLab/DOVESEI).