OpenRaiser/Intern-Atlas huggingface dataset:
Summarization of Investment Reports Using Pre-trained Model
Venue: arXiv cs.CL 2024
Year: 2024

In this paper, we attempt to summarize monthly reports as investment reports. Fund managers have a wide range of tasks, one of which is the preparation of investment reports. In addition to preparing monthly reports on fund management, fund managers prepare management reports that summarize these monthly reports every six months or once a year. The preparation of fund reports is a labor-intensive and time-consuming task. Therefore, in this paper, we tackle investment summarization from monthly reports using transformer-based models. There are two main types of summarization methods: extractive summarization and abstractive summarization, and this study constructs both methods and examines which is more useful in summarizing investment reports.
Demystifying Device-specific Compatibility Issues in Android Apps
Venue: arXiv cs.SE 2024
Year: 2024

The Android ecosystem is profoundly fragmented due to the frequent updates of the Android system and the prevalent customizations by mobile device manufacturers. Previous research primarily focused on identifying and repairing evolution-induced API compatibility issues, with limited consideration of devices-specific compatibility issues (DSC issues). To fill this gap, we conduct an empirical study of 197 DSC issues collected from 94 open-source repositories on GitHub. We introduce a new perspective for comprehending these issues by categorizing them into two principal groups, Functionality Breaks, and OEM Features, based on their manifestations and root causes. The functionality break issues disrupt standard Android system behaviors, lead to crashes or unexpected behaviors on specific devices, and require developers to implement workarounds to preserve the original functionality. The OEM feature issues involve the introduction of device-specific functionalities or features beyond the basic Android system. The different nature of functionality break issues and OEM feature issues lead to unique challenges in addressing them. Common solutions for functionality break issues involve calling additional APIs, substituting problematic ones, or using specific parameters, while resolving OEM feature issues often relies on Android inter-component communication methods and reflection, with additional unconventional strategies. Such observations highlight the distinctive challenges in addressing DSC issues in Android apps and will facilitate the future development of testing and analysis tools targeting these issues. Our study demonstrates that Functionality break and OEM feature issues have different characteristics, and future research may need to investigate them separately.
Facilitating the Parametric Definition of Geometric Properties in Programming-Based CAD
Venue: arXiv cs.HC 2024
Year: 2024

Parametric Computer-aided design (CAD) enables the creation of reusable models by integrating variables into geometric properties, facilitating customization without a complete redesign. However, creating parametric designs in programming-based CAD presents significant challenges. Users define models in a code editor using a programming language, with the application generating a visual representation in a viewport. This process involves complex programming and arithmetic expressions to describe geometric properties, linking various object properties to create parametric designs. Unfortunately, these applications lack assistance, making the process unnecessarily demanding. We propose a solution that allows users to retrieve parametric expressions from the visual representation for reuse in the code, streamlining the design process. We demonstrated this concept through a proof-of-concept implemented in the programming-based CAD application, OpenSCAD, and conducted an experiment with 11 users. Our findings suggest that this solution could significantly reduce design errors, improve interactivity and engagement in the design process, and lower the entry barrier for newcomers by reducing the mathematical skills typically required in programming-based CAD applications
Efficient Solutions For An Intriguing Failure of LLMs: Long Context Window Does Not Mean LLMs Can Analyze Long Sequences Flawlessly
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) have demonstrated remarkable capabilities in comprehending and analyzing lengthy sequential inputs, owing to their extensive context windows that allow processing millions of tokens in a single forward pass. However, this paper uncovers a surprising limitation: LLMs fall short when handling long input sequences. We investigate this issue using three datasets and two tasks (sentiment analysis and news categorization) across various LLMs, including Claude 3, Gemini Pro, GPT 3.5 Turbo, Llama 3 Instruct, and Mistral Instruct models. To address this limitation, we propose and evaluate ad-hoc solutions that substantially enhance LLMs' performance on long input sequences by up to 50%, while reducing API cost and latency by up to 93% and 50%, respectively.
Safe Semi-Supervised Contrastive Learning Using In-Distribution Data as Positive Examples
Venue: arXiv cs.LG 2024
Year: 2024

Semi-supervised learning methods have shown promising results in solving many practical problems when only a few labels are available. The existing methods assume that the class distributions of labeled and unlabeled data are equal; however, their performances are significantly degraded in class distribution mismatch scenarios where out-of-distribution (OOD) data exist in the unlabeled data. Previous safe semi-supervised learning studies have addressed this problem by making OOD data less likely to affect training based on labeled data. However, even if the studies effectively filter out the unnecessary OOD data, they can lose the basic information that all data share regardless of class. To this end, we propose to apply a self-supervised contrastive learning approach to fully exploit a large amount of unlabeled data. We also propose a contrastive loss function with coefficient schedule to aggregate as an anchor the labeled negative examples of the same class into positive examples. To evaluate the performance of the proposed method, we conduct experiments on image classification datasets - CIFAR-10, CIFAR-100, Tiny ImageNet, and CIFAR-100+Tiny ImageNet - under various mismatch ratios. The results show that self-supervised contrastive learning significantly improves classification accuracy. Moreover, aggregating the in-distribution examples produces better representation and consequently further improves classification accuracy.
Impact of Major Health Events on Pharmaceutical Stocks: A Comprehensive Analysis Using Macroeconomic and Market Indicators
Venue: arXiv cs.CE 2024
Year: 2024

This study investigates the impact of significant health events on pharmaceutical stock performance, employing a comprehensive analysis incorporating macroeconomic and market indicators. Using Ordinary Least Squares (OLS) regression, we evaluate the effects of thirteen major health events since 2000, including the Anthrax attacks, SARS outbreak, H1N1 pandemic, and COVID-19 pandemic, on the pharmaceutical sector. The analysis covers different phases of each event beginning, peak, and ending to capture their temporal influence on stock prices. Our findings reveal distinct patterns in stock performance, driven by market reactions to the initial news, peak impact, and eventual resolution of these crises. We also examine scenarios with and without key macroeconomic (MA) and market (MI) indicators to isolate their contributions. This detailed examination provides valuable insights for investors, policymakers, and stakeholders in understanding the interplay between major health events and health market dynamics, guiding better decision-making during future health related disruptions.
Brief state of the art in social information mining: Practical application in analysis of trends in French legislative 2024
Venue: arXiv cs.CY 2024
Year: 2024

The analysis of social media information has undergone significant evolution in the last decade due to advancements in artificial intelligence (AI) and machine learning (ML). This paper provides an overview of the state-of-the-art techniques in social media mining, with a practical application in analyzing trends in the 2024 French legislative elections. We leverage natural language processing (NLP) tools to gauge public opinion by extracting and analyzing comments and reactions from the AgoraVox platform. The study reveals that the National Rally party, led by Marine Le Pen, maintains a high level of engagement on social media, outperforming traditional parties. This trend is corroborated by user interactions, indicating a strong digital presence. The results highlight the utility of advanced AI models, such as transformers and large language models (LLMs), in capturing nuanced public sentiments and predicting political leanings, demonstrating their potential in real-time reputation management and crisis response.
MAO: A Framework for Process Model Generation with Multi-Agent Orchestration
Venue: arXiv cs.AI 2024
Year: 2024

Process models are frequently used in software engineering to describe business requirements, guide software testing and control system improvement. However, traditional process modeling methods often require the participation of numerous experts, which is expensive and time-consuming. Therefore, the exploration of a more efficient and cost-effective automated modeling method has emerged as a focal point in current research. This article explores a framework for automatically generating process models with multi-agent orchestration (MAO), aiming to enhance the efficiency of process modeling and offer valuable insights for domain experts. Our framework MAO leverages large language models as the cornerstone for multi-agent, employing an innovative prompt strategy to ensure efficient collaboration among multi-agent. Specifically, 1) generation. The first phase of MAO is to generate a slightly rough process model from the text description; 2) refinement. The agents would continuously refine the initial process model through multiple rounds of dialogue; 3) reviewing. Large language models are prone to hallucination phenomena among multi-turn dialogues, so the agents need to review and repair semantic hallucinations in process models; 4) testing. The representation of process models is diverse. Consequently, the agents utilize external tools to test whether the generated process model contains format errors, namely format hallucinations, and then adjust the process model to conform to the output paradigm. The experiments demonstrate that the process models generated by our framework outperform existing methods and surpass manual modeling by 89%, 61%, 52%, and 75% on four different datasets, respectively.
A Semi-supervised Multi-channel Graph Convolutional Network for Query Classification in E-commerce
Venue: arXiv cs.CL 2024
Year: 2024

Query intent classification is an essential module for customers to find desired products on the e-commerce application quickly. Most existing query intent classification methods rely on the users' click behavior as a supervised signal to construct training samples. However, these methods based entirely on posterior labels may lead to serious category imbalance problems because of the Matthew effect in click samples. Compared with popular categories, it is difficult for products under long-tail categories to obtain traffic and user clicks, which makes the models unable to detect users' intent for products under long-tail categories. This in turn aggravates the problem that long-tail categories cannot obtain traffic, forming a vicious circle. In addition, due to the randomness of the user's click, the posterior label is unstable for the query with similar semantics, which makes the model very sensitive to the input, leading to an unstable and incomplete recall of categories. In this paper, we propose a novel Semi-supervised Multi-channel Graph Convolutional Network (SMGCN) to address the above problems from the perspective of label association and semi-supervised learning. SMGCN extends category information and enhances the posterior label by utilizing the similarity score between the query and categories. Furthermore, it leverages the co-occurrence and semantic similarity graph of categories to strengthen the relations among labels and weaken the influence of posterior label instability. We conduct extensive offline and online A/B experiments, and the experimental results show that SMGCN significantly outperforms the strong baselines, which shows its effectiveness and practicality.
RobNODDI: Robust NODDI Parameter Estimation with Adaptive Sampling under Continuous Representation
Venue: arXiv cs.CV 2024
Year: 2024

Neurite Orientation Dispersion and Density Imaging (NODDI) is an important imaging technology used to evaluate the microstructure of brain tissue, which is of great significance for the discovery and treatment of various neurological diseases. Current deep learning-based methods perform parameter estimation through diffusion magnetic resonance imaging (dMRI) with a small number of diffusion gradients. These methods speed up parameter estimation and improve accuracy. However, the diffusion directions used by most existing deep learning models during testing needs to be strictly consistent with the diffusion directions during training. This results in poor generalization and robustness of deep learning models in dMRI parameter estimation. In this work, we verify for the first time that the parameter estimation performance of current mainstream methods will significantly decrease when the testing diffusion directions and the training diffusion directions are inconsistent. A robust NODDI parameter estimation method with adaptive sampling under continuous representation (RobNODDI) is proposed. Furthermore, long short-term memory (LSTM) units and fully connected layers are selected to learn continuous representation signals. To this end, we use a total of 100 subjects to conduct experiments based on the Human Connectome Project (HCP) dataset, of which 60 are used for training, 20 are used for validation, and 20 are used for testing. The test results indicate that RobNODDI improves the generalization performance and robustness of the deep learning model, enhancing the stability and flexibility of deep learning NODDI parameter estimatimation applications.
CACE-Net: Co-guidance Attention and Contrastive Enhancement for Effective Audio-Visual Event Localization
Venue: arXiv cs.CV 2024
Year: 2024

The audio-visual event localization task requires identifying concurrent visual and auditory events from unconstrained videos within a network model, locating them, and classifying their category. The efficient extraction and integration of audio and visual modal information have always been challenging in this field. In this paper, we introduce CACE-Net, which differs from most existing methods that solely use audio signals to guide visual information. We propose an audio-visual co-guidance attention mechanism that allows for adaptive bi-directional cross-modal attentional guidance between audio and visual information, thus reducing inconsistencies between modalities. Moreover, we have observed that existing methods have difficulty distinguishing between similar background and event and lack the fine-grained features for event classification. Consequently, we employ background-event contrast enhancement to increase the discrimination of fused feature and fine-tuned pre-trained model to extract more refined and discernible features from complex multimodal inputs. Specifically, we have enhanced the model's ability to discern subtle differences between event and background and improved the accuracy of event classification in our model. Experiments on the AVE dataset demonstrate that CACE-Net sets a new benchmark in the audio-visual event localization task, proving the effectiveness of our proposed methods in handling complex multimodal learning and event localization in unconstrained videos. Code is available at https://github.com/Brain-Cog-Lab/CACE-Net.
Representation Bias of Adolescents in AI: A Bilingual, Bicultural Study
Venue: arXiv cs.CY 2024
Year: 2024

Popular and news media often portray teenagers with sensationalism, as both a risk to society and at risk from society. As AI begins to absorb some of the epistemic functions of traditional media, we study how teenagers in two countries speaking two languages: 1) are depicted by AI, and 2) how they would prefer to be depicted. Specifically, we study the biases about teenagers learned by static word embeddings (SWEs) and generative language models (GLMs), comparing these with the perspectives of adolescents living in the U.S. and Nepal. We find English-language SWEs associate teenagers with societal problems, and more than 50% of the 1,000 words most associated with teenagers in the pretrained GloVe SWE reflect such problems. Given prompts about teenagers, 30% of outputs from GPT2-XL and 29% from LLaMA-2-7B GLMs discuss societal problems, most commonly violence, but also drug use, mental illness, and sexual taboo. Nepali models, while not free of such associations, are less dominated by social problems. Data from workshops with N=13 U.S. adolescents and N=18 Nepalese adolescents show that AI presentations are disconnected from teenage life, which revolves around activities like school and friendship. Participant ratings of how well 20 trait words describe teens are decorrelated from SWE associations, with Pearson's r=.02, n.s. in English FastText and r=.06, n.s. in GloVe; and r=.06, n.s. in Nepali FastText and r=-.23, n.s. in GloVe. U.S. participants suggested AI could fairly present teens by highlighting diversity, while Nepalese participants centered positivity. Participants were optimistic that, if it learned from adolescents, rather than media sources, AI could help mitigate stereotypes. Our work offers an understanding of the ways SWEs and GLMs misrepresent a developmentally vulnerable group and provides a template for less sensationalized characterization.
Optimal and efficient text counterfactuals using Graph Neural Networks
Venue: arXiv cs.CL 2024
Year: 2024

As NLP models become increasingly integral to decision-making processes, the need for explainability and interpretability has become paramount. In this work, we propose a framework that achieves the aforementioned by generating semantically edited inputs, known as counterfactual interventions, which change the model prediction, thus providing a form of counterfactual explanations for the model. We test our framework on two NLP tasks - binary sentiment classification and topic classification - and show that the generated edits are contrastive, fluent and minimal, while the whole process remains significantly faster that other state-of-the-art counterfactual editors.
Shaping Rewards, Shaping Routes: On Multi-Agent Deep Q-Networks for Routing in Satellite Constellation Networks
Venue: arXiv cs.NI 2024
Year: 2024

Effective routing in satellite mega-constellations has become crucial to facilitate the handling of increasing traffic loads, more complex network architectures, as well as the integration into 6G networks. To enhance adaptability as well as robustness to unpredictable traffic demands, and to solve dynamic routing environments efficiently, machine learning-based solutions are being considered. For network control problems, such as optimizing packet forwarding decisions according to Quality of Service requirements and maintaining network stability, deep reinforcement learning techniques have demonstrated promising results. For this reason, we investigate the viability of multi-agent deep Q-networks for routing in satellite constellation networks. We focus specifically on reward shaping and quantifying training convergence for joint optimization of latency and load balancing in static and dynamic scenarios. To address identified drawbacks, we propose a novel hybrid solution based on centralized learning and decentralized control.
Configuring Safe Spiking Neural Controllers for Cyber-Physical Systems through Formal Verification
Venue: arXiv cs.ET 2024
Year: 2024

Spiking Neural Networks (SNNs) are a subclass of neuromorphic models that have great potential to be used as controllers in Cyber-Physical Systems (CPSs) due to their energy efficiency. They can benefit from the prevalent approach of first training an Artificial Neural Network (ANN) and then translating to an SNN with subsequent hyperparameter tuning. The tuning is required to ensure that the resulting SNN is accurate with respect to the ANN in terms of metrics like Mean Squared Error (MSE). However, SNN controllers for safety-critical CPSs must also satisfy safety specifications, which are not guaranteed by the conversion approach. In this paper, we propose a solution which tunes the $temporal$ $window$ hyperparameter of the translated SNN to ensure both accuracy and compliance with the safe range specification that requires the SNN outputs to remain within a safe range. The core verification problem is modelled using mixed-integer linear programming (MILP) and is solved with Gurobi. When the controller fails to meet the range specification, we compute tight bounds on the SNN outputs as feedback for the CPS developer. To mitigate the high computational cost of verification, we integrate data-driven steps to minimize verification calls. Our approach provides designers with the confidence to safely integrate energy-efficient SNN controllers into modern CPSs. We demonstrate our approach with experimental results on five different benchmark neural controllers.
Blockchain-Enabled Dynamic Spectrum Sharing for Satellite and Terrestrial Communication Networks
Venue: arXiv cs.DC 2024
Year: 2024

Dynamic spectrum sharing (DSS) between satellite and terrestrial networks has increasingly engaged the academic and industrial sectors. Nevertheless, facilitating secure, efficient and scalable sharing continues to pose a pivotal challenge. Emerging as a promising technology to bridge the trust gap among multiple participants, blockchain has been envisioned to enable DSS in a decentralized manner. However, satellites with limited resources may struggle to support the frequent interactions required by blockchain networks. Additionally,given the extensive coverage of satellites, spectrum sharing needs vary by regions, challenging traditional blockchain approaches to accommodate differences. In this work, a partitioned, self-governed, and customized dynamic spectrum sharing approach (PSC-DSS) is proposed for spectrum sharing between satellite access networks and terrestrial access networks. This approach establishes a sharded and tiered architecture which allows various regions to manage spectrum autonomously while jointly maintaining a single blockchain ledger. Moreover, a spectrum-consensus integrated mechanism, which decouples DSS process and couples it with blockchain consensus protocol, is designed to enable regions to conduct DSS transactions in parallel and dynamically innovate spectrum sharing schemes without affecting others. Furthermore, a theoretical framework is derived to justify the stability performance of PSC-DSS. Finally, simulations and experiments are conducted to validate the advantageous performance of PSC-DSS in terms of low-overhead, high efficiency, and robust stability.