OpenRaiser/Intern-Atlas huggingface dataset:
Stability of P2P Networks Under Greedy Peering (Full Version)
Venue: arXiv cs.GT 2024
Year: 2024

Major cryptocurrency networks have relied on random peering choice rules for making connections in their peer-to-peer networks. Generally, these choices have good properties, particularly for open, permissionless networks. Random peering choices however do not take into account that some actors may choose to optimize who they connect to such that they are quicker to hear about information being propagated in the network. In this paper, we explore the dynamics of such greedy strategies. We study a model in which nodes select peers with the objective of minimizing their average distance to a designated subset of nodes in the network, and consider the impact of several factors including the peer selection process, degree constraints, and the size of the designated subset. The latter is particularly interesting in the context of blockchain networks as generally only a subset of nodes are the propagation source for content. We first analyze an idealized version of the game where each node has full knowledge of the current network and aims to select the $d$ best connections, and prove the existence of equilibria under various model assumptions. Since in reality nodes only have local knowledge based on their peers' behavior, we also study a greedy protocol which runs in rounds, with each node replacing its worst-performing edge with a new random edge. We exactly characterize stability properties of networks that evolve with this peering rule and derive regimes where stability is possible and even inevitable. We also run extensive simulations with this peering rule examining both how the network evolves and how different network parameters affect the stability properties of the network. Our findings generally show that the only stable networks that arise from greedy peering choices are low-diameter and result in disparate performance for nodes in the network.
Doing AI: Algorithmic decision support as a human activity
Venue: arXiv cs.HC 2024
Year: 2024

Algorithmic decision support (ADS), using Machine-Learning-based AI, is becoming a major part of many processes. Organizations introduce ADS to improve decision-making and use available data, thereby possibly limiting deviations from the normative "homo economicus" and the biases that characterize human decision-making. However, a closer look at the development and use of ADS systems in organizational settings reveals that they necessarily involve a series of largely unspecified human decisions. They begin with deliberations for which decisions to use ADS, continue with choices while developing and deploying the ADS, and end with decisions on how to use the ADS output in an organization's operations. The paper presents an overview of these decisions and some relevant behavioral phenomena. It points out directions for further research, which is essential for correctly assessing the processes and their vulnerabilities. Understanding these behavioral aspects is important for successfully implementing ADS in organizations.
Using construction waste hauling trucks' GPS data to classify earthwork-related locations: A Chengdu case study
Venue: arXiv cs.LG 2024
Year: 2024

Earthwork-related locations (ERLs), such as construction sites, earth dumping ground, and concrete mixing stations, are major sources of urban dust pollution (particulate matters). The effective management of ERLs is crucial and requires timely and efficient tracking of these locations throughout the city. This work aims to identify and classify urban ERLs using GPS trajectory data of over 16,000 construction waste hauling trucks (CWHTs), as well as 58 urban features encompassing geographic, land cover, POI and transport dimensions. We compare several machine learning models and examine the impact of various spatial-temporal features on classification performance using real-world data in Chengdu, China. The results demonstrate that 77.8% classification accuracy can be achieved with a limited number of features. This classification framework was implemented in the Alpha MAPS system in Chengdu, which has successfully identified 724 construction cites/earth dumping ground, 48 concrete mixing stations, and 80 truck parking locations in the city during December 2023, which has enabled local authority to effectively manage urban dust pollution at low personnel costs.
Gilbert-Varshamov Bound for Codes in $L_1$ Metric using Multivariate Analytic Combinatorics
Venue: arXiv cs.IT 2024
Year: 2024

Analytic combinatorics in several variables refers to a suite of tools that provide sharp asymptotic estimates for certain combinatorial quantities. In this paper, we apply these tools to determine the Gilbert--Varshamov lower bound on the rate of optimal codes in $L_1$ metric. Several different code spaces are analyzed, including the simplex and the hypercube in $\mathbb{Z^n}$, all of which are inspired by concrete data storage and transmission models such as the sticky insertion channel, the permutation channel, the adjacent transposition (bit-shift) channel, the multilevel flash memory channel, etc.
Environment Semantic Communication: Enabling Distributed Sensing Aided Networks
Venue: arXiv cs.IT 2024
Year: 2024

Millimeter-wave (mmWave) and terahertz (THz) communication systems require large antenna arrays and use narrow directive beams to ensure sufficient receive signal power. However, selecting the optimal beams for these large antenna arrays incurs a significant beam training overhead, making it challenging to support applications involving high mobility. In recent years, machine learning (ML) solutions have shown promising results in reducing the beam training overhead by utilizing various sensing modalities such as GPS position and RGB images. However, the existing approaches are mainly limited to scenarios with only a single object of interest present in the wireless environment and focus only on co-located sensing, where all the sensors are installed at the communication terminal. This brings key challenges such as the limited sensing coverage compared to the coverage of the communication system and the difficulty in handling non-line-of-sight scenarios. To overcome these limitations, our paper proposes the deployment of multiple distributed sensing nodes, each equipped with an RGB camera. These nodes focus on extracting environmental semantics from the captured RGB images. The semantic data, rather than the raw images, are then transmitted to the basestation. This strategy significantly alleviates the overhead associated with the data storage and transmission of the raw images. Furthermore, semantic communication enhances the system's adaptability and responsiveness to dynamic environments, allowing for prioritization and transmission of contextually relevant information. Experimental results on the DeepSense 6G dataset demonstrate the effectiveness of the proposed solution in reducing the sensing data transmission overhead while accurately predicting the optimal beams in realistic communication environments.
Zero-shot cross-lingual transfer in instruction tuning of large language models
Venue: arXiv cs.CL 2024
Year: 2024

Instruction tuning (IT) is widely used to teach pretrained large language models (LLMs) to follow arbitrary instructions, but is under-studied in multilingual settings. In this work, we conduct a systematic study of zero-shot cross-lingual transfer in IT, when an LLM is instruction-tuned on English-only data and then tested on user prompts in other languages. We advocate for the importance of evaluating various aspects of model responses in multilingual instruction following and investigate the influence of different model configuration choices. We find that cross-lingual transfer does happen successfully in IT even if all stages of model training are English-centric, but only if multiliguality is taken into account in hyperparameter tuning and with large enough IT data. English-trained LLMs are capable of generating correct-language, comprehensive and helpful responses in other languages, but suffer from low factuality and may occasionally have fluency errors.
Design of an Analog Memory Cell in 0.25 micron CMOS process
Venue: arXiv cs.OH 2024
Year: 2024

CMOS VLSI technology is the most dominant integration methodology prevailing in the world today. Various signal-processing blocks are made using analog or digital design techniques in MOS VLSI. An important component is the Memory unit used to store data. In the project a memory cell has been built up using analog design method. A capacitor is used as the basic storage device. The main idea behind analog memory is that the analog value of the charge or voltage stored in the capacitor is the data stored. So the dielectric quality of the capacitor becomes important here to determine how effectively it can store some charge. Analog memory is a trade off between hardware cost, chip area and accuracy or quality of storage. The circuit of analog memory cell was developed starting from the idea that required voltage will be stored in a capacitor and MOS transistors were used as switches. A given technology of integration was used and hence the dielectric property of the capacitor was fixed. By suitable circuit configuration the analog voltage value was written to the capacitor, read out when required and the charge loss was also refreshed. The results obtained are as given in the thesis.
HumanEval on Latest GPT Models -- 2024
Venue: arXiv cs.CL 2024
Year: 2024

In 2023, we are using the latest models of GPT-4 to advance program synthesis. The large language models have significantly improved the state-of-the-art for this purpose. To make these advancements more accessible, we have created a repository that connects these models to Huamn Eval. This dataset was initally developed to be used with a language model called CODEGEN on natural and programming language data. The utility of these trained models is showcased by demonstrating their competitive performance in zero-shot Python code generation on HumanEval tasks compared to previous state-of-the-art solutions. Additionally, this gives way to developing more multi-step paradigm synthesis. This benchmark features 160 diverse problem sets factorized into multistep prompts that our analysis shows significantly improves program synthesis over single-turn inputs. All code is open source at https://github.com/daniel442li/gpt-human-eval .
The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operative
Venue: arXiv cs.CR 2024
Year: 2024

Due to their unprecedented ability to process and respond to various types of data, Multimodal Large Language Models (MLLMs) are constantly defining the new boundary of Artificial General Intelligence (AGI). As these advanced generative models increasingly form collaborative networks for complex tasks, the integrity and security of these systems are crucial. Our paper, ``The Wolf Within'', explores a novel vulnerability in MLLM societies - the indirect propagation of malicious content. Unlike direct harmful output generation for MLLMs, our research demonstrates how a single MLLM agent can be subtly influenced to generate prompts that, in turn, induce other MLLM agents in the society to output malicious content. Our findings reveal that, an MLLM agent, when manipulated to produce specific prompts or instructions, can effectively ``infect'' other agents within a society of MLLMs. This infection leads to the generation and circulation of harmful outputs, such as dangerous instructions or misinformation, across the society. We also show the transferability of these indirectly generated prompts, highlighting their possibility in propagating malice through inter-agent communication. This research provides a critical insight into a new dimension of threat posed by MLLMs, where a single agent can act as a catalyst for widespread malevolent influence. Our work underscores the urgent need for developing robust mechanisms to detect and mitigate such covert manipulations within MLLM societies, ensuring their safe and ethical utilization in societal applications.
Using Harmonics for Low-Cost Jamming
Venue: arXiv cs.CR 2024
Year: 2024

The digitalisation of the modern schooling system has led to multiple schools and organisations buying similar hardware. Electronic equipment like wireless microphones, projectors, touchscreen displays etc., have been almost standardised with a few well-known brands leading the market. This has led to the adoption of common frequency ranges between brands with many sticking between 600-670 MHz. The popularity of low-cost computing devices like the Raspberry Pi which has been used in a plethora of applications has also taken the path of being used as low-cost transmitters. There have been many implementations where the Raspberry Pi has been used as the target device but few cases where the PI is the actual threat. In this paper, we explore the use of the Raspberry Pi as a stealth radio frequency jamming device to disable wireless conference microphones. Harmonics were used to achieve frequencies outside the Pi's transmission frequency by taking advantage of its unfiltered transmission.
Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMs
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs), used in creative writing, code generation, and translation, generate text based on input sequences but are vulnerable to jailbreak attacks, where crafted prompts induce harmful outputs. Most jailbreak prompt methods use a combination of jailbreak templates followed by questions to ask to create jailbreak prompts. However, existing jailbreak prompt designs generally suffer from excessive semantic differences, resulting in an inability to resist defenses that use simple semantic metrics as thresholds. Jailbreak prompts are semantically more varied than the original questions used for queries. In this paper, we introduce a Semantic Mirror Jailbreak (SMJ) approach that bypasses LLMs by generating jailbreak prompts that are semantically similar to the original question. We model the search for jailbreak prompts that satisfy both semantic similarity and jailbreak validity as a multi-objective optimization problem and employ a standardized set of genetic algorithms for generating eligible prompts. Compared to the baseline AutoDAN-GA, SMJ achieves attack success rates (ASR) that are at most 35.4% higher without ONION defense and 85.2% higher with ONION defense. SMJ's better performance in all three semantic meaningfulness metrics of Jailbreak Prompt, Similarity, and Outlier, also means that SMJ is resistant to defenses that use those metrics as thresholds.
On Evaluation Protocols for Data Augmentation in a Limited Data Scenario
Venue: arXiv cs.CL 2024
Year: 2024

Textual data augmentation (DA) is a prolific field of study where novel techniques to create artificial data are regularly proposed, and that has demonstrated great efficiency on small data settings, at least for text classification tasks. In this paper, we challenge those results, showing that classical data augmentation (which modify sentences) is simply a way of performing better fine-tuning, and that spending more time doing so before applying data augmentation negates its effect. This is a significant contribution as it answers several questions that were left open in recent years, namely~: which DA technique performs best (all of them as long as they generate data close enough to the training set, as to not impair training) and why did DA show positive results (facilitates training of network). We further show that zero- and few-shot DA via conversational agents such as ChatGPT or LLama2 can increase performances, confirming that this form of data augmentation is preferable to classical methods.
Chain-of-Thought Unfaithfulness as Disguised Accuracy
Venue: arXiv cs.CL 2024
Year: 2024

Understanding the extent to which Chain-of-Thought (CoT) generations align with a large language model's (LLM) internal computations is critical for deciding whether to trust an LLM's output. As a proxy for CoT faithfulness, Lanham et al. (2023) propose a metric that measures a model's dependence on its CoT for producing an answer. Within a single family of proprietary models, they find that LLMs exhibit a scaling-then-inverse-scaling relationship between model size and their measure of faithfulness, and that a 13 billion parameter model exhibits increased faithfulness compared to models ranging from 810 million to 175 billion parameters in size. We evaluate whether these results generalize as a property of all LLMs. We replicate the experimental setup in their section focused on scaling experiments with three different families of models and, under specific conditions, successfully reproduce the scaling trends for CoT faithfulness they report. However, after normalizing the metric to account for a model's bias toward certain answer choices, unfaithfulness drops significantly for smaller less-capable models. This normalized faithfulness metric is also strongly correlated ($R^2$=0.74) with accuracy, raising doubts about its validity for evaluating faithfulness.
Learning Inverse Kinodynamics for Autonomous Vehicle Drifting
Venue: arXiv cs.RO 2024
Year: 2024

In this work, we explore a data-driven learning-based approach to learning the kinodynamic model of a small autonomous vehicle, and observe the effect it has on motion planning, specifically autonomous drifting. When executing a motion plan in the real world, there are numerous causes for error, and what is planned is often not what is executed on the actual car. Learning a kinodynamic planner based off of inertial measurements and executed commands can help us learn the world state. In our case, we look towards the realm of drifting; it is a complex maneuver that requires a smooth enough surface, high enough speed, and a drastic change in velocity. We attempt to learn the kinodynamic model for these drifting maneuvers, and attempt to tighten the slip of the car. Our approach is able to learn a kinodynamic model for high-speed circular navigation, and is able to avoid obstacles on an autonomous drift at high speed by correcting an executed curvature for loose drifts. We seek to adjust our kinodynamic model for success in tighter drifts in future work.
Re-Examine Distantly Supervised NER: A New Benchmark and a Simple Approach
Venue: arXiv cs.CL 2024
Year: 2024

Distantly-Supervised Named Entity Recognition (DS-NER) uses knowledge bases or dictionaries for annotations, reducing manual efforts but rely on large human labeled validation set. In this paper, we introduce a real-life DS-NER dataset, QTL, where the training data is annotated using domain dictionaries and the test data is annotated by domain experts. This dataset has a small validation set, reflecting real-life scenarios. Existing DS-NER approaches fail when applied to QTL, which motivate us to re-examine existing DS-NER approaches. We found that many of them rely on large validation sets and some used test set for tuning inappropriately. To solve this issue, we proposed a new approach, token-level Curriculum-based Positive-Unlabeled Learning (CuPUL), which uses curriculum learning to order training samples from easy to hard. This method stabilizes training, making it robust and effective on small validation sets. CuPUL also addresses false negative issues using the Positive-Unlabeled learning paradigm, demonstrating improved performance in real-life applications.
Reinforcement Learning with Elastic Time Steps
Venue: arXiv cs.RO 2024
Year: 2024

Traditional Reinforcement Learning (RL) policies are typically implemented with fixed control rates, often disregarding the impact of control rate selection. This can lead to inefficiencies as the optimal control rate varies with task requirements. We propose the Multi-Objective Soft Elastic Actor-Critic (MOSEAC), an off-policy actor-critic algorithm that uses elastic time steps to dynamically adjust the control frequency. This approach minimizes computational resources by selecting the lowest viable frequency. We show that MOSEAC converges and produces stable policies at the theoretical level, and validate our findings in a real-time 3D racing game. MOSEAC significantly outperformed other variable time step approaches in terms of energy efficiency and task effectiveness. Additionally, MOSEAC demonstrated faster and more stable training, showcasing its potential for real-world RL applications in robotics.
Mirror: A Multiple-perspective Self-Reflection Method for Knowledge-rich Reasoning
Venue: arXiv cs.CL 2024
Year: 2024

While Large language models (LLMs) have the capability to iteratively reflect on their own outputs, recent studies have observed their struggles with knowledge-rich problems without access to external resources. In addition to the inefficiency of LLMs in self-assessment, we also observe that LLMs struggle to revisit their predictions despite receiving explicit negative feedback. Therefore, We propose Mirror, a Multiple-perspective self-reflection method for knowledge-rich reasoning, to avoid getting stuck at a particular reflection iteration. Mirror enables LLMs to reflect from multiple-perspective clues, achieved through a heuristic interaction between a Navigator and a Reasoner. It guides agents toward diverse yet plausibly reliable reasoning trajectory without access to ground truth by encouraging (1) diversity of directions generated by Navigator and (2) agreement among strategically induced perturbations in responses generated by the Reasoner. The experiments on five reasoning datasets demonstrate that Mirror's superiority over several contemporary self-reflection approaches. Additionally, the ablation study studies clearly indicate that our strategies alleviate the aforementioned challenges.
CLoVe: Encoding Compositional Language in Contrastive Vision-Language Models
Venue: arXiv cs.CV 2024
Year: 2024

Recent years have witnessed a significant increase in the performance of Vision and Language tasks. Foundational Vision-Language Models (VLMs), such as CLIP, have been leveraged in multiple settings and demonstrated remarkable performance across several tasks. Such models excel at object-centric recognition yet learn text representations that seem invariant to word order, failing to compose known concepts in novel ways. However, no evidence exists that any VLM, including large-scale single-stream models such as GPT-4V, identifies compositions successfully. In this paper, we introduce a framework to significantly improve the ability of existing models to encode compositional language, with over 10% absolute improvement on compositionality benchmarks, while maintaining or improving the performance on standard object-recognition and retrieval benchmarks. Our code and pre-trained models are publicly available at https://github.com/netflix/clove.