OpenRaiser/Intern-Atlas huggingface dataset:
Supply Chain Due Diligence Risk Assessment for the EU: A Network Approach to estimate expected effectiveness of the planned EU directive
Venue: arXiv cs.SI 2023
Year: 2023

Globalization has had undesirable effects on the labor standards embedded in the products we consume. This paper proposes an ex-ante evaluation of supply chain due diligence regulations, such as the EU Corporate Sustainable Due Diligence Directive (CSDDD). We construct a full-scale network model derived from structural business statistics of 30 million EU firms to quantify the likelihood of links to firms potentially involved in human rights abuses in the European supply chain. The 900 million supply links of these firms are modeled in a way that is consistent with multiregional input-output data, EU import data, and stylized facts of firm-level production networks. We find that this network exhibits a small world effect with three degrees of separation, meaning that most firms are no more than three steps away from each other in the network. Consequently we find that about 8.5% of EU companies are at risk of having child or forced labor in the first tier of their supply chains, about 82.4% are likely to have such offenders at the second tier and more than 99.1% have such offenders at the third tier. We also profile companies by country, sector, and size for the likelihood of having human rights violations or child and forced labor violations at a given tier in their supply chain, revealing considerable heterogeneity across EU companies. Our results show that supply chain due diligence regulations that focus on monitoring individual buyer-supplier links, as currently proposed in the CSDDD, are likely to be ineffective due to a high degree of redundancy and the fact that individual company value chains cannot be properly isolated from the global supply network. Rather, to maximize cost-effectiveness without compromising due diligence coverage, we suggest that regulations should focus on monitoring individual suppliers.
DiffSLVA: Harnessing Diffusion Models for Sign Language Video Anonymization
Venue: arXiv cs.CV 2023
Year: 2023

Since American Sign Language (ASL) has no standard written form, Deaf signers frequently share videos in order to communicate in their native language. However, since both hands and face convey critical linguistic information in signed languages, sign language videos cannot preserve signer privacy. While signers have expressed interest, for a variety of applications, in sign language video anonymization that would effectively preserve linguistic content, attempts to develop such technology have had limited success, given the complexity of hand movements and facial expressions. Existing approaches rely predominantly on precise pose estimations of the signer in video footage and often require sign language video datasets for training. These requirements prevent them from processing videos 'in the wild,' in part because of the limited diversity present in current sign language video datasets. To address these limitations, our research introduces DiffSLVA, a novel methodology that utilizes pre-trained large-scale diffusion models for zero-shot text-guided sign language video anonymization. We incorporate ControlNet, which leverages low-level image features such as HED (Holistically-Nested Edge Detection) edges, to circumvent the need for pose estimation. Additionally, we develop a specialized module dedicated to capturing facial expressions, which are critical for conveying essential linguistic information in signed languages. We then combine the above methods to achieve anonymization that better preserves the essential linguistic content of the original signer. This innovative methodology makes possible, for the first time, sign language video anonymization that could be used for real-world applications, which would offer significant benefits to the Deaf and Hard-of-Hearing communities. We demonstrate the effectiveness of our approach with a series of signer anonymization experiments.
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
Venue: arXiv cs.CL 2023
Year: 2023

Large language models (LLMs) can potentially democratize access to medical knowledge. While many efforts have been made to harness and improve LLMs' medical knowledge and reasoning capacities, the resulting models are either closed-source (e.g., PaLM, GPT-4) or limited in scale (<= 13B parameters), which restricts their abilities. In this work, we improve access to large-scale medical LLMs by releasing MEDITRON: a suite of open-source LLMs with 7B and 70B parameters adapted to the medical domain. MEDITRON builds on Llama-2 (through our adaptation of Nvidia's Megatron-LM distributed trainer), and extends pretraining on a comprehensively curated medical corpus, including selected PubMed articles, abstracts, and internationally-recognized medical guidelines. Evaluations using four major medical benchmarks show significant performance gains over several state-of-the-art baselines before and after task-specific finetuning. Overall, MEDITRON achieves a 6% absolute performance gain over the best public baseline in its parameter class and 3% over the strongest baseline we finetuned from Llama-2. Compared to closed-source LLMs, MEDITRON-70B outperforms GPT-3.5 and Med-PaLM and is within 5% of GPT-4 and 10% of Med-PaLM-2. We release our code for curating the medical pretraining corpus and the MEDITRON model weights to drive open-source development of more capable medical LLMs.
How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs
Venue: arXiv cs.CV 2023
Year: 2023

This work focuses on the potential of Vision LLMs (VLLMs) in visual reasoning. Different from prior studies, we shift our focus from evaluating standard performance to introducing a comprehensive safety evaluation suite, covering both out-of-distribution (OOD) generalization and adversarial robustness. For the OOD evaluation, we present two novel VQA datasets, each with one variant, designed to test model performance under challenging conditions. In exploring adversarial robustness, we propose a straightforward attack strategy for misleading VLLMs to produce visual-unrelated responses. Moreover, we assess the efficacy of two jailbreaking strategies, targeting either the vision or language component of VLLMs. Our evaluation of 21 diverse models, ranging from open-source VLLMs to GPT-4V, yields interesting observations: 1) Current VLLMs struggle with OOD texts but not images, unless the visual information is limited; and 2) These VLLMs can be easily misled by deceiving vision encoders only, and their vision-language training often compromise safety protocols. We release this safety evaluation suite at https://github.com/UCSC-VLAA/vllm-safety-benchmark.
Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models
Venue: arXiv cs.CV 2023
Year: 2023

Video-based large language models (Video-LLMs) have been recently introduced, targeting both fundamental improvements in perception and comprehension, and a diverse range of user inquiries. In pursuit of the ultimate goal of achieving artificial general intelligence, a truly intelligent Video-LLM model should not only see and understand the surroundings, but also possess human-level commonsense, and make well-informed decisions for the users. To guide the development of such a model, the establishment of a robust and comprehensive evaluation system becomes crucial. To this end, this paper proposes \textit{Video-Bench}, a new comprehensive benchmark along with a toolkit specifically designed for evaluating Video-LLMs. The benchmark comprises 10 meticulously crafted tasks, evaluating the capabilities of Video-LLMs across three distinct levels: Video-exclusive Understanding, Prior Knowledge-based Question-Answering, and Comprehension and Decision-making. In addition, we introduce an automatic toolkit tailored to process model outputs for various tasks, facilitating the calculation of metrics and generating convenient final scores. We evaluate 8 representative Video-LLMs using \textit{Video-Bench}. The findings reveal that current Video-LLMs still fall considerably short of achieving human-like comprehension and analysis of real-world videos, offering valuable insights for future research directions. The benchmark and toolkit are available at: \url{https://github.com/PKU-YuanGroup/Video-Bench}.
Data Analytics with Differential Privacy
Venue: arXiv cs.LG 2023
Year: 2023

Differential privacy is the state-of-the-art definition for privacy, guaranteeing that any analysis performed on a sensitive dataset leaks no information about the individuals whose data are contained therein. In this thesis, we develop differentially private algorithms to analyze distributed and streaming data. In the distributed model, we consider the particular problem of learning -- in a distributed fashion -- a global model of the data, that can subsequently be used for arbitrary analyses. We build upon PrivBayes, a differentially private method that approximates the high-dimensional distribution of a centralized dataset as a product of low-order distributions, utilizing a Bayesian Network model. We examine three novel approaches to learning a global Bayesian Network from distributed data, while offering the differential privacy guarantee to all local datasets. Our work includes a detailed theoretical analysis of the distributed, differentially private entropy estimator which we use in one of our algorithms, as well as a detailed experimental evaluation, using both synthetic and real-world data. In the streaming model, we focus on the problem of estimating the density of a stream of users, which expresses the fraction of all users that actually appear in the stream. We offer one of the strongest privacy guarantees for the streaming model, user-level pan-privacy, which ensures that the privacy of any user is protected, even against an adversary that observes the internal state of the algorithm. We provide a detailed analysis of an existing, sampling-based algorithm for the problem and propose two novel modifications that significantly improve it, both theoretically and experimentally, by optimally using all the allocated "privacy budget."
Picogrid: An experimental platform for prosumer microgrids
Venue: arXiv cs.NI 2023
Year: 2023

The Microgrid paradigm is gaining momentum as one of the key pieces of technology for expanding clean energy access and improving energy resilience. Most of the interest in this pertains to distinct entities that either generate electricity or act as loads, i.e., distinct producers and consumers. Remote community microgrids and emerging transactive energy service models with interconnected prosumers do not clearly fit into this paradigm. Notwithstanding various publications that present concepts and simulations, there has been a dearth of experimental platforms to study them, due to practical challenges. This paper presents the `Picogrid' - an experimental platform particularly designed for dc prosumer microgrids. It is a low-power, low-cost hardware platform that enables interconnecting multiple prosumer entities in a bench-top setup. Each prosumer sends data to a cloud dashboard and can receive set points for optimal operation from a remote computer system, lending itself to use in a virtual lab setup. The platform enables implementation of custom power profiles based on real-world generation and demand datasets. Features of the platform are demonstrated using simulation and experimental results.
Community Battery Energy Storage Systems for Enhancing Distribution System Operation: A Multi-objective Optimization Approach
Venue: arXiv cs.NI 2023
Year: 2023

The growing penetration of distributed energy resources (DERs) in distribution networks (DNs) raises new operational challenges, particularly in terms of reliability and voltage regulation. In response to these challenges, we introduce an innovative DN operation framework with multi-objective optimization, leveraging community battery energy storage systems (C-BESS). The proposed framework targets two key operational objectives: first, to minimize voltage deviation, which is a concern for a distribution network service provider (DNSP), and second, to maximize the utilization of DERs on the demand side. Recognizing the conflicting nature of these objectives, we utilize C-BESS to enhance the system's adaptability to dynamically adjust DN operations. The multi-objective optimization problem is solved using the non-dominated sorting genetic algorithm-II (NSGA-II). Case studies using real-world data are conducted to validate the effectiveness of the proposed framework. The results show significant improvements in voltage regulation and DER utilization, demonstrating the potential of C-BESS in enabling more reliable DN operation. Our findings contribute to the ongoing discourse on the role of C-BESS in DN operation enhancement and DER integration.
BAGEL: Backdoor Attacks against Federated Contrastive Learning
Venue: arXiv cs.CR 2023
Year: 2023

Federated Contrastive Learning (FCL) is an emerging privacy-preserving paradigm in distributed learning for unlabeled data. In FCL, distributed parties collaboratively learn a global encoder with unlabeled data, and the global encoder could be widely used as a feature extractor to build models for many downstream tasks. However, FCL is also vulnerable to many security threats (e.g., backdoor attacks) due to its distributed nature, which are seldom investigated in existing solutions. In this paper, we study the backdoor attack against FCL as a pioneer research, to illustrate how backdoor attacks on distributed local clients act on downstream tasks. Specifically, in our system, malicious clients can successfully inject a backdoor into the global encoder by uploading poisoned local updates, thus downstream models built with this global encoder will also inherit the backdoor. We also investigate how to inject backdoors into multiple downstream models, in terms of two different backdoor attacks, namely the \textit{centralized attack} and the \textit{decentralized attack}. Experiment results show that both the centralized and the decentralized attacks can inject backdoors into downstream models effectively with high attack success rates. Finally, we evaluate two defense methods against our proposed backdoor attacks in FCL, which indicates that the decentralized backdoor attack is more stealthy and harder to defend.
Vision-Based Incoming Traffic Estimator Using Deep Neural Network on General Purpose Embedded Hardware
Venue: arXiv cs.CV 2023
Year: 2023

Traffic management is a serious problem in many cities around the world. Even the suburban areas are now experiencing regular traffic congestion. Inappropriate traffic control wastes fuel, time, and the productivity of nations. Though traffic signals are used to improve traffic flow, they often cause problems due to inappropriate or obsolete timing that does not tally with the actual traffic intensity at the intersection. Traffic intensity determination based on statistical methods only gives the average intensity expected at any given time. However, to control traffic accurately, it is required to know the real-time traffic intensity. In this research, image processing and machine learning have been used to estimate actual traffic intensity in real time. General-purpose electronic hardware has been used for in-situ image processing based on the edge-detection method. A deep neural network (DNN) was trained to infer traffic intensity in each image in real time. The trained DNN estimated traffic intensity accurately in 90% of the real-time images during road tests. The electronic system was implemented on a Raspberry Pi single-board computer; hence, it is cost-effective for large-scale deployment.
Effective Quantization for Diffusion Models on CPUs
Venue: arXiv cs.CV 2023
Year: 2023

Diffusion models have gained popularity for generating images from textual descriptions. Nonetheless, the substantial need for computational resources continues to present a noteworthy challenge, contributing to time-consuming processes. Quantization, a technique employed to compress deep learning models for enhanced efficiency, presents challenges when applied to diffusion models. These models are notably more sensitive to quantization compared to other model types, potentially resulting in a degradation of image quality. In this paper, we introduce a novel approach to quantize the diffusion models by leveraging both quantization-aware training and distillation. Our results show the quantized models can maintain the high image quality while demonstrating the inference efficiency on CPUs. The code is publicly available at: https://github.com/intel/intel-extension-for-transformers.
Investigating Lane-Free Traffic with a Dynamic Driving Simulator
Venue: arXiv cs.RO 2023
Year: 2023

Lane-free traffic (LFT) is a new traffic system that relies on connected and automated vehicles (CAV) to increase road capacity and utilization by removing traditional lane markings using coordinated maneuvering of CAVs in LFT strategies. LFT is based on two main principles: upstream nudging and vehicles moving without adhering to any lane markings. By leveraging CAV capabilities to communicate and exchange information, LFT represents a promising future traffic system. While current research uses LFT simulations in two-dimensional space, driving simulators are necessary to investigate human behavior and perceived safety in LFT. This paper proposes a conceptual framework for LFT driving simulations and describes the assumptions, requirements, and recent technological developments that make it possible to investigate the human perspective and acceptance of LFT. Additionally, we propose a scenario matrix that can act as a test guide to building driving simulation scenarios for the LFT.
Leveraging Artificial Intelligence Technology for Mapping Research to Sustainable Development Goals: A Case Study
Venue: arXiv cs.DL 2023
Year: 2023

The number of publications related to the Sustainable Development Goals (SDGs) continues to grow. These publications cover a diverse spectrum of research, from humanities and social sciences to engineering and health. Given the imperative of funding bodies to monitor outcomes and impacts, linking publications to relevant SDGs is critical but remains time-consuming and difficult given the breadth and complexity of the SDGs. A publication may relate to several goals (interconnection feature of goals), and therefore require multidisciplinary knowledge to tag accurately. Machine learning approaches are promising and have proven particularly valuable for tasks such as manual data labeling and text classification. In this study, we employed over 82,000 publications from an Australian university as a case study. We utilized a similarity measure to map these publications onto Sustainable Development Goals (SDGs). Additionally, we leveraged the OpenAI GPT model to conduct the same task, facilitating a comparative analysis between the two approaches. Experimental results show that about 82.89% of the results obtained by the similarity measure overlap (at least one tag) with the outputs of the GPT model. The adopted model (similarity measure) can complement GPT model for SDG classification. Furthermore, deep learning methods, which include the similarity measure used here, are more accessible and trusted for dealing with sensitive data without the use of commercial AI services or the deployment of expensive computing resources to operate large language models. Our study demonstrates how a crafted combination of the two methods can achieve reliable results for mapping research to the SDGs.
Conditions for Length Generalization in Learning Reasoning Skills
Venue: arXiv cs.AI 2023
Year: 2023

Reasoning is a fundamental capability of AI agents. Recently, large language models (LLMs) have shown remarkable abilities to perform reasoning tasks. However, numerous evaluations of the reasoning capabilities of LLMs have also showed some limitations. An outstanding limitation is length generalization, meaning that when trained on reasoning problems of smaller lengths or sizes, the resulting models struggle with problems of larger sizes or lengths. This potentially indicates some theoretical limitations of generalization in learning reasoning skills. These evaluations and their observations motivated us to perform a theoretical study of the length generalization problem. This work focuses on reasoning tasks that can be formulated as Markov dynamic processes (MDPs) and/or directed acyclic graphs (DAGs). It identifies and proves conditions that decide whether the length generalization problem can be solved or not for a reasoning task in a particular representation. Experiments are also conducted to verify the theoretical results.
Towards Designing Spatial Robots that are Architecturally Motivated
Venue: arXiv cs.RO 2023
Year: 2023

While robots are increasingly integrated into the built environment, little is known how their qualities can meaningfully influence our spaces to facilitate enjoyable and agreeable interaction, rather than robotic settings that are driven by functional goals. Motivated by the premise that future robots should be aware of architectural sensitivities, we developed a set of exploratory studies that combine methods from both architectural and interaction design. While we empirically discovered that dynamically moving spatial elements, which we coin as spatial robots, can indeed create unique life-sized affordances that encourage or resist human activities, we also encountered many unforeseen design challenges originated from how ordinary users and experts perceived spatial robots. This discussion thus could inform similar design studies in the areas of human-building architecture (HBI) or responsive and interactive architecture.
Reward Shaping for Improved Learning in Real-time Strategy Game Play
Venue: arXiv cs.LG 2023
Year: 2023

We investigate the effect of reward shaping in improving the performance of reinforcement learning in the context of the real-time strategy, capture-the-flag game. The game is characterized by sparse rewards that are associated with infrequently occurring events such as grabbing or capturing the flag, or tagging the opposing player. We show that appropriately designed reward shaping functions applied to different game events can significantly improve the player's performance and training times of the player's learning algorithm. We have validated our reward shaping functions within a simulated environment for playing a marine capture-the-flag game between two players. Our experimental results demonstrate that reward shaping can be used as an effective means to understand the importance of different sub-tasks during game-play towards winning the game, to encode a secondary objective functions such as energy efficiency into a player's game-playing behavior, and, to improve learning generalizable policies that can perform well against different skill levels of the opponent.