OpenRaiser/Intern-Atlas huggingface dataset:
Generative Large Language Models Are All-purpose Text Analytics Engines: Text-to-text Learning Is All Your Need
Venue: arXiv cs.CL 2023
Year: 2023

Objective To solve major clinical natural language processing (NLP) tasks using a unified text-to-text learning architecture based on a generative large language model (LLM) via prompt tuning. Methods We formulated 7 key clinical NLP tasks as text-to-text learning and solved them using one unified generative clinical LLM, GatorTronGPT, developed using GPT-3 architecture and trained with up to 20 billion parameters. We adopted soft prompts (i.e., trainable vectors) with frozen LLM, where the LLM parameters were not updated (i.e., frozen) and only the vectors of soft prompts were updated, known as prompt tuning. We added additional soft prompts as a prefix to the input layer, which were optimized during the prompt tuning. We evaluated the proposed method using 7 clinical NLP tasks and compared them with previous task-specific solutions based on Transformer models. Results and Conclusion The proposed approach achieved state-of-the-art performance for 5 out of 7 major clinical NLP tasks using one unified generative LLM. Our approach outperformed previous task-specific transformer models by ~3% for concept extraction and 7% for relation extraction applied to social determinants of health, 3.4% for clinical concept normalization, 3.4~10% for clinical abbreviation disambiguation, and 5.5~9% for natural language inference. Our approach also outperformed a previously developed prompt-based machine reading comprehension (MRC) model, GatorTron-MRC, for clinical concept and relation extraction. The proposed approach can deliver the ``one model for all`` promise from training to deployment using a unified generative LLM.
Stellar: Systematic Evaluation of Human-Centric Personalized Text-to-Image Methods
Venue: arXiv cs.CV 2023
Year: 2023

In this work, we systematically study the problem of personalized text-to-image generation, where the output image is expected to portray information about specific human subjects. E.g., generating images of oneself appearing at imaginative places, interacting with various items, or engaging in fictional activities. To this end, we focus on text-to-image systems that input a single image of an individual to ground the generation process along with text describing the desired visual context. Our first contribution is to fill the literature gap by curating high-quality, appropriate data for this task. Namely, we introduce a standardized dataset (Stellar) that contains personalized prompts coupled with images of individuals that is an order of magnitude larger than existing relevant datasets and where rich semantic ground-truth annotations are readily available. Having established Stellar to promote cross-systems fine-grained comparisons further, we introduce a rigorous ensemble of specialized metrics that highlight and disentangle fundamental properties such systems should obey. Besides being intuitive, our new metrics correlate significantly more strongly with human judgment than currently used metrics on this task. Last but not least, drawing inspiration from the recent works of ELITE and SDXL, we derive a simple yet efficient, personalized text-to-image baseline that does not require test-time fine-tuning for each subject and which sets quantitatively and in human trials a new SoTA. For more information, please visit our project's website: https://stellar-gen-ai.github.io/.
ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank
Venue: arXiv cs.CV 2023
Year: 2023

Artistic style transfer aims to repaint the content image with the learned artistic style. Existing artistic style transfer methods can be divided into two categories: small model-based approaches and pre-trained large-scale model-based approaches. Small model-based approaches can preserve the content strucuture, but fail to produce highly realistic stylized images and introduce artifacts and disharmonious patterns; Pre-trained large-scale model-based approaches can generate highly realistic stylized images but struggle with preserving the content structure. To address the above issues, we propose ArtBank, a novel artistic style transfer framework, to generate highly realistic stylized images while preserving the content structure of the content images. Specifically, to sufficiently dig out the knowledge embedded in pre-trained large-scale models, an Implicit Style Prompt Bank (ISPB), a set of trainable parameter matrices, is designed to learn and store knowledge from the collection of artworks and behave as a visual prompt to guide pre-trained large-scale models to generate highly realistic stylized images while preserving content structure. Besides, to accelerate training the above ISPB, we propose a novel Spatial-Statistical-based self-Attention Module (SSAM). The qualitative and quantitative experiments demonstrate the superiority of our proposed method over state-of-the-art artistic style transfer methods.
Compute-in-Memory based Neural Network Accelerators for Safety-Critical Systems: Worst-Case Scenarios and Protections
Venue: arXiv cs.LG 2023
Year: 2023

Emerging non-volatile memory (NVM)-based Computing-in-Memory (CiM) architectures show substantial promise in accelerating deep neural networks (DNNs) due to their exceptional energy efficiency. However, NVM devices are prone to device variations. Consequently, the actual DNN weights mapped to NVM devices can differ considerably from their targeted values, inducing significant performance degradation. Many existing solutions aim to optimize average performance amidst device variations, which is a suitable strategy for general-purpose conditions. However, the worst-case performance that is crucial for safety-critical applications is largely overlooked in current research. In this study, we define the problem of pinpointing the worst-case performance of CiM DNN accelerators affected by device variations. Additionally, we introduce a strategy to identify a specific pattern of the device value deviations in the complex, high-dimensional value deviation space, responsible for this worst-case outcome. Our findings reveal that even subtle device variations can precipitate a dramatic decline in DNN accuracy, posing risks for CiM-based platforms in supporting safety-critical applications. Notably, we observe that prevailing techniques to bolster average DNN performance in CiM accelerators fall short in enhancing worst-case scenarios. In light of this issue, we propose a novel worst-case-aware training technique named A-TRICE that efficiently combines adversarial training and noise-injection training with right-censored Gaussian noise to improve the DNN accuracy in the worst-case scenarios. Our experimental results demonstrate that A-TRICE improves the worst-case accuracy under device variations by up to 33%.
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
Venue: arXiv cs.LG 2023
Year: 2023

Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct a multi-task model that performs well across diverse tasks. Recent research, exemplified by task arithmetic, highlights that this multi-task model can be derived through arithmetic operations on task vectors. Nevertheless, current merging techniques frequently resolve potential conflicts among parameters from task-specific models by evaluating individual attributes, such as the parameters' magnitude or sign, overlooking their collective impact on the overall functionality of the model. In this work, we propose the CONtinuous relaxation of disCRETE (Concrete) subspace learning method to identify a common low-dimensional subspace and utilize its shared information to track the interference problem without sacrificing much performance. Specifically, we model the problem as a bi-level optimization problem and introduce a meta-learning framework to find the Concrete subspace mask through gradient-based techniques. At the upper level, we focus on learning a shared Concrete mask to identify the subspace, while at the inner level, model merging is performed to maximize the performance of the merged model. We conduct extensive experiments on both vision domain and language domain, and the results demonstrate the effectiveness of our method. The code is available at https://github.com/tanganke/subspace_fusion
Recent Advances in Deterministic Human Motion Prediction: A Review
Venue: arXiv cs.CV 2023
Year: 2023

In recent years, with the continuous advancement of deep learning and the emergence of large-scale human motion datasets, human motion prediction technology has gradually gained prominence in various fields such as human-computer interaction, autonomous driving, sports analysis, and personnel tracking. This article introduces common model architectures in this domain along with their respective advantages and disadvantages. It also systematically summarizes recent research innovations, focusing on in-depth discussions of relevant papers in these areas, thereby highlighting forward-looking insights into the field's development. Furthermore, this paper provides a comprehensive overview of existing methods, commonly used datasets, and evaluation metrics in this field. Finally, it discusses some of the current limitations in the field and proposes potential future research directions to address these challenges and promote further advancements in human motion prediction.
Transforms for Multiplicative and Fractional Programming with Broad Applications in Edge Computing and Communication Networks
Venue: arXiv cs.CC 2023
Year: 2023

Multiplicative Programming (MP) pertains to a spectrum of optimization problems that involve product term(s). As computational paradigms of communication systems continue to evolve, particularly concerning the offloading strategies of computationally intensive tasks simultaneously to centralized or decentralized servers, designing or optimizing effective communication systems with MP techniques becomes increasingly indispensable. Similarly, Fractional Programming (FP) is another significant branch in the optimization domain, addressing various essential scenarios in communication. For instance, in minimization optimization problems, transmission power and processing delay of communication systems are considered critical metrics. In a very recent JSAC paper by Zhao et al. [2], an innovative transform (Zhao's Optimization Transform) was proposed for solving the minimization of MP and FP problems. Nevertheless, the resolution of optimization problems in communication systems encounters several limitations when adopting Zhao's optimization transform, especially in MP problems. Primarily, objective functions proposed in these optimization problems typically involve sum-of-products terms and the optimization variables are always discrete leading to NP-hard problems. Furthermore, multiple functions mapping to the non-negative domain in these scenarios can result in auxiliary variables being zero values, while the same situation is avoidable in FP problems due to the presence of these functions in the denominator. In this paper, we introduce an updated transform, building on the foundations of Zhao's original method, designed to effectively overcome these challenges by reformulating the original problem into a series of convex or concave problems. This introduced problem reformulation provides a superior iteration algorithm with demonstrable convergence to a stationary point.
A Primer on RecoNIC: RDMA-enabled Compute Offloading on SmartNIC
Venue: arXiv cs.DC 2023
Year: 2023

Today's data centers consist of thousands of network-connected hosts, each with CPUs and accelerators such as GPUs and FPGAs. These hosts also contain network interface cards (NICs), operating at speeds of 100Gb/s or higher, that are used to communicate with each other. We propose RecoNIC, an FPGA-based RDMA-enabled SmartNIC platform that is designed for compute acceleration while minimizing the overhead associated with data copies (in CPU-centric accelerator systems) by bringing network data as close to computation as possible. Since RDMA is the defacto transport-layer protocol for improved communication in data center workloads, RecoNIC includes an RDMA offload engine for high throughput and low latency data transfers. Developers have the flexibility to design their accelerators using RTL, HLS or Vitis Networking P4 within the RecoNIC's programmable compute blocks. These compute blocks can access host memory as well as memory in remote peers through the RDMA offload engine. Furthermore, the RDMA offload engine is shared by both the host and compute blocks, which makes RecoNIC a very flexible platform. Lastly, we have open-sourced RecoNIC for the research community to enable experimentation with RDMA-based applications and use-cases.
Interpretable Long Term Waypoint-Based Trajectory Prediction Model
Venue: arXiv cs.AI 2023
Year: 2023

Predicting the future trajectories of dynamic agents in complex environments is crucial for a variety of applications, including autonomous driving, robotics, and human-computer interaction. It is a challenging task as the behavior of the agent is unknown and intrinsically multimodal. Our key insight is that the agents behaviors are influenced not only by their past trajectories and their interaction with their immediate environment but also largely with their long term waypoint (LTW). In this paper, we study the impact of adding a long-term goal on the performance of a trajectory prediction framework. We present an interpretable long term waypoint-driven prediction framework (WayDCM). WayDCM first predict an agent's intermediate goal (IG) by encoding his interactions with the environment as well as his LTW using a combination of a Discrete choice Model (DCM) and a Neural Network model (NN). Then, our model predicts the corresponding trajectories. This is in contrast to previous work which does not consider the ultimate intent of the agent to predict his trajectory. We evaluate and show the effectiveness of our approach on the Waymo Open dataset.
Security and Reliability Evaluation of Countermeasures implemented using High-Level Synthesis
Venue: arXiv cs.CR 2023
Year: 2023

As the complexity of digital circuits increases, High-Level Synthesis (HLS) is becoming a valuable tool to increase productivity and design reuse by utilizing relevant Electronic Design Automation (EDA) flows, either for Application-Specific Integrated Circuits (ASIC) or for Field Programmable Gate Arrays (FPGA). Side Channel Analysis (SCA) and Fault Injection (FI) attacks are powerful hardware attacks, capable of greatly weakening the theoretical security levels of secure implementations. Furthermore, critical applications demand high levels of reliability including fault tolerance. The lack of security and reliability driven optimizations in HLS tools makes it necessary for the HLS-based designs to validate that the properties of the algorithm and the countermeasures have not been compromised due to the HLS flow. In this work, we provide results on the resilience evaluation of HLS-based FPGA implementations for the aforementioned threats. As a test case, we use multiple versions of an on-the-fly SBOX algorithm integrating different countermeasures (hiding and masking), written in C and implemented using Vivado HLS. We perform extensive evaluations for all the designs and their optimization scenarios. The results provide evidence of issues arising due to HLS optimizations on the security and the reliability of cryptographic implementations. Furthermore, the results put HLS algorithms to the test of designing secure accelerators and can lead to improving them towards the goal of increasing productivity in the domain of secure and reliable cryptographic implementations.
Rank-Metric Codes and Their Parameters
Venue: arXiv cs.IT 2023
Year: 2023

We present the theory of linear rank-metric codes from the point of view of their fundamental parameters. These are: the minimum rank distance, the rank distribution, the maximum rank, the covering radius, and the field size. The focus of this chapter is on the interplay among these parameters and on their significance for the code's (combinatorial) structure. The results covered in this chapter span from the theory of optimal codes and anticodes to very recent developments on the asymptotic density of MRD codes.
HoLLiE C -- A Multifunctional Bimanual Mobile Robot Supporting Versatile Care Applications
Venue: arXiv cs.RO 2023
Year: 2023

Care robotics as a research field has developed a lot in recent years, driven by the rapidly increasing need for it. However, these technologies are mostly limited to a very concrete and usually relatively simple use case. The bimanual robot House of Living Labs intelligent Escort (HoLLiE) includes an omnidirectional mobile platform. This paper presents how HoLLiE is adapted, by flexible software and hardware modules, for different care applications. The design goal of HoLLiE was to be human-like but abstract enough to ensure a high level of acceptance, which is very advantageous for its use in hospitals. After a short retrospect of previous generations of HoLLiE, it is highlighted how the current version is equipped with a variety of additional sensors and actuators to allow a wide range of possible applications. Then, the software stack of HoLLiE is depicted, with the focus on navigation and force sensitive intention recognition.
Measuring Approximate Functional Dependencies: a Comparative Study
Venue: arXiv cs.DB 2023
Year: 2023

Approximate functional dependencies (AFDs) are functional dependencies (FDs) that "almost" hold in a relation. While various measures have been proposed to quantify the level to which an FD holds approximately, they are difficult to compare and it is unclear which measure is preferable when one needs to discover FDs in real-world data, i.e., data that only approximately satisfies the FD. In response, this paper formally and qualitatively compares AFD measures. We obtain a formal comparison through a novel presentation of measures in terms of Shannon and logical entropy. Qualitatively, we perform a sensitivity analysis w.r.t. structural properties of input relations and quantitatively study the effectiveness of AFD measures for ranking AFDs on real world data. Based on this analysis, we give clear recommendations for the AFD measures to use in practice.
BoschAI @ Causal News Corpus 2023: Robust Cause-Effect Span Extraction using Multi-Layer Sequence Tagging and Data Augmentation
Venue: arXiv cs.CL 2023
Year: 2023

Understanding causality is a core aspect of intelligence. The Event Causality Identification with Causal News Corpus Shared Task addresses two aspects of this challenge: Subtask 1 aims at detecting causal relationships in texts, and Subtask 2 requires identifying signal words and the spans that refer to the cause or effect, respectively. Our system, which is based on pre-trained transformers, stacked sequence tagging, and synthetic data augmentation, ranks third in Subtask 1 and wins Subtask 2 with an F1 score of 72.8, corresponding to a margin of 13 pp. to the second-best system.
PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization
Venue: arXiv cs.CV 2023
Year: 2023

Recent advancements in personalized image generation using diffusion models have been noteworthy. However, existing methods suffer from inefficiencies due to the requirement for subject-specific fine-tuning. This computationally intensive process hinders efficient deployment, limiting practical usability. Moreover, these methods often grapple with identity distortion and limited expression diversity. In light of these challenges, we propose PortraitBooth, an innovative approach designed for high efficiency, robust identity preservation, and expression-editable text-to-image generation, without the need for fine-tuning. PortraitBooth leverages subject embeddings from a face recognition model for personalized image generation without fine-tuning. It eliminates computational overhead and mitigates identity distortion. The introduced dynamic identity preservation strategy further ensures close resemblance to the original image identity. Moreover, PortraitBooth incorporates emotion-aware cross-attention control for diverse facial expressions in generated images, supporting text-driven expression editing. Its scalability enables efficient and high-quality image creation, including multi-subject generation. Extensive results demonstrate superior performance over other state-of-the-art methods in both single and multiple image generation scenarios.
Embodied Carbon Accounting through Spatial-Temporal Embodied Carbon Models
Venue: arXiv cs.AR 2023
Year: 2023

Embodied carbon is the total carbon released from the processes associated with a product from cradle to gate. In many industry sectors, embodied carbon dominates the overall carbon footprint. Embodied carbon accounting, i.e., to estimate the embodied carbon of a product, has become an important research topic. Existing studies derive the embodied carbon through life cycle analysis (LCA) reports. Current LCA reports only provide the carbon emission of a product class, e.g., 28nm CPU, yet a product instance can be manufactured from diverse regions and in diverse time periods, e.g., a winter period of Ireland (Intel). It is known that the carbon emission depends on the electricity generation process which has spatial and temporal dynamics. Therefore, the embodied carbon of a specific product instance can largely differ from its product class. In this paper, we present new spatial-temporal embodied carbon models for embodied carbon accounting. We observe significant differences between current embodied carbon models and our spatial-temporal embodied carbon models, e.g., for 7nm CPU the difference can be 13.69%.
SciCat: A Curated Dataset of Scientific Software Repositories
Venue: arXiv cs.SE 2023
Year: 2023

The proliferation of open-source scientific software for science and research presents opportunities and challenges. In this paper, we introduce the SciCat dataset -- a comprehensive collection of Free-Libre Open Source Software (FLOSS) projects, designed to address the need for a curated repository of scientific and research software. This collection is crucial for understanding the creation of scientific software and aiding in its development. To ensure extensive coverage, our approach involves selecting projects from a pool of 131 million deforked repositories from the World of Code data source. Subsequently, we analyze README.md files using OpenAI's advanced language models. Our classification focuses on software designed for scientific purposes, research-related projects, and research support software. The SciCat dataset aims to become an invaluable tool for researching science-related software, shedding light on emerging trends, prevalent practices, and challenges in the field of scientific software development. Furthermore, it includes data that can be linked to the World of Code, GitHub, and other platforms, providing a solid foundation for conducting comparative studies between scientific and non-scientific software.