OpenRaiser/Intern-Atlas huggingface dataset:
Temporal-spatial Correlation Attention Network for Clinical Data Analysis in Intensive Care Unit
Venue: arXiv cs.LG 2023
Year: 2023

In recent years, medical information technology has made it possible for electronic health record (EHR) to store fairly complete clinical data. This has brought health care into the era of "big data". However, medical data are often sparse and strongly correlated, which means that medical problems cannot be solved effectively. With the rapid development of deep learning in recent years, it has provided opportunities for the use of big data in healthcare. In this paper, we propose a temporal-saptial correlation attention network (TSCAN) to handle some clinical characteristic prediction problems, such as predicting death, predicting length of stay, detecting physiologic decline, and classifying phenotypes. Based on the design of the attention mechanism model, our approach can effectively remove irrelevant items in clinical data and irrelevant nodes in time according to different tasks, so as to obtain more accurate prediction results. Our method can also find key clinical indicators of important outcomes that can be used to improve treatment options. Our experiments use information from the Medical Information Mart for Intensive Care (MIMIC-IV) database, which is open to the public. Finally, we have achieved significant performance benefits of 2.0\% (metric) compared to other SOTA prediction methods. We achieved a staggering 90.7\% on mortality rate, 45.1\% on length of stay. The source code can be find: \url{https://github.com/yuyuheintju/TSCAN}.
Algebraic Smooth Occluding Contours
Venue: arXiv cs.GR 2023
Year: 2023

Computing occluding contours is a key building block of non-photorealistic rendering, but producing contours with consistent visibility has been notoriously challenging. This paper describes the first general-purpose smooth surface construction for which the occluding contours can be computed in closed form. For a given input mesh and camera viewpoint, we produce a $G^1$ piecewise-quadratic surface approximating the mesh. We show how the image-space occluding contours of this representation may then be described as piecewise rational curves. We show that this method produces smooth contours with consistent visibility much more efficiently than the state-of-the-art.
Optimized Vectorization Implementation of CRYSTALS-Dilithium
Venue: arXiv cs.CR 2023
Year: 2023

CRYSTALS-Dilithium is a lattice-based signature scheme to be standardized by NIST as the primary post-quantum signature algorithm. In this work, we make a thorough study of optimizing the implementations of Dilithium by utilizing the Advanced Vector Extension (AVX) instructions, specifically AVX2 and the latest AVX-512. We first present an improved parallel small polynomial multiplication with tailored early evaluation (PSPM-TEE) to further speed up the signing procedure. Our PSPM algorithm outperform the NTT by 47%-66% in AVX2 and AVX-512 implementation. We then present a tailored reduction method that is simpler and faster than Montgomery reduction. We minimize the CPU cycles of tailored reduction AVX-512 implementation by using AVX-512IFMA. Finally, we propose a fully and highly vectorized implementation of Dilithium using AVX-512. This is achieved by carefully vectorizing most of Dilithium functions with the AVX-512 instructions in order to improve efficiency both for time and for space simultaneously. With all the optimization efforts, our AVX-512 implementation improves the performance by 43.2%/39.3%/45.6% in key generation, 36.6%/41.6%/43.7% in signing, and 45.3%/46.5%/47.4% in verification for the parameter sets of Dilithium2/3/5 respectively. To the best of our knowledge, our AVX-512 implementation has the best performance for Dilithium on the Intel x86-64 CPU platform to date.
Generative Adversarial Networks for Data Augmentation
Venue: arXiv cs.AI 2023
Year: 2023

One way to expand the available dataset for training AI models in the medical field is through the use of Generative Adversarial Networks (GANs) for data augmentation. GANs work by employing a generator network to create new data samples that are then assessed by a discriminator network to determine their similarity to real samples. The discriminator network is taught to differentiate between actual and synthetic samples, while the generator system is trained to generate data that closely resemble real ones. The process is repeated until the generator network can produce synthetic data that is indistinguishable from genuine data. GANs have been utilized in medical image analysis for various tasks, including data augmentation, image creation, and domain adaptation. They can generate synthetic samples that can be used to increase the available dataset, especially in cases where obtaining large amounts of genuine data is difficult or unethical. However, it is essential to note that the use of GANs in medical imaging is still an active area of research to ensure that the produced images are of high quality and suitable for use in clinical settings.
ADMM-based Detector for Large-scale MIMO Code-domain NOMA Systems
Venue: arXiv cs.IT 2023
Year: 2023

Large-scale multi-input multi-output (MIMO) code domain non-orthogonal multiple access (CD-NOMA) techniques are one of the potential candidates to address the next-generation wireless needs such as massive connectivity, and high reliability. This work focuses on two primary CD-NOMA techniques: sparse-code multiple access (SCMA) and dense-code multiple access (DCMA). One of the primary challenges in implementing MIMO-CD-NOMA systems is designing the optimal detector with affordable computation cost and complexity. This paper proposes an iterative linear detector based on the alternating direction method of multipliers (ADMM). First, the maximum likelihood (ML) detection problem is converted into a sharing optimization problem. The set constraint in the ML detection problem is relaxed into the box constraint sharing problem. An alternative variable is introduced via the penalty term, which compensates for the loss incurred by the constraint relaxation. The system models, i.e., the relation between the input signal and the received signal, are reformulated so that the proposed sharing optimization problem can be readily applied. The ADMM is a robust algorithm to solve the sharing problem in a distributed manner. The proposed detector leverages the distributive nature to reduce per-iteration cost and time. An ADMM-based linear detector is designed for three MIMO-CD-NOMA systems: single input multi output CD-NOMA (SIMO-CD-NOMA), spatial multiplexing CD-NOMA (SMX-CD-NOMA), and spatial modulated CD-NOMA (SM-CD-NOMA). The impact of various system parameters and ADMM parameters on computational complexity and symbol error rate (SER) has been thoroughly examined through extensive Monte Carlo simulations.
On the Empirical Evidence of Microservice Logical Coupling. A Registered Report
Venue: arXiv cs.SE 2023
Year: 2023

[Context] Coupling is a widely discussed metric by software engineers while developing complex software systems, often referred to as a crucial factor and symptom of a poor or good design. Nevertheless, measuring the logical coupling among microservices and analyzing the interactions between services is non-trivial because it demands runtime information in the form of log files, which are not always accessible. [Objective and Method] In this work, we propose the design of a study aimed at empirically validating the Microservice Logical Coupling (MLC) metric presented in our previous study. In particular, we plan to empirically study Open Source Systems (OSS) built using a microservice architecture. [Results] The result of this work aims at corroborating the effectiveness and validity of the MLC metric. Thus, we will gather empirical evidence and develop a methodology to analyze and support the claims regarding the MLC metric. Furthermore, we establish its usefulness in evaluating and understanding the logical coupling among microservices.
Unsupervised Low Light Image Enhancement Using SNR-Aware Swin Transformer
Venue: arXiv cs.CV 2023
Year: 2023

Image captured under low-light conditions presents unpleasing artifacts, which debilitate the performance of feature extraction for many upstream visual tasks. Low-light image enhancement aims at improving brightness and contrast, and further reducing noise that corrupts the visual quality. Recently, many image restoration methods based on Swin Transformer have been proposed and achieve impressive performance. However, on one hand, trivially employing Swin Transformer for low-light image enhancement would expose some artifacts, including over-exposure, brightness imbalance and noise corruption, etc. On the other hand, it is impractical to capture image pairs of low-light images and corresponding ground-truth, i.e. well-exposed image in same visual scene. In this paper, we propose a dual-branch network based on Swin Transformer, guided by a signal-to-noise ratio prior map which provides the spatial-varying information for low-light image enhancement. Moreover, we leverage unsupervised learning to construct the optimization objective based on Retinex model, to guide the training of proposed network. Experimental results demonstrate that the proposed model is competitive with the baseline models.
Simulating Noisy Quantum Circuits for Cryptographic Algorithms
Venue: arXiv cs.CR 2023
Year: 2023

The emergence of noisy intermediate-scale quantum (NISQ) computers has important consequences for cryptographic algorithms. It is theoretically well-established that key algorithms used in cybersecurity are vulnerable to quantum computers due to the fact that theoretical security guarantees, designed based on algorithmic complexity for classical computers, are not sufficient for quantum circuits. Many different quantum algorithms have been developed, which have potentially broad applications on future computing systems. However, this potential depends on the continued maturation of quantum hardware, which remains an area of active research and development. Theoretical limits provide an upper bound on the performance for algorithms. In practice, threats to encryption can only be accurately be assessed in the context of the rapidly evolving hardware and software landscape. Software co-design refers to the concurrent design of software and hardware as a way to understand the limitations of current capabilities and develop effective strategies to advance the state of the art. Since the capabilities for classical computation currently exceed quantum capabilities, quantum emulation techniques can play an important role in the co-design process. In this paper, we describe how the {\em cuQuantum} environment can support quantum algorithm co-design activities using widely-available commodity hardware. We describe how emulation techniques can be used to assess the impact of noise on algorithms of interest, and identify limitations associated with current hardware. We present our analysis in the context of areas of priority for cybersecurity and cryptography in particular since these algorithms are extraordinarily consequential for securing information in the digital world.
A two-way translation system of Chinese sign language based on computer vision
Venue: arXiv cs.CV 2023
Year: 2023

As the main means of communication for deaf people, sign language has a special grammatical order, so it is meaningful and valuable to develop a real-time translation system for sign language. In the research process, we added a TSM module to the lightweight neural network model for the large Chinese continuous sign language dataset . It effectively improves the network performance with high accuracy and fast recognition speed. At the same time, we improve the Bert-Base-Chinese model to divide Chinese sentences into words and mapping the natural word order to the statute sign language order, and finally use the corresponding word videos in the isolated sign language dataset to generate the sentence video, so as to achieve the function of text-to-sign language translation. In the last of our research we built a system with sign language recognition and translation functions, and conducted performance tests on the complete dataset. The sign language video recognition accuracy reached about 99.3% with a time of about 0.05 seconds, and the sign language generation video time was about 1.3 seconds. The sign language system has good performance performance and is feasible.
On the Budgeted Hausdorff Distance Problem
Venue: arXiv cs.CG 2023
Year: 2023

$ \newcommand{\Re}{\mathbb{R}} \newcommand{\reals}{\mathbb{R}} \newcommand{\SetX}{\mathsf{X}} \newcommand{\optX}[1]{#1^\star} \newcommand{\Qopt}{\Mh{\optX{Q}}} \newcommand{\rad}{r} \newcommand{\Mh}[1]{#1} \newcommand{\query}{q} \newcommand{\eps}{\varepsilon} \newcommand{\VorX}[1]{\mathcal{V} \pth{#1}} \newcommand{Polygon}{\mathsf{P}} \newcommand{\IntRange}[1]{[ #1 ]} \newcommand{\Space}{\overline{\mathsf{m}}} \newcommand{\pth}[2][\!]{#1\left({#2}\right)} \newcommand{\polylog}{\mathrm{polylog}} \newcommand{\N}{\mathbb N} \newcommand{\Z}{\mathbb Z} \newcommand{\pt}{p} \newcommand{\distY}[2]{\left\| {#1} - {#2} \right\|} \newcommand{\ptq}{q} \newcommand{\RunningTime}{O\bigl(n^{3/2} \sqrt{k} \log^{3/2} n + kn \log^2 n\bigr)} \newcommand{\pts}{s}$ Given a set $P$ of $n$ points in the plane, and a parameter $k$, we present an algorithm, whose running time is $\RunningTime$, with high probability, that computes a subset $\Qopt \subseteq P$ of $k$ points, that minimizes the Hausdorff distance between the convex-hulls of $\Qopt$ and $P$. This is the first subquadratic algorithm for this problem if $k$ is small.
The effects of increasing velocity on the tractive performance of planetary rovers
Venue: arXiv cs.RO 2023
Year: 2023

An emerging paradigm is being embraced in the conceptualization of future planetary exploration missions. Ambitious objectives and increasingly demanding mission constraints stress the importance associated with faster surface mobility. Driving speeds approaching or surpassing 1 m/s have been rarely used and their effect on performance is today unclear. This study presents experimental evidence and preliminary observations on the impact that increasing velocity has on the tractive performance of planetary rovers. Single-wheel driving tests were conducted using two different metallic, grousered wheels-one rigid and one flexible-over two different soils, olivine sand and CaCO3-based silty soil. Experiments were conducted at speeds between 0.01-1 m/s throughout an ample range of slip ratios (5-90%). Three performance metrics were evaluated: drawbar pull coefficient, wheel sinkage, and tractive efficiency. Results showed similar data trends among all the cases investigated. Drawbar pull and tractive efficiency considerably decreased for speeds beyond 0.2 m/s. Wheel sinkage, unlike what published evidence suggested, increased with increasing velocities. The flexible wheel performed the best at 1m/s, exhibiting 2 times higher drawbar pull and efficiency with 18% lower sinkage under low slip conditions. Although similar data trends were obtained, a different wheel-soil interactive behavior was observed when driving over the different soils. Overall, despite the performance reduction experienced at higher velocities, a speed in the range of 0.2-0.3 m/s would enable 5-10 times faster traverses, compared to current rovers driving capability, while only diminishing drawbar pull and efficiency by 7%. The measurements collected and the analysis presented here lay the groundwork for initial stages in the development of new locomotion subsystems for planetary surface exploration. At the same time...
Stubborn Lexical Bias in Data and Models
Venue: arXiv cs.CL 2023
Year: 2023

In NLP, recent work has seen increased focus on spurious correlations between various features and labels in training data, and how these influence model behavior. However, the presence and effect of such correlations are typically examined feature by feature. We investigate the cumulative impact on a model of many such intersecting features. Using a new statistical method, we examine whether such spurious patterns in data appear in models trained on the data. We select two tasks -- natural language inference and duplicate-question detection -- for which any unigram feature on its own should ideally be uninformative, which gives us a large pool of automatically extracted features with which to experiment. The large size of this pool allows us to investigate the intersection of features spuriously associated with (potentially different) labels. We then apply an optimization approach to *reweight* the training data, reducing thousands of spurious correlations, and examine how doing so affects models trained on the reweighted data. Surprisingly, though this method can successfully reduce lexical biases in the training data, we still find strong evidence of corresponding bias in the trained models, including worsened bias for slightly more complex features (bigrams). We close with discussion about the implications of our results on what it means to "debias" training data, and how issues of data quality can affect model bias.
ATEM: A Topic Evolution Model for the Detection of Emerging Topics in Scientific Archives
Venue: arXiv cs.IR 2023
Year: 2023

This paper presents ATEM, a novel framework for studying topic evolution in scientific archives. ATEM is based on dynamic topic modeling and dynamic graph embedding techniques that explore the dynamics of content and citations of documents within a scientific corpus. ATEM explores a new notion of contextual emergence for the discovery of emerging interdisciplinary research topics based on the dynamics of citation links in topic clusters. Our experiments show that ATEM can efficiently detect emerging cross-disciplinary topics within the DBLP archive of over five million computer science articles.
ROME: Testing Image Captioning Systems via Recursive Object Melting
Venue: arXiv cs.SE 2023
Year: 2023

Image captioning (IC) systems aim to generate a text description of the salient objects in an image. In recent years, IC systems have been increasingly integrated into our daily lives, such as assistance for visually-impaired people and description generation in Microsoft Powerpoint. However, even the cutting-edge IC systems (e.g., Microsoft Azure Cognitive Services) and algorithms (e.g., OFA) could produce erroneous captions, leading to incorrect captioning of important objects, misunderstanding, and threats to personal safety. The existing testing approaches either fail to handle the complex form of IC system output (i.e., sentences in natural language) or generate unnatural images as test cases. To address these problems, we introduce Recursive Object MElting (Rome), a novel metamorphic testing approach for validating IC systems. Different from existing approaches that generate test cases by inserting objects, which easily make the generated images unnatural, Rome melts (i.e., remove and inpaint) objects. Rome assumes that the object set in the caption of an image includes the object set in the caption of a generated image after object melting. Given an image, Rome can recursively remove its objects to generate different pairs of images. We use Rome to test one widely-adopted image captioning API and four state-of-the-art (SOTA) algorithms. The results show that the test cases generated by Rome look much more natural than the SOTA IC testing approach and they achieve comparable naturalness to the original images. Meanwhile, by generating test pairs using 226 seed images, Rome reports a total of 9,121 erroneous issues with high precision (86.47%-92.17%). In addition, we further utilize the test cases generated by Rome to retrain the Oscar, which improves its performance across multiple evaluation metrics.
Fine-Tuning Language Models with Advantage-Induced Policy Alignment
Venue: arXiv cs.CL 2023
Year: 2023

Reinforcement learning from human feedback (RLHF) has emerged as a reliable approach to aligning large language models (LLMs) to human preferences. Among the plethora of RLHF techniques, proximal policy optimization (PPO) is of the most widely used methods. Despite its popularity, however, PPO may suffer from mode collapse, instability, and poor sample efficiency. We show that these issues can be alleviated by a novel algorithm that we refer to as Advantage-Induced Policy Alignment (APA), which leverages a squared error loss function based on the estimated advantages. We demonstrate empirically that APA consistently outperforms PPO in language tasks by a large margin, when a separate reward model is employed as the evaluator. In addition, compared with PPO, APA offers a more stable form of control over the deviation from the model's initial policy, ensuring that the model improves its performance without collapsing to deterministic output. In addition to empirical results, we also provide a theoretical justification supporting the design of our loss function.
Large Language Model Augmented Narrative Driven Recommendations
Venue: arXiv cs.IR 2023
Year: 2023

Narrative-driven recommendation (NDR) presents an information access problem where users solicit recommendations with verbose descriptions of their preferences and context, for example, travelers soliciting recommendations for points of interest while describing their likes/dislikes and travel circumstances. These requests are increasingly important with the rise of natural language-based conversational interfaces for search and recommendation systems. However, NDR lacks abundant training data for models, and current platforms commonly do not support these requests. Fortunately, classical user-item interaction datasets contain rich textual data, e.g., reviews, which often describe user preferences and context - this may be used to bootstrap training for NDR models. In this work, we explore using large language models (LLMs) for data augmentation to train NDR models. We use LLMs for authoring synthetic narrative queries from user-item interactions with few-shot prompting and train retrieval models for NDR on synthetic queries and user-item interaction data. Our experiments demonstrate that this is an effective strategy for training small-parameter retrieval models that outperform other retrieval and LLM baselines for narrative-driven recommendation.
Proteus: Simulating the Performance of Distributed DNN Training
Venue: arXiv cs.DC 2023
Year: 2023

DNN models are becoming increasingly larger to achieve unprecedented accuracy, and the accompanying increased computation and memory requirements necessitate the employment of massive clusters and elaborate parallelization strategies to accelerate DNN training. In order to better optimize the performance and analyze the cost, it is indispensable to model the training throughput of distributed DNN training. However, complex parallelization strategies and the resulting complex runtime behaviors make it challenging to construct an accurate performance model. In this paper, we present Proteus, the first standalone simulator to model the performance of complex parallelization strategies through simulation execution. Proteus first models complex parallelization strategies with a unified representation named Strategy Tree. Then, it compiles the strategy tree into a distributed execution graph and simulates the complex runtime behaviors, comp-comm overlap and bandwidth sharing, with a Hierarchical Topo-Aware Executor (HTAE). We finally evaluate Proteus across a wide variety of DNNs on three hardware configurations. Experimental results show that Proteus achieves $3.0\%$ average prediction error and preserves order for training throughput of various parallelization strategies. Compared to state-of-the-art approaches, Proteus reduces prediction error by up to $133.8\%$.