OpenRaiser/Intern-Atlas huggingface dataset:
NSF RESUME HPC Workshop: High-Performance Computing and Large-Scale Data Management in Service of Epidemiological Modeling
Venue: arXiv cs.CY 2023
Year: 2023

The NSF-funded Robust Epidemic Surveillance and Modeling (RESUME) project successfully convened a workshop entitled "High-performance computing and large-scale data management in service of epidemiological modeling" at the University of Chicago on May 1-2, 2023. This was part of a series of workshops designed to foster sustainable and interdisciplinary co-design for predictive intelligence and pandemic prevention. The event brought together 31 experts in epidemiological modeling, high-performance computing (HPC), HPC workflows, and large-scale data management to develop a shared vision for capabilities needed for computational epidemiology to better support pandemic prevention. Through the workshop, participants identified key areas in which HPC capabilities could be used to improve epidemiological modeling, particularly in supporting public health decision-making, with an emphasis on HPC workflows, data integration, and HPC access. The workshop explored nascent HPC workflow and large-scale data management approaches currently in use for epidemiological modeling and sought to draw from approaches used in other domains to determine which practices could be best adapted for use in epidemiological modeling. This report documents the key findings and takeaways from the workshop.
Topological $k$-metrics
Venue: arXiv cs.CG 2023
Year: 2023

Metric spaces $(X, d)$ are ubiquitous objects in mathematics and computer science that allow for capturing (pairwise) distance relationships $d(x, y)$ between points $x, y \in X$. Because of this, it is natural to ask what useful generalizations there are of metric spaces for capturing "$k$-wise distance relationships" $d(x_1, \ldots, x_k)$ among points $x_1, \ldots, x_k \in X$ for $k > 2$. To that end, Gähler (Math. Nachr., 1963) (and perhaps others even earlier) defined $k$-metric spaces, which generalize metric spaces, and most notably generalize the triangle inequality $d(x_1, x_2) \leq d(x_1, y) + d(y, x_2)$ to the "simplex inequality" $d(x_1, \ldots, x_k) \leq \sum_{i=1}^k d(x_1, \ldots, x_{i-1}, y, x_{i+1}, \ldots, x_k)$. (The definition holds for any fixed $k \geq 2$, and a $2$-metric space is just a (standard) metric space.) In this work, we introduce strong $k$-metric spaces, $k$-metric spaces that satisfy a topological condition stronger than the simplex inequality, which makes them "behave nicely." We also introduce coboundary $k$-metrics, which generalize $\ell_p$ metrics (and in fact all finite metric spaces induced by norms) and minimum bounding chain $k$-metrics, which generalize shortest path metrics (and capture all strong $k$-metrics). Using these definitions, we prove analogs of a number of fundamental results about embedding finite metric spaces including Fréchet embedding (isometric embedding into $\ell_{\infty}$) and isometric embedding of all tree metrics into $\ell_1$. We also study relationships between families of (strong) $k$-metrics, and show that natural quantities, like simplex volume, are strong $k$-metrics.
Maximizing Network Connectivity for UAV Communications via Reconfigurable Intelligent Surfaces
Venue: arXiv cs.IT 2023
Year: 2023

It is anticipated that integrating unmanned aerial vehicles (UAVs) with reconfigurable intelligent surfaces (RISs), resulting in RIS-assisted UAV networks, will offer improved network connectivity against node failures for the beyond 5G networks. In this context, we utilize a RIS to provide path diversity and alternative connectivity options for information flow from user equipment (UE) to UAVs by adding more links to the network, thereby maximizing its connectivity. This paper employs the algebraic connectivity metric, which is adjusted by the reflected links of the RIS, to formulate the problem of maximizing the network connectivity in two cases. First, we consider formulating the problem for one UE, which is solved optimally using a linear search. Then, we consider the problem of a more general case of multiple UEs, which has high computational complexity. To tackle this problem, we formulate the problem of maximizing the network connectivity as a semi-definite programming (SDP) optimization problem that can be solved efficiently in polynomial time. In both cases, our proposed solutions find the best combination between UE(s) and UAVs through the RIS. As a result, it tunes the phase shifts of the RIS to direct the signals of the UEs to the appropriate UAVs, thus maximizing the network connectivity. Simulation results are conducted to assess the performance of the proposed solutions compared to the existing solutions.
Sci-CoT: Leveraging Large Language Models for Enhanced Knowledge Distillation in Small Models for Scientific QA
Venue: arXiv cs.CL 2023
Year: 2023

Large Language Models (LLMs) have shown outstanding performance across wide range of downstream tasks. This competency is attributed to their substantial parameter size and pre-training on extensive corpus. Moreover, LLMs have exhibited enhanced reasoning capabilities in tackling complex reasoning tasks, owing to the utilization of a method named ``Chain-of-Thought (CoT) prompting''. This method is designed to generate intermediate reasoning steps that guide the inference of the final answer. However, it is essential to highlight that these advanced reasoning abilities appear to emerge in models with a minimum of 10 billion parameters, thereby limiting its efficacy in situations where computational resources are constrained. In this paper, we investigate the possibility of transferring the reasoning capabilities of LLMs to smaller models via knowledge distillation. Specifically, we propose Sci-CoT, a two-stage framework that separates the processes of generating rationales and inferring answers. This method enables a more efficient use of rationales during the answer inference stage, leading to improved performance on scientific question-answering tasks. Utilizing Sci-CoT, our 80-million parameter model is able to exceed the performance of BLOOM-176B in the ARC-Easy dataset under the few shot setting.
A Comparative Study of Open-Source Large Language Models, GPT-4 and Claude 2: Multiple-Choice Test Taking in Nephrology
Venue: arXiv cs.CL 2023
Year: 2023

In recent years, there have been significant breakthroughs in the field of natural language processing, particularly with the development of large language models (LLMs). These LLMs have showcased remarkable capabilities on various benchmarks. In the healthcare field, the exact role LLMs and other future AI models will play remains unclear. There is a potential for these models in the future to be used as part of adaptive physician training, medical co-pilot applications, and digital patient interaction scenarios. The ability of AI models to participate in medical training and patient care will depend in part on their mastery of the knowledge content of specific medical fields. This study investigated the medical knowledge capability of LLMs, specifically in the context of internal medicine subspecialty multiple-choice test-taking ability. We compared the performance of several open-source LLMs (Koala 7B, Falcon 7B, Stable-Vicuna 13B, and Orca Mini 13B), to GPT-4 and Claude 2 on multiple-choice questions in the field of Nephrology. Nephrology was chosen as an example of a particularly conceptually complex subspecialty field within internal medicine. The study was conducted to evaluate the ability of LLM models to provide correct answers to nephSAP (Nephrology Self-Assessment Program) multiple-choice questions. The overall success of open-sourced LLMs in answering the 858 nephSAP multiple-choice questions correctly was 17.1% - 25.5%. In contrast, Claude 2 answered 54.4% of the questions correctly, whereas GPT-4 achieved a score of 73.3%. We show that current widely used open-sourced LLMs do poorly in their ability for zero-shot reasoning when compared to GPT-4 and Claude 2. The findings of this study potentially have significant implications for the future of subspecialty medical training and patient care.
FaceSkin: A Privacy Preserving Facial skin patch Dataset for multi Attributes classification
Venue: arXiv cs.CV 2023
Year: 2023

Human facial skin images contain abundant textural information that can serve as valuable features for attribute classification, such as age, race, and gender. Additionally, facial skin images offer the advantages of easy collection and minimal privacy concerns. However, the availability of well-labeled human skin datasets with a sufficient number of images is limited. To address this issue, we introduce a dataset called FaceSkin, which encompasses a diverse range of ages and races. Furthermore, to broaden the application scenarios, we incorporate synthetic skin-patches obtained from 2D and 3D attack images, including printed paper, replays, and 3D masks. We evaluate the FaceSkin dataset across distinct categories and present experimental results demonstrating its effectiveness in attribute classification, as well as its potential for various downstream tasks, such as Face anti-spoofing and Age estimation.
SUnAA: Sparse Unmixing using Archetypal Analysis
Venue: arXiv cs.CV 2023
Year: 2023

This paper introduces a new sparse unmixing technique using archetypal analysis (SUnAA). First, we design a new model based on archetypal analysis. We assume that the endmembers of interest are a convex combination of endmembers provided by a spectral library and that the number of endmembers of interest is known. Then, we propose a minimization problem. Unlike most conventional sparse unmixing methods, here the minimization problem is non-convex. We minimize the optimization objective iteratively using an active set algorithm. Our method is robust to the initialization and only requires the number of endmembers of interest. SUnAA is evaluated using two simulated datasets for which results confirm its better performance over other conventional and advanced techniques in terms of signal-to-reconstruction error. SUnAA is also applied to Cuprite dataset and the results are compared visually with the available geological map provided for this dataset. The qualitative assessment demonstrates the successful estimation of the minerals abundances and significantly improves the detection of dominant minerals compared to the conventional regression-based sparse unmixing methods. The Python implementation of SUnAA can be found at: https://github.com/BehnoodRasti/SUnAA.
Appending Data to Blockchain is not Sufficient for Non-repudiation of Receipt
Venue: arXiv cs.CR 2023
Year: 2023

Exchanging data while ensuring non-repudiation is a challenge, especially if no trusted third party exists. Blockchain promises to provide many of the required guarantees, which is why it has been used in many non-repudiable data exchange protocols. Specifically, some authors propose to append data to blockchain to achieve non-repudiation of receipt. In this position paper, we show that this approach is insufficient. While appending data to blockchain can guarantee non-repudiation of origin in some cases, it is not sufficient for non-repudiation of receipt. For confidential data, we find a catch-22 that makes it impossible. For non-confidential data, meanwhile, plausible deniability remains. We discuss potential solutions and suggest smart contracts as a promising approach.
Enhancement of Satellite-to-Phone Link Budget by Using Distributed Beamforming
Venue: arXiv cs.NI 2023
Year: 2023

Small satellites in Low Earth Orbit (LEO) attract much attention from both industry and academia. The latest production and launch technologies constantly drive the development of LEO constellations. However, the wideband signal, except text messages, cannot be transmitted directly from an LEO satellite to a standard mobile cellular phone due to the insufficient link budget. The current LEO constellation network has to use an extra ground device to receive the signal from the satellite first and then forward the signal to the User Equipment (UE). To achieve direct network communications between LEO satellites and UE, we propose a novel distributed beamforming technology based on the superposition of electromagnetic (EM) waves radiated from multiple satellites that can significantly enhance the link budget in this paper. EM full-wave simulation and Monte Carlo simulation results are provided to verify the effectiveness of the proposed method. The simulation results show a nearly 6 dB enhancement using two radiation sources and an almost 12 dB enhancement using four sources. The received power enhancement could be doubled compared to the diversity gain in Multiple-Input and Single-Output (MISO). Furthermore, other practical application challenges, such as the synchronization and Doppler effect, are also presented.
"This (Smart) Town Ain't Big Enough": Smart Small Towns and Digital Twins for Sustainable Urban and Regional Development
Venue: arXiv cs.CY 2023
Year: 2023

One of the major challenges today lies in the creation of governance concepts for regional development that not only promote growth but, at the same time, ensure promotion of inclusiveness, fairness, and resilience. Digital twins can support policymakers in developing smart, sustainable solutions for cities and regions and, therefore, urban and non-urban environments. The project SCiNDTiLA (Smart Cities aNd Digital Twins in Lower Austria) aims to define the state-of-the-art in the field of smart cities, identify interdependencies, critical components and stakeholders, and provide a roadmap for smart cities with application to both smaller-scale urban and non-urban environments. SCiNDTiLA uses the foundations of complexity theory and computational social science methods to model Austrian towns and regions as smart cities/regions and thus as systems of socio-technical interaction to guide policy decision-making toward sustainable development.
TSSR: A Truncated and Signed Square Root Activation Function for Neural Networks
Venue: arXiv cs.CV 2023
Year: 2023

Activation functions are essential components of neural networks. In this paper, we introduce a new activation function called the Truncated and Signed Square Root (TSSR) function. This function is distinctive because it is odd, nonlinear, monotone and differentiable. Its gradient is continuous and always positive. Thanks to these properties, it has the potential to improve the numerical stability of neural networks. Several experiments confirm that the proposed TSSR has better performance than other stat-of-the-art activation functions. The proposed function has significant implications for the development of neural network models and can be applied to a wide range of applications in fields such as computer vision, natural language processing, and speech recognition.
Improved Approximation Algorithms for the Joint Replenishment Problem with Outliers, and with Fairness Constraints
Venue: arXiv cs.DS 2023
Year: 2023

The joint replenishment problem (JRP) is a classical inventory management problem. We consider a natural generalization with outliers, where we are allowed to reject (that is, not service) a subset of demand points. In this paper, we are motivated by issues of fairness - if we do not serve all of the demands, we wish to ``spread out the pain'' in a balanced way among customers, communities, or any specified market segmentation. One approach is to constrain the rejections allowed, and to have separate bounds for each given customer. In our most general setting, we consider a set of C features, where each demand point has an associated rejection cost for each feature, and we have a given bound on the allowed rejection cost incurred in total for each feature. This generalizes a model of fairness introduced in earlier work on the Colorful k-Center problem in which (analogously) each demand point has a given color, and we bound the number of rejections of each color class. We give the first constant approximation algorithms for the fairness-constrained JRP with a constant number of features; specifically, we give a 2.86-approximation algorithm in this case. Even for the special case in which we bound the total (weighted) number of outliers, this performance guarantee improves upon bounds previously known for this case. Our approach is an LP-based algorithm that splits the instance into two subinstances. One is solved by a novel iterative rounding approach and the other by pipage-based rounding. The standard LP relaxation has an unbounded integrality gap, and hence another key element of our algorithm is to strengthen the relaxation by correctly guessing key attributes of the optimal solution, which are sufficiently concise, so that we can enumerate over all possible guesses in polynomial time - albeit exponential in C, the number of features.
Emotion-Conditioned Text Generation through Automatic Prompt Optimization
Venue: arXiv cs.CL 2023
Year: 2023

Conditional natural language generation methods often require either expensive fine-tuning or training a large language model from scratch. Both are unlikely to lead to good results without a substantial amount of data and computational resources. Prompt learning without changing the parameters of a large language model presents a promising alternative. It is a cost-effective approach, while still achieving competitive results. While this procedure is now established for zero- and few-shot text classification and structured prediction, it has received limited attention in conditional text generation. We present the first automatic prompt optimization approach for emotion-conditioned text generation with instruction-fine-tuned models. Our method uses an iterative optimization procedure that changes the prompt by adding, removing, or replacing tokens. As objective function, we only require a text classifier that measures the realization of the conditional variable in the generated text. We evaluate the method on emotion-conditioned text generation with a focus on event reports and compare it to manually designed prompts that also act as the seed for the optimization procedure. The optimized prompts achieve 0.75 macro-average F1 to fulfill the emotion condition in contrast to manually designed seed prompts with only 0.22 macro-average F1.
Information-Theoretic Characterization of Vowel Harmony: A Cross-Linguistic Study on Word Lists
Venue: arXiv cs.CL 2023
Year: 2023

We present a cross-linguistic study that aims to quantify vowel harmony using data-driven computational modeling. Concretely, we define an information-theoretic measure of harmonicity based on the predictability of vowels in a natural language lexicon, which we estimate using phoneme-level language models (PLMs). Prior quantitative studies have relied heavily on inflected word-forms in the analysis of vowel harmony. We instead train our models using cross-linguistically comparable lemma forms with little or no inflection, which enables us to cover more under-studied languages. Training data for our PLMs consists of word lists with a maximum of 1000 entries per language. Despite the fact that the data we employ are substantially smaller than previously used corpora, our experiments demonstrate the neural PLMs capture vowel harmony patterns in a set of languages that exhibit this phenomenon. Our work also demonstrates that word lists are a valuable resource for typological research, and offers new possibilities for future studies on low-resource, under-studied languages.
Why Data Science Projects Fail
Venue: arXiv cs.LG 2023
Year: 2023

Data Science is a modern Data Intelligence practice, which is the core of many businesses and helps businesses build smart strategies around to deal with businesses challenges more efficiently. Data Science practice also helps in automating business processes using the algorithm, and it has several other benefits, which also deliver in a non-profitable framework. In regards to data science, three key components primarily influence the effective outcome of a data science project. Those are 1.Availability of Data 2.Algorithm 3.Processing power or infrastructure
An In-Depth Analysis of Discretization Methods for Communication Learning using Backpropagation with Multi-Agent Reinforcement Learning
Venue: arXiv cs.LG 2023
Year: 2023

Communication is crucial in multi-agent reinforcement learning when agents are not able to observe the full state of the environment. The most common approach to allow learned communication between agents is the use of a differentiable communication channel that allows gradients to flow between agents as a form of feedback. However, this is challenging when we want to use discrete messages to reduce the message size, since gradients cannot flow through a discrete communication channel. Previous work proposed methods to deal with this problem. However, these methods are tested in different communication learning architectures and environments, making it hard to compare them. In this paper, we compare several state-of-the-art discretization methods as well as a novel approach. We do this comparison in the context of communication learning using gradients from other agents and perform tests on several environments. In addition, we present COMA-DIAL, a communication learning approach based on DIAL and COMA extended with learning rate scaling and adapted exploration. Using COMA-DIAL allows us to perform experiments on more complex environments. Our results show that the novel ST-DRU method, proposed in this paper, achieves the best results out of all discretization methods across the different environments. It achieves the best or close to the best performance in each of the experiments and is the only method that does not fail on any of the tested environments.
Differentially Private Graph Neural Network with Importance-Grained Noise Adaption
Venue: arXiv cs.LG 2023
Year: 2023

Graph Neural Networks (GNNs) with differential privacy have been proposed to preserve graph privacy when nodes represent personal and sensitive information. However, the existing methods ignore that nodes with different importance may yield diverse privacy demands, which may lead to over-protect some nodes and decrease model utility. In this paper, we study the problem of importance-grained privacy, where nodes contain personal data that need to be kept private but are critical for training a GNN. We propose NAP-GNN, a node-importance-grained privacy-preserving GNN algorithm with privacy guarantees based on adaptive differential privacy to safeguard node information. First, we propose a Topology-based Node Importance Estimation (TNIE) method to infer unknown node importance with neighborhood and centrality awareness. Second, an adaptive private aggregation method is proposed to perturb neighborhood aggregation from node-importance-grain. Third, we propose to privately train a graph learning algorithm on perturbed aggregations in adaptive residual connection mode over multi-layers convolution for node-wise tasks. Theoretically analysis shows that NAP-GNN satisfies privacy guarantees. Empirical experiments over real-world graph datasets show that NAP-GNN achieves a better trade-off between privacy and accuracy.