OpenRaiser/Intern-Atlas huggingface dataset:
HNMblock: Blockchain technology powered Healthcare Network Model for epidemiological monitoring, medical systems security, and wellness
Venue: arXiv cs.CR 2024
Year: 2024

In the ever-evolving healthcare sector, the widespread adoption of Internet of Things and wearable technologies facilitates remote patient monitoring. However, the existing client/server infrastructure poses significant security and privacy challenges, necessitating strict adherence to healthcare data regulations. To combat these issues, a decentralized approach is imperative, and blockchain technology emerges as a compelling solution for strengthening Internet of Things and medical systems security. This paper introduces HNMblock, a model that elevates the realms of epidemiological monitoring, medical system security, and wellness enhancement. By harnessing the transparency and immutability inherent in blockchain, HNMblock empowers real-time, tamper-proof tracking of epidemiological data, enabling swift responses to disease outbreaks. Furthermore, it fortifies the security of medical systems through advanced cryptographic techniques and smart contracts, with a paramount focus on safeguarding patient privacy. HNMblock also fosters personalized healthcare, encouraging patient involvement and data-informed decision-making. The integration of blockchain within the healthcare domain, as exemplified by HNMblock, holds the potential to revolutionize data management, epidemiological surveillance, and wellness, as meticulously explored in this research article.
Fast UCB-type algorithms for stochastic bandits with heavy and super heavy symmetric noise
Venue: arXiv cs.LG 2024
Year: 2024

In this study, we propose a new method for constructing UCB-type algorithms for stochastic multi-armed bandits based on general convex optimization methods with an inexact oracle. We derive the regret bounds corresponding to the convergence rates of the optimization methods. We propose a new algorithm Clipped-SGD-UCB and show, both theoretically and empirically, that in the case of symmetric noise in the reward, we can achieve an $O(\log T\sqrt{KT\log T})$ regret bound instead of $O\left (T^{\frac{1}{1+α}} K^{\fracα{1+α}} \right)$ for the case when the reward distribution satisfies $\mathbb{E}_{X \in D}[|X|^{1+α}] \leq σ^{1+α}$ ($α\in (0, 1])$, i.e. perform better than it is assumed by the general lower bound for bandits with heavy-tails. Moreover, the same bound holds even when the reward distribution does not have the expectation, that is, when $α<0$.
Modeling of Key Quality Indicators for End-to-End Network Management: Preparing for 5G
Venue: arXiv cs.NI 2024
Year: 2024

Thanks to evolving cellular telecommunication networks, providers can deploy a wide range of services. Soon, 5G mobile networks will be available to handle all types of services and applications for vast numbers of users through their mobile equipment. To effectively manage new 5G systems, end-to-end (E2E) performance analysis and optimization will be key features. However, estimating the end-user experience is not an easy task for network operators. The amount of end-user performance information operators can measure from the network is limited, complicating this approach. Here we explore the calculation of service metrics [known as key quality indicators (KQIs)] from classic low-layer measurements and parameters. We propose a complete machine-learning (ML) modeling framework. This system's low-layer metrics can be applied to measure service-layer performance. To assess the approach, we implemented and evaluated the proposed system on a real cellular network testbed.
Echoes of Socratic Doubt: Embracing Uncertainty in Calibrated Evidential Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

We present a novel statistical approach to incorporating uncertainty awareness in model-free distributional reinforcement learning involving quantile regression-based deep Q networks. The proposed algorithm, $\textit{Calibrated Evidential Quantile Regression in Deep Q Networks (CEQR-DQN)}$, aims to address key challenges associated with separately estimating aleatoric and epistemic uncertainty in stochastic environments. It combines deep evidential learning with quantile calibration based on principles of conformal inference to provide explicit, sample-free computations of $\textit{global}$ uncertainty as opposed to $\textit{local}$ estimates based on simple variance, overcoming limitations of traditional methods in computational and statistical efficiency and handling of out-of-distribution (OOD) observations. Tested on a suite of miniaturized Atari games (i.e., MinAtar), CEQR-DQN is shown to surpass similar existing frameworks in scores and learning speed. Its ability to rigorously evaluate uncertainty improves exploration strategies and can serve as a blueprint for other algorithms requiring uncertainty awareness.
Towards Quantifying the Preconditioning Effect of Adam
Venue: arXiv cs.LG 2024
Year: 2024

There is a notable dearth of results characterizing the preconditioning effect of Adam and showing how it may alleviate the curse of ill-conditioning -- an issue plaguing gradient descent (GD). In this work, we perform a detailed analysis of Adam's preconditioning effect for quadratic functions and quantify to what extent Adam can mitigate the dependence on the condition number of the Hessian. Our key finding is that Adam can suffer less from the condition number but at the expense of suffering a dimension-dependent quantity. Specifically, for a $d$-dimensional quadratic with a diagonal Hessian having condition number $κ$, we show that the effective condition number-like quantity controlling the iteration complexity of Adam without momentum is $\mathcal{O}(\min(d, κ))$. For a diagonally dominant Hessian, we obtain a bound of $\mathcal{O}(\min(d \sqrt{d κ}, κ))$ for the corresponding quantity. Thus, when $d < \mathcal{O}(κ^p)$ where $p = 1$ for a diagonal Hessian and $p = 1/3$ for a diagonally dominant Hessian, Adam can outperform GD (which has an $\mathcal{O}(κ)$ dependence). On the negative side, our results suggest that Adam can be worse than GD for a sufficiently non-diagonal Hessian even if $d \ll \mathcal{O}(κ^{1/3})$; we corroborate this with empirical evidence. Finally, we extend our analysis to functions satisfying per-coordinate Lipschitz smoothness and a modified version of the Polyak-Łojasiewicz condition.
Natural Language Reinforcement Learning
Venue: arXiv cs.CL 2024
Year: 2024

Reinforcement Learning (RL) has shown remarkable abilities in learning policies for decision-making tasks. However, RL is often hindered by issues such as low sample efficiency, lack of interpretability, and sparse supervision signals. To tackle these limitations, we take inspiration from the human learning process and introduce Natural Language Reinforcement Learning (NLRL), which innovatively combines RL principles with natural language representation. Specifically, NLRL redefines RL concepts like task objectives, policy, value function, Bellman equation, and policy iteration in natural language space. We present how NLRL can be practically implemented with the latest advancements in large language models (LLMs) like GPT-4. Initial experiments over tabular MDPs demonstrate the effectiveness, efficiency, and also interpretability of the NLRL framework.
Link-aware link prediction over temporal graph by pattern recognition
Venue: arXiv cs.AI 2024
Year: 2024

A temporal graph can be considered as a stream of links, each of which represents an interaction between two nodes at a certain time. On temporal graphs, link prediction is a common task, which aims to answer whether the query link is true or not. To do this task, previous methods usually focus on the learning of representations of the two nodes in the query link. We point out that the learned representation by their models may encode too much information with side effects for link prediction because they have not utilized the information of the query link, i.e., they are link-unaware. Based on this observation, we propose a link-aware model: historical links and the query link are input together into the following model layers to distinguish whether this input implies a reasonable pattern that ends with the query link. During this process, we focus on the modeling of link evolution patterns rather than node representations. Experiments on six datasets show that our model achieves strong performances compared with state-of-the-art baselines, and the results of link prediction are interpretable. The code and datasets are available on the project website: https://github.com/lbq8942/TGACN.
Towards Fast Stochastic Sampling in Diffusion Generative Models
Venue: arXiv cs.LG 2024
Year: 2024

Diffusion models suffer from slow sample generation at inference time. Despite recent efforts, improving the sampling efficiency of stochastic samplers for diffusion models remains a promising direction. We propose Splitting Integrators for fast stochastic sampling in pre-trained diffusion models in augmented spaces. Commonly used in molecular dynamics, splitting-based integrators attempt to improve sampling efficiency by cleverly alternating between numerical updates involving the data, auxiliary, or noise variables. However, we show that a naive application of splitting integrators is sub-optimal for fast sampling. Consequently, we propose several principled modifications to naive splitting samplers for improving sampling efficiency and denote the resulting samplers as Reduced Splitting Integrators. In the context of Phase Space Langevin Diffusion (PSLD) [Pandey \& Mandt, 2023] on CIFAR-10, our stochastic sampler achieves an FID score of 2.36 in only 100 network function evaluations (NFE) as compared to 2.63 for the best baselines.
CPSDBench: A Large Language Model Evaluation Benchmark and Baseline for Chinese Public Security Domain
Venue: arXiv cs.AI 2024
Year: 2024

Large Language Models (LLMs) have demonstrated significant potential and effectiveness across multiple application domains. To assess the performance of mainstream LLMs in public security tasks, this study aims to construct a specialized evaluation benchmark tailored to the Chinese public security domain--CPSDbench. CPSDbench integrates datasets related to public security collected from real-world scenarios, supporting a comprehensive assessment of LLMs across four key dimensions: text classification, information extraction, question answering, and text generation. Furthermore, this study introduces a set of innovative evaluation metrics designed to more precisely quantify the efficacy of LLMs in executing tasks related to public security. Through the in-depth analysis and evaluation conducted in this research, we not only enhance our understanding of the performance strengths and limitations of existing models in addressing public security issues but also provide references for the future development of more accurate and customized LLM models targeted at applications in this field.
Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Survey
Venue: arXiv cs.LG 2024
Year: 2024

The precise prediction of molecular properties is essential for advancements in drug development, particularly in virtual screening and compound optimization. The recent introduction of numerous deep learning-based methods has shown remarkable potential in enhancing molecular property prediction (MPP), especially improving accuracy and insights into molecular structures. Yet, two critical questions arise: does the integration of domain knowledge augment the accuracy of molecular property prediction and does employing multi-modal data fusion yield more precise results than unique data source methods? To explore these matters, we comprehensively review and quantitatively analyze recent deep learning methods based on various benchmarks. We discover that integrating molecular information significantly improves molecular property prediction (MPP) for both regression and classification tasks. Specifically, regression improvements, measured by reductions in root mean square error (RMSE), are up to 4.0%, while classification enhancements, measured by the area under the receiver operating characteristic curve (ROC-AUC), are up to 1.7%. We also discover that enriching 2D graphs with 1D SMILES boosts multi-modal learning performance for regression tasks by up to 9.1%, and augmenting 2D graphs with 3D information increases performance for classification tasks by up to 13.2%, with both enhancements measured using ROC-AUC. The two consolidated insights offer crucial guidance for future advancements in drug discovery.
Data Quality Aware Approaches for Addressing Model Drift of Semantic Segmentation Models
Venue: arXiv cs.CV 2024
Year: 2024

In the midst of the rapid integration of artificial intelligence (AI) into real world applications, one pressing challenge we confront is the phenomenon of model drift, wherein the performance of AI models gradually degrades over time, compromising their effectiveness in real-world, dynamic environments. Once identified, we need techniques for handling this drift to preserve the model performance and prevent further degradation. This study investigates two prominent quality aware strategies to combat model drift: data quality assessment and data conditioning based on prior model knowledge. The former leverages image quality assessment metrics to meticulously select high-quality training data, improving the model robustness, while the latter makes use of learned feature vectors from existing models to guide the selection of future data, aligning it with the model's prior knowledge. Through comprehensive experimentation, this research aims to shed light on the efficacy of these approaches in enhancing the performance and reliability of semantic segmentation models, thereby contributing to the advancement of computer vision capabilities in real-world scenarios.
CLIPPER: Robust Data Association without an Initial Guess
Venue: arXiv cs.RO 2024
Year: 2024

Identifying correspondences in noisy data is a critically important step in estimation processes. When an informative initial estimation guess is available, the data association challenge is less acute; however, the existence of a high-quality initial guess is rare in most contexts. We explore graph-theoretic formulations for data association, which do not require an initial estimation guess. Existing graph-theoretic approaches optimize over unweighted graphs, discarding important consistency information encoded in weighted edges, and frequently attempt to solve NP-hard problems exactly. In contrast, we formulate a new optimization problem that fully leverages weighted graphs and seeks the densest edge-weighted clique. We introduce two relaxations to this problem: a convex semidefinite relaxation which we find to be empirically tight, and a fast first-order algorithm called CLIPPER which frequently arrives at nearly-optimal solutions in milliseconds. When evaluated on point cloud registration problems, our algorithms remain robust up to at least 95% outliers while existing algorithms begin breaking down at 80% outliers. Code is available at https://mit-acl.github.io/clipper.
Fundamental Problems on Bounded-Treewidth Graphs: The Real Source of Hardness
Venue: arXiv cs.CC 2024
Year: 2024

It is known for many algorithmic problems that if a tree decomposition of width $t$ is given in the input, then the problem can be solved with exponential dependence on $t$. A line of research by Lokshtanov, Marx, and Saurabh [SODA 2011] produced lower bounds showing that in many cases known algorithms achieve the best possible exponential dependence on $t$, assuming the SETH. The main message of our paper is showing that the same lower bounds can be obtained in a more restricted setting: a graph consisting of a block of $t$ vertices connected to components of constant size already has the same hardness as a general tree decomposition of width $t$. Formally, a $(σ,δ)$-hub is a set $Q$ of vertices such that every component of $Q$ has size at most $σ$ and is adjacent to at most $δ$ vertices of $Q$. $\bullet$ For every $ε> 0$, there are $σ,δ> 0$ such that Independent Set/Vertex Cover cannot be solved in time $(2-ε)^p\cdot n$, even if a $(σ,δ)$-hub of size $p$ is given in the input, assuming the SETH. This matches the earlier tight lower bounds parameterized by the width of the tree decomposition. Similar tight bounds are obtained for Odd Cycle Transversal, Max Cut, $q$-Coloring, and edge/vertex deletions versions of $q$-Coloring. $\bullet$ For every $ε>0$, there are $σ,δ> 0$ such that Triangle-Partition cannot be solved in time $(2-ε)^p\cdot n$, even if a $(σ,δ)$-hub of size $p$ is given in the input, assuming the Set Cover Conjecture (SCC). In fact, we prove that this statement is equivalent to the SCC, thus it is unlikely that this could be proved assuming the SETH. $\bullet$ For Dominating Set, we can prove a non-tight lower bound ruling out $(2-ε)^p\cdot n^{O(1)}$ algorithms, assuming either the SETH or the SCC, but this does not match the $3^p\cdot n^{O(1)}$ upper bound.
Beyond the Headlines: Understanding Sentiments and Morals Impacting Female Employment in Spain
Venue: arXiv cs.CY 2024
Year: 2024

After decades of improvements in the employment conditions of females in Spain, this process came to a sudden stop with the Great Spanish Recession of 2008. In this contribution, we analyse a large longitudinal corpus of national and regional news outlets employing advanced Natural Language Processing techniques to capture the valence of mentions of gender inequality expressed in the Spanish press. The automatic analysis of the news articles does indeed capture the known hardships faced by females in the Spanish labour market. Our approach can be straightforwardly generalised to other topics of interest. Assessing the sentiment and moral values expressed in the articles, we notice that females are, in the majority of cases, concerned more than males when there is a deterioration in the overall labour market conditions, based on newspaper articles. This behaviour has been present in the entire period of study (2000--2022) and looked particularly pronounced during the economic crisis of 2008 and the recent COVID-19 pandemic. Most of the time, this phenomenon looks to be more pronounced at the regional level, perhaps caused by a significant focus on local labour markets rather than on aggregate statistics or because, in local contexts, females might suffer more from an isolation or discrimination condition. Our findings contribute to a deeper understanding of the gender inequalities in Spain using alternative data, informing policymakers and stakeholders.
Data Distribution-based Curriculum Learning
Venue: arXiv cs.LG 2024
Year: 2024

The order of training samples can have a significant impact on the performance of a classifier. Curriculum learning is a method of ordering training samples from easy to hard. This paper proposes the novel idea of a curriculum learning approach called Data Distribution-based Curriculum Learning (DDCL). DDCL uses the data distribution of a dataset to build a curriculum based on the order of samples. Two types of scoring methods known as DDCL (Density) and DDCL (Point) are used to score training samples thus determining their training order. DDCL (Density) uses the sample density to assign scores while DDCL (Point) utilises the Euclidean distance for scoring. We evaluate the proposed DDCL approach by conducting experiments on multiple datasets using a neural network, support vector machine and random forest classifier. Evaluation results show that the application of DDCL improves the average classification accuracy for all datasets compared to standard evaluation without any curriculum. Moreover, analysis of the error losses for a single training epoch reveals that convergence is faster when using DDCL over the no curriculum method.
A Novel Gaussian Min-Max Theorem and its Applications
Venue: arXiv cs.LG 2024
Year: 2024

A celebrated result by Gordon allows one to compare the min-max behavior of two Gaussian processes if certain inequality conditions are met. The consequences of this result include the Gaussian min-max (GMT) and convex Gaussian min-max (CGMT) theorems which have had far-reaching implications in high-dimensional statistics, machine learning, non-smooth optimization, and signal processing. Both theorems rely on a pair of Gaussian processes, first identified by Slepian, that satisfy Gordon's comparison inequalities. In this paper, we identify such a new pair. The resulting theorems extend the classical GMT and CGMT Theorems from the case where the underlying Gaussian matrix in the primary process has iid rows to where it has independent but non-identically-distributed ones. The new CGMT is applied to the problems of multi-source Gaussian regression, as well as to binary classification of general Gaussian mixture models.
Structured Satellite-UAV-Terrestrial Networks for 6G Internet of Things
Venue: arXiv cs.IT 2024
Year: 2024

The upcoming sixth generation (6G) wireless communication network is envisioned to cover space, air, and maritime areas, in addition to urban-centered terrestrial coverage by the fifth generation (5G) network, to support intelligent Internet of Things (IoT). Towards this end, we investigate structured integration of satellites, unmanned aerial vehicles (UAVs), and terrestrial networks, aiming to serve future universal IoT possibly with a massive number of devices in the coverage holes of current 5G. The hybrid satellite-UAV-terrestrial network usually leads to high system complexity, due to the heterogeneity and dynamics of space/air/ground links. With a systematic thinking, we propose to create and exploit hierarchies for the integrated network. Four basic structures are discussed by learning from the synergies in our human body. To orchestrate multiple heterogeneous basic structures, we further propose a process-oriented on-demand coverage method, which characterizes the system behavior as a series of events over time and is able to tackle the system complexity elaborately. We also outline open issues for promoting the agility and intelligence of structured satellite-UAV-terrestrial networks in the making.
Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate
Venue: arXiv cs.CL 2024
Year: 2024

Fact-checking research has extensively explored verification but less so the generation of natural-language explanations, crucial for user trust. While Large Language Models (LLMs) excel in text generation, their capability for producing faithful explanations in fact-checking remains underexamined. Our study investigates LLMs' ability to generate such explanations, finding that zero-shot prompts often result in unfaithfulness. To address these challenges, we propose the Multi-Agent Debate Refinement (MADR) framework, leveraging multiple LLMs as agents with diverse roles in an iterative refining process aimed at enhancing faithfulness in generated explanations. MADR ensures that the final explanation undergoes rigorous validation, significantly reducing the likelihood of unfaithful elements and aligning closely with the provided evidence. Experimental results demonstrate that MADR significantly improves the faithfulness of LLM-generated explanations to the evidence, advancing the credibility and trustworthiness of these explanations.