OpenRaiser/Intern-Atlas huggingface dataset:
ClusterLog: Clustering Logs for Effective Log-based Anomaly Detection
Venue: arXiv cs.DC 2023
Year: 2023

With the increasing prevalence of scalable file systems in the context of High Performance Computing (HPC), the importance of accurate anomaly detection on runtime logs is increasing. But as it currently stands, many state-of-the-art methods for log-based anomaly detection, such as DeepLog, have encountered numerous challenges when applied to logs from many parallel file systems (PFSes), often due to their irregularity and ambiguity in time-based log sequences. To circumvent these problems, this study proposes ClusterLog, a log pre-processing method that clusters the temporal sequence of log keys based on their semantic similarity. By grouping semantically and sentimentally similar logs, this approach aims to represent log sequences with the smallest amount of unique log keys, intending to improve the ability of a downstream sequence-based model to effectively learn the log patterns. The preliminary results of ClusterLog indicate not only its effectiveness in reducing the granularity of log sequences without the loss of important sequence information but also its generalizability to different file systems' logs.
DECISIVE Benchmarking Data Report: sUAS Performance Results from Phase I
Venue: arXiv cs.RO 2023
Year: 2023

This report reviews all results derived from performance benchmarking conducted during Phase I of the Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations (DECISIVE) project by the University of Massachusetts Lowell, using the test methods specified in the DECISIVE Test Methods Handbook v1.1 for evaluating small unmanned aerial systems (sUAS) performance in subterranean and constrained indoor environments, spanning communications, field readiness, interface, obstacle avoidance, navigation, mapping, autonomy, trust, and situation awareness. Using those 20 test methods, over 230 tests were conducted across 8 sUAS platforms: Cleo Robotics Dronut X1P (P = prototype), FLIR Black Hornet PRS, Flyability Elios 2 GOV, Lumenier Nighthawk V3, Parrot ANAFI USA GOV, Skydio X2D, Teal Golden Eagle, and Vantage Robotics Vesper. Best in class criteria is specified for each applicable test method and the sUAS that match this criteria are named for each test method, including a high-level executive summary of their performance.
Joint Service Caching and Computing Resource Allocation for Edge Computing-Enabled Networks
Venue: arXiv cs.NI 2023
Year: 2023

In this paper, we consider the service caching and the computing resource allocation in edge computing (EC) enabled networks. We introduce a random service caching design considering multiple types of latency sensitive services and the base stations (BSs)' service caching storage. We then derive a successful service probability (SSP). We also formulate a SSP maximization problem subject to the service caching distribution and the computing resource allocation. Then, we show that the optimization problem is nonconvex and develop a novel algorithm to obtain the stationary point of the SSP maximization problem by adopting the parallel successive convex approximation (SCA). Moreover, to further reduce the computational complexity, we also provide a low complex algorithm that can obtain the near-optimal solution of the SSP maximization problem in high computing capability region. Finally, from numerical simulations, we show that proposed solutions achieve higher SSP than baseline schemes. Moreover, we show that the near-optimal solution achieves reliable performance in the high computing capability region. We also explore the impacts of target delays, a BSs' service cache size, and an EC servers' computing capability on the SSP.
Subject-Independent Classification of Brain Signals using Skip Connections
Venue: arXiv cs.HC 2023
Year: 2023

Untapped potential for new forms of human-to-human communication can be found in the active research field of studies on the decoding of brain signals of human speech. A brain-computer interface system can be implemented using electroencephalogram signals because it poses more less clinical risk and can be acquired using portable instruments. One of the most interesting tasks for the brain-computer interface system is decoding words from the raw electroencephalogram signals. Before a brain-computer interface may be used by a new user, current electroencephalogram-based brain-computer interface research typically necessitates a subject-specific adaption stage. In contrast, the subject-independent situation is one that is highly desired since it allows a well-trained model to be applied to new users with little or no precalibration. The emphasis is on creating an efficient decoder that may be employed adaptively in subject-independent circumstances in light of this crucial characteristic. Our proposal is to explicitly apply skip connections between convolutional layers to enable the flow of mutual information between layers. To do this, we add skip connections between layers, allowing the mutual information to flow throughout the layers. The output of the encoder is then passed through the fully-connected layer to finally represent the probabilities of the 13 classes. In this study, overt speech was used to record the electroencephalogram data of 16 participants. The results show that when the skip connection is present, the classification performance improves notably.
Job recommendations: benchmarking of collaborative filtering methods for classifieds
Venue: arXiv cs.IR 2023
Year: 2023

Classifieds provide many challenges for recommendation methods, due to the limited information regarding users and items. In this paper, we explore recommendation methods for classifieds using the example of OLX Jobs. The goal of the paper is to benchmark different recommendation methods for jobs classifieds in order to improve advertisements' conversion rate and user satisfaction. In our research, we implemented methods that are scalable and represent different approaches to recommendation, namely ALS, LightFM, Prod2Vec, RP3beta, and SLIM. We performed a laboratory comparison of methods with regard to accuracy, diversity, and scalability (memory and time consumption during training and in prediction). Online A/B tests were also carried out by sending millions of messages with recommendations to evaluate models in a real-world setting. In addition, we have published the dataset that we created for the needs of our research. To the best of our knowledge, this is the first dataset of this kind. The dataset contains 65,502,201 events performed on OLX Jobs by 3,295,942 users, who interacted with (displayed, replied to, or bookmarked) 185,395 job ads in two weeks of 2020. We demonstrate that RP3beta, SLIM, and ALS perform significantly better than Prod2Vec and LightFM when tested in a laboratory setting. Online A/B tests also demonstrated that sending messages with recommendations generated by the ALS and RP3beta models increases the number of users contacting advertisers. Additionally, RP3beta had a 20% greater impact on this metric than ALS.
On the Ergodic Secrecy Capacity of Reconfigurable Intelligent Surface Aided Wireless Systems Under Mixture Gamma Fading
Venue: arXiv cs.IT 2023
Year: 2023

This paper presents a quantified assessment of the physical layer security capabilities of reconfigurable intelligent surface (RIS)-aided wireless systems under eavesdropping. Specifically, we derive a closed-form expression for the ergodic secrecy capacity (ESC) that is adaptable to different types of fading and RIS size. The channels between the transmitter (TX) and RIS, the RIS and legitimate receiver as well as the TX and eavesdropper are assumed to follow independent mixture Gamma (MG) distributions. Note that MG is capable of modeling a large variety of well-known distributions, including Gaussian, Rayleigh, Nakagami-m, Rice, and others. The results reveal that as the RIS size increases, although the legitimate links diversity order increases, the ESC gain decreases.
Global Nash Equilibrium in Non-convex Multi-player Game: Theory and Algorithms
Venue: arXiv cs.GT 2023
Year: 2023

Wide machine learning tasks can be formulated as non-convex multi-player games, where Nash equilibrium (NE) is an acceptable solution to all players, since no one can benefit from changing its strategy unilaterally. Attributed to the non-convexity, obtaining the existence condition of global NE is challenging, let alone designing theoretically guaranteed realization algorithms. This paper takes conjugate transformation to the formulation of non-convex multi-player games, and casts the complementary problem into a variational inequality (VI) problem with a continuous pseudo-gradient mapping. We then prove the existence condition of global NE: the solution to the VI problem satisfies a duality relation. Based on this VI formulation, we design a conjugate-based ordinary differential equation (ODE) to approach global NE, which is proved to have an exponential convergence rate. To make the dynamics more implementable, we further derive a discretized algorithm. We apply our algorithm to two typical scenarios: multi-player generalized monotone game and multi-player potential game. In the two settings, we prove that the step-size setting is required to be $\mathcal{O}(1/k)$ and $\mathcal{O}(1/\sqrt k)$ to yield the convergence rates of $\mathcal{O}(1/ k)$ and $\mathcal{O}(1/\sqrt k)$, respectively. Extensive experiments in robust neural network training and sensor localization are in full agreement with our theory.
Identification, explanation and clinical evaluation of hospital patient subtypes
Venue: arXiv cs.LG 2023
Year: 2023

We present a pipeline in which unsupervised machine learning techniques are used to automatically identify subtypes of hospital patients admitted between 2017 and 2021 in a large UK teaching hospital. With the use of state-of-the-art explainability techniques, the identified subtypes are interpreted and assigned clinical meaning. In parallel, clinicians assessed intra-cluster similarities and inter-cluster differences of the identified patient subtypes within the context of their clinical knowledge. By confronting the outputs of both automatic and clinician-based explanations, we aim to highlight the mutual benefit of combining machine learning techniques with clinical expertise.
xURLCC in 6g with meshed RAN
Venue: arXiv cs.NI 2023
Year: 2023

5G Ultra-Reliable Low Latency Communications Technology (URLLC) will not be able to provide extremely reliable low latency services to the complex networks in 6G. Moreover, URLLC that began with 5G has to be refined and improved in 6G to provide xURLCC (extreme URLCC) with sub-millisecond latency, for supporting diverse mission-critical applications. This paper aims to highlight the importance of peer-to-peer mesh connectivity for services that require xURLLC. Deploying mesh connectivity among RAN nodes would add significant value to the current 5G New Radio (5G NR) enabling 6G to increase flexibility and reliability of the networks while reducing the inherent latency introduced by the core network. To provide a mesh connectivity in RAN, the nodes should be able to communicate with each other directly and be independent from the mobile core network so that data can be directly exchanged between base stations (gNBs) whereas certain aspects of signalling procedure including data session establishment will be managed by RAN itself. In this paper, we introduce several architectural choices for a mesh network topology that could potentially be crucial to a number of applications. In addition, three possible options to create mesh connectivity in RAN are provided, and their pros and cons are discussed in detail.
Position Regression for Unsupervised Anomaly Detection
Venue: arXiv cs.CV 2023
Year: 2023

In recent years, anomaly detection has become an essential field in medical image analysis. Most current anomaly detection methods for medical images are based on image reconstruction. In this work, we propose a novel anomaly detection approach based on coordinate regression. Our method estimates the position of patches within a volume, and is trained only on data of healthy subjects. During inference, we can detect and localize anomalies by considering the error of the position estimate of a given patch. We apply our method to 3D CT volumes and evaluate it on patients with intracranial haemorrhages and cranial fractures. The results show that our method performs well in detecting these anomalies. Furthermore, we show that our method requires less memory than comparable approaches that involve image reconstruction. This is highly relevant for processing large 3D volumes, for instance, CT or MRI scans.
Everything is Connected: Graph Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

In many ways, graphs are the main modality of data we receive from nature. This is due to the fact that most of the patterns we see, both in natural and artificial systems, are elegantly representable using the language of graph structures. Prominent examples include molecules (represented as graphs of atoms and bonds), social networks and transportation networks. This potential has already been seen by key scientific and industrial groups, with already-impacted application areas including traffic forecasting, drug discovery, social network analysis and recommender systems. Further, some of the most successful domains of application for machine learning in previous years -- images, text and speech processing -- can be seen as special cases of graph representation learning, and consequently there has been significant exchange of information between these areas. The main aim of this short survey is to enable the reader to assimilate the key concepts in the area, and position graph representation learning in a proper context with related fields.
ETLP: Event-based Three-factor Local Plasticity for online learning with neuromorphic hardware
Venue: arXiv cs.NE 2023
Year: 2023

Neuromorphic perception with event-based sensors, asynchronous hardware and spiking neurons is showing promising results for real-time and energy-efficient inference in embedded systems. The next promise of brain-inspired computing is to enable adaptation to changes at the edge with online learning. However, the parallel and distributed architectures of neuromorphic hardware based on co-localized compute and memory imposes locality constraints to the on-chip learning rules. We propose in this work the Event-based Three-factor Local Plasticity (ETLP) rule that uses (1) the pre-synaptic spike trace, (2) the post-synaptic membrane voltage and (3) a third factor in the form of projected labels with no error calculation, that also serve as update triggers. We apply ETLP with feedforward and recurrent spiking neural networks on visual and auditory event-based pattern recognition, and compare it to Back-Propagation Through Time (BPTT) and eProp. We show a competitive performance in accuracy with a clear advantage in the computational complexity for ETLP. We also show that when using local plasticity, threshold adaptation in spiking neurons and a recurrent topology are necessary to learn spatio-temporal patterns with a rich temporal structure. Finally, we provide a proof of concept hardware implementation of ETLP on FPGA to highlight the simplicity of its computational primitives and how they can be mapped into neuromorphic hardware for online learning with low-energy consumption and real-time interaction.
PyOED: An Extensible Suite for Data Assimilation and Model-Constrained Optimal Design of Experiments
Venue: arXiv cs.MS 2023
Year: 2023

This paper describes PyOED, a highly extensible scientific package that enables developing and testing model-constrained optimal experimental design (OED) for inverse problems. Specifically, PyOED aims to be a comprehensive Python toolkit for model-constrained OED. The package targets scientists and researchers interested in understanding the details of OED formulations and approaches. It is also meant to enable researchers to experiment with standard and innovative OED technologies with a wide range of test problems (e.g., simulation models). OED, inverse problems (e.g., Bayesian inversion), and data assimilation (DA) are closely related research fields, and their formulations overlap significantly. Thus, PyOED is continuously being expanded with a plethora of Bayesian inversion, DA, and OED methods as well as new scientific simulation models, observation error models, and observation operators. These pieces are added such that they can be permuted to enable testing OED methods in various settings of varying complexities. The PyOED core is completely written in Python and utilizes the inherent object-oriented capabilities; however, the current version of PyOED is meant to be extensible rather than scalable. Specifically, PyOED is developed to enable rapid development and benchmarking of OED methods with minimal coding effort and to maximize code reutilization. This paper provides a brief description of the PyOED layout and philosophy and provides a set of exemplary test cases and tutorials to demonstrate the potential of the package.
A Quantum EL Theorem
Venue: arXiv cs.CC 2023
Year: 2023

In this paper, we prove a quantum version of the EL Theorem. It states that non-exotic projections of large rank must have simple quantum states in their images. A consequence to this is there is no way to communicate a quantum source with corresponding large enough von Neumann entropy without using simple quantum states.
Projective Integral Updates for High-Dimensional Variational Inference
Venue: arXiv cs.LG 2023
Year: 2023

Variational inference is an approximation framework for Bayesian inference that seeks to improve quantified uncertainty in predictions by optimizing a simplified distribution over parameters to stand in for the full posterior. Capturing model variations that remain consistent with training data enables more robust predictions by reducing parameter sensitivity. This work introduces a fixed-point optimization for variational inference that is applicable when every feasible log density can be expressed as a linear combination of functions from a given basis. In such cases, the optimizer becomes a fixed-point of projective integral updates. When the basis spans univariate quadratics in each parameter, feasible densities are Gaussian and the projective integral updates yield quasi-Newton variational Bayes (QNVB). Other bases and updates are also possible. As these updates require high-dimensional integration, this work first proposes an efficient quasirandom quadrature sequence for mean-field distributions. Each iterate of the sequence contains two evaluation points that combine to correctly integrate all univariate quadratics and, if the mean-field factors are symmetric, all univariate cubics. More importantly, averaging results over short subsequences achieves periodic exactness on a much larger space of multivariate quadratics. The corresponding variational updates require 4 loss evaluations with standard (not second-order) backpropagation to eliminate error terms from over half of all multivariate quadratic basis functions. This integration technique is motivated by first proposing stochastic blocked mean-field quadratures, which may be useful in other contexts. A PyTorch implementation of QNVB allows for better control over model uncertainty during training than competing methods. Experiments demonstrate superior generalizability for multiple learning problems and architectures.
Machine Learning for Relaying Topology: Optimization of IoT Network with Energy Harvesting
Venue: arXiv cs.NI 2023
Year: 2023

In this paper, we examine the internet of things system which is dedicated for smart cities, smart factory, and connected cars, etc. To support such systems in wide area with low power consumption, energy harvesting technology without wired charging infrastructure is one of the important issues for longevity of networks. In consideration of the fact that the position and amount of energy charged for each device might be unbalanced according to the distribution of nodes and energy sources, the problem of maximizing the minimum throughput among all nodes becomes a NP-hard challenging issue. To overcome this complexity, we propose a machine learning based relaying topology algorithm with a novel backward-pass rate assessment method to present proper learning direction and an iterative balancing time slot allocation algorithm which can utilize the node with sufficient energy as the relay. To validate the proposed scheme, we conducted simulations on the system model we established, thus confirm that the proposed scheme is stable and superior to conventional schemes.
Cohere: Managing Differential Privacy in Large Scale Systems
Venue: arXiv cs.CR 2023
Year: 2023

The need for a privacy management layer in today's systems started to manifest with the emergence of new systems for privacy-preserving analytics and privacy compliance. As a result, many independent efforts have emerged that try to provide system support for privacy. Recently, the scope of privacy solutions used in systems has expanded to encompass more complex techniques such as Differential Privacy (DP). The use of these solutions in large-scale systems imposes new challenges and requirements. Careful planning and coordination are necessary to ensure that privacy guarantees are maintained across a wide range of heterogeneous applications and data systems. This requires new solutions for managing and allocating scarce and non-replenishable privacy resources. In this paper, we introduce Cohere, a new system that simplifies the use of DP in large-scale systems. Cohere implements a unified interface that allows heterogeneous applications to operate on a unified view of users' data. In this work, we further address two pressing system challenges that arise in the context of real-world deployments: ensuring the continuity of privacy-based applications (i.e., preventing privacy budget depletion) and effectively allocating scarce shared privacy resources (i.e., budget) under complex preferences. Our experiments show that Cohere achieves a 6.4--28x improvement in utility compared to the state-of-the-art across a range of complex workloads.
Towards Multi-robot Exploration: A Decentralized Strategy for UAV Forest Exploration
Venue: arXiv cs.RO 2023
Year: 2023

Efficient exploration strategies are vital in tasks such as search-and-rescue missions and disaster surveying. Unmanned Aerial Vehicles (UAVs) have become particularly popular in such applications, promising to cover large areas at high speeds. Moreover, with the increasing maturity of onboard UAV perception, research focus has been shifting toward higher-level reasoning for single- and multi-robot missions. However, autonomous navigation and exploration of previously unknown large spaces still constitutes an open challenge, especially when the environment is cluttered and exhibits large and frequent occlusions due to high obstacle density, as is the case of forests. Moreover, the problem of long-distance wireless communication in such scenes can become a limiting factor, especially when automating the navigation of a UAV swarm. In this spirit, this work proposes an exploration strategy that enables UAVs, both individually and in small swarms, to quickly explore complex scenes in a decentralized fashion. By providing the decision-making capabilities to each UAV to switch between different execution modes, the proposed strategy strikes a great balance between cautious exploration of yet completely unknown regions and more aggressive exploration of smaller areas of unknown space. This results in full coverage of forest areas of variable density, consistently faster than the state of the art. Demonstrating successful deployment with a single UAV as well as a swarm of up to three UAVs, this work sets out the basic principles for multi-root exploration of cluttered scenes, with up to 65% speed up in the single UAV case and 40% increase in explored area for the same mission time in multi-UAV setups.