OpenRaiser/Intern-Atlas huggingface dataset:
A review of Generative Adversarial Networks for Electronic Health Records: applications, evaluation measures and data sources
Venue: arXiv cs.LG 2022
Year: 2022

Electronic Health Records (EHRs) are a valuable asset to facilitate clinical research and point of care applications; however, many challenges such as data privacy concerns impede its optimal utilization. Deep generative models, particularly, Generative Adversarial Networks (GANs) show great promise in generating synthetic EHR data by learning underlying data distributions while achieving excellent performance and addressing these challenges. This work aims to review the major developments in various applications of GANs for EHRs and provides an overview of the proposed methodologies. For this purpose, we combine perspectives from healthcare applications and machine learning techniques in terms of source datasets and the fidelity and privacy evaluation of the generated synthetic datasets. We also compile a list of the metrics and datasets used by the reviewed works, which can be utilized as benchmarks for future research in the field. We conclude by discussing challenges in GANs for EHRs development and proposing recommended practices. We hope that this work motivates novel research development directions in the intersection of healthcare and machine learning.
Compressing CNN Kernels for Videos Using Tucker Decompositions: Towards Lightweight CNN Applications
Venue: arXiv cs.LG 2022
Year: 2022

Convolutional Neural Networks (CNN) are the state-of-the-art in the field of visual computing. However, a major problem with CNNs is the large number of floating point operations (FLOPs) required to perform convolutions for large inputs. When considering the application of CNNs to video data, convolutional filters become even more complex due to the extra temporal dimension. This leads to problems when respective applications are to be deployed on mobile devices, such as smart phones, tablets, micro-controllers or similar, indicating less computational power. Kim et al. (2016) proposed using a Tucker-decomposition to compress the convolutional kernel of a pre-trained network for images in order to reduce the complexity of the network, i.e. the number of FLOPs. In this paper, we generalize the aforementioned method for application to videos (and other 3D signals) and evaluate the proposed method on a modified version of the THETIS data set, which contains videos of individuals performing tennis shots. We show that the compressed network reaches comparable accuracy, while indicating a memory compression by a factor of 51. However, the actual computational speed-up (factor 1.4) does not meet our theoretically derived expectation (factor 6).
Hybrid Active-Passive Reconfigurable Intelligent Surface-Assisted Multi-User MISO Systems
Venue: arXiv cs.IT 2022
Year: 2022

We consider a multi-user multiple-input single-output (MISO) communications system which is assisted by a hybrid active-passive reconfigurable intelligent surface (RIS). Unlike conventional passive RISs, hybrid RIS is equipped with a few active elements with the ability to reflect and amplify incident signals to significantly improve the system performance. Towards a fairness-oriented design, we maximize the minimum rate among all users through jointly optimizing the transmit beamforming vectors and RIS reflecting/amplifying coefficients. Combining tools from block coordinate ascent and successive convex approximation, the challenging nonconvex problem is efficiently solved by a low-complexity iterative algorithm. The numerical results show that a hybrid RIS with 4 active elements out of a total of 50 elements with a power budget of -1 dBm offers an improvement of up to 80% to the considered system, while that achieved by a fully passive RIS is only 27%.
Neural Forecasting of the Italian Sovereign Bond Market with Economic News
Venue: arXiv cs.LG 2022
Year: 2022

In this paper we employ economic news within a neural network framework to forecast the Italian 10-year interest rate spread. We use a big, open-source, database known as Global Database of Events, Language and Tone to extract topical and emotional news content linked to bond markets dynamics. We deploy such information within a probabilistic forecasting framework with autoregressive recurrent networks (DeepAR). Our findings suggest that a deep learning network based on Long-Short Term Memory cells outperforms classical machine learning techniques and provides a forecasting performance that is over and above that obtained by using conventional determinants of interest rates alone.
A Robust Approach for the Decomposition of High-Energy-Consuming Industrial Loads with Deep Learning
Venue: arXiv cs.LG 2022
Year: 2022

The knowledge of the users' electricity consumption pattern is an important coordinating mechanism between the utility company and the electricity consumers in terms of key decision makings. The load decomposition is therefore crucial to reveal the underlying relationship between the load consumption and its characteristics. However, load decomposition is conventionally performed on the residential and commercial loads, and adequate consideration has not been given to the high-energy-consuming industrial loads leading to inefficient results. This paper thus focuses on the load decomposition of the industrial park loads (IPL). The commonly used parameters in a conventional method are however inapplicable in high-energy-consuming industrial loads. Therefore, a more robust approach is developed comprising a three-algorithm model to achieve this goal on the IPL. First, the improved variational mode decomposition (IVMD) algorithm is introduced to denoise the training data of the IPL and improve its stability. Secondly, the convolutional neural network (CNN) and simple recurrent units (SRU) joint algorithms are used to achieve a non-intrusive and non-invasive decomposition process of the IPL using a double-layer deep learning network based on the IPL characteristics. Specifically, CNN is used to extract the IPL data characteristics while the improved long and short-term memory (LSTM) network, SRU, is adopted to develop the decomposition model and further train the load data. Through the robust decomposition process, the underlying relationship in the load consumption is extracted. The results obtained from the numerical examples show that this approach outperforms the state-of-the-art in the conventional decomposition process.
Towards More Efficient EfficientDets and Low-Light Real-Time Marine Debris Detection
Venue: arXiv cs.CV 2022
Year: 2022

Marine debris is a problem both for the health of marine environments and for the human health since tiny pieces of plastic called "microplastics" resulting from the debris decomposition over the time are entering the food chain at any levels. For marine debris detection and removal, autonomous underwater vehicles (AUVs) are a potential solution. In this letter, we focus on the efficiency of AUV vision for real-time and low-light object detection. First, we improved the efficiency of a class of state-of-the-art object detectors, namely EfficientDets, by 1.5% AP on D0, 2.6% AP on D1, 1.2% AP on D2 and 1.3% AP on D3 without increasing the GPU latency. Subsequently, we created and made publicly available a dataset for the detection of in-water plastic bags and bottles and trained our improved EfficientDets on this and another dataset for marine debris detection. Finally, we investigated how the detector performance is affected by low-light conditions and compared two low-light underwater image enhancement strategies both in terms of accuracy and latency. Source code and dataset are publicly available.
Dataset and Case Studies for Visual Near-Duplicates Detection in the Context of Social Media
Venue: arXiv cs.IR 2022
Year: 2022

The massive spread of visual content through the web and social media poses both challenges and opportunities. Tracking visually-similar content is an important task for studying and analyzing social phenomena related to the spread of such content. In this paper, we address this need by building a dataset of social media images and evaluating visual near-duplicates retrieval methods based on image retrieval and several advanced visual feature extraction methods. We evaluate the methods using a large-scale dataset of images we crawl from social media and their manipulated versions we generated, presenting promising results in terms of recall. We demonstrate the potential of this method in two case studies: one that shows the value of creating systems supporting manual content review, and another that demonstrates the usefulness of automatic large-scale data analysis.
Development Frameworks for Microservice-based Applications: Evaluation and Comparison
Venue: arXiv cs.SE 2022
Year: 2022

The microservice architectural style has gained much attention from both academia and industry recently as a novel way to design, develop, and deploy cloud-native applications. This concept encourages the decomposition of a monolith into multiple independently deployable units. A typical microservices-based application is formed of two service types: functional services, which provide the core business logic, and infrastructure services, which provide essential functionalities for a microservices ecosystem. To improve developers' productivity, many software frameworks have been developed to provide those reusable infrastructure services, allowing programmers to focus on implementing microservices in arbitrary ways. In this work, we made use of four open source frameworks to develop a cloud-based application in order to compare and evaluate their usability and practicability. While all selected frameworks promote asynchronous microservice design in general, there are differences in the ways each implements services. This leads to interoperability issues, such as message topic naming convention. Additionally, a key finding is the long startup times of JVM-based services that might reduce application's resiliency and portability. Some other advantages come directly from the programming language, such as the ability of Go to generate native binary executables, which results in very small and compact Docker images (up to 78\% smaller compared to other languages).
On Cyclic Solutions to the Min-Max Latency Multi-Robot Patrolling Problem
Venue: arXiv cs.CG 2022
Year: 2022

We consider the following surveillance problem: Given a set $P$ of $n$ sites in a metric space and a set of $k$ robots with the same maximum speed, compute a patrol schedule of minimum latency for the robots. Here a patrol schedule specifies for each robot an infinite sequence of sites to visit (in the given order) and the latency $L$ of a schedule is the maximum latency of any site, where the latency of a site $s$ is the supremum of the lengths of the time intervals between consecutive visits to $s$. When $k=1$ the problem is equivalent to the travelling salesman problem (TSP) and thus it is NP-hard. We have two main results. We consider cyclic solutions in which the set of sites must be partitioned into $\ell$ groups, for some~$\ell \leq k$, and each group is assigned a subset of the robots that move along the travelling salesman tour of the group at equal distance from each other. Our first main result is that approximating the optimal latency of the class of cyclic solutions can be reduced to approximating the optimal travelling salesman tour on some input, with only a $1+\varepsilon$ factor loss in the approximation factor and an $O\left(\left( k/\varepsilon \right)^k\right)$ factor loss in the runtime, for any $\varepsilon >0$. Our second main result shows that an optimal cyclic solution is a $2(1-1/k)$-approximation of the overall optimal solution. Note that for $k=2$ this implies that an optimal cyclic solution is optimal overall. The results have a number of consequences. For the Euclidean version of the problem, for instance, combining our results with known results on Euclidean TSP, yields a PTAS for approximating an optimal cyclic solution, and it yields a $(2(1-1/k)+\varepsilon)$-approximation of the optimal unrestricted solution. If the conjecture mentioned above is true, then our algorithm is actually a PTAS for the general problem in the Euclidean setting.
Relational Diagrams: a pattern-preserving diagrammatic representation of non-disjunctive Relational Queries
Venue: arXiv cs.DB 2022
Year: 2022

Analyzing relational languages by their logical expressiveness is well understood. Something not well understood or even formalized is the vague concept of relational query patterns. What are query patterns? And how can we reason about query patterns across different relational languages, irrespective of their syntax and their procedural or declarative nature? In this paper, we formalize the concept of query patterns with a variant of pattern-preserving mappings between the relational atoms of queries. This formalism allows us to analyze the relative pattern expressiveness of relational query languages and to create a hierarchy of languages with equal logical expressiveness yet different pattern expressiveness. In this analysis, relational calculus can expressive more patterns than the basic operators of relational algebra. We additionally contribute an intuitive, complete, and sound diagrammatic representation of safe relational calculus that is not only relationally complete, but can also express all logical patterns for the large and useful fragment of non-disjunctive relational calculus. Among all diagrammatic representations for relational queries that we are aware of, this is the only one that is relationally complete and that can represent all logical patterns in the non-disjunctive fragment.
Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts
Venue: arXiv cs.CL 2022
Year: 2022

Generative commonsense reasoning (GCR) in natural language is to reason about the commonsense while generating coherent text. Recent years have seen a surge of interest in improving the generation quality of commonsense reasoning tasks. Nevertheless, these approaches have seldom investigated diversity in the GCR tasks, which aims to generate alternative explanations for a real-world situation or predict all possible outcomes. Diversifying GCR is challenging as it expects to generate multiple outputs that are not only semantically different but also grounded in commonsense knowledge. In this paper, we propose MoKGE, a novel method that diversifies the generative reasoning by a mixture of expert (MoE) strategy on commonsense knowledge graphs (KG). A set of knowledge experts seek diverse reasoning on KG to encourage various generation outputs. Empirical experiments demonstrated that MoKGE can significantly improve the diversity while achieving on par performance on accuracy on two GCR benchmarks, based on both automatic and human evaluations.
On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernel
Venue: arXiv cs.LG 2022
Year: 2022

Deep Gaussian Process (DGP) as a model prior in Bayesian learning intuitively exploits the expressive power in function composition. DGPs also offer diverse modeling capabilities, but inference is challenging because marginalization in latent function space is not tractable. With Bochner's theorem, DGP with squared exponential kernel can be viewed as a deep trigonometric network consisting of the random feature layers, sine and cosine activation units, and random weight layers. In the wide limit with a bottleneck, we show that the weight space view yields the same effective covariance functions which were obtained previously in function space. Also, varying the prior distributions over network parameters is equivalent to employing different kernels. As such, DGPs can be translated into the deep bottlenecked trig networks, with which the exact maximum a posteriori estimation can be obtained. Interestingly, the network representation enables the study of DGP's neural tangent kernel, which may also reveal the mean of the intractable predictive distribution. Statistically, unlike the shallow networks, deep networks of finite width have covariance deviating from the limiting kernel, and the inner and outer widths may play different roles in feature learning. Numerical simulations are present to support our findings.
Refined Hardness of Distance-Optimal Multi-Agent Path Finding
Venue: arXiv cs.MA 2022
Year: 2022

We study the computational complexity of multi-agent path finding (MAPF). Given a graph $G$ and a set of agents, each having a start and target vertex, the goal is to find collision-free paths minimizing the total distance traveled. To better understand the source of difficulty of the problem, we aim to study the simplest and least constrained graph class for which it remains hard. To this end, we restrict $G$ to be a 2D grid, which is a ubiquitous abstraction, as it conveniently allows for modeling well-structured environments (e.g., warehouses). Previous hardness results considered highly constrained 2D grids having only one vertex unoccupied by an agent, while the most restricted hardness result that allowed multiple empty vertices was for (non-grid) planar graphs. We therefore refine previous results by simultaneously considering both 2D grids and multiple empty vertices. We show that even in this case distance-optimal MAPF remains NP-hard, which settles an open problem posed by Banfi et al. (2017). We present a reduction directly from 3-SAT using simple gadgets, making our proof arguably more informative than previous work in terms of potential progress towards positive results. Furthermore, our reduction is the first linear one for the case where $G$ is planar, appearing nearly four decades after the first related result. This allows us to go a step further and exploit the Exponential Time Hypothesis (ETH) to obtain an exponential lower bound for the running time of the problem. Finally, as a stepping stone towards our main results, we prove the NP-hardness of the monotone case, in which agents move one by one with no intermediate stops.
Hercules: Heterogeneity-Aware Inference Serving for At-Scale Personalized Recommendation
Venue: arXiv cs.DC 2022
Year: 2022

Personalized recommendation is an important class of deep-learning applications that powers a large collection of internet services and consumes a considerable amount of datacenter resources. As the scale of production-grade recommendation systems continues to grow, optimizing their serving performance and efficiency in a heterogeneous datacenter is important and can translate into infrastructure capacity saving. In this paper, we propose Hercules, an optimized framework for personalized recommendation inference serving that targets diverse industry-representative models and cloud-scale heterogeneous systems. Hercules performs a two-stage optimization procedure - offline profiling and online serving. The first stage searches the large under-explored task scheduling space with a gradient-based search algorithm achieving up to 9.0x latency-bounded throughput improvement on individual servers; it also identifies the optimal heterogeneous server architecture for each recommendation workload. The second stage performs heterogeneity-aware cluster provisioning to optimize resource mapping and allocation in response to fluctuating diurnal loads. The proposed cluster scheduler in Hercules achieves 47.7% cluster capacity saving and reduces the provisioned power by 23.7% over a state-of-the-art greedy scheduler.
The Unexplored Treasure Trove of Phabricator Code Review
Venue: arXiv cs.SE 2022
Year: 2022

Phabricator is a modern code collaboration tool used by popular projects like FreeBSD and Mozilla. However, unlike the other well-known code review environments, such as Gerrit or GitHub, there is no readily accessible public code review dataset for Phabricator. This paper describes our experience mining code reviews from five different projects that use Phabricator (Blender, FreeBSD, KDE, LLVM, and Mozilla). We discuss the challenges associated with the data retrieval process and our solutions, resulting in a dataset with details regarding 317,476 Phabricator code reviews. Our dataset is available in both JSON and MySQL database dump formats. The dataset enables analyses of the history of code reviews at a more granular level than other platforms. In addition, given that the projects we mined are publicly accessible via the Conduit API, our dataset can be used as a foundation to fetch additional details and insights.
Constrained Precision Tuning
Venue: arXiv cs.SE 2022
Year: 2022

Precision tuning or customized precision number representations is emerging, in these recent years, as one of the most promising techniques that has a positive impact on the footprint of programs concerning energy consumption, bandwidth usage and computation time of numerical programs. In contrast to the uniform precision, mixed precision tuning assigns different finite-precision types to each variable and arithmetic operation of a program and offers many additional optimization opportunities. However, this technique introduces new challenge related to the cost of operations or type conversions which can overload the program execution after tuning. In this article, we extend our tool POP (Precision OPtimizer), with efficient ways to limit the number of drawbacks of mixed precision and to achieve best compromise between performance and memory consumption. On a popular set of tests from the FPBench suite, we discuss the results obtained by POP.
Conformance Checking Over Stochastically Known Logs
Venue: arXiv cs.AI 2022
Year: 2022

With the growing number of devices, sensors and digital systems, data logs may become uncertain due to, e.g., sensor reading inaccuracies or incorrect interpretation of readings by processing programs. At times, such uncertainties can be captured stochastically, especially when using probabilistic data classification models. In this work we focus on conformance checking, which compares a process model with an event log, when event logs are stochastically known. Building on existing alignment-based conformance checking fundamentals, we mathematically define a stochastic trace model, a stochastic synchronous product, and a cost function that reflects the uncertainty of events in a log. Then, we search for an optimal alignment over the reachability graph of the stochastic synchronous product for finding an optimal alignment between a model and a stochastic process observation. Via structured experiments with two well-known process mining benchmarks, we explore the behavior of the suggested stochastic conformance checking approach and compare it to a standard alignment-based approach as well as to an approach that creates a lower bound on performance. We envision the proposed stochastic conformance checking approach as a viable process mining component for future analysis of stochastic event logs.
A Unified Framework for Rank-based Evaluation Metrics for Link Prediction in Knowledge Graphs
Venue: arXiv cs.LG 2022
Year: 2022

The link prediction task on knowledge graphs without explicit negative triples in the training data motivates the usage of rank-based metrics. Here, we review existing rank-based metrics and propose desiderata for improved metrics to address lack of interpretability and comparability of existing metrics to datasets of different sizes and properties. We introduce a simple theoretical framework for rank-based metrics upon which we investigate two avenues for improvements to existing metrics via alternative aggregation functions and concepts from probability theory. We finally propose several new rank-based metrics that are more easily interpreted and compared accompanied by a demonstration of their usage in a benchmarking of knowledge graph embedding models.