OpenRaiser/Intern-Atlas huggingface dataset:
Software Micro-Rejuvenation for Android Mobile Systems
Venue: arXiv cs.SE 2021
Year: 2021

Software aging -- the phenomenon affecting many long-running systems, causing performance degradation or an increasing failure rate over mission time, and eventually leading to failure - is known to affect mobile devices and their operating systems, too. Software rejuvenation -- the technique typically used to counteract aging -- may compromise the user's perception of availability and reliability of the personal device, if applied at a coarse grain, e.g., by restarting applications or, worse, rebooting the entire device. This article proposes a configurable micro-rejuvenation technique to counteract software aging in Android-based mobile devices, acting at a fine-grained level, namely on in-memory system data structures. The technique is engineered in two phases. Before releasing the (customized) Android version, a heap profiling facility is used by the manufacturer's developers to identify potentially bloating data structures in Android services and to instrument their code. After release, an aging detection and rejuvenation service will safely clean up the bloating data structures, with a negligible impact on user perception and device availability, as neither the device nor operating system's processes are restarted. The results of experiments show the ability of the technique to provide significant gains in aging mobile operating system responsiveness and time to failure.
A Methodology for a Scalable, Collaborative, and Resource-Efficient Platform to Facilitate Healthcare AI Research
Venue: arXiv cs.SE 2021
Year: 2021

Healthcare AI holds the potential to increase patient safety, augment efficiency and improve patient outcomes, yet research is often limited by data access, cohort curation, and tooling for analysis. Collection and translation of electronic health record data, live data, and real-time high resolution device data can be challenging and time-consuming. The development of real-world AI tools requires overcoming challenges in data acquisition, scarce hospital resources and high needs for data governance. These bottlenecks may result in resource-heavy needs and long delays in research and development of AI systems. We present a system and methodology to accelerate data acquisition, dataset development and analysis, and AI model development. We created an interactive platform that relies on a scalable microservice backend. This system can ingest 15,000 patient records per hour, where each record represents thousands of multimodal measurements, text notes, and high resolution data. Collectively, these records can approach a terabyte of data. The system can further perform cohort generation and preliminary dataset analysis in 2-5 minutes. As a result, multiple users can collaborate simultaneously to iterate on datasets and models in real time. We anticipate that this approach will drive real-world AI model development, and, in the long run, meaningfully improve healthcare delivery.
Interpretable Design of Reservoir Computing Networks using Realization Theory
Venue: arXiv cs.LG 2021
Year: 2021

The reservoir computing networks (RCNs) have been successfully employed as a tool in learning and complex decision-making tasks. Despite their efficiency and low training cost, practical applications of RCNs rely heavily on empirical design. In this paper, we develop an algorithm to design RCNs using the realization theory of linear dynamical systems. In particular, we introduce the notion of $α$-stable realization, and provide an efficient approach to prune the size of a linear RCN without deteriorating the training accuracy. Furthermore, we derive a necessary and sufficient condition on the irreducibility of number of hidden nodes in linear RCNs based on the concepts of controllability and observability matrices. Leveraging the linear RCN design, we provide a tractable procedure to realize RCNs with nonlinear activation functions. Finally, we present numerical experiments on forecasting time-delay systems and chaotic systems to validate the proposed RCN design methods and demonstrate their efficacy.
Automated Customization of On-Thing Inference for Quality-of-Experience Enhancement
Venue: arXiv cs.LG 2021
Year: 2021

The rapid uptake of intelligent applications is pushing deep learning (DL) capabilities to Internet-of-Things (IoT). Despite the emergence of new tools for embedding deep neural networks (DNNs) into IoT devices, providing satisfactory Quality of Experience (QoE) to users is still challenging due to the heterogeneity in DNN architectures, IoT devices, and user preferences. This paper studies automated customization for DL inference on IoT devices (termed as on-thing inference), and our goal is to enhance user QoE by configuring the on-thing inference with an appropriate DNN for users under different usage scenarios. The core of our method is a DNN selection module that learns user QoE patterns on-the-fly and identifies the best-fit DNN for on-thing inference with the learned knowledge. It leverages a novel online learning algorithm, NeuralUCB, that has excellent generalization ability for handling various user QoE patterns. We also embed the knowledge transfer technique in NeuralUCB to expedite the learning process. However, NeuralUCB frequently solicits QoE ratings from users, which incurs non-negligible inconvenience. To address this problem, we design feedback solicitation schemes to reduce the number of QoE solicitations while maintaining the learning efficiency of NeuralUCB. A pragmatic problem, aggregated QoE, is further investigated to improve the practicality of our framework. We conduct experiments on both synthetic and real-world data. The results indicate that our method efficiently learns the user QoE pattern with few solicitations and provides drastic QoE enhancement for IoT devices.
CGAN-EB: A Non-parametric Empirical Bayes Method for Crash Hotspot Identification Using Conditional Generative Adversarial Networks: A Simulated Crash Data Study
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, a new non-parametric empirical Bayes approach called CGAN-EB is proposed for approximating empirical Bayes (EB) estimates in traffic locations (e.g., road segments) which benefits from the modeling advantages of deep neural networks, and its performance is compared in a simulation study with the traditional approach based on negative binomial model (NB-EB). The NB-EB uses negative binomial model in order to model the crash data and is the most common approach in practice. To model the crash data in the proposed CGAN-EB, conditional generative adversarial network is used, which is a powerful deep neural network based method that can model any types of distributions. A number of simulation experiments are designed and conducted to evaluate the CGAN-EB performance in different conditions and compare it with the NB-EB. The results show that CGAN-EB performs as well as NB-EB when conditions favor the NB-EB model (i.e. data conform to the assumptions of the NB model) and outperforms NB-EB in experiments reflecting conditions frequently encountered in practice, specifically low sample means, and when crash frequency does not follow a log-linear relationship with covariates.
Slowing Down for Performance and Energy: An OS-Centric Study in Network Driven Workloads
Venue: arXiv cs.OS 2021
Year: 2021

This paper studies three fundamental aspects of an OS that impact the performance and energy efficiency of network processing: 1) batching, 2) processor energy settings, and 3) the logic and instructions of the OS networking paths. A network device's interrupt delay feature is used to induce batching and processor frequency is manipulated to control the speed of instruction execution. A baremetal library OS is used to explore OS path specialization. This study shows how careful use of batching and interrupt delay results in 2X energy and performance improvements across different workloads. Surprisingly, we find polling can be made energy efficient and can result in gains up to 11X over baseline Linux. We developed a methodology and a set of tools to collect system data in order to understand how energy is impacted at a fine-grained granularity. This paper identifies a number of other novel findings that have implications in OS design for networked applications and suggests a path forward to consider energy as a focal point of systems research.
The state of optics research in Colombia: a scientometric analysis
Venue: arXiv cs.DL 2021
Year: 2021

Although there is a perception in the community that the state of optics research in Colombia has reached a mature state with international recognition, to date, there is no study that supports this view quantitatively. We aimed to assess the state of optics research in Colombia based on international journal publications. Therefore, we determined scientometric indicators using the research articles published by authors with Colombian affiliation in journals indexed in the Scopus database belonging to the Atomic and Molecular Physics and Optics subject category. The research output has increased dramatically in the past two decades, with an average of 169 articles per year since 2016. Most of these articles are published in high-impact journals. A little over 10\% of that research is in the top 10\% most cited in the world. Over 25 higher education institutions contribute significantly to this research with many international and national collaborations. The normalized citation impact for Colombian optics research is 0.95, only five points below the world average, and ranked second in Latin America, only superseded by Chile (1.33). Our results show that optics research is an established research area in Colombia with high impact and many active groups from different institutions spread throughout the country.
Optimal Fully Dynamic $k$-Centers Clustering
Venue: arXiv cs.DS 2021
Year: 2021

We present the first algorithm for fully dynamic $k$-centers clustering in an arbitrary metric space that maintains an optimal $2+ε$ approximation in $O(k \cdot \operatorname{polylog}(n,Δ))$ amortized update time. Here, $n$ is an upper bound on the number of active points at any time, and $Δ$ is the aspect ratio of the data. Previously, the best known amortized update time was $O(k^2\cdot \operatorname{polylog}(n,Δ))$, and is due to Chan, Gourqin, and Sozio. We demonstrate that the runtime of our algorithm is optimal up to $\operatorname{polylog}(n,Δ)$ factors, even for insertion-only streams, which closes the complexity of fully dynamic $k$-centers clustering. In particular, we prove that any algorithm for $k$-clustering tasks in arbitrary metric spaces, including $k$-means, $k$-medians, and $k$-centers, must make at least $Ω(n k)$ distance queries to achieve any non-trivial approximation factor. Despite the lower bound for arbitrary metrics, we demonstrate that an update time sublinear in $k$ is possible for metric spaces which admit locally sensitive hash functions (LSH). Namely, we demonstrate a black-box transformation which takes a locally sensitive hash family for a metric space and produces a faster fully dynamic $k$-centers algorithm for that space. In particular, for a large class of metrics including Euclidean space, $\ell_p$ spaces, the Hamming Metric, and the Jaccard Metric, for any $c > 1$, our results yield a $c(4+ε)$ approximate $k$-centers solution in $O(n^{1/c} \cdot \operatorname{polylog}(n,Δ))$ amortized update time, simultaneously for all $k \geq 1$. Previously, the only known comparable result was a $O(c \log n)$ approximation for Euclidean space due to Schmidt and Sohler, running in the same amortized update time.
Towards a Unified Foundation Model: Jointly Pre-Training Transformers on Unpaired Images and Text
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we explore the possibility of building a unified foundation model that can be adapted to both vision-only and text-only tasks. Starting from BERT and ViT, we design a unified transformer consisting of modality-specific tokenizers, a shared transformer encoder, and task-specific output heads. To efficiently pre-train the proposed model jointly on unpaired images and text, we propose two novel techniques: (i) We employ the separately-trained BERT and ViT models as teachers and apply knowledge distillation to provide additional, accurate supervision signals for the joint training; (ii) We propose a novel gradient masking strategy to balance the parameter updates from the image and text pre-training losses. We evaluate the jointly pre-trained transformer by fine-tuning it on image classification tasks and natural language understanding tasks, respectively. The experiments show that the resultant unified foundation transformer works surprisingly well on both the vision-only and text-only tasks, and the proposed knowledge distillation and gradient masking strategy can effectively lift the performance to approach the level of separately-trained models.
Do you trust experts on Twitter?: Successful correction of COVID-19-related misinformation
Venue: arXiv cs.SI 2021
Year: 2021

This study focuses on how scientifically-correct information is disseminated through social media, and how misinformation can be corrected. We have identified examples on Twitter where scientific terms that have been misused have been rectified and replaced by scientifically-correct terms through the interaction of users. The results show that the percentage of correct terms ("variant" or "COVID-19 variant") being used instead of the incorrect terms ("strain") on Twitter has already increased since the end of December 2020. This was about a month before the release of an official statement by the Japanese Association for Infectious Diseases regarding the correct terminology, and the use of terms on social media was faster than it was in television. Some Twitter users who quickly started using the correct term were more likely to retweet messages sent by leading influencers on Twitter, rather than messages sent by traditional media or portal sites. However, a few Twitter users continued to use wrong terms even after March 2021, even though the use of the correct terms was widespread. Further analysis of their tweets revealed that they were quoting sources that differed from that of other users. This study empirically verified that self-correction occurs even on Twitter, which is often known as a "hotbed for spreading rumors." The results of this study also suggest that influencers with expertise can influence the direction of public opinion on social media and that the media that users usually cite can also affect the possibility of behavioral changes.
Logics in fungal mycelium networks
Venue: arXiv cs.ET 2021
Year: 2021

The living mycelium networks are capable of efficient sensorial fusion over very large areas and distributed decision making. The information processing in the mycelium networks is implemented via propagation of electrical and chemical signals en pair with morphological changes in the mycelium structure. These information processing mechanisms are manifested in experimental laboratory findings that show that the mycelium networks exhibit rich dynamics of neuron-like spiking behaviour and a wide range of non-linear electrical properties. On an example of a single real colony of \emph{Aspergillus niger}, we demonstrate that the non-linear transformation of electrical signals and trains of extracellular voltage spikes can be used to implement logical gates and circuits. The approaches adopted include numerical modelling of excitation propagation on the mycelium network, representation of the mycelium network as a resistive and capacitive (RC) network and an experimental laboratory study on mining logical circuits in mycelium bound composites.
Compensating trajectory bias for unsupervised patient stratification using adversarial recurrent neural networks
Venue: arXiv cs.LG 2021
Year: 2021

Electronic healthcare records are an important source of information which can be used in patient stratification to discover novel disease phenotypes. However, they can be challenging to work with as data is often sparse and irregularly sampled. One approach to solve these limitations is learning dense embeddings that represent individual patient trajectories using a recurrent neural network autoencoder (RNN-AE). This process can be susceptible to unwanted data biases. We show that patient embeddings and clusters using previously proposed RNN-AE models might be impacted by a trajectory bias, meaning that results are dominated by the amount of data contained in each patients trajectory, instead of clinically relevant details. We investigate this bias on 2 datasets (from different hospitals) and 2 disease areas as well as using different parts of the patient trajectory. Our results using 2 previously published baseline methods indicate a particularly strong bias in case of an event-to-end trajectory. We present a method that can overcome this issue using an adversarial training scheme on top of a RNN-AE. Our results show that our approach can reduce the trajectory bias in all cases.
Federated Learning for Face Recognition with Gradient Correction
Venue: arXiv cs.CV 2021
Year: 2021

With increasing appealing to privacy issues in face recognition, federated learning has emerged as one of the most prevalent approaches to study the unconstrained face recognition problem with private decentralized data. However, conventional decentralized federated algorithm sharing whole parameters of networks among clients suffers from privacy leakage in face recognition scene. In this work, we introduce a framework, FedGC, to tackle federated learning for face recognition and guarantees higher privacy. We explore a novel idea of correcting gradients from the perspective of backward propagation and propose a softmax-based regularizer to correct gradients of class embeddings by precisely injecting a cross-client gradient term. Theoretically, we show that FedGC constitutes a valid loss function similar to standard softmax. Extensive experiments have been conducted to validate the superiority of FedGC which can match the performance of conventional centralized methods utilizing full training dataset on several popular benchmark datasets.
Quantifying Multimodality in World Models
Venue: arXiv cs.LG 2021
Year: 2021

Model-based Deep Reinforcement Learning (RL) assumes the availability of a model of an environment's underlying transition dynamics. This model can be used to predict future effects of an agent's possible actions. When no such model is available, it is possible to learn an approximation of the real environment, e.g. by using generative neural networks, sometimes also called World Models. As most real-world environments are stochastic in nature and the transition dynamics are oftentimes multimodal, it is important to use a modelling technique that is able to reflect this multimodal uncertainty. In order to safely deploy such learning systems in the real world, especially in an industrial context, it is paramount to consider these uncertainties. In this work, we analyze existing and propose new metrics for the detection and quantification of multimodal uncertainty in RL based World Models. The correct modelling & detection of uncertain future states lays the foundation for handling critical situations in a safe way, which is a prerequisite for deploying RL systems in real-world settings.
Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated Memory
Venue: arXiv cs.DC 2021
Year: 2021

Memory disaggregation architecture physically separates CPU and memory into independent components, which are connected via high-speed RDMA networks, greatly improving resource utilization of databases. However, such an architecture poses unique challenges to data indexing in databases due to limited RDMA semantics and near-zero computation power at memory-side. Existing indexes supporting disaggregated memory either suffer from low write performance, or require hardware modification. This paper presents Sherman, a write-optimized distributed B+Tree index on disaggregated memory that delivers high performance with commodity RDMA NICs. Sherman combines RDMA hardware features and RDMA-friendly software techniques to boost index write performance from three angles. First, to reduce round trips, Sherman coalesces dependent RDMA commands by leveraging in-order delivery property of RDMA. Second, to accelerate concurrent accesses, Sherman introduces a hierarchical lock that exploits on-chip memory of RDMA NICs. Finally, to mitigate write amplification, Sherman tailors the data structure layout of B+Tree with a two-level version mechanism. Our evaluation shows that, Sherman is one order of magnitude faster in terms of both throughput and 99th percentile latency on typical write-intensive workloads, compared with state-of-the-art designs.
OMAD: Object Model with Articulated Deformations for Pose Estimation and Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

Articulated objects are pervasive in daily life. However, due to the intrinsic high-DoF structure, the joint states of the articulated objects are hard to be estimated. To model articulated objects, two kinds of shape deformations namely the geometric and the pose deformation should be considered. In this work, we present a novel category-specific parametric representation called Object Model with Articulated Deformations (OMAD) to explicitly model the articulated objects. In OMAD, a category is associated with a linear shape function with shared shape basis and a non-linear joint function. Both functions can be learned from a large-scale object model dataset and fixed as category-specific priors. Then we propose an OMADNet to predict the shape parameters and the joint states from an object's single observation. With the full representation of the object shape and joint states, we can address several tasks including category-level object pose estimation and the articulated object retrieval. To evaluate these tasks, we create a synthetic dataset based on PartNet-Mobility. Extensive experiments show that our simple OMADNet can serve as a strong baseline for both tasks.
Speeding up enclave transitions for IO-intensive applications
Venue: arXiv cs.CR 2021
Year: 2021

Process-based confidential computing enclaves such as Intel SGX can be used to protect the confidentiality and integrity of workloads, without the overhead of virtualisation. However, they introduce a notable performance overhead, especially when it comes to transitions in and out of the enclave context. Such overhead makes the use of enclaves impractical for running IO-intensive applications, such as network packet processing or biological sequence analysis. We build on earlier approaches to improve the IO performance of work-loads in Intel SGX enclaves and propose the SGX-Bundler library, which helps reduce the cost of both individual single enclave transitions well as of the total number of enclave transitions in trusted applications running in Intel SGX enclaves. We describe the implementation of the SGX-Bundler library, evaluate its performance and demonstrate its practicality using the case study of Open vSwitch, a widely used software switch implementation.