OpenRaiser/Intern-Atlas huggingface dataset:
Bipartite Graph Reasoning GANs for Person Pose and Facial Image Synthesis
Venue: arXiv cs.CV 2022
Year: 2022

We present a novel bipartite graph reasoning Generative Adversarial Network (BiGraphGAN) for two challenging tasks: person pose and facial image synthesis. The proposed graph generator consists of two novel blocks that aim to model the pose-to-pose and pose-to-image relations, respectively. Specifically, the proposed bipartite graph reasoning (BGR) block aims to reason the long-range cross relations between the source and target pose in a bipartite graph, which mitigates some of the challenges caused by pose deformation. Moreover, we propose a new interaction-and-aggregation (IA) block to effectively update and enhance the feature representation capability of both a person's shape and appearance in an interactive way. To further capture the change in pose of each part more precisely, we propose a novel part-aware bipartite graph reasoning (PBGR) block to decompose the task of reasoning the global structure transformation with a bipartite graph into learning different local transformations for different semantic body/face parts. Experiments on two challenging generation tasks with three public datasets demonstrate the effectiveness of the proposed methods in terms of objective quantitative scores and subjective visual realness. The source code and trained models are available at https://github.com/Ha0Tang/BiGraphGAN.
Distributed and secure linear algebra -- Master Thesis
Venue: arXiv cs.CR 2022
Year: 2022

Cryptography is the discipline that allows securing of the exchange of information. In this internship, we will focus on a certain branch of this discipline, secure computation in a network. The main goal of this internship, illustrated in this report, is to adapt a roster of protocols intended to do linear algebra. We want to adapt them to do algebra for matrices with polynomial coefficients. We then wish to make a complete analysis of the different complexities of these protocols.
Formalizing the presumption of independence
Venue: arXiv cs.AI 2022
Year: 2022

Mathematical proof aims to deliver confident conclusions, but a very similar process of deduction can be used to make uncertain estimates that are open to revision. A key ingredient in such reasoning is the use of a "default" estimate of $\mathbb{E}[XY] = \mathbb{E}[X] \mathbb{E}[Y]$ in the absence of any specific information about the correlation between $X$ and $Y$, which we call *the presumption of independence*. Reasoning based on this heuristic is commonplace, intuitively compelling, and often quite successful -- but completely informal. In this paper we introduce the concept of a heuristic estimator as a potential formalization of this type of defeasible reasoning. We introduce a set of intuitively desirable coherence properties for heuristic estimators that are not satisfied by any existing candidates. Then we present our main open problem: is there a heuristic estimator that formalizes intuitively valid applications of the presumption of independence without also accepting spurious arguments?
Textual Data Augmentation for Patient Outcomes Prediction
Venue: arXiv cs.CL 2022
Year: 2022

Deep learning models have demonstrated superior performance in various healthcare applications. However, the major limitation of these deep models is usually the lack of high-quality training data due to the private and sensitive nature of this field. In this study, we propose a novel textual data augmentation method to generate artificial clinical notes in patients' Electronic Health Records (EHRs) that can be used as additional training data for patient outcomes prediction. Essentially, we fine-tune the generative language model GPT-2 to synthesize labeled text with the original training data. More specifically, We propose a teacher-student framework where we first pre-train a teacher model on the original data, and then train a student model on the GPT-augmented data under the guidance of the teacher. We evaluate our method on the most common patient outcome, i.e., the 30-day readmission rate. The experimental results show that deep models can improve their predictive performance with the augmented data, indicating the effectiveness of the proposed architecture.
Reinforcement Learning Enhanced Weighted Sampling for Accurate Subgraph Counting on Fully Dynamic Graph Streams
Venue: arXiv cs.DB 2022
Year: 2022

As the popularity of graph data increases, there is a growing need to count the occurrences of subgraph patterns of interest, for a variety of applications. Many graphs are massive in scale and also fully dynamic (with insertions and deletions of edges), rendering exact computation of these counts to be infeasible. Common practice is, instead, to use a small set of edges as a sample to estimate the counts. Existing sampling algorithms for fully dynamic graphs sample the edges with uniform probability. In this paper, we show that we can do much better if we sample edges based on their individual properties. Specifically, we propose a weighted sampling algorithm called WSD for estimating the subgraph count in a fully dynamic graph stream, which samples the edges based on their weights that indicate their importance and reflect their properties. We determine the weights of edges in a data-driven fashion, using a novel method based on reinforcement learning. We conduct extensive experiments to verify that our technique can produce estimates with smaller errors while often running faster compared with existing algorithms.
EPViSA: Efficient Auction Design for Real-time Physical-Virtual Synchronization in the Metaverse
Venue: arXiv cs.NI 2022
Year: 2022

Metaverse can obscure the boundary between the physical and virtual worlds. Specifically, for the Metaverse in vehicular networks, i.e., the vehicular Metaverse, vehicles are no longer isolated physical spaces but interfaces that extend the virtual worlds to the physical world. Accessing the Metaverse via autonomous vehicles (AVs), drivers and passengers can immerse in and interact with 3D virtual objects overlaying views of streets on head-up displays (HUD) via augmented reality (AR). The seamless, immersive, and interactive experience rather relies on real-time multi-dimensional data synchronization between physical entities, i.e., AVs, and virtual entities, i.e., Metaverse billboard providers (MBPs). However, mechanisms to allocate and match synchronizing AV and MBP pairs to roadside units (RSUs) in a synchronization service market, which consists of the physical and virtual submarkets, are vulnerable to adverse selection. In this paper, we propose an enhanced second-score auction-based mechanism, named EPViSA, to allocate physical and virtual entities in the synchronization service market of the vehicular Metaverse. The EPViSA mechanism can determine synchronizing AV and MBP pairs simultaneously while protecting participants from adverse selection and thus achieving high total social welfare. We propose a synchronization scoring rule to eliminate the external effects from the virtual submarkets. Then, a price scaling factor is introduced to enhance the allocation of synchronizing virtual entities in the virtual submarkets. Finally, rigorous analysis and extensive experimental results demonstrate that EPViSA can achieve at least 96\% of the social welfare compared to the omniscient benchmark while ensuring strategy-proof and adverse selection free through a simulation testbed.
Choice Trees: Representing Nondeterministic, Recursive, and Impure Programs in Coq
Venue: arXiv cs.PL 2022
Year: 2022

This paper introduces Choice Trees (ctrees), a monad for modeling nondeterministic, recursive, and impure programs in Coq. Inspired by Xia et al.'s itrees, this novel data structure embeds computations into coinductive trees with three kind of nodes: external events, and two variants of nondeterministic branching. This apparent redundancy allows us to provide shallow embedding of denotational models with internal choice in the style of ccs, while recovering an inductive LTS view of the computation. CTrees inherit a vast collection of bisimulation and refinement tools, with respect to which we establish a rich equational theory. We connect ctrees to the itrees infrastructure by showing how a monad morphism embedding the former into the latter permits to use ctrees to implement nondeterministic effects. We demonstrate the utility of ctrees by using them to model concurrency semantics in two case studies: ccs and cooperative multithreading.
Benefits, Challenges, and Research Topics: A Multi-vocal Literature Review of Kubernetes
Venue: arXiv cs.SE 2022
Year: 2022

Context: Kubernetes is an open source software that helps in automated deployment of software and orchestration of containers. With Kubernetes, IT organizations, such as IBM, Pinterest, and Spotify have experienced an increase in release frequency. Objective: The goal of this paper is to inform practitioners and researchers on benefits and challenges of Kubernetes usage by conducting a multi-vocal literature review of Kubernetes. Methodology: We conduct a multi-vocal literature review (MLR) where we use 321 Kubernetes-related Internet artifacts to identify benefits and challenges perceived by practitioners. In our MLR, we also analyze 105 peer-reviewed publications to identify the research topics addressed by the research community. Findings: We find 8 benefits that include service level objective (SLO)-based scalability and self-healing containers. Our identified 15 challenges related to Kubernetes include unavailability of diagnostics and security tools and attack surface reduction. We observe researchers to address 14 research topics related to Kubernetes, which includes efficient resource utilization. We also identify 9 challenges that are under-explored in research publications, which include cultural change, hardware compatibility, learning curve, maintenance, and testing.
Approximating the lambda-low-density value
Venue: arXiv cs.CG 2022
Year: 2022

The use of realistic input models has gained popularity in the theory community. Assuming a realistic input model often precludes complicated hypothetical inputs, and the analysis yields bounds that better reflect the behaviour of algorithms in practice. One of the most popular models for polygonal curves and line segments is $λ$-low-density. To select the most efficient algorithm for a certain input, one often needs to compute the $λ$-low-density value, or at least an approximate value. In this paper, we show that given a set of $n$ line segments in $\mathbb{R}^2$ one can compute a $3$-approximation of the $λ$-low density value in $O(n \log n + λn)$ time. We also show how to maintain a $3$-approximation of the $λ$-low density value while allowing insertions of new segments in $O(\log n + λ^2)$ amortized time per update. Finally, we argue that many real-world data sets have a small $λ$-low density value, warranting the recent development of specialised algorithms. This is done by computing approximate $λ$-low density values for $12$ real-world data sets.
Treatment-RSPN: Recurrent Sum-Product Networks for Sequential Treatment Regimes
Venue: arXiv cs.LG 2022
Year: 2022

Sum-product networks (SPNs) have recently emerged as a novel deep learning architecture enabling highly efficient probabilistic inference. Since their introduction, SPNs have been applied to a wide range of data modalities and extended to time-sequence data. In this paper, we propose a general framework for modelling sequential treatment decision-making behaviour and treatment response using recurrent sum-product networks (RSPNs). Models developed using our framework benefit from the full range of RSPN capabilities, including the abilities to model the full distribution of the data, to seamlessly handle latent variables, missing values and categorical data, and to efficiently perform marginal and conditional inference. Our methodology is complemented by a novel variant of the expectation-maximization algorithm for RSPNs, enabling efficient training of our models. We evaluate our approach on a synthetic dataset as well as real-world data from the MIMIC-IV intensive care unit medical database. Our evaluation demonstrates that our approach can closely match the ground-truth data generation process on synthetic data and achieve results close to neural and probabilistic baselines while using a tractable and interpretable model.
Learning predictive checklists from continuous medical data
Venue: arXiv cs.LG 2022
Year: 2022

Checklists, while being only recently introduced in the medical domain, have become highly popular in daily clinical practice due to their combined effectiveness and great interpretability. Checklists are usually designed by expert clinicians that manually collect and analyze available evidence. However, the increasing quantity of available medical data is calling for a partially automated checklist design. Recent works have taken a step in that direction by learning predictive checklists from categorical data. In this work, we propose to extend this approach to accomodate learning checklists from continuous medical data using mixed-integer programming approach. We show that this extension outperforms a range of explainable machine learning baselines on the prediction of sepsis from intensive care clinical trajectories.
Few-shot Metric Learning: Online Adaptation of Embedding for Retrieval
Venue: arXiv cs.CV 2022
Year: 2022

Metric learning aims to build a distance metric typically by learning an effective embedding function that maps similar objects into nearby points in its embedding space. Despite recent advances in deep metric learning, it remains challenging for the learned metric to generalize to unseen classes with a substantial domain gap. To tackle the issue, we explore a new problem of few-shot metric learning that aims to adapt the embedding function to the target domain with only a few annotated data. We introduce three few-shot metric learning baselines and propose the Channel-Rectifier Meta-Learning (CRML), which effectively adapts the metric space online by adjusting channels of intermediate layers. Experimental analyses on miniImageNet, CUB-200-2011, MPII, as well as a new dataset, miniDeepFashion, demonstrate that our method consistently improves the learned metric by adapting it to target classes and achieves a greater gain in image retrieval when the domain gap from the source classes is larger.
TriDoNet: A Triple Domain Model-driven Network for CT Metal Artifact Reduction
Venue: arXiv cs.NI 2022
Year: 2022

Recent deep learning-based methods have achieved promising performance for computed tomography metal artifact reduction (CTMAR). However, most of them suffer from two limitations: (i) the domain knowledge is not fully embedded into the network training; (ii) metal artifacts lack effective representation models. The aforementioned limitations leave room for further performance improvement. Against these issues, we propose a novel triple domain model-driven CTMAR network, termed as TriDoNet, whose network training exploits triple domain knowledge, i.e., the knowledge of the sinogram, CT image, and metal artifact domains. Specifically, to explore the non-local repetitive streaking patterns of metal artifacts, we encode them as an explicit tight frame sparse representation model with adaptive thresholds. Furthermore, we design a contrastive regularization (CR) built upon contrastive learning to exploit clean CT images and metal-affected images as positive and negative samples, respectively. Experimental results show that our TriDoNet can generate superior artifact-reduced CT images.
Grafting Pre-trained Models for Multimodal Headline Generation
Venue: arXiv cs.CV 2022
Year: 2022

Multimodal headline utilizes both video frames and transcripts to generate the natural language title of the videos. Due to a lack of large-scale, manually annotated data, the task of annotating grounded headlines for video is labor intensive and impractical. Previous researches on pre-trained language models and video-language models have achieved significant progress in related downstream tasks. However, none of them can be directly applied to multimodal headline architecture where we need both multimodal encoder and sentence decoder. A major challenge in simply gluing language model and video-language model is the modality balance, which is aimed at combining visual-language complementary abilities. In this paper, we propose a novel approach to graft the video encoder from the pre-trained video-language model on the generative pre-trained language model. We also present a consensus fusion mechanism for the integration of different components, via inter/intra modality relation. Empirically, experiments show that the grafted model achieves strong results on a brand-new dataset collected from real-world applications.
SA-DPSGD: Differentially Private Stochastic Gradient Descent based on Simulated Annealing
Venue: arXiv cs.CR 2022
Year: 2022

Differential privacy (DP) provides a formal privacy guarantee that prevents adversaries with access to machine learning models from extracting information about individual training points. Differentially private stochastic gradient descent (DPSGD) is the most popular training method with differential privacy in image recognition. However, existing DPSGD schemes lead to significant performance degradation, which prevents the application of differential privacy. In this paper, we propose a simulated annealing-based differentially private stochastic gradient descent scheme (SA-DPSGD) which accepts a candidate update with a probability that depends both on the update quality and on the number of iterations. Through this random update screening, we make the differentially private gradient descent proceed in the right direction in each iteration, and result in a more accurate model finally. In our experiments, under the same hyperparameters, our scheme achieves test accuracies 98.35%, 87.41% and 60.92% on datasets MNIST, FashionMNIST and CIFAR10, respectively, compared to the state-of-the-art result of 98.12%, 86.33% and 59.34%. Under the freely adjusted hyperparameters, our scheme achieves even higher accuracies, 98.89%, 88.50% and 64.17%. We believe that our method has a great contribution for closing the accuracy gap between private and non-private image classification.
Buying Privacy: User Perceptions of Privacy Threats from Mobile Apps
Venue: arXiv cs.HC 2022
Year: 2022

As technology and technology companies have grown in power, ubiquity, and societal influence, some companies -- and notably some mobile apps -- have come to be perceived as privacy threats. Prior work has considered how various factors impact perceptions of threat, including social factors, political speech, and user-interface design. In this work, we investigate how user-visible context clues impact perceptions about whether a mobile application application poses a privacy threat. We conduct a user study with 2109 users in which we find that users depend on context clues -- such as presence of advertising and occurrence (and timing of payment) -- to determine the extent to which a mobile app poses a privacy threat. We also quantify how accurately user assessments match published data collection practices, and we identify a commonly-held misconception about how payments are processed. This work provides new insight into how users assess the privacy threat posed by mobile apps and into social norms around data collection.
Learning to Model Multimodal Semantic Alignment for Story Visualization
Venue: arXiv cs.CV 2022
Year: 2022

Story visualization aims to generate a sequence of images to narrate each sentence in a multi-sentence story, where the images should be realistic and keep global consistency across dynamic scenes and characters. Current works face the problem of semantic misalignment because of their fixed architecture and diversity of input modalities. To address this problem, we explore the semantic alignment between text and image representations by learning to match their semantic levels in the GAN-based generative model. More specifically, we introduce dynamic interactions according to learning to dynamically explore various semantic depths and fuse the different-modal information at a matched semantic level, which thus relieves the text-image semantic misalignment problem. Extensive experiments on different datasets demonstrate the improvements of our approach, neither using segmentation masks nor auxiliary captioning networks, on image quality and story consistency, compared with state-of-the-art methods.
Temporal patterns in insulin needs for Type 1 diabetes
Venue: arXiv cs.LG 2022
Year: 2022

Type 1 Diabetes (T1D) is a chronic condition where the body produces little or no insulin, a hormone required for the cells to use blood glucose (BG) for energy and to regulate BG levels in the body. Finding the right insulin dose and time remains a complex, challenging and as yet unsolved control task. In this study, we use the OpenAPS Data Commons dataset, which is an extensive dataset collected in real-life conditions, to discover temporal patterns in insulin need driven by well-known factors such as carbohydrates as well as potentially novel factors. We utilised various time series techniques to spot such patterns using matrix profile and multi-variate clustering. The better we understand T1D and the factors impacting insulin needs, the more we can contribute to building data-driven technology for T1D treatments.
Parallel Automatic History Matching Algorithm Using Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Reformulating the history matching problem from a least-square mathematical optimization problem into a Markov Decision Process introduces a method in which reinforcement learning can be utilized to solve the problem. This method provides a mechanism where an artificial deep neural network agent can interact with the reservoir simulator and find multiple different solutions to the problem. Such formulation allows for solving the problem in parallel by launching multiple concurrent environments enabling the agent to learn simultaneously from all the environments at once, achieving significant speed up.
Enabling Autonomous Teams and Continuous Deployment at Scale
Venue: arXiv cs.SE 2022
Year: 2022

In this article, we give advice on transitioning to a more agile delivery model for large-scale agile development projects based on experience from the Parental Benefit Project of the Norwegian Labour and Welfare Administration. The project modernized a central part of the organizations IT portfolio and included up to ten development teams working in parallel. The project successfully changed from using a delivery model which combined traditional project management elements and agile methods to a more agile delivery model with autonomous teams and continuous deployment. This transition was completed in tandem with the project execution. We identify key lessons learned which will be useful for other organizations considering similar changes and report how the new delivery model reduced risk and opened up a range of new possibilities for delivering the benefits of digitalization.
Findings of the Covid-19 MLIA Machine Translation Task
Venue: arXiv cs.CL 2022
Year: 2022

This work presents the results of the machine translation (MT) task from the Covid-19 MLIA @ Eval initiative, a community effort to improve the generation of MT systems focused on the current Covid-19 crisis. Nine teams took part in this event, which was divided in two rounds and involved seven different language pairs. Two different scenarios were considered: one in which only the provided data was allowed, and a second one in which the use of external resources was allowed. Overall, best approaches were based on multilingual models and transfer learning, with an emphasis on the importance of applying a cleaning process to the training data.