OpenRaiser/Intern-Atlas huggingface dataset:
Deep Generative Models for Downlink Channel Estimation in FDD Massive MIMO Systems
Venue: arXiv cs.IT 2022
Year: 2022

It is well accepted that acquiring downlink channel state information in frequency division duplexing (FDD) massive multiple-input multiple-output (MIMO) systems is challenging because of the large overhead in training and feedback. In this paper, we propose a deep generative model (DGM)-based technique to address this challenge. Exploiting the partial reciprocity of uplink and downlink channels, we first estimate the frequency-independent underlying channel parameters, i.e., the magnitudes of path gains, delays, angles-of-arrivals (AoAs) and angles-of-departures (AoDs), via uplink training, since these parameters are common in both uplink and downlink. Then, the frequency-specific underlying channel parameters, namely, the phase of each propagation path, are estimated via downlink training using a very short training signal. In the first step, we incorporate the underlying distribution of the channel parameters as a prior into our channel estimation algorithm. We use DGMs to learn this distribution. Simulation results indicate that our proposed DGM-based channel estimation technique outperforms, by a large gap, the conventional channel estimation techniques in practical ranges of signal-to-noise ratio (SNR). In addition, a near-optimal performance is achieved using only few downlink pilot measurements.
Do Small Code Changes Merge Faster? A Multi-Language Empirical Investigation
Venue: arXiv cs.SE 2022
Year: 2022

Code velocity, or the speed with which code changes are integrated into a production environment, plays a crucial role in Continuous Integration and Continuous Deployment. Many studies report factors influencing code velocity. However, solutions to increase code velocity are unclear. Meanwhile, the industry continues to issue guidelines on "ideal" code change size, believing it increases code velocity despite lacking evidence validating the practice. Surprisingly, this fundamental question has not been studied to date. This study investigates the practicality of improving code velocity by optimizing pull request size and composition (ratio of insertions, deletions, and modifications). We start with a hypothesis that a moderate correlation exists between pull request size and time-to-merge. We selected 100 most popular, actively developed projects from 10 programming languages on GitHub. We analyzed our dataset of 845,316 pull requests by size, composition, and context to explore its relationship to time-to-merge - a proxy to measure code velocity. Our study shows that pull request size and composition do not relate to time-to-merge. Regardless of the contextual factors that can influence pull request size or composition (e.g., programming language), the observation holds. Pull request data from two other platforms: Gerrit and Phabricator (401,790 code reviews) confirms the lack of relationship. This negative result as in "... eliminate useless hypotheses ..." challenges a widespread belief by showing that small code changes do not merge faster to increase code velocity.
Mining Code Review Data to Understand Waiting Times Between Acceptance and Merging: An Empirical Analysis
Venue: arXiv cs.SE 2022
Year: 2022

Increasing code velocity (or the speed with which code changes are reviewed and merged) is integral to speeding up development and contributes to the work satisfaction of engineers. While factors affecting code change acceptance have been investigated in the past, solutions to decrease the code review lifetime are less understood. This study investigates the code review process to quantify delays and investigate opportunities to potentially increase code velocity. We study the temporal characteristics of half a million code reviews hosted on Gerrit and Phabricator, starting from the first response, to a decision to accept or reject the changes, and until the changes are merged into a target branch. We identified two types of time delays: (a) the wait time from the proposal of code changes until first response, and (b) the wait time between acceptance and merging. Our study indicates that reducing the time between acceptance and merging has the potential to speed up Phabricator code reviews by 29-63%. Small code changes and changes made by authors with a large number of previously accepted code reviews have a higher chance of being immediately accepted, without code review iterations. Our analysis suggests that switching from manual to automatic merges can help increase code velocity.
HealthPrompt: A Zero-shot Learning Paradigm for Clinical Natural Language Processing
Venue: arXiv cs.CL 2022
Year: 2022

Deep learning algorithms are dependent on the availability of large-scale annotated clinical text datasets. The lack of such publicly available datasets is the biggest bottleneck for the development of clinical Natural Language Processing(NLP) systems. Zero-Shot Learning(ZSL) refers to the use of deep learning models to classify instances from new classes of which no training data have been seen before. Prompt-based learning is an emerging ZSL technique where we define task-based templates for NLP tasks. We developed a novel prompt-based clinical NLP framework called HealthPrompt and applied the paradigm of prompt-based learning on clinical texts. In this technique, rather than fine-tuning a Pre-trained Language Model(PLM), the task definitions are tuned by defining a prompt template. We performed an in-depth analysis of HealthPrompt on six different PLMs in a no-data setting. Our experiments prove that prompts effectively capture the context of clinical texts and perform remarkably well without any training data.
The Transitive Information Theory and its Application to Deep Generative Models
Venue: arXiv cs.LG 2022
Year: 2022

Paradoxically, a Variational Autoencoder (VAE) could be pushed in two opposite directions, utilizing powerful decoder model for generating realistic images but collapsing the learned representation, or increasing regularization coefficient for disentangling representation but ultimately generating blurry examples. Existing methods narrow the issues to the rate-distortion trade-off between compression and reconstruction. We argue that a good reconstruction model does learn high capacity latents that encode more details, however, its use is hindered by two major issues: the prior is random noise which is completely detached from the posterior and allow no controllability in the generation; mean-field variational inference doesn't enforce hierarchy structure which makes the task of recombining those units into plausible novel output infeasible. As a result, we develop a system that learns a hierarchy of disentangled representation together with a mechanism for recombining the learned representation for generalization. This is achieved by introducing a minimal amount of inductive bias to learn controllable prior for the VAE. The idea is supported by here developed transitive information theory, that is, the mutual information between two target variables could alternately be maximized through the mutual information to the third variable, thus bypassing the rate-distortion bottleneck in VAE design. In particular, we show that our model, named SemafoVAE (inspired by the similar concept in computer science), could generate high-quality examples in a controllable manner, perform smooth traversals of the disentangled factors and intervention at a different level of representation hierarchy.
Compilable Neural Code Generation with Compiler Feedback
Venue: arXiv cs.CL 2022
Year: 2022

Automatically generating compilable programs with (or without) natural language descriptions has always been a touchstone problem for computational linguistics and automated software engineering. Existing deep-learning approaches model code generation as text generation, either constrained by grammar structures in decoder, or driven by pre-trained language models on large-scale code corpus (e.g., CodeGPT, PLBART, and CodeT5). However, few of them account for compilability of the generated programs. To improve compilability of the generated programs, this paper proposes COMPCODER, a three-stage pipeline utilizing compiler feedback for compilable code generation, including language model fine-tuning, compilability reinforcement, and compilability discrimination. Comprehensive experiments on two code generation tasks demonstrate the effectiveness of our proposed approach, improving the success rate of compilation from 44.18 to 89.18 in code completion on average and from 70.3 to 96.2 in text-to-code generation, respectively, when comparing with the state-of-the-art CodeGPT.
Manifold Modeling in Quotient Space: Learning An Invariant Mapping with Decodability of Image Patches
Venue: arXiv cs.CV 2022
Year: 2022

This study proposes a framework for manifold learning of image patches using the concept of equivalence classes: manifold modeling in quotient space (MMQS). In MMQS, we do not consider a set of local patches of the image as it is, but rather the set of their canonical patches obtained by introducing the concept of equivalence classes and performing manifold learning on their canonical patches. Canonical patches represent equivalence classes, and their auto-encoder constructs a manifold in the quotient space. Based on this framework, we produce a novel manifold-based image model by introducing rotation-flip-equivalence relations. In addition, we formulate an image reconstruction problem by fitting the proposed image model to a corrupted observed image and derive an algorithm to solve it. Our experiments show that the proposed image model is effective for various self-supervised image reconstruction tasks, such as image inpainting, deblurring, super-resolution, and denoising.
TiSAT: Time Series Anomaly Transformer
Venue: arXiv cs.LG 2022
Year: 2022

While anomaly detection in time series has been an active area of research for several years, most recent approaches employ an inadequate evaluation criterion leading to an inflated F1 score. We show that a rudimentary Random Guess method can outperform state-of-the-art detectors in terms of this popular but faulty evaluation criterion. In this work, we propose a proper evaluation metric that measures the timeliness and precision of detecting sequential anomalies. Moreover, most existing approaches are unable to capture temporal features from long sequences. Self-attention based approaches, such as transformers, have been demonstrated to be particularly efficient in capturing long-range dependencies while being computationally efficient during training and inference. We also propose an efficient transformer approach for anomaly detection in time series and extensively evaluate our proposed approach on several popular benchmark datasets.
Unpacking Invisible Work Practices, Constraints, and Latent Power Relationships in Child Welfare through Casenote Analysis
Venue: arXiv cs.HC 2022
Year: 2022

Caseworkers are trained to write detailed narratives about families in Child-Welfare (CW) which informs collaborative high-stakes decision-making. Unlike other administrative data, these narratives offer a more credible source of information with respect to workers' interactions with families as well as underscore the role of systemic factors in decision-making. SIGCHI researchers have emphasized the need to understand human discretion at the street-level to be able to design human-centered algorithms for the public sector. In this study, we conducted computational text analysis of casenotes at a child-welfare agency in the midwestern United States and highlight patterns of invisible street-level discretionary work and latent power structures that have direct implications for algorithm design. Casenotes offer a unique lens for policymakers and CW leadership towards understanding the experiences of on-the-ground caseworkers. As a result of this study, we highlight how street-level discretionary work needs to be supported by sociotechnical systems developed through worker-centered design. This study offers the first computational inspection of casenotes and introduces them to the SIGCHI community as a critical data source for studying complex sociotechnical systems.
A Systematic Literature Review on Blockchain Enabled Federated Learning Framework for Internet of Vehicles
Venue: arXiv cs.CR 2022
Year: 2022

While the convergence of Artificial Intelligence (AI) techniques with improved information technology systems ensured enormous benefits to the Internet of Vehicles (IoVs) systems, it also introduced an increased amount of security and privacy threats. To ensure the security of IoVs data, privacy preservation methodologies have gained significant attention in the literature. However, these strategies also need specific adjustments and modifications to cope with the advances in IoVs design. In the interim, Federated Learning (FL) has been proven as an emerging idea to protect IoVs data privacy and security. On the other hand, Blockchain technology is showing prominent possibilities with secured, dispersed, and auditable data recording and sharing schemes. In this paper, we present a comprehensive survey on the application and implementation of Blockchain-Enabled Federated Learning frameworks for IoVs. Besides, probable issues, challenges, solutions, and future research directions for BC-Enabled FL frameworks for IoVs are also presented. This survey can further be used as the basis for developing modern BC-Enabled FL solutions to resolve different data privacy issues and scenarios of IoVs.
Online Deep Metric Learning via Mutual Distillation
Venue: arXiv cs.CV 2022
Year: 2022

Deep metric learning aims to transform input data into an embedding space, where similar samples are close while dissimilar samples are far apart from each other. In practice, samples of new categories arrive incrementally, which requires the periodical augmentation of the learned model. The fine-tuning on the new categories usually leads to poor performance on the old, which is known as "catastrophic forgetting". Existing solutions either retrain the model from scratch or require the replay of old samples during the training. In this paper, a complete online deep metric learning framework is proposed based on mutual distillation for both one-task and multi-task scenarios. Different from the teacher-student framework, the proposed approach treats the old and new learning tasks with equal importance. No preference over the old or new knowledge is caused. In addition, a novel virtual feature estimation approach is proposed to recover the features assumed to be extracted by the old models. It allows the distillation between the new and the old models without the replay of old training samples or the holding of old models during the training. A comprehensive study shows the superior performance of our approach with the support of different backbones.
Organisations (de-)centralised to a greater or lesser degree for allocating cities in two Multiple Travelling Salesmen Problems
Venue: arXiv cs.GT 2022
Year: 2022

Decisions in organisations may be made either by a Central Authority (CA), e.g., in a hierarchy, or by the agents in a decentralised way, e.g., in a heterarchy. Since both kinds of organisations have their advantages (e.g., optimality for centralised organisations and reactivity for decentralised ones), our goal is ultimately to understand when and how to use each of them. Our previous work proposed a variant of the Multiple Travelling Salesmen Problem, which we now call MTSPs . We use the subscript "s" to refer to salesmen's selfishness when they minimise their individual route length. If, on the contrary, they are assumed to be benevolent, we add subscript "b" and thus the term MTSPb to refer to the traditional MTSP in which the salesmen minimise the total route length. This article shows how to obtain such benevolent agents by slightly modifying selfish agents. We can then compare organisations which are (de-)centralised to a greater or lesser degree, which enables us to carry out the allocation of cities in the MTSPb . The first experiment shows that the relative efficiency (ranking) of the organisations differs between MTSPb and MTSPs . Since reactivity fosters decentralisation, the second experiment gradually reduces the time taken for it to impact this ranking. Both experiments show that pure centralisation is either the best or the worst option, and that the zone between the two situations is very narrow.
Realizing Implicit Computational Complexity
Venue: arXiv cs.CC 2022
Year: 2022

This abstract aims at presenting an ongoing effort to apply a novel typing mechanism stemming from Implicit Computational Complexity (ICC), that tracks dependencies between variables in three different ways, at different stages of maturation.The first and third projects bend the original typing discipline to gain finer-grained view on statements independence, to optimize loops by hoisting invariant and by splitting loops "horizontally" to parallelize them more efficiently.The second project refines and implements the original analysis to obtain a fast, modular static analyzer.All three projects aims at pushing the original type system, inspired from ICC, to its limits, to assess how ICC can in practice leads to original, sometimes orthogonal, approaches.
Beyond the Badge: Reproducibility Engineering as a Lifetime Skill
Venue: arXiv cs.SE 2022
Year: 2022

Ascertaining reproducibility of scientific experiments is receiving increased attention across disciplines. We argue that the necessary skills are important beyond pure scientific utility, and that they should be taught as part of software engineering (SWE) education. They serve a dual purpose: Apart from acquiring the coveted badges assigned to reproducible research, reproducibility engineering is a lifetime skill for a professional industrial career in computer science. SWE curricula seem an ideal fit for conveying such capabilities, yet they require some extensions, especially given that even at flagship conferences like ICSE, only slightly more than one-third of the technical papers (at the 2021 edition) receive recognition for artefact reusability. Knowledge and capabilities in setting up engineering environments that allow for reproducing artefacts and results over decades (a standard requirement in many traditional engineering disciplines), writing semi-literate commit messages that document crucial steps of a decision-making process and that are tightly coupled with code, or sustainably taming dynamic, quickly changing software dependencies, to name a few: They all contribute to solving the scientific reproducibility crisis, and enable software engineers to build sustainable, long-term maintainable, software-intensive, industrial systems. We propose to teach these skills at the undergraduate level, on par with traditional SWE topics.
Use of Digital Technologies in Public Health Responses to Tackle Covid-19: the Bangladesh Perspective
Venue: arXiv cs.CY 2022
Year: 2022

This paper aims to study the fight against COVID-19 in Bangladesh and digital intervention initiatives. To achieve the purpose of our research, we conducted a methodical review of online content. We have reviewed the first digital intervention that COVID-19 has been used to fight against worldwide. Then we reviewed the initiatives that have been taken in Bangladesh. Our paper has shown that while Bangladesh can take advantage of the digital intervention approach, it will require rigorous collaboration between government organizations and universities to get the most out of it. Public health can become increasingly digital in the future, and we are reviewing international alignment requirements. This exploration also focused on the strategies for controlling, evaluating, and using digital technology to strengthen epidemic management and future preparations for COVID-19.
TrueType Transformer: Character and Font Style Recognition in Outline Format
Venue: arXiv cs.CV 2022
Year: 2022

We propose TrueType Transformer (T3), which can perform character and font style recognition in an outline format. The outline format, such as TrueType, represents each character as a sequence of control points of stroke contours and is frequently used in born-digital documents. T3 is organized by a deep neural network, so-called Transformer. Transformer is originally proposed for sequential data, such as text, and therefore appropriate for handling the outline data. In other words, T3 directly accepts the outline data without converting it into a bitmap image. Consequently, T3 realizes a resolution-independent classification. Moreover, since the locations of the control points represent the fine and local structures of the font style, T3 is suitable for font style classification, where such structures are very important. In this paper, we experimentally show the applicability of T3 in character and font style recognition tasks, while observing how the individual control points contribute to classification results.
EyeLoveGAN: Exploiting domain-shifts to boost network learning with cycleGANs
Venue: arXiv cs.CV 2022
Year: 2022

This paper presents our contribution to the REFUGE challenge 2020. The challenge consisted of three tasks based on a dataset of retinal images: Segmentation of optic disc and cup, classification of glaucoma, and localization of fovea. We propose employing convolutional neural networks for all three tasks. Segmentation is performed using a U-Net, classification is performed by a pre-trained InceptionV3 network, and fovea detection is performed by employing stacked hour-glass for heatmap prediction. The challenge dataset contains images from three different data sources. To enhance performance, cycleGANs were utilized to create a domain-shift between the data sources. These cycleGANs move images across domains, thus creating artificial images which can be used for training.
A Contribution-based Device Selection Scheme in Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

In a Federated Learning (FL) setup, a number of devices contribute to the training of a common model. We present a method for selecting the devices that provide updates in order to achieve improved generalization, fast convergence, and better device-level performance. We formulate a min-max optimization problem and decompose it into a primal-dual setup, where the duality gap is used to quantify the device-level performance. Our strategy combines \emph{exploration} of data freshness through a random device selection with \emph{exploitation} through simplified estimates of device contributions. This improves the performance of the trained model both in terms of generalization and personalization. A modified Truncated Monte-Carlo (TMC) method is applied during the exploitation phase to estimate the device's contribution and lower the communication overhead. The experimental results show that the proposed approach has a competitive performance, with lower communication overhead and competitive personalization performance against the baseline schemes.
Semantic Norm Recognition and its application to Portuguese Law
Venue: arXiv cs.CL 2022
Year: 2022

Being able to clearly interpret legal texts and fully understanding our rights, obligations and other legal norms has become progressively more important in the digital society. However, simply giving citizens access to the laws is not enough, as there is a need to provide meaningful information that cater to their specific queries and needs. For this, it is necessary to extract the relevant semantic information present in legal texts. Thus, we introduce the SNR (Semantic Norm Recognition) system, an automatic semantic information extraction system trained on a domain-specific (legal) text corpus taken from Portuguese Consumer Law. The SNR system uses the Portuguese Bert (BERTimbau) and was trained on a legislative Portuguese corpus. We demonstrate how our system achieved good results (81.44\% F1-score) on this domain-specific corpus, despite existing noise, and how it can be used to improve downstream tasks such as information retrieval.