OpenRaiser/Intern-Atlas huggingface dataset:
Federated Learning Hyper-Parameter Tuning from a System Perspective
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning (FL) is a distributed model training paradigm that preserves clients' data privacy. It has gained tremendous attention from both academia and industry. FL hyper-parameters (e.g., the number of selected clients and the number of training passes) significantly affect the training overhead in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters imposes a heavy burden on FL practitioners because applications have different training preferences. In this paper, we propose FedTune, an automatic FL hyper-parameter tuning algorithm tailored to applications' diverse system requirements in FL training. FedTune iteratively adjusts FL hyper-parameters during FL training and can be easily integrated into existing FL systems. Through extensive evaluations of FedTune for diverse applications and FL aggregation algorithms, we show that FedTune is lightweight and effective, achieving 8.48%-26.75% system overhead reduction compared to using fixed FL hyper-parameters. This paper assists FL practitioners in designing high-performance FL training solutions. The source code of FedTune is available at https://github.com/DataSysTech/FedTune.
Estimating Regression Predictive Distributions with Sample Networks
Venue: arXiv cs.LG 2022
Year: 2022

Estimating the uncertainty in deep neural network predictions is crucial for many real-world applications. A common approach to model uncertainty is to choose a parametric distribution and fit the data to it using maximum likelihood estimation. The chosen parametric form can be a poor fit to the data-generating distribution, resulting in unreliable uncertainty estimates. In this work, we propose SampleNet, a flexible and scalable architecture for modeling uncertainty that avoids specifying a parametric form on the output distribution. SampleNets do so by defining an empirical distribution using samples that are learned with the Energy Score and regularized with the Sinkhorn Divergence. SampleNets are shown to be able to well-fit a wide range of distributions and to outperform baselines on large-scale real-world regression tasks.
Parallel Repetition for the GHZ Game: Exponential Decay
Venue: arXiv cs.CC 2022
Year: 2022

We show that the value of the $n$-fold repeated GHZ game is at most $2^{-Ω(n)}$, improving upon the polynomial bound established by Holmgren and Raz. Our result is established via a reduction to approximate subgroup type questions from additive combinatorics.
Sketch-Guided Text-to-Image Diffusion Models
Venue: arXiv cs.CV 2022
Year: 2022

Text-to-Image models have introduced a remarkable leap in the evolution of machine learning, demonstrating high-quality synthesis of images from a given text-prompt. However, these powerful pretrained models still lack control handles that can guide spatial properties of the synthesized images. In this work, we introduce a universal approach to guide a pretrained text-to-image diffusion model, with a spatial map from another domain (e.g., sketch) during inference time. Unlike previous works, our method does not require to train a dedicated model or a specialized encoder for the task. Our key idea is to train a Latent Guidance Predictor (LGP) - a small, per-pixel, Multi-Layer Perceptron (MLP) that maps latent features of noisy images to spatial maps, where the deep features are extracted from the core Denoising Diffusion Probabilistic Model (DDPM) network. The LGP is trained only on a few thousand images and constitutes a differential guiding map predictor, over which the loss is computed and propagated back to push the intermediate images to agree with the spatial map. The per-pixel training offers flexibility and locality which allows the technique to perform well on out-of-domain sketches, including free-hand style drawings. We take a particular focus on the sketch-to-image translation task, revealing a robust and expressive way to generate images that follow the guidance of a sketch of arbitrary style or domain. Project page: sketch-guided-diffusion.github.io
Design and Prototyping Distributed CNN Inference Acceleration in Edge Computing
Venue: arXiv cs.CV 2022
Year: 2022

For time-critical IoT applications using deep learning, inference acceleration through distributed computing is a promising approach to meet a stringent deadline. In this paper, we implement a working prototype of a new distributed inference acceleration method HALP using three raspberry Pi 4. HALP accelerates inference by designing a seamless collaboration among edge devices (EDs) in Edge Computing. We maximize the parallelization between communication and computation among the collaborative EDs by optimizing the task partitioning ratio based on the segment-based partitioning. Experimental results show that the distributed inference HALP achieves 1.7x inference acceleration for VGG-16. Then, we combine distributed inference with conventional neural network model compression by setting up different shrinking hyperparameters for MobileNet-V1. In this way, we can further accelerate inference but at the cost of inference accuracy loss. To strike a balance between latency and accuracy, we propose dynamic model selection to select a model which provides the highest accuracy within the latency constraint. It is shown that the model selection with distributed inference HALP can significantly improve service reliability compared to the conventional stand-alone computation.
On the Linguistic and Computational Requirements for Creating Face-to-Face Multimodal Human-Machine Interaction
Venue: arXiv cs.HC 2022
Year: 2022

In this study, conversations between humans and avatars are linguistically, organizationally, and structurally analyzed, focusing on what is necessary for creating face-to-face multimodal interfaces for machines. We videorecorded thirty-four human-avatar interactions, performed complete linguistic microanalysis on video excerpts, and marked all the occurrences of multimodal actions and events. Statistical inferences were applied to data, allowing us to comprehend not only how often multimodal actions occur but also how multimodal events are distributed between the speaker (emitter) and the listener (recipient). We also observed the distribution of multimodal occurrences for each modality. The data show evidence that double-loop feedback is established during a face-to-face conversation. This led us to propose that knowledge from Conversation Analysis (CA), cognitive science, and Theory of Mind (ToM), among others, should be incorporated into the ones used for describing human-machine multimodal interactions. Face-to-face interfaces require an additional control layer to the multimodal fusion layer. This layer has to organize the flow of conversation, integrate the social context into the interaction, as well as make plans concerning 'what' and 'how' to progress on the interaction. This higher level is best understood if we incorporate insights from CA and ToM into the interface system.
Detecting Entities in the Astrophysics Literature: A Comparison of Word-based and Span-based Entity Recognition Methods
Venue: arXiv cs.CL 2022
Year: 2022

Information Extraction from scientific literature can be challenging due to the highly specialised nature of such text. We describe our entity recognition methods developed as part of the DEAL (Detecting Entities in the Astrophysics Literature) shared task. The aim of the task is to build a system that can identify Named Entities in a dataset composed by scholarly articles from astrophysics literature. We planned our participation such that it enables us to conduct an empirical comparison between word-based tagging and span-based classification methods. When evaluated on two hidden test sets provided by the organizer, our best-performing submission achieved $F_1$ scores of 0.8307 (validation phase) and 0.7990 (testing phase).
Lessons Learned to Improve the UX Practices in Agile Projects Involving Data Science and Process Automation
Venue: arXiv cs.SE 2022
Year: 2022

Context: User-Centered Design and Agile methodologies focus on human issues. Nevertheless, agile methodologies focus on contact with contracting customers and generating value for them. Usually, the communication between end users and the agile team is mediated by customers. However, they do not know the problems end users face in their routines. Hence, UX issues are typically identified only after the implementation, during user testing and validation. Objective: Aiming to improve the understanding and definition of the problem in agile projects, this research investigates the practices and difficulties experienced by agile teams during the development of data science and process automation projects. Also, we analyze the benefits and the teams' perceptions regarding user participation in these projects. Method: We collected data from four agile teams in an academia-industry collaboration focusing on delivering data science and process automation solutions. Therefore, we applied a carefully designed questionnaire answered by developers, scrum masters, and UX designers. In total, 18 subjects answered the questionnaire. Results: From the results, we identify practices used by the teams to define and understand the problem and to represent the solution. The practices most often used are prototypes and meetings with stakeholders. Another practice that helped the team to understand the problem was using Lean Inceptions. Also, our results present some specific issues regarding data science projects. Conclusion: We observed that end-user participation can be critical to understanding and defining the problem. They help to define elements of the domain and barriers in the implementation. We identified a need for approaches that facilitate user-team communication in data science projects and the need for more detailed requirements representations to support data science solutions.
Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism
Venue: arXiv cs.LG 2022
Year: 2022

Transformer models have achieved state-of-the-art performance on various domains of applications and gradually becomes the foundations of the advanced large deep learning (DL) models. However, how to train these models over multiple GPUs efficiently is still challenging due to a large number of parallelism choices. Existing DL systems either rely on manual efforts to make distributed training plans or apply parallelism combinations within a very limited search space. In this approach, we propose Galvatron, a new system framework that incorporates multiple popular parallelism dimensions and automatically finds the most efficient hybrid parallelism strategy. To better explore such a rarely huge search space, we 1) involve a decision tree to make decomposition and pruning based on some reasonable intuitions, and then 2) design a dynamic programming search algorithm to generate the optimal plan. Evaluations on four representative Transformer workloads show that Galvatron could perform automatically distributed training with different GPU memory budgets. Among all evluated scenarios, Galvatron always achieves superior system throughput compared to previous work with limited parallelism.
Synthesis Cost-Optimal Targeted Mutant Protein Libraries
Venue: arXiv cs.DS 2022
Year: 2022

Protein variant libraries produced by site-directed mutagenesis are a useful tool utilized by protein engineers to explore variants with potentially improved properties, such as activity and stability. These libraries are commonly built by selecting residue positions and alternative beneficial mutations for each position. All possible combinations are then constructed and screened, by incorporating degenerate codons at mutation sites. These degenerate codons often encode additional unwanted amino acids or even STOP codons. Our study aims to take advantage of annealing based recombination of oligonucleotides during synthesis and utilize multiple degenerate codons per mutation site to produce targeted protein libraries devoid of unwanted variants. Toward this goal we created an algorithm to calculate the minimum number of degenerate codons necessary to specify any given amino acid set, and a dynamic programming method that uses this algorithm to optimally partition a DNA target sequence with degeneracies into overlapping oligonucleotides, such that the total cost of synthesis of the target mutant protein library is minimized. Computational experiments show that, for a modest increase in DNA synthesis costs, beneficial variant yields in produced mutant libraries are increased by orders of magnitude, an effect particularly pronounced in large combinatorial libraries.
Fault-Tolerant Offline Multi-Agent Path Planning
Venue: arXiv cs.RO 2022
Year: 2022

We study a novel graph path planning problem for multiple agents that may crash at runtime, and block part of the workspace. In our setting, agents can detect neighboring crashed agents, and change followed paths at runtime. The objective is then to prepare a set of paths and switching rules for each agent, ensuring that all correct agents reach their destinations without collisions or deadlocks, despite unforeseen crashes of other agents. Such planning is attractive to build reliable multi-robot systems. We present problem formalization, theoretical analysis such as computational complexities, and how to solve this offline planning problem.
On DNA Codes Over the Non-Chain Ring $\mathbb{Z}_4+u\mathbb{Z}_4+u^2\mathbb{Z}_4$ with $u^3=1$
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we present a novel design strategy of DNA codes with length $3n$ over the non-chain ring $R=\mathbb{Z}_4+u\mathbb{Z}_4+u^2\mathbb{Z}_4$ with $64$ elements and $u^3=1$, where $n$ denotes the length of a code over $R$. We first study and analyze a distance conserving map defined over the ring $R$ into the length-$3$ DNA sequences. Then, we derive some conditions on the generator matrix of a linear code over $R$, which leads to a DNA code with reversible, reversible-complement, homopolymer $2$-run-length, and $\frac{w}{3n}$-GC-content constraints for integer $w$ ($0\leq w\leq 3n$). Finally, we propose a new construction of DNA codes using Reed-Muller type generator matrices. This allows us to obtain DNA codes with reversible, reversible-complement, homopolymer $2$-run-length, and $\frac{2}{3}$-GC-content constraints.
Spatial-Temporal Attention Network for Open-Set Fine-Grained Image Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Triggered by the success of transformers in various visual tasks, the spatial self-attention mechanism has recently attracted more and more attention in the computer vision community. However, we empirically found that a typical vision transformer with the spatial self-attention mechanism could not learn accurate attention maps for distinguishing different categories of fine-grained images. To address this problem, motivated by the temporal attention mechanism in brains, we propose a spatial-temporal attention network for learning fine-grained feature representations, called STAN, where the features learnt by implementing a sequence of spatial self-attention operations corresponding to multiple moments are aggregated progressively. The proposed STAN consists of four modules: a self-attention backbone module for learning a sequence of features with self-attention operations, a spatial feature self-organizing module for facilitating the model training, a spatial-temporal feature learning module for aggregating the re-organized features via a Long Short-Term Memory network, and a context-aware module that is implemented as the forget block of the spatial-temporal feature learning module for preserving/forgetting the long-term memory by utilizing contextual information. Then, we propose a STAN-based method for open-set fine-grained recognition by integrating the proposed STAN network with a linear classifier, called STAN-OSFGR. Extensive experimental results on 3 fine-grained datasets and 2 coarse-grained datasets demonstrate that the proposed STAN-OSFGR outperforms 9 state-of-the-art open-set recognition methods significantly in most cases.
Combinatorial Civic Crowdfunding with Budgeted Agents: Welfare Optimality at Equilibrium and Optimal Deviation
Venue: arXiv cs.GT 2022
Year: 2022

Civic Crowdfunding (CC) uses the ``power of the crowd'' to garner contributions towards public projects. As these projects are non-excludable, agents may prefer to ``free-ride,'' resulting in the project not being funded. For single project CC, researchers propose to provide refunds to incentivize agents to contribute, thereby guaranteeing the project's funding. These funding guarantees are applicable only when agents have an unlimited budget. This work focuses on a combinatorial setting, where multiple projects are available for CC and agents have a limited budget. We study certain specific conditions where funding can be guaranteed. Further, funding the optimal social welfare subset of projects is desirable when every available project cannot be funded due to budget restrictions. We prove the impossibility of achieving optimal welfare at equilibrium for any monotone refund scheme. We then study different heuristics that the agents can use to contribute to the projects in practice. Through simulations, we demonstrate the heuristics' performance as the average-case trade-off between welfare obtained and agent utility.
On picking sequences for chores
Venue: arXiv cs.GT 2022
Year: 2022

We consider the problem of allocating $m$ indivisible chores to $n$ agents with additive disvaluation (cost) functions. It is easy to show that there are picking sequences that give every agent (that uses the greedy picking strategy) a bundle of chores of disvalue at most twice her share value (maximin share, MMS, for agents of equal entitlement, and anyprice share, APS, for agents of arbitrary entitlement). Aziz, Li and Wu (2022) designed picking sequences that improve this ratio to $\frac{5}{3}$ for the case of equal entitlement. We design picking sequences that improve the ratio to~1.733 for the case of arbitrary entitlement, and to $\frac{8}{5}$ for the case of equal entitlement. (In fact, computer assisted analysis suggests that the ratio is smaller than $1.543$ in the equal entitlement case.) We also prove a lower bound of $\frac{3}{2}$ on the obtainable ratio when $n$ is sufficiently large. Additional contributions of our work include improved guarantees in the equal entitlement case when $n$ is small; introduction of the chore share as a convenient proxy to other share notions for chores; introduction of ex-ante notions of envy for risk averse agents; enhancements to our picking sequences that eliminate such envy; showing that a known allocation algorithm (not based on picking sequences) for the equal entitlement case gives each agent a bundle of disvalue at most $\frac{4n-1}{3n}$ times her APS (previously, this ratio was shown for this algorithm with respect to the easier benchmark of the MMS).
Generating 2D and 3D Master Faces for Dictionary Attacks with a Network-Assisted Latent Space Evolution
Venue: arXiv cs.CR 2022
Year: 2022

A master face is a face image that passes face-based identity authentication for a high percentage of the population. These faces can be used to impersonate, with a high probability of success, any user, without having access to any user information. We optimize these faces for 2D and 3D face verification models, by using an evolutionary algorithm in the latent embedding space of the StyleGAN face generator. For 2D face verification, multiple evolutionary strategies are compared, and we propose a novel approach that employs a neural network to direct the search toward promising samples, without adding fitness evaluations. The results we present demonstrate that it is possible to obtain a considerable coverage of the identities in the LFW or RFW datasets with less than 10 master faces, for six leading deep face recognition systems. In 3D, we generate faces using the 2D StyleGAN2 generator and predict a 3D structure using a deep 3D face reconstruction network. When employing two different 3D face recognition systems, we are able to obtain a coverage of 40%-50%. Additionally, we present the generation of paired 2D RGB and 3D master faces, which simultaneously match 2D and 3D models with high impersonation rates.
Quantum Software Engineering: A New Genre of Computing
Venue: arXiv cs.SE 2022
Year: 2022

Quantum computing (QC) is no longer only a scientific interest but is rapidly becoming an industrially available technology that can potentially tackle the limitations of classical computing. Over the last few years, major technology giants have invested in developing hardware and programming frameworks to develop quantum-specific applications. QC hardware technologies are gaining momentum, however, operationalizing the QC technologies trigger the need for software-intensive methodologies, techniques, processes, tools, roles, and responsibilities for developing industrial-centric quantum software applications. This paper presents the vision of the quantum software engineering (QSE) life cycle consisting of quantum requirements engineering, quantum software design, quantum software implementation, quantum software testing, and quantum software maintenance. This paper particularly calls for joint contributions of software engineering research and industrial community to present real-world solutions to support the entire quantum software development activities. The proposed vision facilitates the researchers and practitioners to propose new processes, reference architectures, novel tools, and practices to leverage quantum computers and develop emerging and next generations of quantum software.
A Vision of DevOps Requirements Change Management Standardization
Venue: arXiv cs.SE 2022
Year: 2022

DevOps (development and operations) aims to shorten the software development process and provide continuous delivery with high software quality. To get the potential gains of DevOps, the software development industry considering global software development (GSD) environment to hire skilled human resources and round-the-clock working hours. However, due to the lack of frequent communication and coordination in GSD, the planning and managing of the requirements change process becomes a challenging task. As in DevOps, requirements are not only shaped by development feedback but also by the operations team. This means requirements affect development, development affects operations and operations affect requirements. However, DevOps in GSD still faces many challenges in terms of requirement management. The purpose of this research project is to develop a DevOps requirement change management and implementation maturity model (DevOps-RCMIMM) that could assist the GSD organizations in modifying and improving their requirement management process in the DevOps process. The development of DevOps-RCMIMM will be based on the existing DevOps and RCM literature, industrial empirical study, and understanding of factors that could impact the implementation of the DevOps requirement change management process in the domain of GSD. This vision study presents the initial results of a systematic literature review that will contribute to the development of maturity levels of the proposed DevOps-RCMIMM.