Reviewerly/exHarmony huggingface dataset:
Demystifying software release note issues on GitHub

Release notes (RNs) summarize main changes between two consecutive software versions and serve as a central source of information when users upgrade software. While producing high quality RNs can be hard and poses a variety of challenges to developers, a comprehensive empirical understanding of these challenges is still lacking. In this paper, we bridge this knowledge gap by manually analyzing 1,731 latest GitHub issues to build a comprehensive taxonomy of RN issues with four dimensions: Content, Presentation, Accessibility, and Production. Among these issues, nearly half (48.47%) of them focus on Production; Content, Accessibility, and Presentation take 25.61%, 17.65%, and 8.27%, respectively. We find that: 1) RN producers are more likely to miss information than to include incorrect information, especially for breaking changes; 2) improper layout may bury important information and confuse users; 3) many users find RNs inaccessible due to link deterioration, lack of notification, and obfuscate RN locations; 4) automating and regulating RN production remains challenging despite the great needs of RN producers. Our taxonomy not only pictures a roadmap to improve RN production in practice but also reveals interesting future research directions for automating RN production.
An Information Theoretic Approach for Attention-Driven Face Forgery Detection

Recently, Deepfake arises as a powerful tool to fool the existing real-world face detection systems, which has received wide attention in both academia and society. Most existing forgery face detection methods use heuristic clues to build a binary forgery detector, which mainly takes advantage of the empirical observation based on abnormal texture, blending clues, or high-frequency noise, etc.. However, heuristic clues only reflect certain aspects of the forgery, which might lead to model bias or sub-optimization. Our recent observations indicate that most of the forgery clues are hidden in the informative region, which can be measured quantitatively by the classic information maximization theory. Motivated by this, we make the first attempt to introduce the self-information metric to enhance the feature representation for forgery detection. The proposed metric can be formulated as a plug-and-play block, termed self-information attention (SIA) module, which can be integrated with most of the top-performance deep models to boost their detection performance. The SIA module can explicitly help the model locate the informative regions and recalibrate channel-wise feature responses, which improves both model's performance and generalization with few additional parameters. Extensive experiments on several large-scale benchmarks demonstrate the superiority of the proposed method against the state-of-the-art competitors.
Discrete time convolution for fast event-based stereo

Inspired by biological retina, dynamical vision sensor transmits events of instantaneous changes of pixel intensity, giving it a series of advantages over traditional frame-based camera, such as high dynamical range, high temporal resolution and low power consumption. However, extracting information from highly asynchronous event data is a challenging task. Inspired by continuous dynamics of biological neuron models, we propose a novel encoding method for sparse events-continuous time convolution (CTC)-which learns to model the spatial feature of the data with intrinsic dynamics. Adopting channel-wise parameterization, temporal dynamics of the model is synchronized on the same feature map and diverges across different ones, enabling it to embed data in a variety of temporal scales. Abstracted from CTC, we further develop discrete time convolution (DTC) which accelerates the process with lower computational cost. We apply these methods to event-based multi- view stereo matching where they surpass state-of-the-art methods on benchmark criteria of the MVSEC dataset. Spatially sparse event data often leads to inaccurate estimation of edges and local contours. To address this problem, we propose a dual-path architecture in which the feature map is complemented by underlying edge information from original events extracted with spatially-adaptive denormal-ization. We demonstrate the superiority of our model in terms of speed (up to 110 FPS), accuracy and robustness, showing a great potential for real-time fast depth estimation. Finally, we perform experiments on the recent DSEC dataset to demonstrate the general usage of our model.
Personalized Edge Intelligence via Federated Self-Knowledge Distillation

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Federated Learning</i> (FL) is an emerging approach in edge computing for collaboratively training machine learning models among multiple devices, which aims to address limited bandwidth, system heterogeneity, and privacy issues in traditional centralized training. However, the existing federated learning methods focus on learning a shared global model for all devices, which may not always be ideal for different devices. Such situations become even worse when each edge device has its own data distribution or task. In this paper, we study personalized federated learning in which our goal is to train models to perform well for individual clients. We observe that the initialization in each communication round causes the forgetting of historical personalized knowledge. Based on this observation, we propose a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Personalized Federated Learning</i> (PFL) framework via self-knowledge distillation, named pFedSD. By allowing clients to distill the knowledge of previous personalized models to current local models, pFedSD accelerates the process of recalling the personalized knowledge for the latest initialized clients. Moreover, self-knowledge distillation provides different views of data in feature space to realize an implicit ensemble of local models. Extensive experiments on various datasets and settings demonstrate the effectiveness and robustness of pFedSD.
Multi-Objective Adaptive Manta-Ray Foraging Optimization for Workflow Scheduling with Selected Virtual Machines Using Time-Series-Based Prediction

Optimization problems are challenging, but the larger challenge is to deal with the energy issue of the cloud, with continuous dynamic load and fluctuating VM performance. The optimization technique aids in efficient task-resource mapping ensuring optimal resource utilization with minimum active hosts and energy consumption. Existing works focused on time-invariant and bounded VM performance with major concentration on minimizing the execution cost and time. A multi-objective adaptive manta-ray foraging optimization (MAMFO) has been proposed in the paper for efficient scheduling with optimum resource utilization and energy consumption. The paper contributes by considering the time-varying VM performance and performance prediction using a dynamic time-series based ARIMA model, filtering out the VMs with larger fluctuating possibility, and employing only the selected VMs to be scheduled using MAMFO to meet the optimization goal with minimum SLA violations. The experimental analysis improves the work efficiency (e.g., energy consumption attained to be 0.405 kWh, and 5.97% of SLA violations).
Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis

Video-to-Video synthesis (Vid2Vid) has achieved remarkable results in generating a photo-realistic video from a sequence of semantic maps. However, this pipeline suffers from high computational cost and long inference latency, which largely depends on two essential factors: 1) network architecture parameters, 2) sequential data stream. Recently, the parameters of image-based generative models have been significantly compressed via more efficient network architectures. Existing methods mainly focus on slimming network architectures and ignore the size of the sequential data stream. Moreover, due to the lack of temporal coherence, image-based compression is not sufficient for the compression of the video task. In this paper, we present a spatial-temporal compression framework, Fast-Vid2Vid, which focuses on data aspects of generative models. It makes the first attempt at time dimension to reduce computational resources and accelerate inference. Specifically, we compress the input data stream spatially and reduce the temporal redundancy. After the proposed spatial-temporal knowledge distillation, our model can synthesize key-frames using the low-resolution data stream. Finally, Fast-Vid2Vid interpolates intermediate frames by motion compensation with slight latency. On standard benchmarks, Fast-Vid2Vid achieves around real-time performance as 20 FPS and saves around $$8\times $$ computational cost on a single V100 GPU. Code and models are publicly available (Project page: https://fast-vid2vid.github.io/ , Code and models: https://github.com/fast-vid2vid/fast-vid2vid ).
Voice Quality Features for Replay Attack Detection

Replay attacks are attempts to get fraudulent access to an automatic speaker verification system. In this paper, we investigate the usefulness of voice quality features to detect replay attacks. The voice quality features are used together with the state-of-the-art constant Q cepstral coefficients (CQCC) features. The two feature sets are fused at the score level. Thus, the log-likelihood scores estimated from the two feature sets are linearly weighted to obtain a single fused score. The fused score is used to classify whether a given speech sample is genuine or spoofed. Our experiments with the ASVspoof 2017 dataset demonstrate that the fusion of log-likelihood scores extracted from the CQCC and voice quality features improve the Equal Error Rate (EER) compared to the baseline system which is based only on CQCC features.
Virtual Correspondence: Humans as a Cue for Extreme-View Geometry

Recovering the spatial layout of the cameras and the geometry of the scene from extreme-view images is a longstanding challenge in computer vision. Prevailing 3D reconstruction algorithms often adopt the image matching paradigm and presume that a portion of the scene is covisible across images, yielding poor performance when there is little overlap among inputs. In contrast, humans can associate visible parts in one image to the corresponding invisible components in another image via prior knowledge of the shapes. Inspired by this fact, we present a novel concept called virtual correspondences (VCs). VCs are a pair of pixels from two images whose camera rays intersect in 3D. Similar to classic correspondences, VCs conform with epipolar geometry; unlike classic correspondences, VCs do not need to be co-visible across views. Therefore VCs can be established and exploited even if images do not overlap. We introduce a method to find virtual correspondences based on humans in the scene. We showcase how VCs can be seamlessly integrated with classic bundle adjustment to recover camera poses across extreme views. Experiments show that our method significantly outperforms state-of-the-art camera pose estimation methods in challenging scenarios and is comparable in the traditional densely captured setup. Our approach also unleashes the potential of multiple down-stream tasks such as scene reconstruction from multi-view stereo and novel view synthesis in extreme-view scenarios <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Project page: https://people.csail.mit.edu/weichium/virtual-correspondence/.
A Taxonomy for Characterizing Blockchain Systems

Blockchain is a promising technology that has received extensive attention from academia and industry in various subject domains. The significant potential of blockchain technology lies in its key characteristics of decentralization, persistency, anonymity, and auditability. However, the development of blockchain applications faces many technical difficulties because such applications are highly complex and heterogeneous, involving cooperation and interoperation between legacy systems and blockchain systems. These difficulties are further exacerbated by the lack of a common understanding of different blockchain solutions and components from a software engineering perspective. This study proposes a taxonomy to classify the eight fundamental components of blockchain systems. These components comprise a platform, network, distributed ledger, smart contract, consensus protocol, digital wallet, token, and node. Under the proposed taxonomy, each component is divided into several aspects (i.e., characteristics) that collectively describe the component. As a result of this elaboration, the taxonomy organizes the eight components into 83 dimensions and 199 characteristics. The taxonomy was validated using illustrative scenarios and a case study. The proposed taxonomy aims to provide a better understanding of the interrelationships among different blockchain components and aspects. Such an understanding can help blockchain application developers to design better blockchain architectures and systems.
A Taxonomy and Lessons Learned From Blockchain Adoption Within the Internet of Energy Paradigm

The concept of the internet of energy (IoE) emerged as an innovative paradigm to encompass all the complex and intertwined notions relevant to the transition of current smart grids towards more decarbonization, digitalization and decentralization. With a focus on the two last aspects, the amount of intelligent devices being connected in a scattered way to the existing power grid is ever-growing. Nevertheless, guaranteeing a cyber-secure and resilient control of these IoE components as well as a seamless and reliable delivery of electricity services, such as renewable energy exchange, electric vehicles charging, demand response, and so forth; might be the bottleneck of current power systems that are largely still functioning following a centralized approach. Thus, the future power grid would gradually incorporate a growing number of distributed-based control schemes to deal with this challenge. And many believe that blockchain could be a key-enabler in this transition, due to its consistent characteristics with multiple requirements of future power systems. In this paper, we provide an extensive state-of-the-art of blockchain-based additions to the IoE. Where, we first introduce various concepts related to blockchain and discuss the rationale behind its adoption in the context of IoE. Then, differently from the existing body of literature surveys, we do not only provide a taxonomy and evaluate a wide range of recent research outputs that integrated blockchain within modern power systems. But we also draw some valuable lessons learned for each studied category and discuss the intersection of blockchain with various emerging paradigms that have the potential of radically impacting the smart grid. In addition, we present some real-world industrial initiatives and ongoing projects built on top of blockchain, dedicated for offering diverse electricity services with a case study of a pilot project on energy trading in Amsterdam. Finally, we discuss the remaining challenges and worthwhile opportunities of deploying blockchain in this particular area, with a focus on the aspect of operational cyber-security.
Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps

Convolutional Neural Networks (CNNs) compression is crucial to deploying these models in edge devices with limited resources. Existing channel pruning algorithms for CNNs have achieved plenty of success on complex models. They approach the pruning problem from various perspectives and use different metrics to guide the pruning process. However, these metrics mainly focus on the model's 'outputs' or 'weights' and neglect its 'interpretations' information. To fill in this gap, we propose to address the channel pruning problem from a novel perspective by leveraging the interpretations of a model to steer the pruning process, thereby utilizing information from both inputs and outputs of the model. However, existing interpretation methods cannot get deployed to achieve our goal as either they are inefficient for pruning or may predict non-coherent explanations. We tackle this challenge by introducing a selector model that predicts real-time smooth saliency masks for pruned models. We parameterize the distribution of explanatory masks by Radial Basis Function (RBF)-like functions to incorporate geometric prior of natural images in our selector model's inductive bias. Thus, we can obtain compact representations of explanations to reduce the computational costs of our pruning method. We leverage our selector model to steer the network pruning by maximizing the similarity of explanatory representations for the pruned and original models. Extensive experiments on CIFAR-10 and ImageNet benchmark datasets demonstrate the efficacy of our proposed method. Our implementations are available at https://github.com/Alii-Ganjj/InterpretationsSteeredPruning .
Think Twice Before Detecting GAN-generated Fake Images from their Spectral Domain Imprints

Accurate detection of the fake but photorealistic images is one of the most challenging tasks to address social, biometrics security and privacy related concerns in our community. Earlier research has underlined the existence of spectral domain artifacts in fake images generated by powerful generative adversarial network (GAN) based methods. Therefore, a number of highly accurate frequency domain methods to detect such GAN generated images have been proposed in the literature. Our study in this paper introduces a pipeline to mitigate the spectral artifacts. We show from our experiments that the artifacts in frequency spectrum of such fake images can be mitigated by proposed methods, which leads to the sharp decrease of performance of spectrum-based detectors. This paper also presents experimental results using a large database of images that are synthesized using BigGAN, CRN, CycleGAN, IMLE, Pro-GAN, StarGAN, StyleGAN and StyleGAN2 (including synthesized high resolution fingerprint images) to illustrate effectiveness of the proposed methods. Furthermore, we select a spatial-domain based fake image detector and observe a notable decrease in the detection performance when proposed method is incorporated. In summary, our insightful analysis and pipeline presented in this paper cautions the forensic community on the reliability of GAN-generated fake image detectors that are based on the analysis of frequency artifacts as these artifacts can be easily mitigated.
Functional Choreographic Programming

Choreographic programming is an emerging programming paradigm for concurrent and distributed systems, where developers write the communications that should be enacted and a compiler then automatically generates a distributed implementation. Currently, the most advanced incarnation of the paradigm is Choral, an object-oriented choreographic programming language that targets Java. Choral deviated significantly from known theories of choreographies, and in particular introduced the possibility of expressing higher-order choreographies that are fully distributed. In this article, we introduce Chor $$\lambda $$ , the first functional choreographic programming language. It is also the first theory that explains the core ideas of higher-order choreographic programming. We show that bridging the gap between practice and theory requires developing a new evaluation strategy and typing discipline for $$\lambda $$ terms that accounts for the distributed nature of computation in choreographies.
Recent Applications of Artificial Intelligence for Sustainable Development in Smart Cities

We are seeing an explosion in artificial intelligence (AI), which may be defined as a technology that mimics the traits often associated with human intelligence. Today, many industries rely on artificial intelligence, including marketing and banking, healthcare and security; robotics and transportation; chatbots; artificial creativity; and manufacturing. AI has recently started to play a significant role in smart city operations. AI is used to describe how well computers can imitate human thought processes. Expert systems, natural language processing, speech recognition, and machine vision are some areas of AI. Smart City uses information and communication technologies to boost the economy, improve the quality of people's lives, and support governing. AI can play a significant role in making cities safer and better for people to live in by giving people more access and control over their own homes, monitoring traffic and managing waste. In this paper, the authors have explored the recent artificial intelligence applications for sustainable development in smart cities. These applications can be used in (1) sustainable environmental plans, with requirements with more and better facilities in the limited available area, and (2) enhanced standard of living for urban residents at a more economical expense.
SwiftDependencyChecker

Vulnerable versions of third party libraries can remain a threat even after a security fix has been published. Developers are hesitant to update dependencies and could unknowingly depend on an outdated vulnerable library version. We developed SwiftDependencyChecker to make data from public vulnerability databases more accessible to Swift developers. SwiftDependencyChecker analyses dependencies declared through CocoaPods, Carthage and Swift PM, queries the NVD database, and shows warnings for used vulnerable library versions in Xcode. Preliminary evaluation of the tool showed that the vulnerability analysis is effective. General feedback from independent developers was encouraging with some improvement potential on performance for the initial analysis of a project.
Attention-Based Dynamic Alignment and Dynamic Distribution Adaptation for Remote Sensing Cross-Domain Scene Classification

Due to the lack of high-quality labeled data and poor generalization ability of supervised models in remote sensing scene classification, cross-domain scene classification is proposed to better utilize the existing knowledge to classify the unlabeled data. Since there is a data distribution difference between the training (source domain) and test (target domain) set, many deep domain adaptation methods have been proposed to reduce such distribution discrepancy. However, existing deep domain adaptation methods usually use the discrepancy metric function to align the marginal distribution and do not consider the effect of each sample in different domains on the network weights. In addition, the existing domain adaptation methods cannot adaptively balance the relative importance of marginal and conditional distributions well. To overcome the above shortcomings, we propose a novel Attention-based Dynamic Alignment and Dynamic Distribution Adaptation (ADA-DDA) method to better align the marginal distribution between different distributions by calculating the dynamic weights of each sample in different domains and dynamically balance the relative importance of marginal and conditional distributions. Moreover, the attention mechanism enables purposeful knowledge transfer, so that the extracted features can be highly discriminative. The experimental results demonstrate that our proposed method is superior to the other state-of-the-art deep domain adaptation methods in the comparison, and outperforms the second place in accuracy by 5.36%.