Reviewerly/exHarmony huggingface dataset:
MP-OPU: A Mixed Precision FPGA-based Overlay Processor for Convolutional Neural Networks

Low precision quantization in convolutional neural network (CNN) inference has been proved effective for reducing computation complexity and bandwidth requirement. Mixed precision CNNs manage to benefit from low precision while maintaining accuracy. In this paper, we propose a Mixed Precision FPGA-based Overlay Processor (MP-OPU) to fully leverage the advantages of mixed precision for both conventional and lightweight CNNs. The micro-architecture of MP-OPU considers sharing of computation core with mixed precision weights and activations to improve computation efficiency. In addition, run-time scheduling of external memory access and data arrangement are optimized to further leverage the advantages of mixed precision data representation. Our experimental results show that MP-OPU reaches 4.92 TOPS peak throughput when implemented on the Xilinx VC709 FPGA (with all DSPs configured to support 2-bit multipliers). Moreover, MP-OPU achieves 12.9 × latency reduction and 2.2 × better throughput/DSP for conventional CNNs while 7.6× latency reduction and 2.9× better throughput/DSP for lightweight CNNs, all on average compared with existing FPGA accelerators/processors, respectively. To the best of our knowledge, this is the first in-depth study on mixed precision FPGA-based overlay processor for both conventional and lightweight CNNs.
RSVQA Meets Bigearthnet: A New, Large-Scale, Visual Question Answering Dataset for Remote Sensing

Visual Question Answering is a new task that can facilitate the extraction of information from images through textual queries: it aims at answering an open-ended question formulated in natural language about a given image. In this work, we introduce a new dataset to tackle the task of visual question answering on remote sensing images: this large-scale, open access dataset extracts image/question/answer triplets from the BigEarthNet dataset. This new dataset contains close to 15 millions samples and is openly available. We present the dataset construction procedure, its characteristics and first results using a deep-learning based methodology. These first results show that the task of visual question answering is challenging and opens new interesting research avenues at the interface of remote sensing and natural language processing. The dataset and the code to create and process it are open and freely available on https://rsvqa.sylvainlobry.com/
StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objects

Geometric organization of objects into semantically meaningful arrangements pervades the built world. As such, assistive robots operating in warehouses, offices, and homes would greatly benefit from the ability to recognize and rearrange objects into these semantically meaningful structures. To be useful, these robots must contend with previously unseen objects and receive instructions without significant programming. While previous works have examined recognizing pairwise semantic relations and sequential manipulation to change these simple relations none have shown the ability to arrange objects into complex structures such as circles or table settings. To address this problem we propose a novel transformer-based neural network, StructFormer, which takes as input a partial-view point cloud of the current object arrangement and a structured language command encoding the desired object configuration. We show through rigorous experiments that StructFormer enables a physical robot to rearrange novel objects into semantically meaningful structures with multi-object relational constraints inferred from the language command.
SVDD boundary and DPC clustering technique-based oversampling approach for handling imbalanced and overlapped data

Imbalanced datasets classification remains an important domain in machine learning. Conventional supervised learning algorithms tend to be biased towards the majority class when addressing imbalanced datasets, thus providing poor classification results on the minority class. The learning task would become more challenging when there are overlapping and within-imbalance issues in imbalanced datasets, which are often the case and have been proven to play more negative impact on the classification performance than between-class imbalance. In this paper, we propose a novel SVDD (Support Vector Data Description) boundary and DPC (Density Peaks Clustering) clustering technique-based oversampling approach (SVDDDPCO) for handling imbalanced and overlapped data. The proposed approach first utilizes support vector data description (SVDD) model with greater penalty constant for the minority class than the majority class to generate the class boundary, and then identifies those misclassified majority or few minority instances by the boundary as potential overlapped or noisy ones and eliminates them. To address the within-balance issues, density peaks clustering (DPC) is used to cluster the minority instances due to its advantage of accurately identifying sub-clusters with different sizes and densities, which facilitates simultaneously combating between-class and within-class imbalance issues caused by various reasons. The minority instances are assigned with different weights inversely proportional to their local densities and their distances to the boundary and then the size for each identified sub-cluster to be oversampled is adaptively determined according to its own size and the weights of its elements. Such strategy aims to generate more synthetic minority instances for borderline and sparser small sub-clusters which are usually informative to the later learning tasks. Finally, oversampling is performed within each sub-cluster with different sizes to not only counteract the within imbalance but also avoid the generation of any noisy or overlapped synthetic instance. Extensive comparison results on various datasets showed the proposed method achieves statistically significant improvements in classification performance relative to state-of-the-art ones.
After Violation But Before Sanction: Understanding Volunteer Moderators' Profiling Processes Toward Violators in Live Streaming Communities

Content moderation is an essential part of online community health and governance. While much of extant research is centered on what happens to the content, moderation also involves the management of violators. This study focuses on how moderators (mods) make decisions about their actions after the violation takes place but before the sanction by examining how they "profile" the violators. Through observations and interviews with volunteer mods on Twitch, we found that mods engage in a complex process of collaborative evidence collection and profile violators into different categories to decide the type and extent of punishment. Mods consider violators' characteristics as well as behavioral history and violation context before taking moderation action. The main purpose of the profiling was to avoid excessive punishment and aim to integrate violators more into the community. We discuss the contributions of profiling to moderation practice and suggest design mechanisms to facilitate mods' profiling processes.
A Statistical Approach to Mining Semantic Similarity for Deep Unsupervised Hashing

The majority of deep unsupervised hashing methods usually first construct pairwise semantic similarity information and then learn to map images into compact hash codes while preserving the similarity structure, which implies that the quality of hash codes highly depends on the constructed semantic similarity structure. However, since the features of images for each kind of semantics usually scatter in high-dimensional space with unknown distribution, previous methods could introduce a large number of false positives and negatives for boundary points of distributions in the local semantic structure based on pairwise cosine distances. Towards this limitation, we propose a general distribution-based metric to depict the pairwise distance between images. Specifically, each image is characterized by its random augmentations that can be viewed as samples from the corresponding latent semantic distribution. Then we estimate the distances between images by calculating the sample distribution divergence of their semantics. By applying this new metric to deep unsupervised hashing, we come up with Distribution-based similArity sTructure rEconstruction (DATE). DATE can generate more accurate semantic similarity information by using non-parametric ball divergence. Moreover, DATE explores both semantic-preserving learning and contrastive learning to obtain high-quality hash codes. Extensive experiments on several widely-used datasets validate the superiority of our DATE.
S3PRL-VC: Open-Source Voice Conversion Framework with Self-Supervised Speech Representations

This paper introduces S3PRL-VC, an open-source voice conversion (VC) framework based on the S3PRL toolkit. In the context of recognition-synthesis VC, self-supervised speech representation (S3R) is valuable in its potential to replace the expensive supervised representation adopted by state-of-the-art VC systems. Moreover, we claim that VC is a good probing task for S3R analysis. In this work, we provide a series of in-depth analyses by benchmarking on the two tasks in VCC2020, namely intra-/cross-lingual any-to-one (A2O) VC, as well as an any-to-any (A2A) setting. We also provide comparisons between not only different S3Rs but also top systems in VCC2020 with supervised representations. Systematic objective and subjective evaluation were conducted, and we show that S3R is comparable with VCC2020 top systems in the A2O setting in terms of similarity, and achieves state-of-the-art in S3R-based A2A VC. We believe the extensive analysis, as well as the toolkit itself, contribute to not only the S3R community but also the VC community. The codebase is now open-sourced <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Tokenfication - The potential of non-fungible tokens (NFT) for game development

In recent years, there has been an increase in the interest in non-fungible tokens (NFT) to purchase digital art and has shaped a new form of art collecting. The art community has a long history of challenges and opportunities with the commercialization of art. More recently, there has also been an interest in the purchase of game art as investments or collecting. With the advent of blockchain currencies, NFTs have introduced a way for investors, collectors, and game developers to purchase or sell game art. However, the potential of NFT for the games and game development industry is manifold. In this paper, the authors will reflect on the potential of NTF's for game developers and content creators.
ByteSGAN: A semi-supervised Generative Adversarial Network for encrypted traffic classification in SDN Edge Gateway

With the rapid development of communication network technology, the types and quantity of network traffic data are accordingly increasing. Network traffic classification has become a non-trivial research task in the area of network management and security, which not only helps to improve the fine-grained network resource allocation but also enables policy-driven network management. As the closest network element to the subscribers, SDN Edge Gateway can tremendously enhance the user experience with the capability of traffic classification. Deep Learning (DL) performs automatic feature extraction without human intervention, which undoubtedly makes it a highly desirable approach for traffic classification, especially encrypted traffic. However, capturing large labeled datasets is cumbersome and time-consuming manual labor. Semi-Supervised learning is an appropriate technique to overcome this problem. With that in mind, we proposed a Generative Adversarial Network (GAN)-based Semi-Supervised Learning Encrypted Traffic Classification method called ByteSGAN embedded in SDN Edge Gateway to achieve the goal of traffic classification in a fine-grained manner to further improve network resource utilization. ByteSGAN can only use a small number of labeled traffic samples and a large number of samples to achieve a good performance of traffic classification by modifying the structure and loss function of the regular GAN discriminator network in a semi-supervised learning way. Based on the public datasets ‘ISCX2012 VPN-non VPN’ and ‘Crossmarket’, two experimental results show that the ByteSGAN can efficiently improve the performance of traffic classifiers and outperform the other supervised learning methods like CNN.
Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation

Semantic segmentation has been continuously investigated in the last ten years, and majority of the established technologies are based on supervised models. In recent years, image-level weakly supervised semantic segmentation (WSSS), including single- and multi-stage process, has attracted large attention due to data labeling efficiency. In this paper, we propose to embed affinity learning of multi-stage approaches in a single-stage model. To be specific, we introduce an adaptive affinity loss to thoroughly learn the local pairwise affinity. As such, a deep neural network is used to deliver comprehensive semantic information in the training phase, whilst improving the performance of the final prediction module. On the other hand, considering the existence of errors in the pseudo labels, we propose a novel label reassign loss to mitigate over-fitting. Extensive experiments are conducted on the PASCAL VOC 2012 dataset to evaluate the effectiveness of our proposed approach that outperforms other standard single-stage methods and achieves comparable performance against several multi-stage methods.
Organizational issues in embracing Agile methods: an empirical assessment

Abstract This study provides empirical evidence to the body of knowledge in Agile methods adoption in small, medium and large organizations in international context. This research explores the factors involved in the adoption of Agile methods in software development organizations. A survey was conducted among Agile professionals to gather survey data from 52 software organizations in seven countries across the world. Statistical techniques are applied towards empirical assessment. Organizational culture, team structure and management support are found to be crucial success factors whereas lack of management support, a large organization size and traditional organizational culture are found to be detrimental for the adoption of Agile approach in an organization. The selection of an appropriate Agile method depends on the project size and, for each size, there are specific methods preferred by different enterprises. Providing better control over the work is viewed as the primary advantage of the Agile methods within large and small organizations, while for the medium-size organizations, the priority is switched to coping with changing user requirements. Majority of the respondents did not consider embracing agile methods as a reason for project failure which indicates that Agile methods are, indeed, beneficial.
Semi-supervised Active Salient Object Detection

In this paper, we propose a novel semi-supervised active salient object detection (SOD) method that actively acquires a small subset of the most discriminative and representative samples for labeling. Two main contributions have been made to prevent the method from being overwhelmed by labeling similar distributed samples. First, we design a saliency encoder-decoder with adversarial discriminator to generate a confidence map, representing the network uncertainty on the current prediction. Then, we select the least confident (discriminative) samples from the unlabeled pool to form the “candidate labeled pool”. Second, we train a Variational Auto-Encoder (VAE) to select and add the most representative data from the “candidate labeled pool” into the labeled pool by comparing their corresponding features in the latent space. Within our framework, these two networks are optimized conditioned on the states of each other progressively. Experimental results on six benchmarking SOD datasets demonstrate that our annotation-efficient learning based salient object detection method, reaching to 14% labeling budget, can be on par with the state-of-the-art fully-supervised deep SOD models. The source code is publicly available via our project page: https://github.com/JingZhang617/Semi-sup-active-self-sup-Learning.
<scp>BOSS</scp>: A new <scp>QoS</scp> aware blockchain assisted framework for secure and smart healthcare as a service

Abstract The latest epidemic of COVID‐19 has significantly impacted both human capital and the global economy, contributing to pandemics and severe global crises. Research into the creation and propagation of the disease is desperately needed. The Internet of Things, cloud computing, and artificial intelligence offer modern technology for real‐time processing for multiple applications such as healthcare applications, transport, traffic control, and so on blockchain is an evolving technology that will dramatically boost transaction protection in finance, supply chain, and other transaction networks. A stable and latency‐sensitive Quality of Service framework for COVID‐19 is the need of an hour. The purpose of this paper is to combine Fog computing and Artificial Intelligence with smart health to establish a reliable platform for early‐stage detection of COVID‐19 infection. A new ensemble‐based classifier is proposed to detect COVID‐19 patients. This research offers a blockchain platform to analyse how the unrelated cases of the COVID‐19 virus can be tracked and identified using peer‐to‐peer, time stamping, and the shared storage advantages of blockchain. In addition to growing patient loyalty, this would effectively enhance the consistency, flexibility, productivity, performance, and effectiveness of healthcare services. The idea of blockchain is used to establish security for the whole framework. Different implementations measure the efficiency of the suggested system. The performance of the proposed framework is evaluated in terms of delay, network usages, RAM usages, and energy consumption. On the other hand, the classifier is evaluated in terms of classifier accuracy, recall, precision, kappa static, and root mean square error. The result shows the performance of the proposed framework and classifier is always better than the traditional frameworks and classifiers.
Design and development of a glass façade cleaning robot

This paper presents a unique design concept, dynamic modeling, and control strategies for efficient coverage path planning of a glass façade cleaning robot (GFCR). The robot design has been conceptualized using mechanisms for adhesion, motion, steering, and cleaning. The dynamic model for robot vertical glass façade cleaning is derived using Lagrangian formulation. A modified particle swarm optimization (PSO) is used to autotune the proportional, integral, and derivative (PID) parameters for the trajectory tracking simulation and it is more efficient and robust compared to the standard PSO algorithm. The path planning algorithm using hybrid PID-PSO approach is also developed for energy-efficient coverage of the robot for glass façade cleaning. The coverage algorithm illustrates the energy-performance of the GFCR for different paths viz., horizontal line sweep (HLS), vertical line sweep (VLS), spiral line sweep (SLS), and special cell diffusion (SCD) motion. Simulation reveals the robot motion for HLS path is the most energy-efficient. The GFCR model with minimum energy consumption has been validated by working trials. The GFCR has potential applications for cleaning high-rise glass façade buildings and photovoltaic (PV) solar panels.
Re-examining the quantum volume test: Ideal distributions, compiler optimizations, confidence intervals, and scalable resource estimations

The quantum volume test is a full-system benchmark for quantum computers that is sensitive to qubit number, fidelity, connectivity, and other quantities believed to be important in building useful devices. The test was designed to produce a single-number measure of a quantum computer's general capability, but a complete understanding of its limitations and operational meaning is still missing. We explore the quantum volume test to better understand its design aspects, sensitivity to errors, passing criteria, and what passing implies about a quantum computer. We elucidate some transient behaviors the test exhibits for small qubit number including the ideal measurement output distributions and the efficacy of common compiler optimizations. We then present an efficient algorithm for estimating the expected heavy output probability under different error models and compiler optimization options, which predicts performance goals for future systems. Additionally, we explore the original confidence interval construction and show that it underachieves the desired coverage level for single shot experiments and overachieves for more typical number of shots. We propose a new confidence interval construction that reaches the specified coverage for typical number of shots and is more efficient in the number of circuits needed to pass the test. We demonstrate these savings with a $QV=2^{10}$ experimental dataset collected from Quantinuum System Model H1-1. Finally, we discuss what the quantum volume test implies about a quantum computer's practical or operational abilities especially in terms of quantum error correction.
Deep learning-based NLP data pipeline for EHR-scanned document information extraction

Scanned documents in electronic health records (EHR) have been a challenge for decades, and are expected to stay in the foreseeable future. Current approaches for processing include image preprocessing, optical character recognition (OCR), and natural language processing (NLP). However, there is limited work evaluating the interaction of image preprocessing methods, NLP models, and document layout.We evaluated 2 key indicators for sleep apnea, Apnea hypopnea index (AHI) and oxygen saturation (SaO2), from 955 scanned sleep study reports. Image preprocessing methods include gray-scaling, dilating, eroding, and contrast. OCR was implemented with Tesseract. Seven traditional machine learning models and 3 deep learning models were evaluated. We also evaluated combinations of image preprocessing methods, and 2 deep learning architectures (with and without structured input providing document layout information), with the goal of optimizing end-to-end performance.Our proposed method using ClinicalBERT reached an AUROC of 0.9743 and document accuracy of 94.76% for AHI, and an AUROC of 0.9523 and document accuracy of 91.61% for SaO2.There are multiple, inter-related steps to extract meaningful information from scanned reports. While it would be infeasible to experiment with all possible option combinations, we experimented with several of the most critical steps for information extraction, including image processing and NLP. Given that scanned documents will likely be part of healthcare for years to come, it is critical to develop NLP systems to extract key information from this data.We demonstrated the proper use of image preprocessing and document layout could be beneficial to scanned document processing.