Reviewerly/exHarmony huggingface dataset:
Blockchain-based Federated Learning with Contribution-Weighted Aggregation for Medical Data Modeling

To promote the sharing of medical data and thus to improve the medical level and resolve the imbalance of medical resource distribution is one of the most significant things of nowa-days society. In this paper, a framework combining the newly emerging techniques, i.e., federated learning and blockchain, is proposed for decentralized medical data modeling and sharing. Federated learning is leveraged for data training and modeling, which will bring more precise medical models but without the leakage of original medical data. It can well solve the privacy concerns from patients. As the underlay for federated learning, blockchain contributes to provide a decentralized, secure, and transparent learning and sharing environment. In particular, a contribution-weighted incentive mechanism is proposed to promote the participation of medical data sharing, where the contributions and corresponding rewards are well considered and guaranteed. Finally, the prototype has been developed and implemented with a public data set of breast images. The results show that the proposed blockchain-enabled federated learning with contribution-weighted aggregation has advantages over the centralized learning approach and federated learning average aggregation in terms of model accuracy and system security.
A Multi-Scale Fusion Residual Encoder-Decoder Approach for Low Illumination Image Enhancement

In a low-light environment, due to the extremely small number of photons and high noise, the sensory light source of the line-scan camera cannot be fully exposed, resulting in a decrease in image quality. To this end, a low illumination image enhancement approach based on multi-scale fusion residual encoder- decoder is proposed, which directly learn the end-to-end mapping between the original sensor RAW light and dark images, and effectively enhance the brightness of the image while completely restoring the original image details and colors. In order to increase the feature diversity and speed up the network training, the network structure is added with residual block. To aggregate the global multi-scale features of the context, a dense context feature aggregation module is designed to make up for the lack of spatial information in the network. Based on the SID dataset, a comparative experiment with other 10 methods shows that proposed method is significantly better than most other methods in visual effects and quantitative evaluation of PSNR and SSIM. While restoring the brightness of the image, the edges and colors of the image are effectively represented, and finally a satisfactory visual quality is obtained under low light enhancement.
DaSEA

Software package managers facilitate reuse and rapid construction of software systems. Since evermore software is distributed via package managers, researchers and practitioners require explicit data of software dependency networks that are opaquely formed by dependency relations between software packages. To reason about increasingly complex software products and ecosystems, researchers and practitioners rely either on publicly available datasets like the seemingly unattended libraries.io [14] or they mine problem-specific data from software ecosystems repeatedly and non-transparently. Therefore, we present the DaSEA dataset, which contains metadata of software packages, their versions, and dependencies from multiple ecosystems (currently six programming languages and five operating system package managers). Alongside the dataset, we provide an extensible open-source tool under the same name that is used to create updated versions of the DaSEA dataset allowing studies of evolution of software ecosystems.
IterVM: Iterative Vision Modeling Module for Scene Text Recognition

Scene text recognition (STR) is a challenging problem due to the imperfect imagery conditions in natural images. State-of-the-art methods utilize both visual cues and linguistic knowledge to tackle this challenging problem. Specifically, they propose iterative language modeling module (IterLM) to repeatedly refine the output sequence from the visual modeling module (VM). Though achieving promising results, the vision modeling module has become the performance bottleneck of these methods. In this paper, we newly propose iterative vision modeling module (IterVM) to further improve the STR accuracy. Specifically, the first VM directly extracts multi-level features from the input image, and the following VMs re-extract multi-level features from the input image and fuse them with the high-level (i.e., the most semantic one) feature extracted by the previous VM. By combining the proposed IterVM with iterative language modeling module, we further propose a powerful scene text recognizer called IterNet. Extensive experiments demonstrate that the proposed IterVM can significantly improve the scene text recognition accuracy, especially on low-quality scene text images. Moreover, the proposed scene text recognizer IterNet achieves new state-of-the-art results on several public benchmarks. Codes and models are available at https://github.com/VDIGPKU/IterNet.
UnweaveNet: Unweaving Activity Stories

Our lives can be seen as a complex weaving of activities; we switch from one activity to another, to maximise our achievements or in reaction to demands placed upon us. Observing a video of unscripted daily activities, we parse the video into its constituent activity threads through a process we call unweaving. To accomplish this, we introduce a video representation explicitly capturing activity threads called a thread bank, along with a neural controller capable of detecting goal changes and resuming of past activities, together forming UnweaveNet. We train and evaluate UnweaveNet on sequences from the unscripted egocentric dataset EPIC-KITCHENS. We propose and showcase the efficacy of pretraining UnweaveNet in a self-supervised manner.
Can Data Visualizations Change Minds? Identifying Mechanisms of Elaborative Thinking and Persuasion

Recent years have seen rapid growth in data-driven communication and the public availability of datasets on a broad set of social issues. Yet despite this unprecedented accessibility, the public often remains divided along partisan or ideological lines and to lack a common understanding of the issues at stake. In this paper we consider the role of data visualizations in communicating scientific evidence, and in particular, their power to persuade in the face of conflicting prior beliefs and attitudes. We describe a recent study showing that strong attitudes about politically polarized topics were associated with less belief change when interacting with statistical data visualizations. Moreover, there was little evidence for attitude change even when people updated their beliefs about specific empirical relationships. We then draw on research in cognitive science to identify elements of visualizations that may produce such attitude change because they encourage elaborative thinking when interacting with data. We argue for further research that considers how broader attitudes-which are tied to social identity, values, and worldviews-affect the power of data visualizations to persuade among communities with diverse ideological and cultural backgrounds.
Deep Factorized Q-Learning for Large Scale Multi-Agent Learning

The value function decomposition is an effective way to alleviate the curse of dimension in Multi-Agent Reinforcement Learning (MARL). However, the existing methods usually either can only provide the low-order approximate decomposition of no more than the second-order, or need to spend a lot of effort to manually design the high-order interaction among agents according to experience. Therefore, the existing methods either tend to bear large decomposition error or are not convenient to use. In this paper, a high-order approximate value function decomposition method is proposed, which can be trained end-to-end. There have some prominent features about this method including low-rank vector exploited to represent value function, both low- and high-order component sharing the same input (i.e., the embedding vector), the model parameters shared among all the agents if they are homogeneous. Experimental results show that our method is effective.
PrivAim: A Dual-Privacy Preserving and Quality-Aware Incentive Mechanism for Federated Learning

Privacy protection and incentive mechanism are two fundamental problems in federated learning (FL), which aim at protecting the privacy of data owners and stimulating them to share more resources, respectively. Recent works have proposed differential privacy (DP) based privacy-preserving incentive mechanisms to solve both problems simultaneously. However, almost all of them took the privacy level as the only incentive item, without considering other factors, such as data quantity and quality. Moreover, an untrusted server can further infer sensitive information from the bids that reflect the true costs of data owners. To solve these problems, in this paper, we propose a dual-privacy preserving and quality-aware incentive mechanism, PrivAim, for federated learning. Specifically, it utilizes differential privacy to protect the local models and true costs against the untrusted parameter server, and carefully designs a multi-dimensional reverse auction mechanism to incentivize data owners with high quality and low cost to participate in FL without knowing the true bids. We theoretically prove that PrivAim satisfies <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\Delta b$</tex-math></inline-formula> -truthfulness, individual rational, computational efficiency, and differential privacy. Extensive experiments show that PrivAim can effectively protect bid privacy, and achieve at least 21% and 6% improvement on social welfare and model accuracy, respectively, compared to the state-of-the-art.
Radar signal recognition using Wavelet Transform and Machine Learning

Automatic signal recognition algorithms turned out to be useful in many areas including intelligent radio, electronic warfare or surveillance systems. There are various new signals and the emitter recognition is a key problem in rapidly changing electromagnetic environment.Using continuous wavelet transform (CWT) turns out to be effective way to extract signal/modulation features for classification purposes. In this paper the recognition possibilities of selected types of radar signals, like linear and stepped frequency modulated signals (LFM, SFM), phase coded waveforms (PCW) with Barker code and rectangular pulses (Rec), are analysed. Two kinds of algorithms are considered. In the first one higher order statistics (HOS) of continuous wavelet transform (CWT) coefficients are proposed as signal features. Principal component analysis (PCA) is considered to reduce number of features and feed-forward neural network is proposed as classifier. In the second one CWT coefficients are treated as an image and the classification process is carried out using a convolutional neural network (CNN). For evaluating the performance of the simulated (in Matlab environment) classification models a confusion matrix is used.
Is Appearance Free Action Recognition Possible?

Intuition might suggest that motion and dynamic information are key to video-based action recognition. In contrast, there is evidence that state-of-the-art deep-learning video understanding architectures are biased toward static information available in single frames. Presently, a methodology and corresponding dataset to isolate the effects of dynamic information in video are missing. Their absence makes it difficult to understand how well contemporary architectures capitalize on dynamic vs. static information. We respond with a novel Appearance Free Dataset (AFD) for action recognition. AFD is devoid of static information relevant to action recognition in a single frame. Modeling of the dynamics is necessary for solving the task, as the action is only apparent through consideration of the temporal dimension. We evaluated 11 contemporary action recognition architectures on AFD as well as its related RGB video. Our results show a notable decrease in performance for all architectures on AFD compared to RGB. We also conducted a complimentary study with humans that shows their recognition accuracy on AFD and RGB is very similar and much better than the evaluated architectures on AFD. Our results motivate a novel architecture that revives explicit recovery of optical flow, within a contemporary design for best performance on AFD and RGB.
tSF: Transformer-Based Semantic Filter for Few-Shot Learning

Few-Shot Learning (FSL) alleviates the data shortage challenge via embedding discriminative target-aware features among plenty seen (base) and few unseen (novel) labeled samples. Most feature embedding modules in recent FSL methods are specially designed for corresponding learning tasks (e.g., classification, segmentation, and object detection), which limits the utility of embedding features. To this end, we propose a light and universal module named transformer-based Semantic Filter (tSF), which can be applied for different FSL tasks. The proposed tSF redesigns the inputs of a transformer-based structure by a semantic filter, which not only embeds the knowledge from whole base set to novel set but also filters semantic features for target category. Furthermore, the parameters of tSF is equal to half of a standard transformer block (less than 1M). In the experiments, our tSF is able to boost the performances in different classic few-shot learning tasks (about $$2\%$$ improvement), especially outperforms the state-of-the-arts on multiple benchmark datasets in few-shot classification task.
Sentiment Analysis of Tweets Using Deep Learning

The coronavirus pandemic has caused a worldwide crisis and a drastic change in day-to-day life activities. Worldwide, people use social media platforms to share and discuss their opinions about the situation. Twitter is one such platform for public conversation around the coronavirus pandemic, the spread of disease, vaccination, non-pharmaceutical interventions, and many other discussions. In this study, we use Twitter social medial data for sentiment analysis. The tweets are collected based on covid-19 related hashtags. This work presents a deep learning-based framework for sentiment analysis using DistilBERT, a distilled version of Bidirectional Encoder Representation from Transformers (BERT), Convolutional Neural Network (CNN), and Long Short Term Memory (LSTM). The results show that transformer-based pre-processing and fine-tuning yield better performance results. The DistilBERT model yields the highest accuracy of 91.46% compared to the CNN and LSTM models.
PrivHAR: Recognizing Human Actions from Privacy-Preserving Lens

The accelerated use of digital cameras prompts an increasing concern about privacy and security, particularly in applications such as action recognition. In this paper, we propose an optimizing framework to provide robust visual privacy protection along the human action recognition pipeline. Our framework parameterizes the camera lens to successfully degrade the quality of the videos to inhibit privacy attributes and protect against adversarial attacks while maintaining relevant features for activity recognition. We validate our approach with extensive simulations and hardware experiments.
A Comparison of RGB and RGNIR Color Spaces for Plastic Waste Detection Using The YOLOv5 Architecture

Plastic waste is a serious environmental issue that damages human health, wildlife, and habitats. Many researchers have come out with multiple solutions on the problem. One of the most efficient ways is to implement machine learning approaches to detect plastic waste in common areas. Deep learning is a powerful machine learning approach that automatically learns image features for object recognition tasks using an object detector. Therefore, this paper proposed a recent object detection model, YOLOv5m, to develop a plastic waste detection model. Two plastic waste datasets, which consist of red, green, and blue (RGB) and red, green, and near-infrared (RGNIR) images, are introduced to train the proposed model. The performance of the proposed model is evaluated using 10-fold cross-validation on the two datasets. The proposed model achieves the best result on RGNIR datasets for validation and testing with an average mAP@0.5:0.95 value of 69.39% and 69.45%, respectively. These results indicate that near-infrared information can be a valuable feature representation in machine learning. This opens more possible opportunities, such as the development of automated plastic detection for the robotic and waste management industry.
Design and Implementation of Indoor Navigation for PENS Visitors Using Augmented Intelligence

Navigation systems are commonly utilized to assist users in reaching their destination, seeing the route, and calculating the distance traveled. The application of indoor navigation attempts to give users with information to help them navigate the environment, particularly new users. In this project, augmented reality is employed to create a user-friendly navigation system that eliminates user confusion. We created an interior navigation system employing Augmented Reality (AR) technology to deliver an excellent user experience. This study employed QR Code, namely tags that exist in each room, to place the room into an initial position. To validate each scanned QR Code, a confusion matrix test must be performed. The accuracy of the QRCode was calculated using the confusion matrix and yielded a value of 96.6 percent. This demonstrates that the categorization findings are accurate.
FrequencyLowCut Pooling - Plug and Play Against Catastrophic Overfitting

Over the last years, Convolutional Neural Networks (CNNs) have been the dominating neural architecture in a wide range of computer vision tasks. From an image and signal processing point of view, this success might be a bit surprising as the inherent spatial pyramid design of most CNNs is apparently violating basic signal processing laws, i.e. Sampling Theorem in their down-sampling operations. However, since poor sampling appeared not to affect model accuracy, this issue has been broadly neglected until model robustness started to receive more attention. Recent work [18] in the context of adversarial attacks and distribution shifts, showed after all, that there is a strong correlation between the vulnerability of CNNs and aliasing artifacts induced by poor down-sampling operations. This paper builds on these findings and introduces an aliasing free down-sampling operation which can easily be plugged into any CNN architecture: FrequencyLowCut pooling. Our experiments show, that in combination with simple and Fast Gradient Sign Method (FGSM) adversarial training, our hyper-parameter free operator substantially improves model robustness and avoids catastrophic overfitting. Our code is available at https://github.com/GeJulia/flc_pooling .
Appearance and Structure Aware Robust Deep Visual Graph Matching: Attack, Defense and Beyond

Despite the recent breakthrough of high accuracy deep graph matching (GM) over visual images, the robustness of deep GM models is rarely studied which yet has been revealed an important issue in modern deep nets, ranging from image recognition to graph learning tasks. We first show that an adversarial attack on keypoint localities and the hidden graphs can cause significant accuracy drop to deep GM models. Accordingly, we propose our defense strategy, namely Appearance and Structure Aware Robust Graph Matching (ASAR-GM). Specifically, orthogonal to de facto adversarial training (AT), we devise the Appearance Aware Regularizer (AAR) on those appearance-similar keypoints between graphs that are likely to confuse. Experimental results show that our ASAR-GM achieves better robustness compared to AT. Moreover, our locality attack can serve as a data augmentation technique, which boosts the state-of-the-art GM models even on the clean test dataset. Code is available at https://github.com/Thinklab-SJTU/RobustMatch.
The Need of Standardised Metadata to Encode Causal Relationships: Towards Safer Data-Driven Machine Learning Biological Solutions

In this paper, we discuss the importance of considering causal relations in the development of machine learning solutions to prevent factors hampering the robustness and generalisation capacity of the models, such as induced biases. This issue often arises when the algorithm decision is affected by confounding factors. In this work, we argue that the integration of research assumptions as causal relationships can help identify potential confounders. Together with metadata information, it can enable meta-comparison of data acquisition pipelines. We call for standardised meta-information practices as a crucial step for proper machine learning solutions development, validation, and data sharing. Such practices include detailing the data acquisition process, aiming for automatic integration of causal relationships and actionable metadata.
Semi-supervised Dual-Domain Adaptation for Semantic Segmentation

Deep learning approaches for semantic segmentation rely primarily on supervised learning approaches and require substantial efforts in producing pixel-level annotations. Further, such approaches may perform poorly when applied to unseen image domains. To cope with these limitations, both unsupervised domain adaptation (UDA) with full source supervision but without target supervision and semi-supervised learning (SSL) with partial supervision have been proposed. While such methods are effective at aligning different feature distributions, there is still a need to efficiently exploit unlabeled data to address the performance gap with respect to fully-supervised methods. In this paper we address semi-supervised domain adaptation (SSDA) for semantic segmentation, where a large amount of labeled source data as well as a small amount of labeled target data are available. We propose a novel and effective two-step semi-supervised dual-domain adaptation (SSDDA) approach to address both cross-and intra-domain gaps in semantic segmentation. The proposed framework is comprised of two mixing modules. First, we conduct a cross-domain adaptation via an image-level mixing strategy, which learns to align the distribution shift of features between the source data and target data. Second, intra-domain adaptation is achieved using a separate student-teacher network which is built to generate category-level data augmentation by mixing unlabeled target data in a way that respects predicted object boundaries. We demonstrate that the proposed approach outperforms state-of-the-art methods on two common synthetic-to-real semantic segmentation benchmarks. An extensive ablation study is provided to further validate the effectiveness of our approach.