Reviewerly/exHarmony huggingface dataset:
Revisiting Superpixels for Active Learning in Semantic Segmentation with Realistic Annotation Costs

State-of-the-art methods for semantic segmentation are based on deep neural networks that are known to be data-hungry. Region-based active learning has shown to be a promising method for reducing data annotation costs. A key design choice for region-based AL is whether to use regularly-shaped regions (e.g., rectangles) or irregularly-shaped region (e.g., superpixels). In this work, we address this question under realistic, click-based measurement of annotation costs. In particular, we revisit the use of super-pixels and demonstrate that the inappropriate choice of cost measure (e.g., the percentage of labeled pixels), may cause the effectiveness of the superpixel-based approach to be under-estimated. We benchmark the superpixel-based approach against the traditional "rectangle+polygon"-based approach with annotation cost measured in clicks, and show that the former outperforms on both Cityscapes and PASCAL VOC. We further propose a class-balanced acquisition function to boost the performance of the superpixel-based approach and demonstrate its effectiveness on the evaluation datasets. Our results strongly argue for the use of superpixel-based AL for semantic segmentation and highlight the importance of using realistic annotation costs in evaluating such methods.
Investigations of Molecular Optical Properties Using Quantum Light and Hong–Ou–Mandel Interferometry

Entangled photon pairs have been used for molecular spectroscopy in the form of entangled two-photon absorption and in quantum interferometry for precise measurements of light source properties and time delays. We present an experiment that combines molecular spectroscopy and quantum interferometry by utilizing the correlations of entangled photons in a Hong–Ou–Mandel (HOM) interferometer to study molecular properties. We find that the HOM signal is sensitive to the presence of a resonant organic sample placed in one arm of the interferometer, and the resulting signal contains information pertaining to the light–matter interaction. We can extract the dephasing time of the coherent response induced by the excitation on a femtosecond time scale. A dephasing time of 102 fs is obtained, which is relatively short compared to times found with similar methods and considering line width broadening and the instrument entanglement time As the measurement is done with coincidence counts as opposed to simply intensity, it is unaffected by even-order dispersion effects, and because interactions with the molecular state affect the photon correlation, the observed measurement contains only these effects and no other classical losses. The experiments are accompanied by theory that predicts the observed temporal shift and captures the entangled photon joint spectral amplitude and the molecule's transmission in the coincidence counting rate. Thus, we present a proof-of-concept experimental method based of entangled photon interferometry that can be used to characterize optical properties in organic molecules and can in the future be expanded on for more complex spectroscopic studies of nonlinear optical properties.
Semi-Deraingan: A New Semi-Supervised Single Image Deraining

Although supervised single image deraining (SID) have obtained impressive results, they still cannot obtain satisfactory results on real images for the weak generalization of rain removal capacity. In this paper, we mainly discuss the semi-supervised SID and propose a new GAN-based deraining network called Semi-DerainGAN, which can use both synthetic and real data in a uniform network based on two supervised and unsupervised processes. For this task, a semi-supervised rain streak learner termed SSRML sharing the same parameters of both processes is derived, which makes the real images contribute more rain streak information, so that the resulted model has a strong generalization power to the real SID task. We also contribute a new real-world rain image dataset called Real200 to alleviate the difference between both synthetic and real image domains. Extensive results on public datasets show that our model can obtain competitive results, especially on the real rain images.
Working Together on Diverse Tasks: A Longitudinal Study on Individual Workload, Presence and Emotional Recognition in Collaborative Virtual Environments

Numerous studies have shown the potential benefits of collaborative virtual environments (CVEs) for distributed teams. However, there are few longitudinal studies on collaboration in immersive virtual environments, and existing studies mostly examine how pairs or groups adapt over time. In a longitudinal study, we examined what does and does not change over time as individual users adapt to collaboration in virtual environments. In our mixed-methods, exploratory study, we matched 20 participants in random pairs over five sessions. We assigned each participant to complete a different collaborative task, with a different partner, in each session. Our quantitative data analysis and qualitative interview data show that adaptation to VR increased significantly over time. Presence ratings did not show change over time, but participants reported developing new ways to communicate in VR. We also identified patterns indicating a relationship between a person’s emotional state and their partner’s ability to recognize their emotion. We conclude with a discussion of our findings and provide design implications and future directions for designers and researchers in the field.
Developing a socio-computational approach to examine toxicity propagation and regulation in COVID-19 discourse on YouTube

As the novel coronavirus (COVID-19) continues to ravage the world at an unprecedented rate, formal recommendations from medical experts are becoming muffled by the avalanche of toxic content posted on social media platforms. This high level of toxic content prevents the dissemination of important and time-sensitive information and jeopardizes the sense of community that online social networks (OSNs) seek to cultivate. In this article, we present techniques to analyze toxic content and actors that propagated it on YouTube during the initial months after COVID-19 information was made public. Our dataset consists of 544 channels, 3,488 videos, 453,111 commenters, and 849,689 comments. We applied topic modeling based on Latent Dirichlet Allocation (LDA) to identify dominant topics and evolving trends within the comments on relevant videos. We conducted social network analysis (SNA) to detect influential commenters, and toxicity analysis to measure the health of the network. SNA allows us to identify the top toxic users in the network, which led to the creation of experiments simulating the impact of removal of these users on toxicity in the network. Through this work, we demonstrate not only how to identify toxic content related to COVID-19 on YouTube and the actors who propagated this toxicity, but also how social media companies and policy makers can use this work. This work is novel in that we devised a set of experiments in an attempt to show how if social media platforms eliminate certain toxic users, they can improve the overall health of the network by reducing the overall toxicity level.
Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation

Open compound domain adaptation (OCDA) is a domain adaptation setting, where target domain is modeled as a compound of multiple unknown homogeneous domains, which brings the advantage of improved generalization to unseen domains. In this work, we propose a principled meta-learning based approach to OCDA for semantic segmentation, MOCDA, by modeling the unlabeled target domain continuously. Our approach consists of four key steps. First, we cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. Then, different sub-target domains are split into independent branches, for which batch normalization parameters are learnt to treat them independently. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. Meanwhile, we learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization. We validate the benefits of our approach by extensive experiments on synthetic-to-real knowledge transfer benchmark, where we achieve the state-of-the-art performance in both compound and open domains.
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models

The computer vision world has been re-gaining enthusiasm in various pre-trained models, including both classical ImageNet supervised pre-training and recently emerged self-supervised pre-training such as simCLR [10] and MoCo [40]. Pre-trained weights often boost a wide range of downstream tasks including classification, detection, and segmentation. Latest studies suggest that pre-training benefits from gigantic model capacity [11]. We are hereby curious and ask: after pre-training, does a pre-trained model indeed have to stay large for its downstream transferability? In this paper, we examine supervised and self-supervised pre-trained models through the lens of the lottery ticket hypothesis (LTH) [31]. LTH identifies highly sparse matching subnetworks that can be trained in isolation from (nearly) scratch yet still reach the full models' performance. We extend the scope of LTH and question whether matching subnetworks still exist in pre-trained computer vision models, that enjoy the same downstream transfer performance. Our extensive experiments convey an overall positive message: from all pre-trained weights obtained by ImageNet classification, simCLR, and MoCo, we are consistently able to locate such matching subnetworks at 59.04% to 96.48% sparsity that transfer universally to multiple downstream tasks, whose performance see no degradation compared to using full pre-trained weights. Further analyses reveal that subnetworks found from different pre-training tend to yield diverse mask structures and perturbation sensitivities. We conclude that the core LTH observations remain generally relevant in the pre-training paradigm of computer vision, but more delicate discussions are needed in some cases. Codes and pre-trained models will be made available at: https://github.com/VITA-Group/CV_LTH_Pre-training.
TARDIS: A Foundation of Time-Lock Puzzles in UC

Time-based primitives like time-lock puzzles (TLP) are finding widespread use in practical protocols, partially due to the surge of interest in the blockchain space where TLPs and related primitives are perceived to solve many problems. Unfortunately, the security claims are often shaky or plainly wrong since these primitives are used under composition. One reason is that TLPs are inherently not UC secure and time is tricky to model and use in the UC model. On the other hand, just specifying standalone notions of the intended task, left alone correctly using standalone notions like non-malleable TLPs only, might be hard or impossible for the given task. And even when possible a standalone secure primitive is harder to apply securely in practice afterwards as its behavior under composition is unclear. The ideal solution would be a model of TLPs in the UC framework to allow simple modular proofs. In this paper we provide a foundation for proving composable security of practical protocols using time-lock puzzles and related timed primitives in the UC model. We construct UC-secure TLPs based on random oracles and show that using random oracles is necessary. In order to prove security, we provide a simple and abstract way to reason about time in UC protocols. Finally, we demonstrate the usefulness of this foundation by constructing applications that are interesting in their own right, such as UC-secure two-party computation with output-independent abort.
Transfer Learning Under High-Dimensional Generalized Linear Models

In this work, we study the transfer learning problem under high-dimensional generalized linear models (GLMs), which aim to improve the fit on target data by borrowing information from useful source data. Given which sources to transfer, we propose a transfer learning algorithm on GLM, and derive its l1/l2-estimation error bounds as well as a bound for a prediction error measure. The theoretical analysis shows that when the target and sources are sufficiently close to each other, these bounds could be improved over those of the classical penalized estimator using only target data under mild conditions. When we don't know which sources to transfer, an algorithm-free transferable source detection approach is introduced to detect informative sources. The detection consistency is proved under the high-dimensional GLM transfer learning setting. We also propose an algorithm to construct confidence intervals of each coefficient component, and the corresponding theories are provided. Extensive simulations and a real-data experiment verify the effectiveness of our algorithms. We implement the proposed GLM transfer learning algorithms in a new R package glmtrans, which is available on CRAN. Supplementary materials for this article are available online.
US Based COVID-19 Tweets Sentiment Analysis Using TextBlob and Supervised Machine Learning Algorithms

In December 2019, a highly contagious disease, Coronavirus disease 2019 (COVID-19) was first detected in Wuhan, China. The disease has spread to 212 countries and territories worldwide. While this epidemic has continued to infect millions of people, several nations have resorted to complete lockdowns. People took social networks during this shutdown to share their opinions, feelings, and find a way to calm down. This study proposed a US-based sentiment analysis of the tweets using machine learning and the lexicon analysis approach. This US-based tweets dataset was collected by RStudio software from 30 January 2020 to 10th May 2020, contains 11858 tweets. We find the label corresponding to each tweet using TextBlob, that is to say, positive, negative, or neutral. To clean up the facts we pre-process the tweets. In a later step, different feature techniques such as bag-of-words (BoW) and term frequency-inverse document frequency (TF-IDF) are used to preserve expressive information. Finally, the random forest, gradient boosting machine, extra tree classifier, logistic regression, and support vector machine models are used to categorize beliefs as being positive, negative, or neutral. Our suggested pipeline output is assessed using accuracy, precision, recall & F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> score. This research study shows how TF-IDF features can increase the performance of the supervised machine learning models and in this work, the gradient boosting machine outperforms the others and achieves high accuracy of 96% when paired with TF-IDF features. This analysis was done to analyze how the situation is being handled by citizens of the United States. The results of the experiments validate the approach’s effectiveness.
Blockchain-Enabled Intelligent Vehicular Edge Computing

Smart vehicles are expected to be equipped with high-dimensional, resource-intensive applications, including platoon control, augmented reality supported gaming, AI-based pedestrian detection, fuel scheduling, and so on, catering to diverse user preferences and enhancing safety and efficiency. These applications pose unique challenges for resource-constrained vehicles due to their intense computation requirements, whereas vehicular edge computing (VEC) networks, consisting of roadside units (RSUs) and MEC servers, contain the capability of providing cloud-like computing experience at vehicular edges while meeting performance requirements in terms of latency and throughput. Moreover, the development of intelligent VEC (IVEC) infrastructure is accelerated due to rapid advancement of AI algorithms in recent years. However, IVEC is prone to attacks, including fake computation feedback, unfair or biased resource allocation in a VEC server, and so on, due to its centralized governance and black box computation (edge computation works like a black box for end users). To combat such security vulnerabilities, we propose a blockchain-based decentralized architecture to enhance transparency in IVEC resource management and leverage edge consumers (e.g., vehicles) with a computation verification option. Additionally, we address the unbalanced load distribution issue and propose a secure IVEC federation model for balancing loads. We also outline the main challenges and provide a brief description of promising research directions to draw the attention of concerned stakeholders and parties in both the blockchain and edge computing domains.
Energy, economic and environmental performance assessment of a grid-tied rooftop system in different cities of India based on 3E analysis

Abstract India is very rich in solar energy, with a total of 3000 sunshine hours annually in most places. The installation of on-grid rooftop electricity-generation photovoltaic (PV) systems is currently undergoing substantial growth and extension as an alternate source of energy that contributes to Indian buildings. This paper analyses the viability of mounting solar PV plants in distinct cities of India in various locations with different climate conditions such as Delhi, Bhopal, Udaipur, Ahmadabad, Thiruvananthapuram, Pune and Madurai. The technical feasibility of installing a 100-kWp system is evaluated using PVsyst software under local climatic conditions. The performance ratio is between 70% and 80%, with a capacity utilization factor of 19–21% and estimated energy output of 170 MWh annually at all sites. The system produces 400–500 kWh of energy daily at a per-unit cost of INR 6–7 (Indian rupees) in all locations. The lifespan of the system is ~25–30 years, reducing about 150–170 tons of carbon-dioxide emission to the atmosphere every year. The payback period of the system is ~5–6 years, which defines its feasibility. This information would encourage organizations and individuals to install such PV plants on the rooftops of buildings to use solar electricity for meeting the energy demands of the country.
Spatially-invariant Style-codes Controlled Makeup Transfer

Transferring makeup from the misaligned reference image is challenging. Previous methods overcome this barrier by computing pixel-wise correspondences between two images, which is inaccurate and computational-expensive. In this paper, we take a different perspective to break down the makeup transfer problem into a two-step extraction-assignment process. To this end, we propose a Style-based Controllable GAN model that consists of three components, each of which corresponds to target style-code encoding, face identity features extraction, and makeup fusion, respectively. In particular, a Part-specific Style Encoder encodes the component-wise makeup style of the reference image into a style-code in an intermediate latent space W. The style-code discards spatial information and therefore is invariant to spatial misalignment. On the other hand, the style-code embeds component-wise information, enabling flexible partial makeup editing from multiple references. This style-code, together with source identity features, is integrated into a Makeup Fusion Decoder equipped with multiple AdaIN layers to generate the final result. Our proposed method demonstrates great flexibility on makeup transfer by supporting makeup removal, shade-controllable makeup transfer, and part-specific makeup transfer, even with large spatial misalignment. Extensive experiments demonstrate the superiority of our approach over state-of-the-art methods. Code is available at https://github.com/makeuptransfer/SCGAN.
Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes

Joint rolling shutter correction and deblurring (RSCD) techniques are critical for the prevalent CMOS cameras. However, current approaches are still based on conventional energy optimization and are developed for static scenes. To enable learning-based approaches to address real-world RSCD problem, we contribute the first dataset, BS-RSCD, which includes both ego-motion and object-motion in dynamic scenes. Real distorted and blurry videos with corresponding ground truth are recorded simultaneously via a beam-splitter-based acquisition system.Since direct application of existing individual rolling shutter correction (RSC) or global shutter deblurring (GSD) methods on RSCD leads to undesirable results due to inherent flaws in the network architecture, we further present the first learning-based model (JCD) for RSCD. The key idea is that we adopt bi-directional warping streams for displacement compensation, while also preserving the non-warped deblurring stream for details restoration. The experimental results demonstrate that JCD achieves state-of-the-art performance on the realistic RSCD dataset (BS-RSCD) and the synthetic RSC dataset (Fastec-RS). The dataset and code are available at https://github.com/zzh-tech/RSCD.
A High Visual Quality Color Image Reversible Data Hiding Scheme Based on B-R-G Embedding Principle and CIEDE2000 Assessment Metric

Reversible data hiding methods for color images are investigated because of the popularity of color images. However, the traditional RDH methods for color images only take PSNR as the assessment metric and pursue a high PSNR value. To further consider the different subjective perception in the R, G, and B channels of the color image, this paper proposes a high visual quality color image reversible data hiding scheme based on the B-R-G embedding principle and the CIEDE2000 assessment metric. In this method, a double-layer least square prediction is proposed to satisfy the sorting requirement and maintain prediction accuracy, and then the B-R-G embedding principle is proposed to improve the visual quality of the marked color images based on the different visual perception in the three channels, and finally, a one-by-one embedding method is proposed to reduce the embedding distortion by maintaining the inter channel correlation. The experimental results show that the proposed method is superior to the state-of-the-art RDH methods for color images.
Robust subspace clustering based on automatic weighted multiple kernel learning

• To handle the noise in the kernel effectively, we design a robust multikernel learning strategy based on CIM. • To handle the noise in the data, we introduce the BDR item to address the destruction of the intrinsic block- diagonal structure of the affinity matrix . • To solve the nonconvex problem of the model caused by the introduction of low-rank kernel constraints, a novel arithmetic AM&GST is designed by combining alternating minimization and generalized soft thresholding . Multiple kernel learning (MKL), which combines a set of prespecified basic kernels to improve the clustering performance, has become an important research topic. Unfortunately, the current methods have the following defects in noisy circumstances. 1) Their clustering performance may be significantly reduced due to the noise in the kernel, which is caused by the lack of a reliable discriminant guideline for basic kernel combinations. 2) The noise from corrupted data or occlusion may destroy the block-diagonal structures of the affinity matrices they obtained, which will affect the clustering performance when using spectral clustering. In this work, to solve the above problems, we propose an automatic weighted multikernel learning-based robust subspace clustering (AWLKSC) algorithm. The model integrates multikernel learning strategies, the Correntropy-Induced Metric (CIM), low rank approximation technology and block diagonal constraints. In addition, an effective AM&GST algorithm, which is integrated by alternating minimization and generalized soft-thresholding, is developed to optimize the AWLKSC. Seven types of noise are considered in the experiments, and the experimental results illustrate that AWLKSC is more effective and robust than several up-to-date single kernel and multiple kernel clustering methods.