Reviewerly/exHarmony huggingface dataset:
Beyond Joint Demosaicking and Denoising: An Image Processing Pipeline for a Pixel-bin Image Sensor

Pixel binning is considered one of the most prominent solutions to tackle the hardware limitation of smartphone cameras. Despite numerous advantages, such an image sensor has to appropriate an artefact-prone non-Bayer colour filter array (CFA) to enable the binning capability. Contrarily, performing essential image signal processing (ISP) tasks like demosaicking and denoising, explicitly with such CFA patterns, makes the reconstruction process notably complicated. In this paper, we tackle the challenges of joint demosaicing and denoising (JDD) on such an image sensor by introducing a novel learning-based method. The proposed method leverages the depth and spatial attention in a deep network. The proposed network is guided by a multi-term objective function, including two novel perceptual losses to produce visually plausible images. On top of that, we stretch the proposed image processing pipeline to comprehensively reconstruct and enhance the images captured with a smartphone camera, which uses pixel binning techniques. The experimental results illustrate that the proposed method can outperform the existing methods by a noticeable margin in qualitative and quantitative comparisons. Code available: https://github.com/sharif-apu/BJDD_CVPR21.
Looking for Trouble: Analyzing Classifier Behavior via Pattern Divergence

Machine learning models may perform differently on different data subgroups, which we represent as itemsets (i.e., conjunctions of simple predicates). The identification of these critical data subgroups plays an important role in many applications, for example model validation and testing, or evaluation of model fairness. Typically, domain expert help is required to identify relevant (or sensitive) subgroups.
Identification of Fake Stereo Audio Using SVM and CNN

The number of channels is one of the important criteria in regard to digital audio quality. Generally, stereo audio with two channels can provide better perceptual quality than mono audio. To seek illegal commercial benefit, one might convert a mono audio system to stereo with fake quality. Identifying stereo-faking audio is a lesser-investigated audio forensic issue. In this paper, a stereo faking corpus is first presented, which is created using the Haas effect technique. Two identification algorithms for fake stereo audio are proposed. One is based on Mel-frequency cepstral coefficient features and support vector machines. The other is based on a specially designed five-layer convolutional neural network. The experimental results on two datasets with five different cut-off frequencies show that the proposed algorithm can effectively detect stereo-faking audio and has good robustness.
Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report

Image denoising is one of the most critical problems in mobile photo processing. While many solutions have been proposed for this task, they are usually working with synthetic data and are too computationally expensive to run on mobile devices. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image denoising solution that can demonstrate high efficiency on smartphone GPUs. For this, the participants were provided with a novel large-scale dataset consisting of noisy-clean image pairs captured in the wild. The runtime of all models was evaluated on the Samsung Exynos 2100 chipset with a powerful Mali GPU capable of accelerating floating-point and quantized neural networks. The proposed solutions are fully compatible with any mobile GPU and are capable of processing 480p resolution images under 40-80 ms while achieving high fidelity results. A detailed description of all models developed in the challenge is provided in this paper.
BurstSketch

Burst is a common pattern in data streams which is characterized by a sudden increase in terms of arrival rate followed by a sudden decrease. Burst detection has attracted extensive attention from the research community. In this paper, we propose a novel sketch, namely BurstSketch, to detect bursts accurately in real time. BurstSketch first uses the technique Running Track to select potential burst items efficiently, and then monitors the potential burst items and capture the key features of burst pattern by a technique called Snapshotting. Experimental results show that our sketch achieves a 1.75 times higher recall rate than the strawman solution.
Hate is the New Infodemic: A Topic-aware Modeling of Hate Speech Diffusion on Twitter

Online hate speech, particularly over microblogging platforms like Twitter, has emerged as arguably the most severe issue of the past decade. Several countries have reported a steep rise in hate crimes infuriated by malicious hate campaigns. While the detection of hate speech is one of the emerging research areas, the generation and spread of topic-dependent hate in the information network remain under-explored. In this work, we focus on exploring user behavior, which triggers the genesis of hate speech on Twitter and how it diffuses via retweets. We crawl a large-scale dataset of tweets, retweets, user activity history, and follower networks, comprising over 161 million tweets from more than 41 million unique users. We also collect over 600k contemporary news articles published online. We characterize different signals of information that govern these dynamics. Our analyses differentiate the diffusion dynamics in the presence of hate from usual information diffusion. This motivates us to formulate the modeling problem in a topic-aware setting with real-world knowledge. For predicting the initiation of hate speech for any given hashtag, we propose multiple feature-rich models, with the best performing one achieving a macro F1 score of 0.65. Meanwhile, to predict the retweet dynamics on Twitter, we propose RETINA, a novel neural architecture that incorporates exogenous influence using scaled dot-product attention. RETINA achieves a macro F1-score of 0.85, outperforming multiple state-of-the-art models. Our analysis reveals the superlative power of RETINA to predict the retweet dynamics of hateful content compared to the existing diffusion models.
DyGLIP: A Dynamic Graph Model with Link Prediction for Accurate Multi-Camera Multiple Object Tracking

Multi-Camera Multiple Object Tracking (MC-MOT) is a significant computer vision problem due to its emerging applicability in several real-world applications. Despite a large number of existing works, solving the data association problem in any MC-MOT pipeline is arguably one of the most challenging tasks. Developing a robust MC-MOT system, however, is still highly challenging due to many practical issues such as inconsistent lighting conditions, varying object movement patterns, or the trajectory occlusions of the objects between the cameras. To address these problems, this work, therefore, proposes a new Dynamic GraphModel with Link Prediction (DyGLIP) approach 1 to solve the data association task.Compared to existing methods, our new model offers several advantages, including better feature representations and the ability to recover from lost tracks during camera transitions.Moreover, our model works gracefully regardless of the overlapping ratios between the cameras.Experimental results show that we outperform existing MC-MOT algorithms by a large margin on several practical datasets.Notably, our model works favorably on online settings but can be extended to an incremental approach for large-scale datasets.
The true costs of digital currencies: Exploring impact beyond energy use

The digital currency Bitcoin is known for its energy hunger and associated carbon footprint. Investors, however, must not neglect further environmental, social, and governance issues related to digital currencies. Therefore, we urge the adoption of a more comprehensive view in assessing the externalities of investments in Bitcoin and other cryptocurrencies.
Mapping lithological units, structural lineaments and alteration zones in the Southern Kibi-Winneba belt of Ghana using integrated geophysical and remote sensing datasets

Geophysical and remote sensing datasets were employed to delineate potential zones of gold mineralization within the southern Kibi-Winneba belt of Ghana. The study area, which is characterized by the Birimian metavolcanic and metasedimentary rocks with prevalence of intrusive granitoids hosts several scientifically unguided operational activities by artisanal miners locally referred to as Galamsey. In this study, various enhancement techniques were employed on geophysical datasets encompassing magnetic, gravity and radiometric to delineate various lithological units, which corroborated with the Birimian formation and its related granitoid intrusives within the study area. The Center for Exploration Targeting grid analysis and Line Module Algorithm techniques were employed respectively on the magnetic and Sentinel 2A datasets to delineate various structural lineaments within the study area. Approximately 72% and 67% structural lineaments delineated respectively from the magnetic and Sentinel 2A datasets trended in the NE-SW and N-S direction, which control gold mineralization within the Birimian system. The prinicpal component analysis based on the Crosta technique was employed on the Sentinel 2A bands to highlight alteration zones within the study area. A potential map of gold mineralization zones within the study area was generated by integrating exploration layers comprising magnetic, gravity, structural density, radiometric and alteration zones delineated by Crosta technique. This mineral potential map categorized the belt into low, moderately low, moderately high and highly prospective regions with area coverage of 600.4 km2, 681.27 km2, 635.95 km2 and 295.20 km2 respectively. The mineralization potential map obtained was validated with available gold geochemical data and locations of artisanal mining operations (Galamsey), which corroborated with the highly potential mineralization zones of gold deposits delineated within the study area.
Defense Against Adversarial Attacks by Reconstructing Images

Convolutional neural networks (CNNs) are vulnerable to being deceived by adversarial examples generated by adding small, human-imperceptible perturbations to a clean image. In this paper, we propose an image reconstruction network that reconstructs an input adversarial example into a clean output image to defend against such adversarial attacks. Due to the powerful learning capabilities of the residual block structure, our model can learn a precise mapping from adversarial examples to reconstructed examples. The use of a perceptual loss greatly suppresses the error amplification effect and improves the performance of our reconstruction network. In addition, by adding randomization layers to the end of the network, the effects of additional noise are further suppressed, especially for iterative attacks. Our model has the following four advantages. 1) It greatly reduces the impact of adversarial perturbations while having little influence on the prediction performance of clean images. 2) During inference phase, it performs better than most existing model-agnostic defense methods. 3) It has better generalization capability. 4) It can be flexibly combined with other methods, such as adversarially trained models.
Blockchain technology awareness on social media: Insights from twitter analytics

This study investigates the affective technology acceptance model applied to the case of blockchain through Twitter text mining. The analysis focuses on mapping the acceptance drivers of the blockchain technology by visualizing the users perception constructs through Blockchain hashtags. More than 5000 relevant tweets per day were collected between December 15, 2020, and January 15, 2021. The Kruskal-Wallis and the Mann-Whitney tests were applied over the frequency of the characteristics and the emotions' measurements to validate the research hypotheses. The results prove that users show more interest in security, shareability, and decentralization characteristics. Therefore, the blockchain technology usefulness is rather perceived in the informational domain, and the blockchain ease of use is further expressed in smart contracts as a use case. Blockchain benefits are more discussed than the drawbacks among Twitter users. Besides, positive feelings with strong emotions of trust and joy dominate among users. In summary, the results show significant awareness of users towards blockchain technology. To the best of the authors' knowledge, this paper is the first study that explores the affective technology acceptance model with user-generated content analysis.
Leveraging Blockchain Technology for Indian Healthcare system: An assessment using value-focused thinking approach

Indian Healthcare system is a complex and distinguished structure due to variety of factors viz. population size it serves, presence of organized and unorganized services of healthcare, economic disparity and governance structure among others. Extant research emphasizes on the aspects like transparency, trust, disintermediation, non-repudiation, benchmarking of processes and practices, traceability and data integrity among others to stimulate the Indian Healthcare system for an improvised and more effective format. Blockchain Technology is recent innovation that offers many of these aforementioned aspects as a feature to the host systems. However, it is critical to understand and articulate the utility of Blockchain Technology specific to the Indian Healthcare system. Further, it is important to gauge the requirement elicitation of key stakeholders for Blockchain Technology adoption in Indian Healthcare system w.r.t any attribute prioritization/customization or design considerations (Consensus mechanism, type of blockchain, smart contracts etc.). There have been some research works on aspects of Blockchain applications in Indian Healthcare but lack a strong perspective of stakeholder requirements. We use Value-Focused Thinking framework based on discussions with the stakeholders to address the above mentioned research premise. We identify Fundamental Objectives and Strategic Objectives from the stakeholder interactions that contribute to the strategic goal. This is a niche work with a significant analytical contribution from the information systems perspective of Blockchain Technology adoption in the Indian Healthcare ecosystem given the light of premises from realms of human agents, behavioural and applications requirement from a stakeholder viewpoint.
Deep Lucas-Kanade Homography for Multimodal Image Alignment

Estimating homography to align image pairs captured by different sensors or image pairs with large appearance changes is an important and general challenge for many computer vision applications. In contrast to others, we propose a generic solution to pixel-wise align multimodal image pairs by extending the traditional Lucas-Kanade algorithm with networks. The key contribution in our method is how we construct feature maps, named as deep Lucas-Kanade feature map (DLKFM). The learned DLKFM can spontaneously recognize invariant features under various appearance-changing conditions. It also has two nice properties for the Lucas-Kanade algorithm: (1) The template feature map keeps brightness consistency with the input feature map, thus the color difference is very small while they are well-aligned. (2) The Lucas-Kanade objective function built on DLKFM has a smooth landscape around ground truth homography parameters, so the iterative solution of the Lucas-Kanade can easily converge to the ground truth. With those properties, directly updating the Lucas-Kanade algorithm on our feature maps will precisely align image pairs with large appearance changes. We share the datasets, code, and demo video online <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
SRDAN: Scale-aware and Range-aware Domain Adaptation Network for Cross-dataset 3D Object Detection

Geometric characteristic plays an important role in the representation of an object in 3D point clouds. For example, large objects often contain more points, while small ones contain fewer points. The points from objects near the capture device are denser, while those from far-range objects are sparser. These issues bring new challenges to 3D object detection, especially under the domain adaptation scenarios. In this work, we propose a new cross-dataset 3D object detection method named Scale-aware and Range-aware Domain Adaptation Network (SRDAN). We take advantage of the geometric characteristics of 3D data (i.e., size and distance), and propose the scale-aware domain alignment and the range-aware domain alignment strategies to guide the distribution alignment between two domains. For scale-aware domain alignment, we design a 3D voxel-based feature pyramid network to extract multi-scale semantic voxel features, and align the features and instances with similar scales between two domains. For range-aware domain alignment, we introduce a range-guided domain alignment module to align the features of objects according to their distance to the capture device. Extensive experiments under three different scenarios demonstrate the effectiveness of our SRDAN approach, and comprehensive ablation study also validates the importance of geometric characteristics for cross-dataset 3D object detection.
AutoProf – I. An automated non-parametric light profile pipeline for modern galaxy surveys

We present an automated non-parametric light profile extraction pipeline called AutoProf. All steps for extracting surface brightness (SB) profiles are included in AutoProf, allowing streamlined analyses of galaxy images. AutoProf improves upon previous non-parametric ellipse fitting implementations with fit-stabilization procedures adapted from machine learning techniques. Additional advanced analysis methods are included in the flexible pipeline for the extraction of alternative brightness profiles (along radial or axial slices), smooth axisymmetric models, and the implementation of decision trees for arbitrarily complex pipelines. Detailed comparisons with widely used photometry algorithms (photutils, XVISTA, and GALFIT) are presented. These comparisons rely on a large collection of late-type galaxy images from the PROBES survey. The direct comparison of SB profiles shows that AutoProf can reliably extract fainter isophotes than other methods on the same images, typically by >2 mag/arcsec^2. Contrasting non-parametric elliptical isophote fitting with simple parametric models also shows that two-component fits (e.g., S\'ersic plus exponential) are insufficient to describe late-type galaxies with high fidelity. It is established that elliptical isophote fitting, and in particular AutoProf, is ideally suited for a broad range of automated isophotal analysis tasks. AutoProf is freely available to the community at: https://github.com/ConnorStoneAstro/AutoProf
A Learning Experience in Inquiry-Based Physics with Immersive Virtual Reality: Student Perceptions and an Interaction Effect Between Conceptual Gains and Attitudinal Profiles

The integration of immersive virtual reality (VR) in authentic science classrooms can result in a totally new learning experience for the students. However, the effect of such a learning experience on students’ conceptual learning gains and their perceptions of the experience, while considering students’ pre-existing science- and digital technologies-related attitudinal profiles, has not been explored to date. In this study, we have enacted a 90-min technology-enhanced inquiry-based intervention with high-school students (n = 107), on the topic of the Special Theory of Relativity in a Physics course, using a learning experience design, structured around an immersive VR simulation. Firstly, we aimed at examining students’ attitudinal profiles and, secondly, at exploring the potential differences of those profiles in relation to conceptual learning gains and perceptions of the learning experience. A clustering analysis has revealed two attitudinal profiles: the low-attitudes profile (n = 48) included students with low science- and digital technologies-related attitudes, and the opposite for the high-attitudes profile (n = 59). Results from a 2 × 2 RM ANOVA indicated a statistically significant interaction between conceptual learning gains and attitudinal profiles. In addition, a one-way MANOVA test showed statistically significant differences between the two profiles in relation to students’ perceptions of the learning experience, with the students of the high-attitude profile outperforming their counterparts. We discuss our findings, focusing on the implications of students’ individual differences in learning and attitudes linked to the integration of immersive VR in inquiry-based instruction.
A Multi-Camera Vehicle Tracking System based on City-Scale Vehicle Re-ID and Spatial-Temporal Information

With the demands of the intelligent city and city-scale traffic management, city-scale multi-camera vehicle tracking (MCVT) has become a vital problem. The MCVT is challenging due to frequent occlusion, similar vehicle models, significant feature variation by different lighting conditions, and viewing perspective in different cameras. This paper proposes an MCVT system composed of single-camera tracking (SCT), vehicle re-identification (Re-ID), and multi-camera tracks matching (MCTM). In the SCT phase, we designed a tracker update strategy and used the Re-ID model in advance. We also adopted a template matching method to re-associate the discontinuous tracklets. As for vehicle Re-ID, we implemented a spatial attention mechanism based on the background model. Then we fully leveraged the labels of synthetic data to train attributes Re-ID models as the attributes features extractor. Finally, we proposed an MCTM method to leverage tracklets representation and spatial-temporal information efficiently. Our system is evaluated both on the City-Scale Multi-Camera Vehicle Re-Identification task (Track 2) and City-Scale Multi-Camera Vehicle Tracking task (Track 3) at the AI City Challenge. Our vehicle Re-ID method has achieved 3rd place of Track 2, with an mAP score of 66.50%, and achieved state-of-the-art results on the VeRi776 dataset. Our MCVT system has achieved 3rd place, yielding 76.51% IDF <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> of Track 3. Experimental results demonstrate that our system has achieved competitive performance for city-scale traffic management.
SSAN: Separable Self-Attention Network for Video Representation Learning

Self-attention has been successfully applied to video representation learning due to the effectiveness of modeling long range dependencies. Existing approaches build the dependencies merely by computing the pairwise correlations along spatial and temporal dimensions simultaneously. However, spatial correlations and temporal correlations represent different contextual information of scenes and temporal reasoning. Intuitively, learning spatial contextual information first will benefit temporal modeling. In this paper, we propose a separable self-attention (SSA) module, which models spatial and temporal correlations sequentially, so that spatial contexts can be efficiently used in temporal modeling. By adding SSA module into 2D CNN, we build a SSA network (SSAN) for video representation learning. On the task of video action recognition, our approach outperforms state-of-the-art methods on Something-Something and Kinetics-400 datasets. Our models often outperform counterparts with shallower network and fewer modalities. We further verify the semantic learning ability of our method in visual-language task of video retrieval, which showcases the homogeneity of video representations and text embeddings. On MSR-VTT and Youcook2 datasets, video representations learnt by SSA significantly improve the state-of-the-art performance.