Reviewerly/exHarmony huggingface dataset:
Highlight-aware two-stream network for single-image SVBRDF acquisition

This paper addresses the task of estimating spatially-varying reflectance (i.e., SVBRDF) from a single, casually captured image. Central to our method is a highlight-aware (HA) convolution operation and a two-stream neural network equipped with proper training losses. Our HA convolution, as a novel variant of standard (ST) convolution, directly modulates convolution kernels under the guidance of automatically learned masks representing potentially overexposed highlight regions. It helps to reduce the impact of strong specular highlights on diffuse components and at the same time, hallucinates plausible contents in saturated regions. Considering that variation of saturated pixels also contains important cues for inferring surface bumpiness and specular components, we design a two-stream network to extract features from two different branches stacked by HA convolutions and ST convolutions, respectively. These two groups of features are further fused in an attention-based manner to facilitate feature selection of each SVBRDF map. The whole network is trained end to end with a new perceptual adversarial loss which is particularly useful for enhancing the texture details. Such a design also allows the recovered material maps to be disentangled. We demonstrate through quantitative analysis and qualitative visualization that the proposed method is effective to recover clear SVBRDFs from a single casually captured image, and performs favorably against state-of-the-arts. Since we impose very few constraints on the capture process, even a non-expert user can create high-quality SVBRDFs that cater to many graphical applications.
BFIM: Performance Measurement of a Blockchain Based Hierarchical Tree Layered Fog-IoT Microservice Architecture

Fog computing complements cloud computing by removing several limitations, such as delays and network bandwidth. It emerged to support Internet of Things (IoT) applications wherein its computations and tasks are carried out at the network's edge. Heterogeneous IoT devices interact with different users throughout a network. However, data security is a crucial concern for IoT, fog and cloud network ecosystems. Since the number of anonymous users increases and new identity disclosures occur within the IoTs, it is becoming challenging to grow mesh networks to deliver end to end communications, as the extended IoT networks resemble a mesh architecture. To reinforce data security over IoTs, we deploy a microservice-based blockchain mechanism for fogs, which works as a decentralized client-server network medium (i.e., secured end device-based communication). We implement a blockchain equipped security scheme to be used with a fog-IoT hierarchical tree-based overlay mesh architecture to address and develop the network performance issues. In this study, we consider encryption and decryption delays from IoT and fog-integrated parts to monitor data records and compare them through the developed security scheme. The blocks of a blockchain offer the desired execution results mainly in terms of the algorithmic efficiency, which correlates with the existing algorithms, namely the Advanced Encryption Standard (AES), the Rivest Shamir Adleman (RSA), and the Data Encryption Standard (DES). Our `BFIM' scheme has an enhanced task scheduling capacity and a more efficient throughput than the AES, DES, RSA resource deliverables (i.e., tasks). Our comprehensive performance evaluation implies that the Blockchain-based Fog IoT Microservice (i.e., BFIM) architecture provides a task delivery efficiency of 78.79% (i.e., task deliverable) and a service delivery efficiency of 83.24% (i.e., task scheduling). The `BFIM' also has an overall process delivery efficiency of 75% (i.e., time delay, throughput) in the fog layer, rather than a central cloud layer running the AES, DES, and RSA algorithms.
Robust Explainability: A tutorial on gradient-based attribution methods for deep neural networks

With the rise of deep neural networks, the challenge of explaining the predictions of these networks has become increasingly recognized. While many methods for explaining the decisions of deep neural networks exist, there is currently no consensus on how to evaluate them. On the other hand, robustness is a popular topic for deep learning research; however, it is hardly talked about in explainability until very recently. In this tutorial paper, we start by presenting gradient-based interpretability methods. These techniques use gradient signals to assign the burden of the decision on the input features. Later, we discuss how gradient-based methods can be evaluated for their robustness and the role that adversarial robustness plays in having meaningful explanations. We also discuss the limitations of gradient-based methods. Finally, we present the best practices and attributes that should be examined before choosing an explainability method. We conclude with the future directions for research in the area at the convergence of robustness and explainability.
Bitcoin, currencies, and fragility

Formulae display:?Mathematical formulae have been encoded as MathML and are displayed in this HTML version using MathJax in order to improve their display. Uncheck the box to turn MathJax off. This feature requires Javascript. Click on a formula to zoom.
Exploring Differences in Student Learning and Behavior Between Real-life and Virtual Reality Chemistry Laboratories

Recent global events and educational trends have led schools to heavily rely on digital media to educate their students. Science classes, in particular, stand to lose substantial learning opportunities without the ability to provide physical laboratory experiences. Virtual reality (VR) technology has the potential to resolve this issue, but little is known if VR environments can produce similar results to real-life (RL) science learning environments. This 2 × 1, between-subjects study compares students' learning results and safety behaviors in VR and RL chemistry laboratories. The study attempts to identify differences in learning experience (i.e., general chemistry content, experiment comprehension, laboratory safety knowledge) and laboratory safety behavior. Results indicate learning general content knowledge, laboratory skills, and procedure-related safety behaviors were comparable between RL and VR conditions, but clean-up behaviors were less frequent in VR. Also, the exploratory, risk-free nature of VR environments may have allowed the learners to elaborate and reflect more on general chemistry content and laboratory safety knowledge than in the RL environment.
When the Timeline Meets the Pipeline: A Survey on Automated Cyberbullying Detection

Web 2.0 helped user-generated platforms to spread widely. Unfortunately, it also allowed for cyberbullying to spread. Cyberbullying has negative effects that could lead to cases of depression and low self-esteem. It has become crucial to develop tools for automated cyberbullying detection. The research on developing these tools has been growing over the last decade, especially with the recent advances in machine learning and natural language processing. Given the large body of work on this topic, it is vital to critically review the literature on cyberbullying within the context of these latest advances. In this paper, we survey the automated detection of cyberbullying. Our survey sheds light on some challenges and limitations for the field. The challenges range from defining cyberbullying, data collection, and feature representation to model selection, training, and evaluation. We also provide some suggestions for improving the task of cyberbullying detection. In addition to the survey, we propose to improve the task of cyberbullying detection by addressing some of the raised limitations: 1) Using recent contextual language models like BERT for the detection of cyberbullying; 2) Using slang-based word embeddings to generate better representations of the cyberbullying-related datasets. Our results show that BERT outperforms state-of-the-art cyberbullying detection models and deep learning models. The results also show that deep learning models initialized with slang-based word embeddings outperform deep learning models initialized with traditional word embeddings.
A Compressed $$\varSigma $$-Protocol Theory for Lattices

We show a lattice-based solution for commit-and-prove transparent circuit zero-knowledge (ZK) with polylog-communication, the first not depending on PCPs. We start from compressed $$\varSigma $$ -protocol theory (CRYPTO 2020), which is built around basic $$\varSigma $$ -protocols for opening an arbitrary linear form on a long secret vector that is compactly committed to. These protocols are first compressed using a recursive “folding-technique” adapted from Bulletproofs, at the expense of logarithmic rounds. Proving in ZK that the secret vector satisfies a given constraint – captured by a circuit – is then by (blackbox) reduction to the linear case, via arithmetic secret-sharing techniques adapted from MPC. Commit-and-prove is also facilitated, i.e., when commitment(s) to the secret vector are created ahead of any circuit-ZK proof. On several platforms (incl. DL) this leads to logarithmic communication. Non-interactive versions follow from Fiat-Shamir. This abstract modular theory strongly suggests that it should somehow be supported by a lattice-platform as well. However, when going through the motions and trying to establish low communication (on a SIS-platform), a certain significant lack in current understanding of multi-round protocols is exposed. Namely, as opposed to the DL-case, the basic $$\varSigma $$ -protocol in question typically has poly-small challenge space. Taking into account the compression-step – which yields non-constant rounds – and the necessity for parallelization to reduce error, there is no known tight result that the compound protocol admits an efficient knowledge extractor. We resolve the state of affairs here by a combination of two novel results which are fully general and of independent interest. The first gives a tight analysis of efficient knowledge extraction in case of non-constant rounds combined with poly-small challenge space, whereas the second shows that parallel repetition indeed forces rapid decrease of knowledge error. Moreover, in our present context, arithmetic secret sharing is not defined over a large finite field but over a quotient of a number ring and this forces our careful adaptation of how the linearization techniques are deployed. We develop our protocols in an abstract framework that is conceptually simple and can be flexibly instantiated. In particular, the framework applies to arbitrary rings and norms.
Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spotting

Soccer broadcast video understanding has been drawing a lot of attention in recent years within data scientists and industrial companies. This is mainly due to the lucrative potential unlocked by effective deep learning techniques developed in the field of computer vision. In this work, we focus on the topic of camera calibration and on its current limitations for the scientific community. More precisely, we tackle the absence of a large-scale calibration dataset and of a public calibration network trained on such a dataset. Specifically, we distill a powerful commercial calibration tool in a recent neural network architecture on the large-scale SoccerNet dataset, composed of untrimmed broadcast videos of 500 soccer games. We further release our distilled network, and leverage it to provide 3 ways of representing the calibration results along with player localization. Finally, we exploit those representations within the current best architecture for the action spotting task of SoccerNetv2, and achieve new state-of-the-art performances.
$$\mathsf {Mac'n'Cheese}$$: Zero-Knowledge Proofs for Boolean and Arithmetic Circuits with Nested Disjunctions

Zero knowledge proofs are an important building block in many cryptographic applications. Unfortunately, when the proof statements become very large, existing zero-knowledge proof systems easily reach their limits: either the computational overhead, the memory footprint, or the required bandwidth exceed levels that would be tolerable in practice. We present an interactive zero-knowledge proof system for boolean and arithmetic circuits, called $$\mathsf {Mac'n'Cheese}$$ , with a focus on supporting large circuits. Our work follows the commit-and-prove paradigm instantiated using information-theoretic MACs based on vector oblivious linear evaluation to achieve high efficiency. We additionally show how to optimize disjunctions, with a general OR transformation for proving the disjunction of m statements that has communication complexity proportional to the longest statement (plus an additive term logarithmic in m). These disjunctions can further be nested, allowing efficient proofs about complex statements with many levels of disjunctions. We also show how to make $$\mathsf {Mac'n'Cheese}$$ non-interactive (after a preprocessing phase) using the Fiat-Shamir transform, and with only a small degradation in soundness. We have implemented the online phase of $$\mathsf {Mac'n'Cheese}$$ and achieve a runtime of 144 ns per AND gate and 1.5 $$\upmu $$ s per multiplication gate in $$\mathbb {F} _{2^{61} - 1} $$ when run over a network with a 95 ms latency and a bandwidth of 31.5 Mbps. In addition, we show that the disjunction optimization improves communication as expected: when proving a boolean circuit with eight branches and each branch containing roughly 1 billion multiplications, $$\mathsf {Mac'n'Cheese}$$ requires only 75 more bytes to communicate than in the single branch case.
Glare: A free and open-source software for generation and assessment of digital speckle pattern

Generating digital speckle image and its corresponding deformed image is the basis of digital image correlation research. At present, however, it still lacks a powerful, easy-to-use, and user-friendly professional software concerning generation and assessment of digital speckle pattern. Researchers have to reimplement the generation algorithms in literature by themselves, which is time-consuming and error-prone. This paper reports a free and open-source software, Glare, for generation and assessment of digital speckle pattern. Glare has functions including generating speckle patterns, rendering deformed images, assessing pattern quality, and presenting pattern recommendations: Glare can generate ellipse, polygon, and Gaussian speckle patterns; can render deformed images with underlying deformation fields of translation, stretch/compression, rotation, sinusoidal deformation, Gaussian deformation, and Portevin-Le Chatelier band deformation; can calculate key pattern quality assessment parameters such as speckle coverage, speckle size, systematic error, and random error; can produce optimized speckle pattern in form of vector image. The software realizes real-time deformed image rendering with the aid of fast initial value estimation algorithm for backward mapping and pattern pre-rendering technique, and improves the computational efficiency of sum of square of subset intensity gradients by integral image method. In general, the software can be used not only for scientific research and engineering applications in digital image correlation community but also for education of experimental mechanics, and therefore has broad prospects.
Digital Transformation for Sustainable Development Goals (SDGs) - A Security, Safety and Privacy Perspective on AI

The main driver of the digital transformation currently underway is undoubtedly artificial intelligence (AI). The potential of AI to benefit humanity and its environment is undeniably enormous. AI can definitely help find new solutions to the most pressing challenges facing our human society in virtually all areas of life: from agriculture and forest ecosystems that affect our entire planet, to the health of every single human being. However, this article highlights a very different aspect. For all its benefits, the large-scale adoption of AI technologies also holds enormous and unimagined potential for new kinds of unforeseen threats. Therefore, all stakeholders, governments, policy makers, and industry, together with academia, must ensure that AI is developed with these potential threats in mind and that the safety, traceability, transparency, explainability, validity, and verifiability of AI applications in our everyday lives are ensured. It is the responsibility of all stakeholders to ensure the use of trustworthy and ethically reliable AI and to avoid the misuse of AI technologies. Achieving this will require a concerted effort to ensure that AI is always consistent with human values and includes a future that is safe in every way for all people on this planet. In this paper, we describe some of these threats and show that safety, security and explainability are indispensable cross-cutting issues and highlight this with two exemplary selected application areas: smart agriculture and smart health.
GAN computers generate arts? A survey on visual arts, music, and literary text generation using generative adversarial network

“Art is the lie that enables us to realize the truth.” – Pablo Picasso. For centuries, humans have dedicated themselves to producing arts to convey their imagination. The advancement in technology and deep learning in particular, has caught the attention of many researchers trying to investigate whether art generation is possible by computers and algorithms. Using generative adversarial networks (GANs), applications such as synthesizing photorealistic human faces and generating captions automatically from images were realized. This survey takes a comprehensive look at the recent works using GANs for generating visual arts, music, and literary text. A performance comparison and description of the various GAN architecture are also presented. Finally, some of the key challenges in GAN-based art generation are highlighted along with recommendations for future work.
Group-aware Contrastive Regression for Action Quality Assessment

Assessing action quality is challenging due to the subtle differences between videos and large variations in scores. Most existing approaches tackle this problem by regressing a quality score from a single video, suffering a lot from the large inter-video score variations. In this paper, we show that the relations among videos can provide important clues for more accurate action quality assessment during both training and inference. Specifically, we reformulate the problem of action quality assessment as regressing the relative scores with reference to another video that has shared attributes (e.g., category and difficulty), instead of learning unreferenced scores. Following this formulation, we propose a new Contrastive Regression (CoRe) framework to learn the relative scores by pair-wise comparison, which highlights the differences between videos and guides the models to learn the key hints for assessment. In order to further exploit the relative information between two videos, we devise a group-aware regression tree to convert the conventional score regression into two easier sub-problems: coarse-to-fine classification and regression in small intervals. To demonstrate the effectiveness of CoRe, we conduct extensive experiments on three mainstream AQA datasets including AQA-7, MTL-AQA and JIGSAWS. Our approach outperforms previous methods by a large margin and establishes new state-of-the-art on all three benchmarks.
A Novel Framework for the Analysis and Design of Heterogeneous Federated Learning

In federated learning, heterogeneity in the clients' local datasets and computation speeds results in large variations in the number of local updates performed by each client in each communication round. Naive weighted aggregation of such models causes objective inconsistency, that is, the global model converges to a stationary point of a mismatched objective function which can be arbitrarily different from the true objective. This paper provides a general framework to analyze the convergence of federated optimization algorithms with heterogeneous local training progress at clients. The analyses are conducted for both smooth non-convex and strongly convex settings, and can also be extended to partial client participation case. Additionally, it subsumes previously proposed methods such as FedAvg and FedProx, and provides the first principled understanding of the solution bias and the convergence slowdown due to objective inconsistency. Using insights from this analysis, we propose FedNova, a normalized averaging method that eliminates objective inconsistency while preserving fast error convergence.
ShipRSImageNet: A Large-Scale Fine-Grained Dataset for Ship Detection in High-Resolution Optical Remote Sensing Images

Ship detection in optical remote sensing images has potential applications in national maritime security, fishing, and defense. Many detectors, including computer vision and geoscience-based methods, have been proposed in the past decade. Recently, deep-learning-based algorithms have also achieved great success in the field of ship detection. However, most of the existing detectors face difficulties in complex environments, small ship detection, and fine-grained ship classification. One reason is that existing datasets have shortcomings in terms of the inadequate number of images, few ship categories, image diversity, and insufficient variations. This article publishes a public ship detection dataset, namely ShipRSImageNet, which contributes an accurately labeled dataset in different scenes with variant categories and image sources. The proposed ShipRSImageNet contains over 3435 images with 17 573 ship instances in 50 categories, elaborately annotated with both horizontal and orientated bounding boxes by experts. From our knowledge, up to now, the proposed ShipRSImageNet is the largest remote sensing dataset for ship detection. Moreover, several state-of-the-art detection algorithms are evaluated on our proposed ShipRSImageNet dataset to give a benchmark for deep-learning-based ship detection methods, which is valuable for assessing algorithm improvement. <xref ref-type="fn" rid="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><sup>1</sup></xref>
Learning lightweight super-resolution networks with weight pruning

Single image super-resolution (SISR) has achieved significant performance improvements due to the deep convolutional neural networks (CNN). However, the deep learning-based method is computationally intensive and memory demanding, which limit its practical deployment, especially for mobile devices. Focusing on this issue, in this paper, we present a novel approach to compress SR networks by weight pruning. To achieve this goal, firstly, we explore a progressive optimization method to gradually zero out the redundant parameters. Then, we construct a sparse-aware attention module by exploring a pruning-based well-suited attention strategy. Finally, we propose an information multi-slicing network which extracts and integrates multi-scale features at a granular level to acquire a more lightweight and accurate SR network. Extensive experiments reflect the pruning method could reduce the model size without a noticeable drop in performance, making it possible to apply the start-of-the-art SR models in the real-world applications. Furthermore, our proposed pruning versions could achieve better accuracy and visual improvements than state-of-the-art methods.