Reviewerly/exHarmony huggingface dataset:
HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers

High-resolution representations (HR) are essential for dense prediction tasks such as segmentation, detection, and pose estimation. Learning HR representations is typically ignored in previous Neural Architecture Search (NAS) methods that focus on image classification. This work proposes a novel NAS method, called HR-NAS, which is able to find efficient and accurate networks for different tasks, by effectively encoding multiscale contextual information while maintaining high-resolution representations. In HR-NAS, we renovate the NAS search space as well as its searching strategy. To better encode multiscale image contexts in the search space of HR-NAS, we first carefully design a lightweight transformer, whose computational complexity can be dynamically changed with respect to different objective functions and computation budgets. To maintain high-resolution representations of the learned networks, HR-NAS adopts a multi-branch architecture that provides convolutional encoding of multiple feature resolutions, inspired by HRNet [73]. Last, we proposed an efficient fine-grained search strategy to train HR-NAS, which effectively explores the search space, and finds optimal architectures given various tasks and computation resources. As shown in Fig. 1 (a), HR-NAS is capable of achieving state-of-the-art trade-offs between performance and FLOPs for three dense prediction tasks and an image classification task, given only small computational budgets. For example, HR-NAS surpasses SqueezeNAS [63] that is specially designed for semantic segmentation while improving efficiency by 45.9%. Code is available at https://github.com/dingmyu/HR-NAS.
Self-Supervised 3D Hand Pose Estimation from monocular RGB via Contrastive Learning

Encouraged by the success of contrastive learning on image classification tasks, we propose a new self-supervised method for the structured regression task of 3D hand pose estimation. Contrastive learning makes use of unlabeled data for the purpose of representation learning via a loss formulation that encourages the learned feature representations to be invariant under any image transformation. For 3D hand pose estimation, it too is desirable to have invariance to appearance transformation such as color jitter. However, the task requires equivariance under affine transformations, such as rotation and translation. To address this issue, we propose an equivariant contrastive objective and demonstrate its effectiveness in the context of 3D hand pose estimation. We experimentally investigate the impact of invariant and equivariant contrastive objectives and show that learning equivariant features leads to better representations for the task of 3D hand pose estimation. Furthermore, we show that standard ResNets with sufficient depth, trained on additional unlabeled data, attain improvements of up to 14.5% in PA-EPE on FreiHAND and thus achieves state-of-the-art performance without any task specific, specialized architectures. Code and models are available at https://ait.ethz.ch/projects/2021/PeCLR/
Explainable AI methods in cyber risk management

Abstract Artificial intelligence (AI) methods are becoming widespread, especially when data are not sufficient to build classical statistical models, as is the case for cyber risk management. However, when applied to regulated industries, such as energy, finance, and health, AI methods lack explainability. Authorities aimed at validating machine learning models in regulated fields will not consider black‐box models, unless they are supplemented with further methods that explain why certain predictions have been obtained, and which are the variables that mostly concur to such predictions. Recently, Shapley values have been introduced for this purpose: They are model agnostic, and powerful, but are not normalized and, therefore, cannot become a standardized procedure. In this paper, we provide an explainable AI model that embeds Shapley values with a statistical normalization, based on Lorenz Zonoids, particularly suited for ordinal measurement variables that can be obtained to assess cyber risk.
Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer

Abstract Deep learning-based predictive quality enables manufacturing companies to make data-driven predictions of the quality of a produced product based on process data. A central challenge is that production processes are subject to continuous changes such as the manufacturing of new products, with the result that previously trained models may no longer perform well in the process. In this paper, we address this problem and propose a method for continual learning in such predictive quality scenarios. We therefore adapt and extend the memory-aware synapses approach to train an artificial neural network across different product variations. Our evaluation in a real-world regression problem in injection molding shows that the approach successfully prevents the neural network from forgetting of previous tasks and improves the training efficiency for new tasks. Moreover, by extending the approach with the transfer of network weights from similar previous tasks, we significantly improve its data efficiency and performance on sparse data. Our code is publicly available to reproduce our results and build upon them.
Kernel multi-attention neural network for knowledge graph embedding

Link prediction is the problem of predicting missing link between entities and relations for knowledge graph. In recent years, some tasks have achieved great success for link prediction, but these tasks are far from expanding entity relation vectors, and cannot predict missing links more efficiently. In this paper, we propose a novel link prediction method called kernel multi-attention neural network for knowledge graph embedding (KMAE) which is able to extend kernel separately in entity and relation attributes. The kernel function uses Gaussian kernel function to expand into more robust entity kernel and relation kernel. In addition, we constructed a novel multi-attention neural network that acts on the entity kernel and relation kernel which can capture local important characteristics. Experiments on FB15k-237 and WN18RR, show that multi-attention fully reflect excellent performance in the task of knowledge graph embedding. Our proposed KMAE achieves better results than previous state-of-the-art link prediction methods.
Continual Adaptation of Visual Representations via Domain Randomization and Meta-learning

Most standard learning approaches lead to fragile models which are prone to drift when sequentially trained on samples of a different nature—the well-known catastrophic forgetting issue. In particular, when a model consecutively learns from different visual domains, it tends to forget the past domains in favor of the most recent ones. In this context, we show that one way to learn models that are inherently more robust against forgetting is domain randomization—for vision tasks, randomizing the current domain’s distribution with heavy image manipulations. Building on this result, we devise a meta-learning strategy where a regularizer explicitly penalizes any loss associated with transferring the model from the current domain to different "auxiliary" meta-domains, while also easing adaptation to them. Such meta-domains are also generated through randomized image manipulations. We empirically demonstrate in a variety of experiments—spanning from classification to semantic segmentation—that our approach results in models that are less prone to catastrophic forgetting when transferred to new domains.
On Adaptive Sketch-and-Project for Solving Linear Systems

We generalize the concept of adaptive sampling rules to the sketch-and-project method for solving linear systems. Analyzing adaptive sampling rules in the sketch-and-project setting yields convergence results that apply to all special cases at once, including the Kaczmarz and coordinate descent. This eliminates the need to separately analyze analogous adaptive sampling rules in each special case. To deduce new sampling rules, we show how the progress of one step of the sketch-and-project method depends directly on a sketched residual. Based on this insight, we derive a (1) max-distance sampling rule, by sampling the sketch with the largest sketched residual, (2) a proportional sampling rule, by sampling proportional to the sketched residual, and finally (3) a capped sampling rule. The capped sampling rule is a generalization of the recently introduced adaptive sampling rules for the Kaczmarz method [Z.-Z. Bai and W.-T. Wu, SIAM J. Sci. Comput., 40 (2018), pp. A592--A606]. We provide a global exponential convergence theorem for each sampling rule and show that the max-distance sampling rule enjoys the fastest convergence. This finding is also verified in extensive numerical experiments that lead us to conclude that the max-distance sampling rule is superior both experimentally and theoretically to the capped sampling rule. We also provide numerical insights into implementing the adaptive strategies so that the per iteration cost is of the same order as using a fixed sampling strategy when the product of the number of sketches with the sketch size is not significantly larger than the number of columns.
NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results

This paper reviews the first challenge on high-dynamic range (HDR) imaging that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2021. This manuscript focuses on the newly introduced dataset, the proposed methods and their results. The challenge aims at estimating a HDR image from one or multiple respective low-dynamic range (LDR) observations, which might suffer from under-or over-exposed regions and different sources of noise. The challenge is composed by two tracks: In Track 1 only a single LDR image is provided as input, whereas in Track 2 three differently-exposed LDR images with inter-frame motion are available. In both tracks, the ultimate goal is to achieve the best objective HDR reconstruction in terms of PSNR with respect to a ground-truth image, evaluated both directly and with a canonical tonemapping operation.
DriveGAN: Towards a Controllable High-Quality Neural Simulation

Realistic simulators are critical for training and verifying robotics systems. While most of the contemporary simulators are hand-crafted, a scaleable way to build simulators is to use machine learning to learn how the environment behaves in response to an action, directly from data. In this work, we aim to learn to simulate a dynamic environment directly in pixel-space, by watching unannotated sequences of frames and their associated actions. We introduce a novel high-quality neural simulator referred to as DriveGAN that achieves controllability by disentangling different components without supervision. In addition to steering controls, it also includes controls for sampling features of a scene, such as the weather as well as the location of non-player objects. Since DriveGAN is a fully differentiable simulator, it further allows for re-simulation of a given video sequence, offering an agent to drive through a recorded scene again, possibly taking different actions. We train DriveGAN on multiple datasets, including 160 hours of real-world driving data. We showcase that our approach greatly surpasses the performance of previous data-driven simulators, and al-lows for new key features not explored before.
Sketch, Ground, and Refine: Top-Down Dense Video Captioning

The dense video captioning task aims to detect and describe a sequence of events in a video for detailed and coherent storytelling. Previous works mainly adopt a "detect-then-describe" framework, which firstly detects event proposals in the video and then generates descriptions for the detected events. However, the definitions of events are diverse which could be as simple as a single action or as complex as a set of events, depending on different semantic con-texts. Therefore, directly detecting events based on video information is ill-defined and hurts the coherency and accuracy of generated dense captions. In this work, we reverse the predominant "detect-then-describe" fashion, proposing a top-down way to first generate paragraphs from a global view and then ground each event description to a video segment for detailed refinement. It is formulated as a Sketch, Ground, and Refine process (SGR). The sketch stage first generates a coarse-grained multi-sentence paragraph to describe the whole video, where each sentence is treated as an event and gets localised in the grounding stage. In the re-fining stage, we improve captioning quality via refinement-enhanced training and dual-path cross attention on both coarse-grained event captions and aligned event segments. The updated event caption can further adjust its segment boundaries. Our SGR model outperforms state-of-the-art methods on ActivityNet Captioning benchmark under traditional and story-oriented dense caption evaluations. Code will be released at github.com/bearcatt/SGR.
Dual Mutual Robust Graph Convolutional Network for Weakly Supervised Node Classification in Social Networks of Internet of People

Social networks are a crucial component of the Internet of People (IoP), which represents cutting-edge of Internet of Things (IoT). Predicting a large number of unknown node labels with few known labels is one of the challenging problems in social network analysis. Fortunately, the graph convolutional network (GCN) and subsequent variants have achieved remarkable performance on semi-supervised node classification (SSNC). However, previous works only focus on the case of clean labels and rarely study the problem of SSNC under noisy labels (SSNCNL), which is a more challenging and practical problem in the realm of weakly supervised learning. To cope with the aforementioned challenge, we present a novel dual mutual robust GCN named DMRGCN with inspiration from deep mutual learning and robust learning in the domain of image recognition. Specifically, we first employ two GCNs with different learning abilities to construct network architecture. Then, we define a joint loss function which consists of a weighted combination of supervised loss, mutual loss, and robust loss. Finally, we train the network under the pseudo-siamese network paradigm. Experimental results on three social network benchmark datasets with different levels of noise on labels demonstrate that DMRGCN outperforms the vanilla GCN and several variants on classification accuracy. In particular, under the two conditions of labels without noise and with noise, the node classification accuracy obtained by our proposed DMRGCN can be 3.05% and 6.44% higher than that of the vanilla GCN, respectively.
Monocular Real-time Full Body Capture with Inter-part Correlations

We present the first method for real-time full body capture that estimates shape and motion of body and hands together with a dynamic 3D face model from a single color image. Our approach uses a new neural network architecture that exploits correlations between body and hands at high computational efficiency. Unlike previous works, our approach is jointly trained on multiple datasets focusing on hand, body or face separately, without requiring data where all the parts are annotated at the same time, which is much more difficult to create at sufficient variety. The possibility of such multi-dataset training enables superior generalization ability. In contrast to earlier monocular full body methods, our approach captures more expressive 3D face geometry and color by estimating the shape, expression, albedo and illumination parameters of a statistical face model. Our method achieves competitive accuracy on public benchmarks, while being significantly faster and providing more complete face reconstructions.
Real-time and light-weighted unsupervised video object segmentation network

Video object segmentation is one of the most practical computer vision tasks, especially in the unsupervised case, which has no manually labeled segmentation mask at the beginning of a video sequence. In this paper, we propose a new real-time unsupervised video object segmentation network. Based on the encoder-decoder framework, we present a Dynamic ASPP module and a RNN-Conv module. The former adds a dynamic selection mechanism into the Astrous Spatial Pyramid Pooling structure, and then the dilated convolutional kernels adaptively select appropriate features according to the scales by the channel attention mechanism. Compared with directly concatenating the dilated convolutional features, dynamically selecting feature maps reduces the amount of parameters and makes the module more efficient. The RNN-Conv module incorporates the RNN units with external convolutional blocks, aggregating the temporal features of a video sequence with the spatial information extracted by the convolutional network. We stack this module to extract deeper spatiotemporal features than the traditional RNN network. This module helps to avoid the gradient disappearance and explosion during network training. We test our network on the popular video object segmentation datasets. The experiment results demonstrate the effectiveness of our model.1
Protecting Intellectual Property of Generative Adversarial Networks from Ambiguity Attacks

Ever since Machine Learning as a Service emerges as a viable business that utilizes deep learning models to generate lucrative revenue, Intellectual Property Right (IPR) has become a major concern because these deep learning models can easily be replicated, shared, and re-distributed by any unauthorized third parties. To the best of our knowledge, one of the prominent deep learning models - Generative Adversarial Networks (GANs) which has been widely used to create photorealistic image are totally unprotected despite the existence of pioneering IPR protection methodology for Convolutional Neural Networks (CNNs). This paper therefore presents a complete protection framework in both black-box and white-box settings to enforce IPR protection on GANs. Empirically, we show that the proposed method does not compromise the original GANs performance (i.e. image generation, image super-resolution, style transfer), and at the same time, it is able to withstand both removal and ambiguity attacks against embedded watermarks. Codes are available at https://github.com/dingsheng-ong/ipr-gan.
Arabic Text Classification Using Convolutional Neural Network and Genetic Algorithms

Arabic documents are massively rising due to numerous contents utilized in websites, social media, and news articles. The classification of such documents in labelled categories is a significant and vital task that deserves more attention. Arabic Text Classification is an emerging research theme in Arabic Natural Language Processing. Recently, Deep Neural Network approaches have successfully been applied to many text classification problems, especially in English Text Classification. Convolutional Neural Network (CNN) is one of the best popular models. However, CNN is not highly applied in Arabic Text Classification. In addition, the recent studies did not achieve a high classification accuracy due to parameter setting issue. To overcome this limitation, a new hybrid classification model for Arabic Text is developed. This paper proposes Genetic Algorithms based Convolutional Neural Network for Arabic Text Classification. Genetic Algorithm is used to optimize the CNN parameters. The proposed model is tested using two large datasets and compared with the state-of-the art studies. The results showed that the classification accuracy achieved an improvement of 4 to 5%.
Self-Supervised Pillar Motion Learning for Autonomous Driving

Autonomous driving can benefit from motion behavior comprehension when interacting with diverse traffic participants in highly dynamic environments. Recently, there has been a growing interest in estimating class-agnostic motion directly from point clouds. Current motion estimation methods usually require vast amount of annotated training data from self-driving scenes. However, manually labeling point clouds is notoriously difficult, error-prone and time-consuming. In this paper, we seek to answer the research question of whether the abundant unlabeled data collections can be utilized for accurate and efficient motion learning. To this end, we propose a learning framework that leverages free supervisory signals from point clouds and paired camera images to estimate motion purely via self-supervision. Our model involves a point cloud based structural consistency augmented with probabilistic motion masking as well as a cross-sensor motion regularization to realize the desired self-supervision. Experiments reveal that our approach performs competitively to supervised methods, and achieves the state-of-the-art result when combining our self-supervised model with supervised fine-tuning.
Factors affecting institutional investors to add crypto-currency to asset portfolios

As crypto-currency has been gained great attention in financial market, it becomes more popular among institutional investment managers’ portfolios. Thus this research makes a pioneering study to investigate the role of crypto-currencies in PE company portfolios and factors affecting the institutional investors’ investment intention in order to enhance the reliability of their investment products. Data were obtained from 253 multinational PE company investment managers who have bought and used crypto-currency. The findings suggest not only important roles or drivers but also the moderating roles of innovativeness on relationships between the investment intention and its antecedents. It suggests that price volatility doesn’t lower institutional investors’ confidence as long as the market can offer timely and accurate price change information to meet investors’ price consciousness. The result also identifies that crypto-currency with high familiarity and trust can be well-suited in institutional investors’ portfolios.
Sentiment Analysis at Online Social Network for Cyber-Malicious Post Reviews Using Machine Learning Techniques

The creating centrality of sentiment assessment agrees with the advancement of the electronic stage, for instance, cyber-vulnerability reviews, gathering discussions for cyber-dangers studies, malicious movement-based overview web diaries, more modest scope sites, and casual associations related to the cyber-lawbreakers exercises study. The choices and approach, used to intrigue this present reality, are generally adjusted to how others see and survey the world about feeling and sentiment. For such an explanation, the ordinarily looking through method is utilized, when it is needed to settle down the decision, for ends, conduction, and evaluation of others. This is certifiable for individuals just as for affiliations and associations and society. This work is a broad inclination assessment which implies the task of ordinary language processing (NLP) to choose if a touch of substance contains some theoretical information and what enthusiastic information it imparts using cyber-malicious post overview, i.e. whether or not the way behind this substance is certain (+) or negative (−). Since radical advances the hoodlums cyber-occasions utilizing on the web interpersonal organization and security offices and validate user blocks it. Understanding the emotions behind the web user delivered substance and instructive file subsequently is of unbelievable help for business and individual use, among others. The task can be coordinated on different levels of substance taking care of, requesting the furthest point of words, sentences or entire educational assortments. Here, the methodology investigates a predominant system for cyber-weaknesses overviews subject to the AI approach.