Reviewerly/exHarmony huggingface dataset:
Multilevel Attention-Based Sample Correlations for Knowledge Distillation

Recently, model compression has been widely used for the deployment of cumbersome deep models on resource-limited edge devices in the performance-demanding industrial Internet of Things (IoT) scenarios. As a simple yet effective model compression technique, knowledge distillation (KD) aims to transfer the knowledge (e.g., sample relationships as the relational knowledge) from a large teacher model to a small student model. However, existing relational KD methods usually build sample correlations directly from the feature maps at a certain middle layer in deep neural networks, which tends to overfit the feature maps of the teacher model and fails to address the most important sample regions. Inspired by this, we argue that the characteristics of important regions are of great importance, and thus, introduce attention maps to construct sample correlations for knowledge distillation. Specifically, with attention maps from multiple middle layers, attention-based sample correlations are newly built upon the most informative sample regions, and can be used as an effective and novel relational knowledge for knowledge distillation. We refer to the proposed method as multilevel attention-based sample correlations for knowledge distillation (or MASCKD). We perform extensive experiments on popular KD datasets for image classification, image retrieval, and person reidentification, where the experimental results demonstrate the effectiveness of the proposed method for relational KD.
A Chaotic Image Encryption Scheme Based on Genetic Central Dogma and KMP Method

In this paper, an image cryptosystem based on genetic central dogma (GCD), Knuth–Morria–Pratt (KMP) algorithm and a chaotic system is developed. The KMP algorithm is firstly used to bind DNA strings to obtain the next array, which participates in the design of the chaotic initial condition, and then the secure chaotic sequences are produced by employing the sliding idea in pattern string matching. In the present procedure, a DNA-level two-way pixel’s shuffle is achieved by a shared stack push operation and it is adopted in the permutation module for the purpose of accelerating the overall pixel’s shuffle. Subsequently, the pixel values are substituted by simulating the process of protein synthesis in GCD, in which the DNA replication and RNA replication form the basis of the DNA-triploid mutation and new RNA mutation rules, respectively. Experimental simulations and extensive cryptanalysis fully vindicate that superior security effects in addition to satisfactory low time complexity can be simultaneously obtained by the proposed confusion-substitution scheme.
A Survey on ensemble learning under the era of deep learning

Due to the dominant position of deep learning (mostly deep neural networks) in various artificial intelligence applications, recently, ensemble learning based on deep neural networks (ensemble deep learning) has shown significant performances in improving the generalization of learning system. However, since modern deep neural networks usually have millions to billions of parameters, the time and space overheads for training multiple base deep learners and testing with the ensemble deep learner are far greater than that of traditional ensemble learning. Though several algorithms of fast ensemble deep learning have been proposed to promote the deployment of ensemble deep learning in some applications, further advances still need to be made for many applications in specific fields, where the developing time and computing resources are usually restricted or the data to be processed is of large dimensionality. An urgent problem needs to be solved is how to take the significant advantages of ensemble deep learning while reduce the required expenses so that many more applications in specific fields can benefit from it. For the alleviation of this problem, it is essential to know about how ensemble learning has developed under the era of deep learning. Thus, in this article, we present fundamental discussions focusing on data analyses of published works, methodologies, recent advances and unattainability of traditional ensemble learning and ensemble deep learning. We hope this article will be helpful to realize the intrinsic problems and technical challenges faced by future developments of ensemble learning under the era of deep learning.
How Elon Musk's Twitter activity moves cryptocurrency markets

Elon Musk, one of the richest individuals in the world, is considered a technological visionary and has a social network of over 110 million followers on social media platform Twitter. He regularly uses his social media presence to communicate on various topics, one of which is cryptocurrency, such as Bitcoin or Dogecoin. Using an event study approach, we analyze to what extent Musk's Twitter activity affects short-term cryptocurrency returns and volume. In other words, we investigate whether cryptocurrency markets exhibit a “Musk Effect”. Based on a sample of 47 cryptocurrency-related Twitter events, we identify significant positive abnormal returns and trading volume following such events. However, we discover that on average, price effects are only significant for Dogecoin-related Tweets but not for Bitcoin. This is because regarding the latter, the significant price effects of positive and negative news cancel each other out. Considered in isolation, non-negative tweets from Musk lead to significantly positive abnormal Bitcoin returns. Individual tweets do raise the price of Bitcoin by 16.9 % or reduce it by almost 11.8 %. Our study shows the significant impact that the social media activity of influential individuals can have on cryptocurrencies. This suggests a conflict between morals, risks of market manipulation and investor protection.
When makers meet the metaverse: Effects of creating NFT metaverse exhibition in maker education

The main purpose of this paper is to conduct empirical research on the effectiveness of application of a metaverse in maker education. While the control group of students participate in a general maker class that encourages making digital content for relevant contest participation, the experimental group was treated to register their digital artwork as an NFT certified by blockchain technology and display them in the metaverse as a form of a virtual exhibition. The study findings show that while there was a significant statistical difference in terms of creative problem-solving and creative curiosity for both groups, only experimental participants showed improvement in creative cognition. In addition, following the addition of the metaverse treatment, a significant statistical improvement was found in terms of a sense of achievement towards their making process as well as ownership of their making products. Furthermore, the sentiment analysis collected from students' reflection papers reveals that the compounding result of their opinions, appraisals, complaints, emotions, or attitudes towards the metaverse exhibition was generally positive. Based on the findings, this paper suggests a new maker education model by expanding the previous one, TMSI (Thinkering, Making, Sharing, Improving), to a concept of TMIOS (Thinkering, Making, Improving, Ownering, Sharing).
A new metaheuristic algorithm: Shrimp and Goby association search algorithm and its application for damage identification in large-scale and complex structures

This paper presents the application of a new Shrimp and Goby Association Search algorithm (SGA) to solve large-scale global optimization problems. The performance of SGA is assessed using 13 benchmark high-dimensional functions, 10 classical benchmark functions, and several real-world engineering applications. For the first time, an efficient optimization approach for structural health monitoring (SHM) in truss-like structures is presented. The proposed approach is applied for damage identification of complex structures. A real structure, namely Canton Tower in Guangzhou, China, is served as an example for damage detection. Interestingly, this tower was the tallest structure in the world in 2009 with a height of 610 m. The great merit of this example is that it provides a real complex structure with a high-dimensional problem to assess the performance of SGA in the real world. The results show that SGA can deal with this problem effectively, at the same time, it operates better to escape from local optima with faster convergence rate than population-based algorithms.
Collaborative adoption of blockchain technology: A supply chain contract perspective

The outbreak of COVID-19 has significantly affected the development of enterprises. In the post-pandemic era, blockchain technology has become one of the important technologies to help enterprises quickly gain market competitiveness. The heavy investment required of supply chain stakeholders to employ blockchain technology has hindered its adoption and application. To tackle this issue, this study aims to facilitate the adoption of blockchain technology in a supply chain consisting of a core enterprise and a small/medium-sized enterprise through an effective supply chain contract. We analyze the performance of a cost-sharing (CS) contract and a revenue-sharing (RS) contract and propose a new hybrid CS-RS contract for better performance. We conduct comparative analyses of the three contracts and find that the hybrid CS-RS contract can more effectively incentivize both parties to reach the highest level of blockchain technology adoption and achieve supply chain coordination.
Contrastive Test-Time Adaptation

Test-time adaptation is a special setting of unsupervised domain adaptation where a trained model on the source domain has to adapt to the target domain without accessing source data. We propose a novel way to leverage self-supervised contrastive learning to facilitate target feature learning, along with an online pseudo labeling scheme with refinement that significantly denoises pseudo labels. The contrastive learning task is applied jointly with pseudo labeling, contrasting positive and negative pairs constructed similarly as MoCo but with source-initialized encoder, and excluding same-class negative pairs indicated by pseudo labels. Meanwhile, we produce pseudo labels online and refine them via soft voting among their nearest neighbors in the target feature space, enabled by maintaining a memory queue. Our method, AdaContrast, achieves state-of-the-art performance on major benchmarks while having several desirable properties compared to existing works, including memory efficiency, insensitivity to hyper-parameters, and better model calibration. Code is released at https://github.com/DianCh/AdaContrast.
Few Could Be Better Than All: Feature Sampling and Grouping for Scene Text Detection

Recently, transformer-based methods have achieved promising progresses in object detection, as they can eliminate the post-processes like NMS and enrich the deep representations. However, these methods cannot well cope with scene text due to its extreme variance of scales and aspect ratios. In this paper, we present a simple yet effective transformer-based architecture for scene text detection. Different from previous approaches that learn robust deep representations of scene text in a holistic manner, our method performs scene text detection based on a few representative features, which avoids the disturbance by background and reduces the computational cost. Specifically, we first select a few representative features at all scales that are highly relevant to foreground text. Then, we adopt a transformer for modeling the relationship of the sampled features, which effectively divides them into reasonable groups. As each feature group corresponds to a text instance, its bounding box can be easily obtained without any post-processing operation. Using the basic feature pyramid network for feature extraction, our method consistently achieves state-of-the-art results on several popular datasets for scene text detection.
I M Avatar: Implicit Morphable Head Avatars from Videos

Traditional 3D morphable face models (3DMMs) provide fine-grained control over expression but cannot easily capture geometric and appearance details. Neural volumetric representations approach photorealism but are hard to animate and do not generalize well to unseen expressions. To tackle this problem, we propose IMavatar (Implicit Morphable avatar), a novel method for learning implicit head avatars from monocular videos. Inspired by the fine-grained control mechanisms afforded by conventional 3DMMs, we represent the expression- and pose-related deformations via learned blendshapes and skinning fields. These attributes are pose-independent and can be used to morph the canonical geometry and texture fields given novel expression and pose parameters. We employ ray marching and iterative root-finding to locate the canonical surface intersection for each pixel. A key contribution is our novel analytical gradient formulation that enables end-to-end training of IMavatars from videos. We show quantitatively and qualitatively that our method improves geometry and covers a more complete expression space compared to state-of-the-art methods. Code and data can be found at https://ait.ethz.ch/projects/2022/IMavatar/.
Generating Diverse and Natural 3D Human Motions from Text

Automated generation of 3D human motions from text is a challenging problem. The generated motions are expected to be sufficiently diverse to explore the text-grounded motion space, and more importantly, accurately depicting the content in prescribed text descriptions. Here we tackle this problem with a two-stage approach: text2length sampling and text2motion generation. Text2length involves sampling from the learned distribution function of motion lengths conditioned on the input text. This is followed by our text2motion module using temporal variational autoen-coder to synthesize a diverse set of human motions of the sampled lengths. Instead of directly engaging with pose sequences, we propose motion snippet code as our internal motion representation, which captures local semantic motion contexts and is empirically shown to facilitate the generation of plausible motions faithful to the input text. Moreover, a large-scale dataset of scripted 3D Human motions, HumanML3D, is constructed, consisting of 14,616 motion clips and 44,970 text descriptions.
C<sup>2</sup> AM: Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localization and Semantic Segmentation

While class activation map (CAM) generated by image classification network has been widely used for weakly su-pervised object localization (WSOL) and semantic segmentation (WSSS), such classifiers usually focus on discriminative object regions. In this paper, we propose Contrastive learning for Class-agnostic Activation Map (C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AM) generation only using unlabeled image data, without the involvement of image-level supervision. The core idea comes from the observation that i) semantic information of fore-ground objects usually differs from their backgrounds; ii) foreground objects with similar appearance or background with similar color/texture have similar representations in the feature space. We form the positive and negative pairs based on the above relations and force the network to disentangle foreground and background with a class-agnostic activation map using a novel contrastive loss. As the network is guided to discriminate cross-image foreground-background, the class-agnostic activation maps learned by our approach generate more complete object regions. We successfully extracted from C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AM class-agnostic object bounding boxes for object localization and background cues to refine CAM generated by classification network for semantic segmentation. Extensive experiments on CUB-200-2011, ImageNet-1K, and PASCAL VOC2012 datasets show that both WSOL and WSSS can benefit from the proposed C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AM. Code will be available at https://github.com/CVI-SZUICCAM.
Implicit Sample Extension for Unsupervised Person Re-Identification

Most existing unsupervised person re-identification (Re-ID) methods use clustering to generate pseudo labels for model training. Unfortunately, clustering sometimes mixes different true identities together or splits the same identity into two or more sub clusters. Training on these noisy clusters substantially hampers the Re-ID accuracy. Due to the limited samples in each identity, we suppose there may lack some underlying information to well reveal the accurate clusters. To discover these information, we propose an Implicit Sample Extension (ISE) method to generate what we call support samples around the cluster boundaries. Specifically, we generate support samples from actual samples and their neighbouring clusters in the embedding space through a progressive linear interpolation (PLI) strategy. PLI controls the generation with two critical factors, i.e., 1) the direction from the actual sample towards its K-nearest clusters and 2) the degree for mixing up the context information from the K-nearest clusters. Meanwhile, given the support samples, ISE further uses a label-preserving loss to pull them towards their corresponding actual samples, so as to compact each cluster. Consequently, ISE reduces the “sub and mixed” clustering errors, thus improving the Re-ID performance. Extensive experiments demonstrate that the proposed method is effective and achieves state-of-the-art performance for unsupervised person Re-ID. Code is available at: https://github.com/PaddlePaddle/PaddleClas.
TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization

The dominant CNN-based methods for cross-view image geo-localization rely on polar transform and fail to model global correlation. We propose a pure transformer-based approach (TransGeo) to address these limitations from a different perspective. TransGeo takes full advantage of the strengths of transformer related to global information modeling and explicit position information encoding. We further leverage the flexibility of transformer input and propose an attention-guided non-uniform cropping method, so that uninformative image patches are removed with negligible drop on performance to reduce computation cost. The saved computation can be reallocated to increase resolution only for informative patches, resulting in performance improvement with no additional computation cost. This “attend and zoom-in” strategy is highly similar to human behavior when observing images. Remarkably, TransGeo achieves state-of-the-art results on both urban and rural datasets, with significantly less computation cost than CNN-based methods. It does not rely on polar transform and infers faster than CNN-based methods. Code is available at https://github.com/Jeff-Zilence/TransGeo2022.
Sentiment Analysis on Twitter Data

With the appearance of online innovation and its development, the Web presently contains an enormous measure of information for Web clients, just as a lot of information being produced. The Web has advanced into a stage for Internet learning, thought trade, and assessment sharing. Individuals utilize long-range informal communication locales, like Twitter, Facebook, and Google+ to impart and communicate their insights on different points, take an interest in conversations with assorted networks, and send messages everywhere on the world. In the subject of opinion analysis of Twitter information, a great deal of work has been finished. This examination centers on conclusion analysis of Twitter information, which is valuable for breaking down data in tweets where assessments are communicated. We use a variety of AI techniques, such as Naive Bayes and support vector machines, to investigate Twitter data streams.
Scaling Up Vision-Language Pretraining for Image Captioning

In recent years, we have witnessed significant performance boost in the image captioning task based on vision-language pre-training (VLP). Scale is believed to be an important factor for this advance. However, most existing work only focuses on pre-training transformers with moderate sizes (e.g., 12 or 24 layers) on roughly 4 million images. In this paper, we present LEMON O, a LargE-scale iMage captiONer, and provide the first empirical study on the scaling behavior of VLP for image captioning. We use the state-of-the-art Vin VL model as our reference model, which consists of an image feature extractor and a transformer model, and scale the transformer both up and down, with model sizes ranging from 13 to 675 million parameters. In terms of data, we conduct experiments with up to 200 million imagetext pairs which are automatically collected from web based on the alt attribute of the image (dubbed as ALT200M <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> The dataset is released at https://github.com/xiaoweihu/ALT200M). Extensive analysis helps to characterize the performance trend as the model size and the pre-training data size increase. We also compare different training recipes, especially for training on large-scale noisy data. As a result, LEMON achieves new state of the arts on several major image captioning benchmarks, including COCO Caption, nocaps, and Conceptual Captions. We also show LEMON can generate captions with long-tail vi-sual concepts when used in a zero-shot manner.
End-to-End Reconstruction-Classification Learning for Face Forgery Detection

Existing face forgery detectors mainly focus on specific forgery patterns like noise characteristics, local textures, or frequency statistics for forgery detection. This causes specialization of learned representations to known forgery patterns presented in the training set, and makes it difficult to detect forgeries with unknown patterns. In this paper, from a new perspective, we propose a forgery detection frame-work emphasizing the common compact representations of genuine faces based on reconstruction-classification learning. Reconstruction learning over real images enhances the learned representations to be aware of forgery patterns that are even unknown, while classification learning takes the charge of mining the essential discrepancy between real and fake images, facilitating the understanding of forgeries. To achieve better representations, instead of only using the encoder in reconstruction learning, we build bipartite graphs over the encoder and decoder features in a multi-scale fashion. We further exploit the reconstruction difference as guidance of forgery traces on the graph output as the final representation, which is fed into the classifier for forgery detection. The reconstruction and classification learning is optimized end-to-end. Extensive experiments on large-scale benchmark datasets demonstrate the superiority of the proposed method over state of the arts.