Reviewerly/exHarmony huggingface dataset:
Pseudo-Supervised Deep Subspace Clustering

Auto-Encoder (AE)-based deep subspace clustering (DSC) methods have achieved impressive performance due to the powerful representation extracted using deep neural networks while prioritizing categorical separability. However, self-reconstruction loss of an AE ignores rich useful relation information and might lead to indiscriminative representation, which inevitably degrades the clustering performance. It is also challenging to learn high-level similarity without feeding semantic labels. Another unsolved problem facing DSC is the huge memory cost due to n×n similarity matrix, which is incurred by the self-expression layer between an encoder and decoder. To tackle these problems, we use pairwise similarity to weigh the reconstruction loss to capture local structure information, while a similarity is learned by the self-expression layer. Pseudo-graphs and pseudo-labels, which allow benefiting from uncertain knowledge acquired during network training, are further employed to supervise similarity learning. Joint learning and iterative training facilitate to obtain an overall optimal solution. Extensive experiments on benchmark datasets demonstrate the superiority of our approach. By combining with the k-nearest neighbors algorithm, we further show that our method can address the large-scale and out-of-sample problems. The source code of our method is available: https://github.com/sckangz/SelfsupervisedSC.
Detecting Compressed Deepfake Videos in Social Networks Using Frame-Temporality Two-Stream Convolutional Network

The development of technologies that can generate Deepfake videos is expanding rapidly. These videos are easily synthesized without leaving obvious traces of manipulation. Though forensically detection in high-definition video datasets has achieved remarkable results, the forensics of compressed videos is worth further exploring. In fact, compressed videos are common in social networks, such as videos from Instagram, Wechat, and Tiktok. Therefore, how to identify compressed Deepfake videos becomes a fundamental issue. In this paper, we propose a two-stream method by analyzing the frame-level and temporality-level of compressed Deepfake videos. Since the video compression brings lots of redundant information to frames, the proposed frame-level stream gradually prunes the network to prevent the model from fitting the compression noise. Aiming at the problem that the temporal consistency in Deepfake videos might be ignored, we apply a temporality-level stream to extract temporal correlation features. When combined with scores from the two streams, our proposed method performs better than the state-of-the-art methods in compressed Deepfake videos detection.
An efficient deep Convolutional Neural Network based detection and classification of Acute Lymphoblastic Leukemia

Automated and accurate diagnosis of Acute Lymphoblastic Leukemia (ALL), blood cancer, is a challenging task. Nowadays, Convolutional Neural Networks (CNNs) have become a preferred approach for medical image analysis. However, for achieving excellent performance, classical CNNs usually require huge databases for proper training. This paper proposes an efficient deep CNNs framework to mitigate this issue and yield more accurate ALL detection. The salient features: depthwise separable convolutions, linear bottleneck architecture, inverted residual, and skip connections make it a faster and preferred approach. In this proposed method, a novel probability-based weight factor is suggested, which has a significant role in efficiently hybridizing MobilenetV2 and ResNet18 with preserving the benefits of both approaches. Its performance is validated using public benchmark datasets: ALLIDB1 and ALLIDB2. The experimental results display that the proposed approach yields the best accuracy (with 70% training and 30% testing) 99.39% and 97.18% in ALLIDB1 and ALLIDB2 datasets, respectively. Similarly, it also achieves the best accuracy (with 50% training and 50% testing) 97.92% and 96.00% in ALLIDB1 and ALLIDB2 datasets, respectively. Moreover, it also achieves the best performance compared to the recent transfer learning-based techniques in both the datasets, in terms of sensitivity, specificity, accuracy, precision, F1 score, and receiver operating characteristic (ROC) in most of the cases.
Neural Prototype Trees for Interpretable Fine-grained Image Recognition

Prototype-based methods use interpretable representations to address the black-box nature of deep learning models, in contrast to post-hoc explanation methods that only approximate such models. We propose the Neural Prototype Tree (ProtoTree), an intrinsically interpretable deep learning method for fine-grained image recognition. ProtoTree combines prototype learning with decision trees, and thus results in a globally interpretable model by design. Additionally, ProtoTree can locally explain a single prediction by outlining a decision path through the tree. Each node in our binary tree contains a trainable prototypical part. The presence or absence of this learned prototype in an image determines the routing through a node. Decision making is therefore similar to human reasoning: Does the bird have a red throat? And an elongated beak? Then it’s a hummingbird! We tune the accuracy-interpretability trade-off using ensemble methods, pruning and binarizing. We apply pruning without sacrificing accuracy, resulting in a small tree with only 8 learned prototypes along a path to classify a bird from 200 species. An ensemble of 5 ProtoTrees achieves competitive accuracy on the CUB-200-2011 and Stanford Cars data sets. Code is available at github.com/M-Nauta/ProtoTree.
Sentiment analysis and classification of Indian farmers’ protest using twitter data

Protests are an integral part of democracy and an important source for citizens to convey their demands and/or dissatisfaction to the government. As citizens become more aware of their rights, there has been an increasing number of protests all over the world for various reasons. With the advancement of technology, there has also been an exponential rise in the use of social media to exchange information and ideas. In this research, we gathered data from the microblogging website Twitter concerning farmers’ protest to understand the sentiments that the public shared on an international level. We used models to categorize and analyze the sentiments based on a collection of around 20,000 tweets on the protest. We conducted our analysis using Bag of Words and TF-IDF and discovered that Bag of Words performed better than TF-IDF. In addition, we also used Naive Bayes, Decision Trees, Random Forests, and Support Vector Machines and also discovered that Random Forest had the highest classification accuracy.
A software engineering perspective on engineering machine learning systems: State of the art and challenges

Advancements in machine learning (ML) lead to a shift from the traditional view of software development, where algorithms are hard-coded by humans, to ML systems materialized through learning from data. Therefore, we need to revisit our ways of developing software systems and consider the particularities required by these new types of systems. The purpose of this study is to systematically identify, analyze, summarize, and synthesize the current state of software engineering (SE) research for engineering ML systems. I performed a systematic literature review (SLR). I systematically selected a pool of 141 studies from SE venues and then conducted a quantitative and qualitative analysis using the data extracted from these studies. The non-deterministic nature of ML systems complicates all SE aspects of engineering ML systems. Despite increasing interest from 2018 onwards, the results reveal that none of the SE aspects have a mature set of tools and techniques. Testing is by far the most popular area among researchers. Even for testing ML systems, engineers have only some tool prototypes and solution proposals with weak experimental proof. Many of the challenges of ML systems engineering were identified through surveys and interviews. Researchers should conduct experiments and case studies, ideally in industrial environments, to further understand these challenges and propose solutions. The results may benefit (1) practitioners in foreseeing the challenges of ML systems engineering; (2) researchers and academicians in identifying potential research questions; and (3) educators in designing or updating SE courses to cover ML systems engineering.
Counterfactual Zero-Shot and Open-Set Visual Recognition

We present a novel counterfactual framework for both Zero-Shot Learning (ZSL) and Open-Set Recognition (OSR), whose common challenge is generalizing to the unseen-classes by only training on the seen-classes. Our idea stems from the observation that the generated samples for unseen-classes are often out of the true distribution, which causes severe recognition rate imbalance between the seen-class (high) and unseen-class (low). We show that the key reason is that the generation is not Counterfactual Faithful, and thus we propose a faithful one, whose generation is from the sample-specific counterfactual question: What would the sample look like, if we set its class attribute to a certain class, while keeping its sample attribute unchanged? Thanks to the faithfulness, we can apply the Consistency Rule to perform unseen/seen binary classification, by asking: Would its counterfactual still look like itself? If "yes", the sample is from a certain class, and "no" otherwise. Through extensive experiments on ZSL and OSR, we demonstrate that our framework effectively mitigates the seen/unseen imbalance and hence significantly improves the overall performance. Note that this framework is orthogonal to existing methods, thus, it can serve as a new baseline to evaluate how ZSL/OSR models generalize. Codes are available at https://github.com/yue-zhongqi/gcm-cf.
Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges

Abstract Most machine learning algorithms are configured by a set of hyperparameters whose values must be carefully chosen and which often considerably impact performance. To avoid a time‐consuming and irreproducible manual process of trial‐and‐error to find well‐performing hyperparameter configurations, various automatic hyperparameter optimization (HPO) methods—for example, based on resampling error estimation for supervised machine learning—can be employed. After introducing HPO from a general perspective, this paper reviews important HPO methods, from simple techniques such as grid or random search to more advanced methods like evolution strategies, Bayesian optimization, Hyperband, and racing. This work gives practical recommendations regarding important choices to be made when conducting HPO, including the HPO algorithms themselves, performance evaluation, how to combine HPO with machine learning pipelines, runtime improvements, and parallelization. This article is categorized under: Algorithmic Development &gt; Statistics Technologies &gt; Machine Learning Technologies &gt; Prediction
Skeleton Aware Multi-modal Sign Language Recognition

Sign language is commonly used by deaf or speech impaired people to communicate but requires significant effort to master. Sign Language Recognition (SLR) aims to bridge the gap between sign language users and others by recognizing signs from given videos. It is an essential yet challenging task since sign language is performed with the fast and complex movement of hand gestures, body posture, and even facial expressions. Recently, skeleton-based action recognition attracts increasing attention due to the independence between the subject and background variation. However, skeleton-based SLR is still under exploration due to the lack of annotations on hand keypoints. Some efforts have been made to use hand detectors with pose estimators to extract hand key points and learn to recognize sign language via Neural Networks, but none of them outperforms RGB-based methods. To this end, we propose a novel Skeleton Aware Multi-modal SLR framework (SAM-SLR) to take advantage of multi-modal information towards a higher recognition rate. Specifically, we propose a Sign Language Graph Convolution Network (SL-GCN) to model the embedded dynamics and a novel Separable Spatial-Temporal Convolution Network (SSTCN) to exploit skeleton features. RGB and depth modalities are also incorporated and assembled into our framework to provide global information that is complementary to the skeleton-based methods SL-GCN and SSTCN. As a result, SAM-SLR achieves the highest performance in both RGB (98.42%) and RGB-D (98.53%) tracks in 2021 Looking at People Large Scale Signer Independent Isolated SLR Challenge. Our code is available at https://github.com/jackyjsy/CVPR21Chal-SLR
Instance Localization for Self-supervised Detection Pretraining

Prior research on self-supervised learning has led to considerable progress on image classification, but often with degraded transfer performance on object detection. The objective of this paper is to advance self-supervised pretrained models specifically for object detection. Based on the inherent difference between classification and detection, we propose a new self-supervised pretext task, called instance localization. Image instances are pasted at various locations and scales onto background images. The pretext task is to predict the instance category given the composited images as well as the foreground bounding boxes. We show that integration of bounding boxes into pretraining promotes better task alignment and architecture alignment for transfer learning. In addition, we propose an augmentation method on the bounding boxes to further enhance the feature alignment. As a result, our model becomes weaker at Imagenet semantic classification but stronger at image patch localization, with an overall stronger pretrained model for object detection. Experimental results demonstrate that our approach yields state-of-the-art transfer learning results for object detection on PASCAL VOC and MSCOCO <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Transformer For Image Quality Assessment

Transformer has become the new standard method in natural language processing (NLP), and it also attracts research interests in computer vision area. In this paper we investigate the application of Transformer in Image Quality (TRIQ) assessment. Following the original Transformer encoder employed in Vision Transformer (ViT), we propose an architecture of using a shallow Transformer encoder on the top of a feature map extracted by convolution neural networks (CNN). Adaptive positional embedding is employed in the Transformer encoder to handle images with arbitrary resolutions. Different settings of Transformer architectures have been investigated on publicly available image quality databases. We have found that the proposed TRIQ architecture achieves outstanding performance. The implementation of TRIQ is published on Github (https://github.com/junyongyou/triq).
Automatic Polyp Segmentation via Multi-scale Subtraction Network

More than 90% of colorectal cancer is gradually transformed from colorectal polyps. In clinical practice, precise polyp segmentation provides important information in the early detection of colorectal cancer. Therefore, automatic polyp segmentation techniques are of great importance for both patients and doctors. Most existing methods are based on U-shape structure and use element-wise addition or concatenation to fuse different level features progressively in decoder. However, both the two operations easily generate plenty of redundant information, which will weaken the complementarity between different level features, resulting in inaccurate localization and blurred edges of polyps. To address this challenge, we propose a multi-scale subtraction network (MSNet) to segment polyp from colonoscopy image. Specifically, we first design a subtraction unit (SU) to produce the difference features between adjacent levels in encoder. Then, we pyramidally equip the SUs at different levels with varying receptive fields, thereby obtaining rich multi-scale difference information. In addition, we build a training-free network “LossNet” to comprehensively supervise the polyp-aware features from bottom layer to top layer, which drives the MSNet to capture the detailed and structural cues simultaneously. Extensive experiments on five benchmark datasets demonstrate that our MSNet performs favorably against most state-of-the-art methods under different evaluation metrics. Furthermore, MSNet runs at a real-time speed of \(\sim \)70fps when processing a \(352 \times 352\) image. The source code will be publicly available at https://github.com/Xiaoqi-Zhao-DLUT/MSNet.
Blockchain and the ‘Internet of Things' for the construction industry: research trends and opportunities

Various applications integrating Blockchain with the Internet of Things (IoT) have emerged in recent years. While industries such as automotive have embraced this integration, application in other areas such as construction remains limited. The scientometric analysis is applied to 648 papers, identifying the use of IoT and Blockchain in engineering, as well as evaluating the progress of research in the construction industry. The qualitative critical review is applied to 88 papers and analyses successful IoT and Blockchain application cases in construction while also highlighting challenges and limitations. Blockchain of Things (BCoT) as a new concept is introduced to exploit the advantages of IoT and Blockchain, and this paper presents potential uses of BCoT in the construction industry. This paper provides researchers with a comprehensive view of related literature and research gaps that offer opportunities for future research.
A Blockchain-Based Auditable Access Control System for Private Data in Service-Centric IoT Environments

Internet of Things (IoT) devices are widely considered in smart cities, intelligent medicine, and intelligent transportation, among other fields that facilitate people's lives, producing a large amount of private data. However, due to the mobility, limited performance, and distributed deployment of IoT, traditional access control methods cannot support the security of private data's access control process in current IoT environments. To address such problems, this article proposes an auditable access control model, based on an attribute-based access control model, and manages the access control policy for private data through the request record, the response record, and the access record stored in the blockchain network. Additionally, a Blockchain-based auditable access control system is also proposed based on the auditable access control model, ensuring private data security in IoT environments and realizing effective management and auditable access to these data. Experimental results show that the proposed system can maintain high throughput while ensuring private data security for real application scenarios in IoT environments.
A critical review of state‐of‐the‐art chatbot designs and applications

Abstract Chatbots are intelligent conversational agents that can interact with users through natural languages. As chatbots can perform a variety of tasks, many companies have committed numerous resources to develop and deploy chatbots to enhance various business processes. However, we lack an up‐to‐date critical review that thoroughly examines both state‐of‐the‐art technologies and innovative applications of chatbots. In this review, we not only critically analyze the various computational approaches used to develop state‐of‐the‐art chatbots, but also thoroughly review the usability and applications of chatbots for various business sectors. We also identify gaps in chatbot‐related studies and propose new research directions to address the shortcomings of existing studies and applications. Our review advances both academic research and practical business applications of state‐of‐the‐art chatbots. We provide guidance for practitioners to fully realize the business value of chatbots and assist in making sensible decisions related to the development and deployment of chatbots in various business contexts. Researchers interested in the design and development of chatbots can also gain useful insights from our critical review and identify fruitful research topics and future research directions based on the research gaps discussed herein. This article is categorized under: Technologies &gt; Machine Learning Application Areas &gt; Business and Industry
Self-Supervised Graph Co-Training for Session-based Recommendation

Session-based recommendation targets next-item prediction by exploiting user behaviors within a short time period. Compared with other recommendation paradigms, session-based recommendation suffers more from the problem of data sparsity due to the very limited short-term interactions. Self-supervised learning, which can discover ground-truth samples from the raw data, holds vast potentials to tackle this problem. However, existing self-supervised recommendation models mainly rely on item/segment dropout to augment data, which are not fit for session-based recommendation because the dropout leads to sparser data, creating unserviceable self-supervision signals. In this paper, for informative session-based data augmentation, we combine self-supervised learning with co-training, and then develop a framework to enhance session-based recommendation. Technically, we first exploit the session-based graph to augment two views that exhibit the internal and external connectivities of sessions, and then we build two distinct graph encoders over the two views, which recursively leverage the different connectivity information to generate ground-truth samples to supervise each other by contrastive learning. In contrast to the dropout strategy, the proposed self-supervised graph co-training preserves the complete session information and fulfills genuine data augmentation. Extensive experiments on multiple benchmark datasets show that, session-based recommendation can be remarkably enhanced under the regime of self-supervised graph co-training, achieving the state-of-the-art performance.
Distributed load forecasting using smart meter data: Federated learning with Recurrent Neural Networks

Load forecasting is essential for energy management, infrastructure planning, grid operation, and budgeting. Large scale smart meter deployments have resulted in ability to collect massive energy data and have created opportunities in sensor-based forecasting. Machine learning (ML) has demonstrated great successes in sensor-based load forecasting; however, when prediction is needed on a smart meter level, typically a single model is trained for each smart meter. With a large number of meters, this becomes computationally expensive or even infeasible. On the other hand, with conventional ML, training a single model for several smart meters requires participants to share their data with the central server. Consequently, this paper proposes federated learning for load forecasting with smart meter data: this strategy enables training a single model with all participating smart meters without the need to share local data. Two alternative federated learning strategies are examined: FedSGD, which performs one step of gradient descent on client before merging updates on the server, and FedAVG, which carries out several steps before the merging. Specifically, residential consumers are diverse what makes training a single model challenging as load profiles vary across consumers. The results show that the FedAVG achieves better accuracy than FedSGD while also requiring fewer communication rounds. Comparing to individual models for each meter and a single central models for all meters, FedAVG achieves comparable or better accuracy.