Reviewerly/exHarmony huggingface dataset:
MT-DETR: Robust End-to-end Multimodal Detection with Confidence Fusion

Due to the trending need for autonomous driving, camera-based object detection has recently attracted lots of attention and successful development. However, there are times when unexpected and severe weather occurs in outdoor environments, making the detection tasks less effective and unexpected. In this case, additional sensors like lidar and radar are adopted to help the camera work in bad weather. However, existing multimodal detection methods do not consider the characteristics of different vehicle sensors to complement each other. Therefore, a novel end-to-end multimodal multistage object detection network called MT-DETR is proposed. Unlike the unimodal object detection networks, MT-DETR adds fusion modules and enhancement modules and adopts a hierarchical fusion mechanism. The Residual Fusion Module (RFM) and Confidence Fusion Module (CFM) are designed to fuse camera, lidar, radar, and time features. The Residual Enhancement Module (REM) reinforces each unimodal branch while a multistage loss is introduced to strengthen each branch’s effectiveness. The synthesis algorithm for generating camera-lidar data pairs in foggy conditions further boosts the performance in unseen adverse weather. Extensive experiments on various weather conditions of the STF dataset demonstrate that MT-DETR outperforms state-of-the-art methods. The generality of MT-DETR has also been confirmed by replacing the feature extractor in the experiments. The code and pre-trained models are available on https://github.com/Chushihyun/MT-DETR.
Anomaly Clustering: Grouping Images into Coherent Clusters of Anomaly Types

We study anomaly clustering, grouping data into coherent clusters of anomaly types. This is different from anomaly detection that aims to divide anomalies from normal data. Unlike object-centered image clustering, anomaly clustering is particularly challenging as anomalous patterns are subtle and local. We present a simple yet effective clustering framework using a patch-based pretrained deep embeddings and off-the-shelf clustering methods. We define a distance function between images, each of which is represented as a bag of embeddings, by the Euclidean distance between weighted averaged embeddings. The weight defines the importance of instances (i.e., patch embeddings) in the bag, which may highlight defective regions. We compute weights in an unsupervised way or in a semi-supervised way when labeled normal data is available. Extensive experimental studies show the effectiveness of the proposed clustering framework along with a novel distance function upon existing multiple instance or deep clustering frameworks. Overall, our framework achieves 0.451 and 0.674 normalized mutual information scores on MVTec object and texture categories and further improve with a few labeled normal data (0.577, 0.669), far exceeding the baselines (0.244, 0.273) or state-of-the-art deep clustering methods (0.176, 0.277).
Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networks

Knowledge distillation which learns a lightweight student model by distilling knowledge from a cumbersome teacher model is an attractive approach for learning compact deep neural networks (DNNs). Recent works further improve student network performance by leveraging multiple teacher networks. However, most of the existing knowledge distillation-based multi-teacher methods use separately pretrained teachers. This limits the collaborative learning between teachers and the mutual learning between teachers and student. Network quantization is another attractive approach for learning compact DNNs. However, most existing network quantization methods are developed and evaluated without considering multi-teacher support to enhance the performance of quantized student model. In this paper, we propose a novel framework that leverages both multi-teacher knowledge distillation and network quantization for learning low bit-width DNNs. The proposed method encourages both collaborative learning between quantized teachers and mutual learning between quantized teachers and quantized student. During learning process, at corresponding layers, knowledge from teachers will form an importance-aware shared knowledge which will be used as input for teachers at subsequent layers and also be used to guide student. Our experimental results on CIFAR-100 and ImageNet datasets show that the compact quantized student models trained with our method achieve competitive results compared to other state-of-the-art methods, and in some cases, surpass the full precision models.
Classification Certainty Maximization for Unsupervised Domain Adaptation

The bi-classifier paradigm is a common practice in unsupervised domain adaptation (UDA), where two classifiers are leveraged to guide the model to learn domain invariant features. Previous approaches only focused on the consistency of the outputs between classifiers, but ignored the classification certainty of each classifier. Therefore, existing methods in some cases may mislead the classifiers into the wrong direction of ambiguous outputs and subsequently undermine the discriminability. To challenge this problem, in this paper we propose Classification Certainty Maximization (CCM) which considers both the joint certainty between classifiers and the individual certainty of each classifier via a novel formulation, and derived their optimal weight ratios by theoretical derivation from the perspective of gradient. In addition, we also propose a dynamic centroid update strategy to mitigate the domain gaps at the feature level. Extensive experiments on four widely used UDA datasets show that CCM performs better than the existing state-of-the-art domain adaptation methods. Notably, our dynamic centroid update strategy can be used as a plug-and-play module for existing bi-classifier domain adaptation methods to boost classification accuracy.
The Covering Radius of the Third-Order Reed-Muller Code RM(3,7) is 20

We prove the covering radius of the third-order Reed-Muller code <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm {RM}(3,7)$ </tex-math></inline-formula> is 20, which was previously known to be between 20 and 23 (inclusive). The covering radius of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm {RM}(3,7)$ </tex-math></inline-formula> is the maximum third-order nonlinearity among all 7-variable Boolean functions. It was known that there exist 7-variable Boolean functions with third-order nonlinearity 20. We prove the third-order nonlinearity cannot achieve 21. According to the classification of the quotient space of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm {RM}(6,6)/\mathrm {RM}(3,6)$ </tex-math></inline-formula> , we classify all 7-variable Boolean functions into 66 types. Firstly, we prove 62 types (among 66) cannot have third-order nonlinearity 21; Secondly, we prove that any function in the remaining 4 types can be transformed into a type (6, 10) function, if its third-order nonlinearity is 21; Finally, we transform type (6, 10) functions into a specific form, and prove the functions in that form cannot achieve the third-order nonlinearity 21 (with the assistance of computers). By the way, we prove that the affine transformation group over any finite field can be generated by two elements.
AI-Enabled Learning Architecture Using Network Traffic Traces over IoT Network: A Comprehensive Review

WiFi and private 5G networks, commonly referred to as P5G, provide Internet of Things (IoT) devices the ability to communicate at fast speeds, with low latency and with a high capacity. Will they coexist and share the burden of delivering a connection to devices at home, on the road, in the workplace, and at a park or a stadium? Or will one replace the other to manage the increase in endpoints and traffic in the enterprise, campus, and manufacturing environments? In this research, we describe IoT device testbeds to collect network traffic in a local area network and cyberspace including beyond 5G/6G network traffic traces at different layers. We also describe research problems and challenges, such as traffic classification and traffic prediction by the traffic traces of devices. An AI-enabled hierarchical learning architecture for the problems above using sources like network packets, frames, and signals from the traffic traces with machine learning models is also presented.
Adaptive Threshold Hierarchical Incremental Learning Method

Traditional deep convolutional neural networks have achieved excellent performance on various machine learning tasks. Still, they perform poorly in continuous data stream environments, where models trained on new datasets often suffer from a significant drop in performance on old datasets, a phenomenon known as “catastrophic forgetting.” Incremental learning can help solve the “catastrophic forgetting” problem in deep learning by learning new knowledge while retaining what has already been learned. In practice, incremental learning algorithms usually need to be deployed on edge devices with limited memory and restricted access to training data, facing the problems of high model complexity and imbalance between old and new categories of data. We propose an Adaptive Threshold Hierarchical Incremental Learning (ATHIL) method to address the above problems. Our proposed method does not require additional data and model storage space during the training process, combines local weight discrete coefficient thresholding and the mean nearest neighbor principle, uses a sparse matrix hierarchical masking network, and flexibly adjusts the network structure according to different tasks to achieve learning multiple image classification tasks in a single network. The experimental results show that the performance of the proposed method significantly outperforms existing methods on fine-grained classification datasets under three evaluation metrics.
A Novel Hybrid Multikey Cryptography Technique for Video Communication

Online video streaming is becoming more widespread in people’s everyday entertainment routines. Protecting copyright and piracy has become a key concern in real-time video streaming systems. This research provides a revolutionary multi-key and hybrid cryptography approach to offer security. This work describes the software implementation of video encryption and decryption employing continuous systems based on the Elliptic Curve Cryptography approach as pseudorandom encryption key generators. This approach creates several keys to encrypt and decode small chunks of video files that are produced dynamically based on the video data. The suggested approach was implemented on the Android platform, where applications for sender and recipients had been created to enable streaming. The security and performance of the proposed system have been examined by implementing it on devices and streaming videos. The outcomes demonstrate superiority in terms of performance and security.
Engineering Semantic Communication: A Survey

As the global demand for data has continued to rise exponentially, some have begun turning to the idea of semantic communication as a means of efficiently meeting this demand. Pushing beyond the boundaries of conventional communication systems, semantic communication focuses on the accurate recovery of the meaning conveyed from source to receiver, as opposed to the accurate recovery of transmitted symbols. In this survey, we aim to provide a comprehensive view of the history and current state of semantic communication and the techniques for engineering this higher level of communication. A survey of the current literature reveals four broad approaches to engineering semantic communication. We term the earliest of these approaches classical semantic information, which seeks to extend information-theoretic results to include semantic information. A second approach makes use of knowledge graphs to achieve semantic communication, and a third utilizes the power of modern deep learning techniques to facilitate this communication. The fourth approach focuses on the significance of information, rather than its meaning, to achieve efficient, goal-oriented communication. We discuss each of these four approaches and their corresponding studies in detail, and provide some challenges and opportunities that pertain to each approach. Finally, we introduce a novel approach to semantic communication, which we term context-based semantic communication. Inspired by the way in which humans naturally communicate with one another, this context-based approach provides a general, optimization-based design framework for semantic communication systems. Together, this survey provides a useful guide for the design and implementation of semantic communication systems.
Occluded Person Re-Identification With Pose Estimation Correction and Feature Reconstruction

In the real-world surveillance system application, the accuracy of person re-identification (Re-ID) still suffers from occlusion. Occluded Re-ID aims to retrieve occluded person images from multiple cameras that do not overlap. To address this issue, we present a multi-branch feature enhancement network in this paper. It has the pose correction module (PCM), the feature reconstruction module (FRM), and the part align module (PAM). PCM builds the key-points confidence corrected mechanism of the self-adaptive weights learning before the part matching stage to strengthen part-level features in the non-occluded region and weaken ones in the occluded region. FRM reconstructs the feature distribution of inter-domain and class-domain using the maximum mean discrepancy to generalize the global features. The proposed feature separation mechanism enables the two branches to attend to distinct pedestrian feature. To increase the power of representation, PAM combines part-level and global features in an outer product. The experimental results on three holistic datasets, two partial datasets, and two occluded datasets demonstrate that our method is superior. It significantly outperforms state-of-the-art by 5.7% and 4.4% rank-1 scores on Occluded-Duke and Occluded-REID, respectively.
Review of Graph-Based Hazardous Event Detection Methods for Autonomous Driving Systems

Automated and autonomous vehicles are often required to operate in complex road environments with potential hazards that may lead to hazardous events causing injury or even death.Therefore, a reliable autonomous hazardous event detection system is a key enabler for highly autonomous vehicles (e.g., Level 4 and 5 autonomous vehicles) to operate without human supervision for significant periods of time.One promising solution to the problem is the use of graph-based methods that are powerful tools for relational reasoning.Using graphs to organise heterogeneous knowledge about the operational environment, link scene entities (e.g., road users, static objects, traffic rules) and describe how they affect each other.Due to a growing interest and opportunity presented by graph-based methods for autonomous hazardous event detection, this paper provides a comprehensive review of the state-of-the-art graph-based methods that we categorise as rule-based, probabilistic, and machine learningdriven.Additionally, we present an in-depth overview of the available datasets to facilitate hazardous event training and evaluation metrics to assess model performance.In doing so, we aim to provide a thorough overview and insight into the key research opportunities and open challenges.
Deriving big geochemical data from high-resolution remote sensing data via machine learning: Application to a tailing storage facility in the Witwatersrand goldfields

Remote sensing data is a cheap form of surficial geoscientific data, and in terms of veracity, velocity and volume, can sometimes be considered big data. Its spatial and spectral resolution continues to improve over time, and some modern satellites, such as the Copernicus Programme's Sentinel-2 remote sensing satellites, offer a spatial resolution of 10 m across many of their spectral bands. The abundance and quality of remote sensing data combined with accumulated primary geochemical data has provided an unprecedented opportunity to inferentially invert remote sensing data into geochemical data. The ability to derive geochemical data from remote sensing data would provide a form of secondary big geochemical data, which can be used for numerous downstream activities, particularly where data timeliness, volume and velocity are important. Major benefactors of secondary geochemical data would be environmental monitoring and applications of artificial intelligence and machine learning in geochemistry, which currently entirely relies on manually derived data that is primarily guided by scientific reduction. Furthermore, it permits the usage of well-established data analysis techniques from geochemistry to remote sensing that allows useable insights to be extracted beyond those typically associated with strictly remote sensing data analysis. Currently, no generally applicable and systematic method to derive chemical elemental concentrations from large-scale remote sensing data have been documented in geosciences. In this paper, we demonstrate that fusing geostatistically-augmented geochemical and remote sensing data produces an abundance of data that enables a more generalized machine learning-based geochemical data generation. We use gold grade data from a South African tailing storage facility (TSF) and data from both the Landsat-8 and Sentinel remote sensing satellites. We show that various machine learning algorithms can be used given the abundance of training data. Consequently, we are able to produce a high resolution (10 m grid size) gold concentration map of the TSF, which demonstrates the potential of our method to be used to guide extraction planning, online resource exploration, environmental monitoring and resource estimation.
Importance of complementary data to histopathological image analysis of oral leukoplakia and carcinoma using deep neural networks

Background Oral cancer is one of the most common types of cancer in men causing mortality if not diagnosed early. In recent years, computer-aided diagnosis (CAD) using artificial intelligence techniques, in particular, deep neural networks have been investigated and several approaches have been proposed to deal with the automated detection of various pathologies using digital images. Recent studies indicate that the fusion of images with the patient’s clinical information is important for the final clinical diagnosis. As such dataset does not yet exist for oral cancer, as far as the authors are aware, a new dataset was collected consisting of histopathological images, demographic and clinical data. This study evaluates the importance of complementary data to histopathological image analysis of oral leukoplakia and carcinoma for CAD. Methods A new dataset (NDB-UFES) was collected from 2011 to 2021 consisting of histopathological images and information. The 237 samples were curated and analyzed by oral pathologists generating the gold standard for classification. State-of-the-art image fusion architectures and complementary data (Concatenation, Mutual Attention, MetaBlock and MetaNet) using the latest deep learning backbones were investigated for 4 distinct tasks to identify oral squamous cell carcinoma, leukoplakia with dysplasia and leukoplakia without dysplasia. We evaluate them using balanced accuracy, precision, recall and area under the ROC curve metrics. Results Experimental results indicate that the best models present balanced accuracy of 83.24% using images, demographic and clinical information with MetaBlock fusion and ResNetV2 backbone. It represents an improvement in performance of 30.68% (19.54 pp) in the task to differentiate samples diagnosed with oral squamous cell carcinoma and leukoplakia with or without dysplasia. Conclusion This study confirms that cured demographic and clinical data positively influence the performance of artificial intelligence models in automated classification of oral cancer.
One Step Closer to Converged Computing: Achieving Scalability with Cloud-Native HPC

As High Performance Computing (HPC) workflows increase in complexity, their designers seek to enable automation and flexibility offered by cloud technologies. Container orchestration through Kubernetes enables highly desirable capabilities but does not satisfy the performance demands of HPC. Kubernetes tools that automate the lifecycle of Message Passing Interface (MPI)-based applications do not scale, and the Kubernetes scheduler does not provide crucial scheduling capabilities. In this work, we detail our efforts to port CORAL-2 benchmark codes to Kubernetes on IBM Cloud and AWS EKS. We describe contributions to the MPI Operator to achieve 3,000-rank scale, a two-orders-of-magnitude improvement to state of the art. We discuss enhancements to Fluence, our scheduler plugin for Kubernetes based on the next-generation, cloud-ready Flux framework. Finally, we compare the placement decisions of Fluence with those of the Kubernetes scheduler and demonstrate that Fluence allows simulated scientific workflows to achieve up to 3× higher performance.
Transfer Naïve Bayes Learning using Augmentation and Stacking for SMS Spam Detection

Short Message Service (SMS) spam, unsolicited messages delivered through phones, is common and prevalent, but difficult to filter out. Naïve Bayes (NB) classifier is a frequently used spam filtering approach for texts, due to its simple but rigorous statistical learning nature and transparency in the decision making. For SMS messages, simple NB classification is ineffective, because SMS texts are short and brief, often contain numerous typos, abbreviations, and slang words. In this paper, we propose, AstNB, a new Augmentation and Stacking combined Transfer learning approach for Naive Bayes (NB) classification. For effective transfer learning from a source domain, e.g. emails, to a target domain, e.g. SMS, AstNB first introduces data augmentation to generate different copies of training data, by combining a target domain sample with a randomly selected source domain instance, followed by training a number of basis classifiers from augmented data. After that, a stacking process is used to generate new feature space by aggregating predictions of basis classifiers and the feature space created from target data. A final classifier is trained to predict unlabeled SMS messages for spam prediction. Experiments and comparisons show that AstNB can effectively transfer knowledge from source domain for SMS spam detection, especially when the target domain has very few labeled messages.