Reviewerly/exHarmony huggingface dataset:
Category-Aware Multimodal Attention Network for Fashion Compatibility Modeling

Fashion compatibility modeling, which is used to estimate the matching degree of a given set of fashion items, has received increasing attention in recent years. However, existing studies often fail to fully leverage multimodal information or ignore the semantic guidance of clothing categories in elevating the reliability of multimodal information. In this paper, we propose a fashion compatibility modeling approach with a category-aware multimodal attention network, termed as FCM-CMAN. In FCM-CMAN, we focus on enriching and aggregating multimodal representations of fashion items by means of the dynamic representations of categories and a contextual attention mechanism simultaneously. Specifically, considering that category correlations are always dynamic and varied for different fashion items, we design a categorical dynamic graph convolutional network to adaptively learn the semantic correlations between categories. When combined with the multi-layered visual outputs of a convolutional neural network and the surrounding contextual information, multiple content-aware category representations and context-aware attention weights are obtained to better characterize fashion items from different aspects. On this basis, two pieces of aware information are integrated by a multimodal factorized bilinear pooling strategy to generate visual-semantic embeddings, which are further improved by a multi-head self-attention mechanism to capture significant elements related to fashion compatibility. Extensive experiments conducted on the FashionVC and ExpFashion datasets demonstrate the superiority of FCM-CMAN over state-of-the-art methods.
Learning-based 3D imaging from single structured-light image

Integrating structured-light technique with deep learning for single-shot 3D imaging has recently gained enormous attention due to its unprecedented robustness. This paper presents an innovative technique of supervised learning-based 3D imaging from a single grayscale structured-light image. The proposed approach uses a single-input, double-output convolutional neural network to transform a regular fringe-pattern image into two intermediate quantities which facilitate the subsequent 3D image reconstruction with high accuracy. A few experiments have been conducted to demonstrate the validity and robustness of the proposed technique.
Novel prediction model on OSCC histopathological images via deep transfer learning combined with Grad-CAM interpretation

This paper proposes a novel model using deep transfer learning to predict oral squamous cell carcinoma (OSCC) histopathological images with gradient-class activation mapping (Grad-CAM) to locate the lesion area in the images. The proposed model utilizes a recent public database of 1224 oral histopathological images of normal and OSCC cells at 100x and 400x magnifications. It is inconsequential to base a decision regarding OSCC prognosis on human evaluation, so an accurate decision needs a deep transfer learning strategy that performs better than other comparisons. It may be possible to solve the problem of predicting oral tumors by exploiting the public source database with a proposed model to reach two main targets. The first target contains ten well-classified algorithms based on a deep learning convolutional neural network (CNN) to build a prediction model for OSCC histopathological images to improve classification accuracy between malignant and normal images. After estimating the performance of the models, the obtained results are compared and the best-performing model is selected. The second target contains a Grad-CAM validation process to locate the lesion area in an OSCC image according to the best model. This validation phase can directly impact the robustness of the entire prediction model for OSCC histopathological images. For experimental results, ResNet-101, with the highest accuracy of 100% at 100x magnification, and EfficientNet-b0, with the highest accuracy of 95.65% at 400x magnification, indicate a high performance in predicting oral cancer compared to other modern trained models. The contribution of this work is based on a combination of CNN and Grad-CAM to handle OSCC histopathological images for the purpose of both classification and validation. The findings gained from this propounded model are severe as the crucial leadership of the clinical community in the early accurate detection of oral cancer.
A novel prediction model based on decomposition-integration and error correction for COVID-19 daily confirmed and death cases

Coronavirus disease (COVID-19) has infected billion people around the world and affected the economy, but most countries are considering reopening, so the COVID-19 daily confirmed and death cases have increased greatly. It is very necessary to predict the COVID-19 daily confirmed and death cases in order to help every country formulate prevention policies. To enhance the prediction performance, this paper proposes a prediction model based on improved variational mode decomposition by sparrow search algorithm (SVMD), improved kernel extreme learning machine by Aquila optimizer algorithm (AO-KELM) and error correction idea, named SVMD-AO-KELM-error for short-term prediction of COVID-19 cases. Firstly, to solve mode number and penalty factor selection of variational mode decomposition (VMD), an improved VMD based on sparrow search algorithm (SSA), named SVMD, is proposed. SVMD decomposes the COVID-19 case data into some intrinsic mode function (IMF) components and residual is considered. Secondly, to properly selected regularization coefficients and kernel parameters of kernel extreme learning machine (KELM) and improve the prediction performance of KELM, an improved KELM by Aquila optimizer (AO) algorithm, named AO-KELM, is proposed. Each component is predicted by AO-KELM. Then, the prediction error of IMF and residual are predicted by AO-KELM to correct prediction results, which is error correction idea. Finally, prediction results of each component and error prediction results are reconstructed to get final prediction results. Through the simulation experiment of the COVID-19 daily confirmed and death cases in Brazil, Mexico, and Russia and comparison with twelve comparative models, simulation experiment gives that SVMD-AO-KELM-error has best prediction accuracy. It also proves that the proposed model can be used to predict the pandemic COVID-19 cases and offers a novel approach for COVID-19 cases prediction.
Automated Machine Learning (AutoML): The Future of Computational Intelligence

Computer science controls every task in today’s environment, and everything in the sector attempts to automate the task. The basic essence of computer science is to minimize human effort and the core of reducing human involvement in any task is to automate. Machine learning (ML) has become an important part of many aspects of our daily life. High-performance machine learning applications, on the other hand, necessitate the use of highly qualified data scientists and domain specialists. Automated machine learning (AutoML) aims to reduce the need for data scientists by allowing domain experts to automatically construct machine learning applications without extensive statistical and machine learning knowledge. This paper provides an overview of existing AutoML methodologies as well as information on related technologies. We present and investigate important AutoML techniques and methodologies along with present challenges and future research directions. We also analyze various security threats that can be posed to the machine learning models and AutoML.
Detecting Kids Cyberbullying Using Transfer Learning Approach: Transformer Fine-Tuning Models

The rising popularity and the usage of social media have given easy access to social media apps such as Facebook, Twitter, Instagram, and YouTube. Numerous entertaining and educative movies and videos are created and uploaded daily to attract a young audience, particularly kids and teenagers. This has caused a significant rise in inappropriate content upload and cyberbullying. The comments received on such videos can adversely impact the mental state, and behaviour child’s if not monitored by the parents. There are researchers done to solve the cyberbullying issue through the classical machine, deep learning, and natural language processing approaches. However, such methods require computational time and hardware while consuming a massive amount of data to train models to improve detection accuracy. This chapter proposes a method to detect cyberbullying on online platforms using the method of transfer learning. Several experiments were carried out using the existing dataset and a variant of pre-trained models based on BERT. Experiment results show that transfer learning outperformed classical machine, deep learning, and learning methods.
An improved deep learning architecture for multi-object tracking systems

Robust and reliable 3D multi-object tracking (MOT) is essential for autonomous driving in crowded urban road scenes. In those scenarios, accurate data association between tracked objects and incoming new detections is crucial. This paper presents a tracking system based on the Kalman filter that uses a deep learning approach to the association problem. The proposed architecture consists of three neural networks. First, a convolutional LSTM network extracts spatiotemporal features from a sequence of detections of the same track. Then, a Siamese network calculates the degree of similarity between all tracks and the new detections found at each new frame. Finally, a recurrent LSTM network is used to extract 3D and bounding box information. This model follows the tracking-by-detection paradigm and has been trained with track sequences to be able to handle missed observations and to reduce identity switches. A validation test was carried out on the Argoverse dataset to validate the performance of the proposed system. The developed deep learning approach could improve current multi-object tracking systems based on classic algorithms like the Kalman filter.
Identifying the Best Approximating Model in Bayesian Phylogenetics: Bayes Factors, Cross-Validation or wAIC?

There is still no consensus as to how to select models in Bayesian phylogenetics, and more generally in applied Bayesian statistics. Bayes factors are often presented as the method of choice, yet other approaches have been proposed, such as cross-validation or information criteria. Each of these paradigms raises specific computational challenges, but they also differ in their statistical meaning, being motivated by different objectives: either testing hypotheses or finding the best-approximating model. These alternative goals entail different compromises, and as a result, Bayes factors, cross-validation, and information criteria may be valid for addressing different questions. Here, the question of Bayesian model selection is revisited, with a focus on the problem of finding the best-approximating model. Several model selection approaches were re-implemented, numerically assessed and compared: Bayes factors, cross-validation (CV), in its different forms (k-fold or leave-one-out), and the widely applicable information criterion (wAIC), which is asymptotically equivalent to leave-one-out cross-validation (LOO-CV). Using a combination of analytical results and empirical and simulation analyses, it is shown that Bayes factors are unduly conservative. In contrast, CV represents a more adequate formalism for selecting the model returning the best approximation of the data-generating process and the most accurate estimates of the parameters of interest. Among alternative CV schemes, LOO-CV and its asymptotic equivalent represented by the wAIC, stand out as the best choices, conceptually and computationally, given that both can be simultaneously computed based on standard Markov chain Monte Carlo runs under the posterior distribution. [Bayes factor; cross-validation; marginal likelihood; model comparison; wAIC.].
Learning Patch-Channel Correspondence for Interpretable Face Forgery Detection

Beyond high accuracy, good interpretability is very critical to deploy a face forgery detection model for visual content analysis. In this paper, we propose learning patch-channel correspondence to facilitate interpretable face forgery detection. Patch-channel correspondence aims to transform the latent features of a facial image into multi-channel interpretable features where each channel mainly encoders a corresponding facial patch. Towards this end, our approach embeds a feature reorganization layer into a deep neural network and simultaneously optimizes classification task and correspondence task via alternate optimization. The correspondence task accepts multiple zero-padding facial patch images and represents them into channel-aware interpretable representations. The task is solved by step-wisely learning channel-wise decorrelation and patch-channel alignment. Channel-wise decorrelation decouples latent features for class-specific discriminative channels to reduce feature complexity and channel correlation, while patch-channel alignment then models the pairwise correspondence between feature channels and facial patches. In this way, the learned model can automatically discover corresponding salient features associated to potential forgery regions during inference, providing discriminative localization of visualized evidences for face forgery detection while maintaining high detection accuracy. Extensive experiments on popular benchmarks clearly demonstrate the effectiveness of the proposed approach in interpreting face forgery detection without sacrificing accuracy. The source code is available at https://github.com/Jae35/IFFD.
Neural Network Fusion Processing and Inverse Mapping to Combine Multisensor Satellite Data and Analyze the Prominent Features

In the last decade, the increase of active and passive earth observation satellites has provided us with more remote sensing data. This fact has led to enhanced interest in the fusion of different satellite data since some of the satellites have properties complementary to others. Fusion techniques can improve the estimation in areas of interest by using the complementary information and inferring unknown parameters. They also have the potential to provide high-resolution detailed classification maps. Thus, we propose a neural network, which combines and analyzes the data obtained from synthetic aperture radar (SAR) and optical sensors to provide high-resolution classification maps. The neural network employs a novel activation function to construct a neural network explainability method termed as inverse mapping for prominent feature analysis. By applying inverse mapping to the data-fusion neural network, we can understand which input features are the prominent contributors for which classification outputs. Inverse mapping realizes backward signal flow based on teacher-signal backpropagation dynamics, which is consistent with its forward processing. It performs the contribution analysis of the data pixel-by-pixel and class-by-class. In this paper, we focus on earthquake damage detection by dealing with SAR and optical sensor data of the 2018 Sulawesi earthquake in Indonesia. The fusion-based results show increased classification accuracy compared to the results of independent sensors. Moreover, we observe that inverse mapping shows reasonable explanations in a consistent manner. It also indicates the contributions of features different from straightforward counterparts, namely pre- and post-seismic features, in the detection of particular classes.
Cross-Modal Retrieval With Partially Mismatched Pairs

In this paper, we study a challenging but less-touched problem in cross-modal retrieval, i.e., partially mismatched pairs (PMPs). Specifically, in real-world scenarios, a huge number of multimedia data (e.g., the Conceptual Captions dataset) are collected from the Internet, and thus it is inevitable to wrongly treat some irrelevant cross-modal pairs as matched. Undoubtedly, such a PMP problem will remarkably degrade the cross-modal retrieval performance. To tackle this problem, we derive a unified theoretical Robust Cross-modal Learning framework (RCL) with an unbiased estimator of the cross-modal retrieval risk, which aims to endow the cross-modal retrieval methods with robustness against PMPs. In detail, our RCL adopts a novel complementary contrastive learning paradigm to address the following two challenges, i.e., the overfitting and underfitting issues. On the one hand, our method only utilizes the negative information which is much less likely false compared with the positive information, thus avoiding the overfitting issue to PMPs. However, these robust strategies could induce underfitting issues, thus making training models more difficult. On the other hand, to address the underfitting issue brought by weak supervision, we present to leverage of all available negative pairs to enhance the supervision contained in the negative information. Moreover, to further improve the performance, we propose to minimize the upper bounds of the risk to pay more attention to hard samples. To verify the effectiveness and robustness of the proposed method, we carry out comprehensive experiments on five widely-used benchmark datasets compared with nine state-of-the-art approaches w.r.t. the image-text and video-text retrieval tasks. The code is available at https://github.com/penghu-cs/RCL.
Distributed and Secure Federated Learning for Wireless Computing Power Networks

The explosively growth of mobile applications imposes much burden on the current computing networks. Wireless Computing Power Network (WCPN), as an emerging computing architecture, can sense and coordinate computing resources through agile wireless communications, and realize distributed intelligence based on federated learning. However, the mobility and heterogeneity of WCPN nodes typically impact the security (e.g., malicious node disturbance) and efficiency of federated learning in WCPN. In light of this, this article proposes a provable secure and decentralized federated learning based on blockchain for WCPN, where nodes can freely participate or leave the WCPN federated training without authorization and security threats. Particularly, we design a blockchain with proof-of-accuracy (PoAcc) consensus scheme to deeply integrate with the federated learning procedure, in which high-accuracy local models have the priority of aggregation, thus accelerating the convergence of federated learning and improving the efficiency of WCPN. The proposed PoAcc is proved to be secure as long as the ratio of honest to malicious nodes is above a lower bound. To further meet the security requirement of PoAcc, we then propose an evolutionary game-based incentive scheme that incentivizes honest nodes to participate the WCPN federated learning under malicious node disturbance. Numerical results show that the proposed scheme ensures the consistency and security of federated learning in WCPN, while outperforming the benchmarks in terms of model accuracy and resource consumption.
Semi-supervised domain adaptation for segmentation models on different monitoring settings

The performance of deep learning models could easily degrade even with slight changes in monitoring settings and environments. Although previous studies have addressed such problems with domain adaptation (DA) methods, this study found that even the state-of-the-art DA methods could not achieve decent adaptation performance in the construction domain. To address the problem, this study presents a novel semi-supervised DA method for semantic segmentation that is built on data augmentation, an unsupervised DA method, and knowledge distillation. Experiments were conducted on the dataset from one construction site (the source domain) and three different construction sites (the target domains). The experimental results demonstrated the effectiveness of the proposed method: the performance is comparable to a model trained with the data in the target domain. The major finding is that DA can be significantly improved when unsupervised DA methods are used with the target domain-specific data augmentation for construction site scenes.
Asynchronous and Synchronous Federated Learning-based UAVs

Unmanned aerial vehicles (UAVs) can be used to support data gathering, training of models, and wireless communication by acting as flying Base Stations (BSs). The usage of UAVs for wireless networks is quickly expanding in areas such as tracking as well as surveillance, defense, leading healthcare deliveries, telecommunications, and so on. As Deep Learning (DL)-assisted approaches require sending raw data from model training devices to UAV servers which is problematic because of device privacy concerns and UAVs' limited processing or communication resources. Federated Learning (FL), which is also called distributed deep learning, is presented as a solution, with the core notion of keeping original or raw data where it is created while transferring the only user's localized trained DL models to a centralized organization for aggregation. We will build a Synchronous Federated Learning (SFL) structure for multi-UAVs and also the comparative analysis of Asynchronous Federated Learning (AFL) and SFL.The SFL methodology will take some time to execute, but there will be no data loss or packet loss. AFL, on the other hand, takes less time but results in packet data loss. Simulation findings suggest that our proposed framework and technique improve global rounds and learning accuracy as compared to AFL.
Analysis of public opinion on food safety in Greater China with big data and machine learning

The Internet contains a wealth of public opinion on food safety, including views on food adulteration, food-borne diseases, agricultural pollution, irregular food distribution, and food production issues. To systematically collect and analyze public opinion on food safety in Greater China, we developed IFoodCloud, which automatically collects data from more than 3,100 public sources. Meanwhile, we constructed sentiment classification models using multiple lexicon-based and machine learning-based algorithms integrated with IFoodCloud that provide an unprecedented rapid means of understanding the public sentiment toward specific food safety incidents. Our best model's F1 score achieved 0.9737, demonstrating its great predictive ability and robustness. Using IFoodCloud, we analyzed public sentiment on food safety in Greater China and the changing trend of public opinion at the early stage of the 2019 Coronavirus Disease pandemic, demonstrating the potential of big data and machine learning for promoting risk communication and decision-making.
MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes

3D human motion capture from monocular RGB images respecting interactions of a subject with complex and possibly deformable environments is a very challenging, illposed and under-explored problem. Existing methods address it only weakly and do not model possible surface deformations often occurring when humans interact with scene surfaces. In contrast, this paper proposes MoCapDe-form, i.e., a new framework for monocular 3D human motion capture that is the first to explicitly model non-rigid deformations of a 3D scene for improved 3D human pose estimation and deformable environment reconstruction. Mo-CapDeform accepts a monocular RGB video and a 3D scene mesh aligned in the camera space. It first localises a subject in the input monocular video along with dense contact labels using a new raycasting based strategy. Next, our human-environment interaction constraints are leveraged to jointly optimise global 3D human poses and non-rigid surface deformations. MoCapDeform achieves superior accuracy than competing methods on several datasets, including our newly recorded one with deforming background scenes.