Reviewerly/exHarmony huggingface dataset:
Representative Color Transform for Image Enhancement

Recently, the encoder-decoder and intensity transformation approaches lead to impressive progress in image enhancement. However, the encoder-decoder often loses details in input images during down-sampling and up-sampling processes. Also, the intensity transformation has a limited capacity to cover color transformation between low-quality and high-quality images. In this paper, we propose a novel approach, called representative color transform (RCT), to tackle these issues in existing methods. RCT determines different representative colors specialized in input images and estimates transformed colors for the representative colors. It then determines enhanced colors using these transformed colors based on the similarity between input and representative colors. Extensive experiments demonstrate that the proposed algorithm outperforms recent state-of-the-art algorithms on various image enhancement problems.
A Unified Objective for Novel Class Discovery

In this paper, we study the problem of Novel Class Discovery (NCD). NCD aims at inferring novel object categories in an unlabeled set by leveraging from prior knowledge of a labeled set containing different, but related classes. Existing approaches tackle this problem by considering multiple objective functions, usually involving specialized loss terms for the labeled and the unlabeled samples respectively, and often requiring auxiliary regularization terms. In this paper we depart from this traditional scheme and introduce a UNified Objective function (UNO) for discovering novel classes, with the explicit purpose of favoring synergy between supervised and unsupervised learning. Using a multi-view self-labeling strategy, we generate pseudo-labels that can be treated homogeneously with ground truth labels. This leads to a single classification objective operating on both known and unknown classes. De-spite its simplicity, UNO outperforms the state of the art by a significant margin on several benchmarks (≈ +10% on CIFAR-100 and +8% on ImageNet). The project page is available at : https://ncd-uno.github.io.
Survey on watermarking methods in the artificial intelligence domain and beyond

Recently, machine/deep learning has become a promising solution for some intelligent tasks. It can be actively used for watermarking but less so for conventional tasks such as prediction, classification and regression. This article presents a comprehensive study on watermarking using trending technologies, such as artificial intelligence, machine learning and deep learning. Also, it briefly discusses the introduction of watermarking, background information and the most interesting and utilized applications. The major role of trending technologies in watermarking has also been highlighted. The contribution of the surveyed scheme is also summarized and compared for different technical perspectives. Lastly, the article highlights recent challenges and directions of potential research that could fill gaps in this area for researchers and developers.
Artificial intelligence based quality of transmission predictive model for cognitive optical networks

Due to the advancements in 5 G technologies, high-definition, and the internet of things (IoT), the capacity demand of optical networks has been exponentially increased. Optical communication networks offer several metrics such as high transmission capacity, low transmission loss, better anti-interference, robustness, etc which offers new opportunities to the communication field. To satisfy the increasing demands of optical networks, effective network resource utilization become essential. So, it is needed to design proper planning tools with superior accuracy for quality of transmission (QoT) in optical networks. Recently, artificial intelligence (AI) techniques pose new opportunities for resolving these issues and machine learning (ML) algorithms offer better performance over the analytical approaches. With this motivation, this paper presents a novel AI based cognitive QoT prediction (AI-CQoT) model for optical communication networks. The proposed AI-CQoT model aims to predict the QoT for the quality of service (QoS) link setup using AI techniques with the transmission equation based synthetic data generation. The proposed model uses the Label weighting extreme learning machine (LW-ELM) model for the prediction process which includes a link and signal characteristics as input features. Besides, the LW-ELM model is trained by the use of transmission equations. For improving the predictive performance of the LW-ELM model, the parameters such as weight matrix W and penalty factor C are optimally tuned by the use of the shuffled shepherd optimization (SSO) algorithm. A detailed experimental validation is performed to highlight the improved performance of the AI-CQoT model and the results are investigated in terms of different performance measures.
Ensemble transfer learning-based multimodal sentiment analysis using weighted convolutional neural networks

Huge amounts of multimodal content and comments in a mixture form of text, image, and emoji are continuously shared by users on various social networks. Most of the comments of the users in these networks have emotional aspects, which make the multimodal sentiment analysis (MSA) an important and attractive research topics in this area. In this paper, an ensemble transfer learning method is exploited to propose a hybrid MSA model based on weighted convolutional neural networks. The extended Dempster–Shafer (Yager) theory is also utilized in the proposed method of this paper to fuse the outputs of text and image classifiers to determine the final polarity at the decision level. The pre-trained VGG16 network is firstly used to extract visual features and fine-tune on the MVSA-Multiple and T4SA datasets for image sentiment classification. The Mask-RCNN model is then exploited to determine the objects in the images and convert them to text. The BERT model receives the output of this step along with the textual descriptions of the images for extracting the text features and embedding the words. The output of the BERT model is then imported into a weighted convolutional neural network ensemble (WCNNE). The texts are classified by several weak learners using the AdaBoost that is an ensemble learning technique in which, classifiers are trained sequentially. The combined use of several weak classifiers results in a strong classification. The WCNNE improves the performance and increases the accuracy of the results. As a fusing phase at the decision level, the outputs of the VGG16 and the WCNNE models will be finally merged using the extended Dempster-Shafer theory to obtain the correct sentiment label. The results of the experiments on the MVSA-Multiple and T4SA datasets show that the proposed model is better than the other compared methods and achieved an appropriate accuracy of 0.9348 on MVSA and 0.9689 on the T4SA datasets. Moreover, the proposed model reduces training time due to the use of transfer learning and the proposed AdaBoostCNN achieves better results compared to the single CNN.
A BERT based dual-channel explainable text emotion recognition system

In this paper, a novel dual-channel system for multi-class text emotion recognition has been proposed, and a novel technique to explain its training & predictions has been developed. The architecture of the proposed system contains the embedding module, dual-channel module, emotion classification module, and explainability module. The embedding module extracts the textual features from the input sentences in the form of embedding vectors using the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model. Then the embedding vectors are fed as the inputs to the dual-channel network containing two network channels made up of convolutional neural network (CNN) and bidirectional long short term memory (BiLSTM) network. The intuition behind using CNN and BiLSTM in both the channels was to harness the goodness of the convolutional layer for feature extraction and the BiLSTM layer to extract text's order and sequence-related information. The outputs of both channels are in the form of embedding vectors which are concatenated and fed to the emotion classification module. The proposed system's architecture has been determined by thorough ablation studies, and a framework has been developed to discuss its computational cost. The emotion classification module learns and projects the emotion embeddings on a hyperplane in the form of clusters. The proposed explainability technique explains the training and predictions of the proposed system by analyzing the inter & intra-cluster distances and the intersection of these clusters. The proposed approach's consistent accuracy, precision, recall, and F1 score results for ISEAR, Aman, AffectiveText, and EmotionLines datasets, ensure its applicability to diverse texts.
Quantum estimation, control and learning: Opportunities and challenges

The development of estimation and control theories for quantum systems is a fundamental task for practical quantum technology. This vision article presents a brief introduction to challenging problems and potential opportunities in the emerging areas of quantum estimation, control and learning. The topics cover quantum state estimation, quantum parameter identification, quantum filtering, quantum open-loop control, quantum feedback control, machine learning for estimation and control of quantum systems, and quantum machine learning.
What drives DeFi prices? Investigating the effects of investor attention

We comparatively assess drivers of DeFi token prices, testing the impact and connectedness between Bitcoin (as a cryptocurrency asset-class leader), Ethereum (as a platform for DeFi), and Google Trends (as a signal of investor attention). While we evidence varying causal linkages between Bitcoin or Ethereum and Defi tokens, these linkages are restricted to bear markets. In contrast, restricted MS-VAR (LR) modelling uncovers bi-causal relationships between DeFi tokens and investor sentiment in both bull and bear markets. We conclude that, rather than platform valuation, investor attention is a dominant driver of DeFi prices, with DeFi being a separate asset class from mainstream cryptocurrencies.
NAFSSR: Stereo Image Super-Resolution Using NAFNet

Stereo image super-resolution aims at enhancing the quality of super-resolution results by utilizing the complementary information provided by binocular systems. To obtain reasonable performance, most methods focus on finely designing modules, loss functions, and etc. to exploit information from another viewpoint. This has the side effect of increasing system complexity, making it difficult for researchers to evaluate new ideas and compare methods. This paper inherits a strong and simple image restoration model, NAFNet, for single-view feature extraction and ex-tends it by adding cross attention modules to fuse features between views to adapt to binocular scenarios. The proposed baseline for stereo image super-resolution is noted as NAFSSR. Furthermore, training/testing strategies are proposed to fully exploit the performance of NAFSSR. Extensive experiments demonstrate the effectiveness of our method. In particular, NAFSSR outperforms the state-of-the-art methods on the KITTI 2012, KITTI 2015, Middle-bury, and Flickr1024 datasets. With NAFSSR, we won 1st place in the NTIRE 2022 Stereo Image Super-resolution Challenge. Codes and models will be released at https://github.com/megviiresearch/NAFNet.
A visual reasoning-based approach for mutual-cognitive human-robot collaboration

Human-robot collaboration (HRC) allows seamless communication and collaboration between humans and robots to fulfil flexible manufacturing tasks in a shared workspace. Nevertheless, existing HRC systems lack an efficient integration of robotic and human cognitions. Empowered by advanced cognitive computing, this paper proposes a visual reasoning-based approach for mutual-cognitive HRC. Firstly, a domain-specific HRC knowledge graph is established. Next, the holistic manufacturing scene is perceived by visual sensors as a temporal graph. Then, a collaborative mode with similar instructions can be inferred by graph embedding. Lastly, mutual-cognitive decisions are immersed into the Augmented Reality execution loop for intuitive HRC support.
A Dual Weighting Label Assignment Scheme for Object Detection

Label assignment (LA), which aims to assign each training sample a positive (pos) and a negative (neg) loss weight, plays an important role in object detection. Existing LA methods mostly focus on the design of pos weighting function, while the neg weight is directly derived from the pos weight. Such a mechanism limits the learning capacity of detectors. In this paper, we explore a new weighting paradigm, termed dual weighting (DW), to specify pos and neg weights separately. We first identify the key influential factors of pos/neg weights by analyzing the evaluation metrics in object detection, and then design the pos and neg weighting functions based on them. Specifically, the pos weight of a sample is determined by the consistency degree between its classification and localization scores, while the neg weight is decomposed into two terms: the probability that it is a neg sample and its importance conditioned on being a neg sample. Such a weighting strategy offers greater flexibility to distinguish between important and less important samples, resulting in a more effective object detector. Equipped with the proposed DW method, a single FCOS-ResNet-50 detector can reach 41.5% mAP on COCO under 1× schedule, outperforming other existing LA methods. It consistently improves the baselines on COCO by a large margin under various backbones without bells and whistles. Code is available at https://github.com/strongwolf/DW.
Auto Color Correction of Underwater Images Utilizing Depth Information

The red spectrum is saliently attenuated due to the absorption and scattering properties of water. The acquired underwater images show severe color cast in underwater scenes. In this letter, we propose a novel color correction method for underwater images, which removes color cast on single pixels based on scene depth. The experimental results demonstrate that our approach can significantly improve the color effect and provide a correct input for the subsequent underwater image defogging methods.
AI Coach: Deep Human Pose Estimation and Analysis for Personalized Athletic Training Assistance

Recent years have witnessed an unprecedented growing of sport videos, as different types of sports activities can be widely-observed (i.e., from professional athletics to personal fitness). Existing approaches by computer vision have predominantly focused on creating experiences of content browsing and searching by video tagging and summarization. These techniques have already enabled a wide-range of applications for sports enthusiasts, such as text-based video search, highlight generation, and so on. In this paper, we take one step further to create an AI coach system to provide personalized athletic training experiences. Especially for sports activities which the training quality largely depends on the correctness of human poses in a video sequence. As sports videos often involve grand challenges of fast movement (e.g., skiing, skating) and complex actions (e.g., gymnastics), we propose to design the system with several distinct features: (1) trajectory extraction for a single human instance by leveraging deep visual tracking, (2) human pose estimation by proposing a novel human joints relation model in spatial and temporal domains, (3) pose correction by abnormal detection and exemplar-based visual suggestions. We have collected sports training videos from 30 sports enthusiasts, namely Freestyle Skiing Aerials dataset (63 clips). We show that the proposed system can lead to a remarkably better user training experience by extensive user studies.
Breast Lesion Classification in Ultrasound Images Using Deep Convolutional Neural Network

In recent years, convolutional neural networks (CNNs) have found many applications in medical image analysis.Having enough labeled data, CNNs could be trained to learn image features and used for object localization, classification, and segmentation.Although there are many interests in building and improving automated systems for medical image analysis, lack of reliable and publicly available biomedical datasets makes such a task difficult.In this work, the effectiveness of CNNs for the classification of breast lesions in ultrasound (US) images will be studied.First, due to a limited number of training data, we use a custom-built CNN with a few hidden layers and apply regularization techniques to improve the performance.Second, we use transfer learning and adapt some pre-trained models for our dataset.The dataset used in this work consists of a limited number of cases, 641 in total, histopathologically categorized (413 benign and 228 malignant lesions).To assess how the results of our classifier generalize on our data set, a 5-fold cross-validation were employed, where in each fold 80% of data were used for training and the 20% for testing.Accuracy and the area under the ROC curve (AUC) were used as the main performance metrics.Before applying any regularizations techniques, we achieved an overall accuracy of 85.98% for tumor classification, and the AUC equal to 0.94.After applying image augmentation and regularization, the accuracy and the AUC increased to 92.05% and 0.97, respectively.Using a pre-trained model, we achieved an overall accuracy of 87.07% and an AUC equal to 0.96.The obtained results demonstrated the effectiveness of our custom architecture for classification of tumors in this small US imaging dataset, surpassing some traditional learning algorithm based on manual feature selection.
Addressing Binary Classification over Class Imbalanced Clinical Datasets Using Computationally Intelligent Techniques

Nowadays, healthcare is the prime need of every human being in the world, and clinical datasets play an important role in developing an intelligent healthcare system for monitoring the health of people. Mostly, the real-world datasets are inherently class imbalanced, clinical datasets also suffer from this imbalance problem, and the imbalanced class distributions pose several issues in the training of classifiers. Consequently, classifiers suffer from low accuracy, precision, recall, and a high degree of misclassification, etc. We performed a brief literature review on the class imbalanced learning scenario. This study carries the empirical performance evaluation of six classifiers, namely Decision Tree, k-Nearest Neighbor, Logistic regression, Artificial Neural Network, Support Vector Machine, and Gaussian Naïve Bayes, over five imbalanced clinical datasets, Breast Cancer Disease, Coronary Heart Disease, Indian Liver Patient, Pima Indians Diabetes Database, and Coronary Kidney Disease, with respect to seven different class balancing techniques, namely Undersampling, Random oversampling, SMOTE, ADASYN, SVM-SMOTE, SMOTEEN, and SMOTETOMEK. In addition to this, the appropriate explanations for the superiority of the classifiers as well as data-balancing techniques are also explored. Furthermore, we discuss the possible recommendations on how to tackle the class imbalanced datasets while training the different supervised machine learning methods. Result analysis demonstrates that SMOTEEN balancing method often performed better over all the other six data-balancing techniques with all six classifiers and for all five clinical datasets. Except for SMOTEEN, all other six balancing techniques almost had equal performance but moderately lesser performance than SMOTEEN.
Stochastic Conjugate Gradient Algorithm With Variance Reduction

Conjugate gradient (CG) methods are a class of important methods for solving linear equations and nonlinear optimization problems. In this paper, we propose a new stochastic CG algorithm with variance reduction1 and we prove its linear convergence with the Fletcher and Reeves method for strongly convex and smooth functions. We experimentally demonstrate that the CG with variance reduction algorithm converges faster than its counterparts for four learning models, which may be convex, nonconvex or nonsmooth. In addition, its area under the curve performance on six large-scale data sets is comparable to that of the LIBLINEAR solver for the L2 -regularized L2 -loss but with a significant improvement in computational efficiency.1CGVR algorithm is available on github: https://github.com/xbjin/cgvr.
Modeling public mood and emotion: Blog and news sentiment and socio-economic phenomena

The development of online virtual communities has raised the importance in analyzing massive volume of text from websites and social networks. This research analyzed financial blogs and online news articles to develop a public mood dynamic prediction model for stock markets, referencing the perspectives of behavioral finance and the characteristics of online financial communities. This research applies big data and opinion mining approaches to the investors' sentiment analysis in Taiwan. The proposed model was verified using experimental datasets from ChinaTimes.com, cnYES.com, Yahoo stock market news, and Google stock market news over an 18 month period. Empirical results indicate the big data analysis techniques to assess emotional content of commentary on current stock or financial issues can effectively forecast stock price movement.
Weird Machines, Exploitability, and Provable Unexploitability

The concept of exploit is central to computer security, particularly in the context of memory corruptions.Yet, in spite of the centrality of the concept and voluminous descriptions of various exploitation techniques or countermeasures, a good theoretical framework for describing and reasoning about exploitation has not yet been put forward.A body of concepts and folk theorems exists in the community of exploitation practitioners; unfortunately, these concepts are rarely written down or made sufficiently precise for people outside of this community to benefit from them.This paper clarifies a number of these concepts, provides a clear definition of exploit, a clear definition of the concept of a weird machine, and how programming of a weird machine leads to exploitation.The papers also shows, somewhat counterintuitively, that it is feasible to design some software in a way that even powerful attackers-with the ability to corrupt memory once-cannot gain an advantage.The approach in this paper is focused on memory corruptions.While it can be applied to many security vulnerabilities introduced by other programming mistakes, it does not address side channel attacks, protocol weaknesses, or security problems that are present by design.