Reviewerly/exHarmony huggingface dataset:
Sentiment Analysis from Images using VGG19 based Transfer Learning Approach

Using image sentiment analysis rather than text sentiment analysis to make different judgments has grown increasingly popular recently. In recent years, transfer learning techniques have been employed extensively in developing end-to-end image sentiment analysis methodologies, which is expected to continue. Deep learning algorithms have produced outstanding outcomes in a variety of applications. Even though image-based sentiment analysis is tricky, there seems to be much space for progress. Research presented here suggests that a VGG-19-based approach, which can easily be used to concentrate on vast body areas, such as the face, offers a significant gain over earlier studies. This research aims to improve image categorization performance by using the well-known deep convolutional neural network, VGG19, and other deep features. Emotion detection and classification with diverse emotions utilizing a VGG-19 (CNN-based) architecture can be easily performed using CK+, FER2013, and JAFFE datasets. The results show that the proposed method can reach accuracy up to 99%. It can study a person's emotional habits and psychological condition using the approach.
Using fuzzy cognitive maps to promote nature-based solutions for water quality improvement in developing-country communities

An adequate strategy for water quality improvement must \nsider a range of political, economic, social, technological, environmental, and legal (PESTEL) concepts. Nature-based solutions have emerged as promising tools to improve water quality while considering these factors. In this context, fuzzy cognitive maps and the PESTEL approach have been merged to i) identify the principal concepts that affect water quality from different perspectives, and ii) theoretically explore the use of artificial floating islands as a measure of nature-based solution combined with different policies, to find strategies to improve water quality given local conditions. For this purpose, three Ecuadorian communities are used as scenarios. The communities are located in different geographical regions, i.e., páramo (an alpine tundra ecosystem), coastal mangrove, and tropical rainforest. From literature-based research, 40 PESTEL concepts are identified, then local experts recognize relevant concepts related to water quality deterioration regarding local conditions, and the communities develop fully democratic social cognitive maps. The cognitive map from the páramo community is constructed with 17 concepts mainly driven by environment (23%) and economy (23%). The major problem identified is natural water pollutants with the highest centrality value ( c i = 12.22). The mangrove community uses 19 concepts influenced by policy (48%), and the major issue is human exposure to environmental pollutants ( c i = 16.27). The rainforest community uses 15 concepts related to the economy (40%), and the major problem is the violation of environmental legislation ( c i = 15.96). Our pioneer work to predict the future of water management shows that in the worst-case scenario, more than 85% of concepts are affected in all communities. However, the implementation of policy strategies in combination with artificial floating islands demonstrates a large potential for improving water quality. With this study, we provide a novel, inclusive, and locally adapted framework to guide future water management and contribute to achieving the Sustainable Development Goal SDG 6. • FCMs offer a novel and locally adapted framework to improve water quality management. • NbS plus policy strategies are useful to achieving the SDG 6 at the community level. • A strategy to improve water quality in communities must consider PESTEL concepts. • Experts and communities can identify key strategies to solve water quality problems.
A Survey on Object Instance Segmentation

Abstract In recent years, instance segmentation has become a key research area in computer vision. This technology has been applied in varied applications such as robotics, healthcare and intelligent driving. Instance segmentation technology not only detects the location of the object but also marks edges for each single instance, which can solve both object detection and semantic segmentation concurrently. Our survey will give a detail introduction to the instance segmentation technology based on deep learning, reinforcement learning and transformers. Further, we will discuss about its development in this field along with the most common datasets used. We will also focus on different challenges and future development scope for instance segmentation. This technology will provide a strong reference for future researchers in our survey paper.
Constructing Strong S-Box by 2D Chaotic Map with Application to Irreversible Parallel Key Expansion

Cryptanalysis of key expansion algorithms in AES and SM4 revealed that (1) there exists weaknesses in their S-Boxes, and (2) the round key expansion algorithm is reversible, i.e. the initial key can be recovered from any round key, which may be explored by the attacker. To solve these problems, first we constructed a nondegenerate 2D exponential hyper chaotic map (2D-EQCM), derived a recursion formula to calculate the number of S-Boxes, and designed a strong S-Box construction algorithm without such weakness. Then based on 2D-EQCM and S-Box, we designed an irreversible parallel key expansion algorithm, which could transform the initial key to any number of relatively independent round keys. Security and statistical analysis demonstrated the flexibility and effectiveness of the proposed irreversible parallel key expansion algorithm.
Automated layer-wise solution for ensemble deep randomized feed-forward neural network

The randomized feed-forward neural network is a single hidden layer feed-forward neural network that enables efficient learning by optimizing only the output weights. The ensemble deep learning framework significantly improves the performance of randomized neural networks. However, the framework’s capabilities are limited by traditional hyper-parameter selection approaches. Meanwhile, different random network architectures, such as the existence or lack of a direct link and the mapping of direct links, can also strongly affect the results. We present an automated learning pipeline for the ensemble deep randomized feed-forward neural network in this paper, which integrates hyper-parameter selection and randomized network architectural search via Bayesian optimization to ensure robust performance. Experiments on 46 UCI tabular datasets show that our strategy produces state-of-the-art performance on various tabular datasets among a range of randomized networks and feed-forward neural networks. We also conduct ablation studies to investigate the impact of various hyper-parameters and network architectures.
Semi-supervised novelty detection with one class SVM for SMS spam detection

The volume of SMS messages sent on a daily basis globally has continued to grow significantly over the past years. Hence, mobile phones are becoming increasingly vulnerable to SMS spam messages, thereby exposing users to the risk of fraud and theft of personal data. Filtering of messages to detect and eliminate SMS spam is now a critical functionality for which different types of machine learning approaches are still being explored. In this paper, we propose a system for detecting SMS spam using a semi-supervised novelty detection approach based on one class SVM classifier. The system is built as an anomaly detector that learns only from normal SMS messages thus enabling detection models to be implemented in the absence of labelled SMS spam training examples. We evaluated our proposed system using a benchmark dataset consisting of 747 SMS spam and 4827 non-spam messages. The results show that our proposed method out-performed the traditional supervised machine learning approaches based on binary, frequency or TF-IDF bag-of-words. The overall accuracy was 98% with 100% SMS spam detection rate and only around 3% false positive rate.
Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch

Photonic neural network has been sought as an alternative solution to surpass the efficiency and speed bottlenecks of electronic neural network. Despite that the integrated Mach-Zehnder Interferometer (MZI) mesh can perform vector-matrix multiplication in photonic neural network, a programmable in-situ nonlinear activation function has not been proposed to date, suppressing further advancement of photonic neural network. Here, we demonstrate an efficient in-situ nonlinear accelerator comprising a unique solution-processed two-dimensional (2D) MoS2 Opto-Resistive RAM Switch (ORS), which exhibits tunable nonlinear resistance switching that allow us to introduce nonlinearity to the photonic neuron which overcomes the linear voltage-power relationship of typical photonic components. Our reconfigurable scheme enables implementation of a wide variety of nonlinear responses. Furthermore, we confirm its feasibility and capability for MNIST handwritten digit recognition, achieving a high accuracy of 91.6%. Our accelerator constitutes a major step towards the realization of in-situ photonic neural network and pave the way for the integration of photonic integrated circuits (PIC).
SEEP: Semantic-enhanced question embeddings pre-training for improving knowledge tracing

Knowledge Tracing (KT) defines the task of diagnosing students’ dynamic knowledge level in exercises. Although existing efforts have leveraged question information, most of them either learn question embeddings during the process of model training or represent questions based on the correlation between questions and concepts, which ignores plentiful implicit information entailed in the student-question-concept interaction and the revelation of fine-grained semantics between the interaction as well as the usage of the students’ historical answers. It is, however, challenging to extract and refine implicit information in the student-question-concept interaction which is highly heterogeneous and complex. To this end, in this paper, we present a novel Semantic-Enhanced Question Embeddings Pre-training (SEEP) method, concentrating on decomposing underlying relation information in the interaction and further fusing information of questions and concepts under different decomposed semantic perspectives to obtain semantic-enhanced question embeddings for improving performances of KT methods. Extensive experiments conducted on two real-world datasets show SEEP has the higher expressive power that enables KT methods to predict students’ performance.
Exploring the Effectiveness of Video Perceptual Representation in Blind Video Quality Assessment

With the rapid growth of in-the-wild videos taken by non-specialists, blind video quality assessment (VQA) has become a challenging and demanding problem. Although lots of efforts have been made to solve this problem, it remains unclear how the human visual system (HVS) relates to the temporal quality of videos. Meanwhile, recent work has found that the frames of natural video transformed into the perceptual domain of the HVS tend to form a straight trajectory of the representations. With the obtained insight that distortion impairs the perceived video quality and results in a curved trajectory of the perceptual representation, we propose a temporal perceptual quality index (TPQI) to measure the temporal distortion by describing the graphic morphology of the representation. Specifically, we first extract the video perceptual representations from the lateral geniculate nucleus (LGN) and primary visual area (V1) of the HVS, and then measure the straightness and compactness of their trajectories to quantify the degradation in naturalness and content continuity of video. Experiments show that the perceptual representation in the HVS is an effective way of predicting subjective temporal quality, and thus TPQI can, for the first time, achieve comparable performance to the spatial quality metric and be even more effective in assessing videos with large temporal variations. We further demonstrate that by combining with NIQE, a spatial quality metric, TPQI can achieve top performance over popular in-the-wild video datasets. More importantly, TPQI does not require any additional information beyond the video being evaluated and thus can be applied to any datasets without parameter tuning. Source code is available at https://github.com/UoLMM/TPQI-VQA.
Skimming, Locating, then Perusing: A Human-Like Framework for Natural Language Video Localization

This paper addresses the problem of natural language video localization (NLVL). Almost all existing works follow the "only look once" framework that exploits a single model to directly capture the complex cross- and self-modal relations among video-query pairs and retrieve the relevant segment. However, we argue that these methods have overlooked two indispensable characteristics of an ideal localization method: 1) Frame-differentiable: considering the imbalance of positive/negative video frames, it is effective to highlight positive frames and weaken negative ones during the localization. 2) Boundary-precise: to predict the exact segment boundary, the model should capture more fine-grained differences between consecutive frames since their variations are often smooth. To this end, inspired by how humans perceive and localize a segment, we propose a two-step human-like framework called Skimming-Locating-Perusing (SLP). SLP consists of a Skimming-and-Locating (SL) module and a Bi-directional Perusing (BP) module. The SL module first refers to the query semantic and selects the best matched frame from the video while filtering out irrelevant frames. Then, the BP module constructs an initial segment based on this frame, and dynamically updates it by exploring its adjacent frames until no frame shares the same activity semantic. Experimental results on three challenging benchmarks show that our SLP is superior to the state-of-the-art methods and localizes more precise segment boundaries.
SPTS: Single-Point Text Spotting

Existing scene text spotting (i.e., end-to-end text detection and recognition) methods rely on costly bounding box annotations (e.g., text-line, word-level, or character-level bounding boxes). For the first time, we demonstrate that training scene text spotting models can be achieved with an extremely low-cost annotation of a single-point for each instance. We propose an end-to-end scene text spotting method that tackles scene text spotting as a sequence prediction task. Given an image as input, we formulate the desired detection and recognition results as a sequence of discrete tokens and use an auto-regressive Transformer to predict the sequence. The proposed method is simple yet effective, which can achieve state-of-the-art results on widely used benchmarks. Most significantly, we show that the performance is not very sensitive to the positions of the point annotation, meaning that it can be much easier to be annotated or even be automatically generated than the bounding box that requires precise positions. We believe that such a pioneer attempt indicates a significant opportunity for scene text spotting applications of a much larger scale than previously possible. The code is available at https://github.com/shannanyinxiang/SPTS.
Bipartite Graph-based Discriminative Feature Learning for Multi-View Clustering

Multi-view clustering is an important technique in machine learning research. Existing methods have improved in clustering performance, most of them learn graph structure depending on all samples, which are high complexity. Bipartite graph-based multi-view clustering can obtain clustering result by establishing the relationship between the sample points and small anchor points, which improve the efficiency of clustering. Most bipartite graph-based clustering methods only focus on topological graph structure learning depending on sample nodes, ignore the influence of node features. In this paper, we propose bipartite graph-based discriminative feature learning for multi-view clustering, which combines bipartite graph learning and discriminative feature learning to a unified framework. Specifically, the bipartite graph learning is proposed via multi-view subspace representation with manifold regularization terms. Meanwhile, our feature learning utilizes data pseudo-labels obtained by fused bipartite graph to seek projection direction, which make the same label be closer and make data points with different labels be far away from each other. At last, the proposed manifold regularization terms establish the relationship between constructed bipartite graph and new data representation. By leveraging the interactions between structure learning and discriminative feature learning, we are able to select more informative features and capture more accurate structure of data for clustering. Extensive experimental results on different scale datasets demonstrate our method achieves better or comparable clustering performance than the results of state-of-the-art methods.
Not All Pixels Are Matched

Visible-Infrared Person Re-Identification (VI-ReID) has become an emerging task for night-time surveillance systems. In order to reduce the cross-modality discrepancy, previous works either align the features via metric learning or generate synthesized cross-modality images by Generative Adversary Network. However, feature-level alignment ignores the heterogeneous data itself while generative framework suffers from the low generation quality, limiting their applications. In this paper, we propose a dense contrastive learning framework (DCLNet), which performs pixel-to-pixel dense alignment acting on the intermediate representations, rather than the final deep feature. It is a new loss function that brings views of positive pixels with same semantic information closer in shallow representation space, whilst pushing views of negative pixels apart. It naturally provides additional dense supervision and captures fine-grained pixel correspondence, reducing the modality gap from a new perspective. To implement it, a Part Aware Parsing (PAP) module and a Semantic Rectification Module (SRM) are introduced to learn and refine a semantic-guided mask, allowing us to efficiently find positive pairs only requiring instance-level supervision. Extensive experiments on the public SYSU-MM01 and RegDB datasets demonstrate the superiority of our pipeline over state-of-the-arts. Code is available at https://github.com/sunhz0117/DCLNet.
TSRFormer

We present a new table structure recognition (TSR) approach, called TSRFormer, to robustly recognizing the structures of complex tables with geometrical distortions from various table images. Unlike previous methods, we formulate table separation line prediction as a line regression problem instead of an image segmentation problem and propose a new two-stage DETR based separator prediction approach, dubbed Sep arator RE gression TR ansformer (SepRETR), to predict separation lines from table images directly. To make the two-stage DETR framework work efficiently and effectively for the separation line prediction task, we propose two improvements: 1) A prior-enhanced matching strategy to solve the slow convergence issue of DETR; 2) A new cross attention module to sample features from a high-resolution convolutional feature map directly so that high localization accuracy is achieved with low computational cost. After separation line prediction, a simple relation network based cell merging module is used to recover spanning cells. With these new techniques, our TSRFormer achieves state-of-the-art performance on several benchmark datasets, including SciTSR, PubTabNet and WTW. Furthermore, we have validated the robustness of our approach to tables with complex structures, borderless cells, large blank spaces, empty or spanning cells as well as distorted or even curved shapes on a more challenging real-world in-house dataset.
Image Quality Assessment: From Mean Opinion Score to Opinion Score Distribution

Recently, many methods have been proposed to predict the image quality which is generally described by the mean opinion score (MOS) of all subjective ratings given to an image. However, few efforts focus on predicting the opinion score distribution of the image quality ratings. In fact, the opinion score distribution reflecting subjective diversity, uncertainty, etc., can provide more subjective information about the image quality than a single MOS, which is worthy of in-depth study. In this paper, we propose a convolutional neural network based on fuzzy theory to predict the opinion score distribution of image quality. The proposed method consists of three main steps: feature extraction, feature fuzzification and fuzzy transfer. Specifically, we first use the pre-trained VGG16 without fully-connected layers to extract image features. Then, the extracted features are fuzzified by fuzzy theory, which is used to model epistemic uncertainty in the process of feature extraction. Finally, a fuzzy transfer network is used to predict the opinion score distribution of image quality by learning the mapping from epistemic uncertainty to the uncertainty existing in the image quality ratings. In addition, a new loss function is designed based on the subjective uncertainty of the opinion score distribution. Extensive experimental results prove the superior prediction performance of our proposed method.
Quantum computing algorithms: getting closer to critical problems in computational biology

The recent biotechnological progress has allowed life scientists and physicians to access an unprecedented, massive amount of data at all levels (molecular, supramolecular, cellular and so on) of biological complexity. So far, mostly classical computational efforts have been dedicated to the simulation, prediction or de novo design of biomolecules, in order to improve the understanding of their function or to develop novel therapeutics. At a higher level of complexity, the progress of omics disciplines (genomics, transcriptomics, proteomics and metabolomics) has prompted researchers to develop informatics means to describe and annotate new biomolecules identified with a resolution down to the single cell, but also with a high-throughput speed. Machine learning approaches have been implemented to both the modelling studies and the handling of biomedical data. Quantum computing (QC) approaches hold the promise to resolve, speed up or refine the analysis of a wide range of these computational problems. Here, we review and comment on recently developed QC algorithms for biocomputing, with a particular focus on multi-scale modelling and genomic analyses. Indeed, differently from other computational approaches such as protein structure prediction, these problems have been shown to be adequately mapped onto quantum architectures, the main limit for their immediate use being the number of qubits and decoherence effects in the available quantum machines. Possible advantages over the classical counterparts are highlighted, along with a description of some hybrid classical/quantum approaches, which could be the closest to be realistically applied in biocomputation.