Reviewerly/exHarmony huggingface dataset:
LAVT: Language-Aware Vision Transformer for Referring Image Segmentation

Referring image segmentation is a fundamental vision-language task that aims to segment out an object referred to by a natural language expression from an image. One of the key challenges behind this task is leveraging the referring expression for highlighting relevant positions in the image. A paradigm for tackling this problem is to leverage a powerful vision-language (“cross-madal”) decoder to fuse features independently extracted from a vision encoder and a language encoder. Recent methods have made remarkable advancements in this paradigm by exploiting Transformers as cross-modal decoders, concurrent to the Transformer's overwhelming success in many other vision-language tasks. Adopting a different approach in this work, we show that significantly better cross-modal alignments can be achieved through the early fusion of linguistic and visual features in intermediate layers of a vision Transformer encoder network. By conducting cross-modal feature fusion in the visual feature encoding stage, we can leverage the well-proven correlation modeling power of a Transformer encoder for excavating helpful multi-modal context. This way, accurate segmentation results are readily harvested with a light-weight mask predictor. Without bells and whistles, our method surpasses the previous state-of-the-art methods on Ref CoCo, RefCOCO+, and G-Ref by large margins.
Attacking Deep Reinforcement Learning With Decoupled Adversarial Policy

While Deep Reinforcement Learning (DRL) has achieved outstanding performance in extensive applications, exploiting its vulnerability with adversarial attacks is essential towards building robust DRL systems. In this work, we aim to propose a novel Decoupled Adversarial Policy (DAP) for attacking the DRL mechanism, whereas the adversarial agent can decompose the adversarial policy into two separate sub-policies: 1) the switch policy which determines if an attacker should launch the attack, and 2) the lure policy which determines the action an attacker induces the victim to take. If the adversarial agent samples an injection action from the switch policy, the attacker can query the pre-constructed database for universal perturbation in the real-time manner, misleading the victim to take the induced action sampled from the lure policy. To train the adversarial agent to learn DAP, we utilize those samples wherein both of the sub-actions from DAP are not restricted by each other or by the external constraint, but can actually affect the attacker’s behaviors. Therefore, we propose trajectory clipping and padding in data pruning, and Decoupled Proximal Policy Optimization (DPPO) in optimizing. Extensive experiments on different Atari games demonstrate the effectiveness of our proposed method. In addition, it can simultaneously implement the real-time and few-steps attack, which outperforms the existing counterparts.
Sentiment analysis of twitter data related to Rinca Island development using Doc2Vec and SVM and logistic regression as classifier

Development on Rinca Island by the Indonesian Government has received a lot of reaction from the community. Masses expressed their opinion through social media, especially Twitter regarding the matter. The research was conducted to analyze the public’s sentiment about this development which was divided into three categories: pro, contra, and neutral. There are two Doc2Vec models used in this research, the distributed model, and the distributed bag of words, and using support vector machines and logistic regression as classifiers. Each combination of the models and classifier has an accuracy rate above 75% and shows that almost all are against the development of Rinca Island.
mSODANet: A network for multi-scale object detection in aerial images using hierarchical dilated convolutions

The object detection in aerial images is one of the most commonly used tasks in the wide-range of computer vision applications. However, the object detection is more challenging due to the following issues: (a) the pixel occupancy vary among the different scales of objects, (b) the distribution of objects is not uniform in aerial images, (c) the appearance of an object varies with different view-points and illumination conditions, and (d) the number of objects, even though they belong to same type, vary across the images. To address these issues, we propose a novel network for multi-scale object detection in aerial images using hierarchical dilated convolutions, called as mSODANet. In particular, we probe hierarchical dilated network using parallel dilated convolutions to learn the contextual information of different types of objects at multiple scales and multiple field-of-views. The introduced hierarchical dilated network captures the visual information of aerial image more effectively and enhances the detection capability of the model. Further, the extensive experiments conducted on three challenging publicly available datasets, i.e., Visdrone2019, DOTA (OBB & HBB), NWPU VHR-10, demonstrate the effectiveness of the proposed mSODANet and achieve the state-of-the-art performance on all three datasets.
Key factors in digital literacy in learning and education: a systematic literature review using text mining

Abstract This research aims at providing an overview of the research field of digital literacy into learning and education. Using text mining, it reviews 1037 research articles published on the topic between 2000 and 2020. This review reveals that there is a plurality of terms associated with digital literacy. Moreover, our research identifies six key factors that define the literature, which are information literacy, developing digital literacy, digital learning, ICT, social media, and twenty-first century digital skills. These factors can be grouped into three main streams, which are 1) digital literacy, 2) digital learning and 3) twenty-first century digital skills. These three streams are supported by informational and technological foundations. These results provide research avenues and offer a framework for digital literacy in education.
Graph Contrastive Clustering

Recently, some contrastive learning methods have been proposed to simultaneously learn representations and clustering assignments, achieving significant improvements. However, these methods do not take the category information and clustering objective into consideration, thus the learned representations are not optimal for clustering and the performance might be limited. Towards this issue, we first propose a novel graph contrastive learning framework, and then apply it to the clustering task, resulting in the Graph Constrastive Clustering (GCC) method. Different from basic contrastive clustering that only assumes an image and its augmentation should share similar representation and clustering assignments, we lift the instance-level consistency to the cluster-level consistency with the assumption that samples in one cluster and their augmentations should all be similar. Specifically, on the one hand, we propose the graph Laplacian based contrastive loss to learn more discriminative and clustering-friendly features. On the other hand, we propose a novel graph-based contrastive learning strategy to learn more compact clustering assignments. Both of them incorporate the latent category information to reduce the intra-cluster variance as well as increase the inter-cluster variance. Experiments on six commonly used datasets demonstrate the superiority of our proposed approach over the state-of-the-art methods. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
An empirical study of handcrafted and dense feature extraction techniques for lung and colon cancer classification from histopathological images

According to a 2020 WHO report, cancer is one of the main causes of deaths worldwide. Among these deaths, lung and colon cancer collectively responsible for nearly 2.735 million deaths. So, detection and classification of lung and colon cancer is one of the utmost priority research areas in the field of biomedical health informatics. In this article, comparative analysis of two feature extraction methodologies has been presented for lung and colon cancer classification. In one approach, six handcrafted features extraction techniques based on colour, texture, shape and structure are presented. Gradient Boosting (GB), SVM-RBF, Multilayer Perceptron (MLP) and Random Forest (RF) classifiers with handcrafted features are trained and tested for lung and colon cancer classification. In another approach, using the notion of transfer learning, seven deep learning frameworks for deep feature extraction from lung and colon cancer histopathological images are presented. The extracted deep features (as input attributes) are applied into conventional GB, SVM-RBF, MLP and RF classifiers for lung and colon cancer classification. However, in contrast to handcrafted features a significant improvement in classifiers performance is observed with features extracted by deep CNN networks. It has been found that the proposed technique obtained excellent results in terms of accuracy, precision, recall, F1 score and ROC-AUC. The RF classifier with DenseNet-121 extracted deep features can identify the lung and colon cancer tissue with an accuracy and recall of 98.60%, precision of 98.63%, F1 score of 0.985 and ROC-AUC of 01.
OadTR: Online Action Detection with Transformers

Most recent approaches for online action detection tend to apply Recurrent Neural Network (RNN) to capture long-range temporal structure. However, RNN suffers from non-parallelism and gradient vanishing, hence it is hard to be optimized. In this paper, we propose a new encoder-decoder framework based on Transformers, named OadTR, to tackle these problems. The encoder attached with a task token aims to capture the relationships and global inter-actions between historical observations. The decoder extracts auxiliary information by aggregating anticipated future clip representations. Therefore, OadTR can recognize current actions by encoding historical information and predicting future context simultaneously. We extensively evaluate the proposed OadTR on three challenging datasets: HDD, TVSeries, and THUMOS14. The experimental results show that OadTR achieves higher training and inference speeds than current RNN based approaches, and significantly outperforms the state-of-the-art methods in terms of both mAP and mcAP. Code is available at https://github.com/wangxiang1230/OadTR.
Towards Flexible Blind JPEG Artifacts Removal

Training a single deep blind model to handle different quality factors for JPEG image artifacts removal has been attracting considerable attention due to its convenience for practical usage. However, existing deep blind methods usually directly reconstruct the image without predicting the quality factor, thus lacking the flexibility to control the output as the non-blind methods. To remedy this problem, in this paper, we propose a flexible blind convolutional neural network, namely FBCNN, that can predict the adjustable quality factor to control the trade-off between artifacts removal and details preservation. Specifically, FBCNN decouples the quality factor from the JPEG image via a decoupler module and then embeds the predicted quality factor into the subsequent reconstructor module through a quality factor attention block for flexible control. Besides, We find existing methods are prone to fail on non-aligned double JPEG images even with only one pixel shift, and we thus propose a double JPEG degradation model to augment the training data. Extensive experiments on single JPEG images, more general double JPEG images and real-world JPEG images demonstrate that our proposed FBCNN achieves favorable performance against state-of-the-art methods in terms of both quantitative metrics and visual quality.
A real time face mask detection system using convolutional neural network

In current times, after the rapid expansion and spread of the COVID-19 outbreak globally, people have experienced severe disruption to their daily lives. One idea to manage the outbreak is to enforce people wear a face mask in public places. Therefore, automated and efficient face detection methods are essential for such enforcement. In this paper, a face mask detection model for static and real time videos has been presented which classifies the images as "with mask" and "without mask". The model is trained and evaluated using the Kaggle data-set. The gathered data-set comprises approximately about 4,000 pictures and attained a performance accuracy rate of 98%. The proposed model is computationally efficient and precise as compared to DenseNet-121, MobileNet-V2, VGG-19, and Inception-V3. This work can be utilized as a digitized scanning tool in schools, hospitals, banks, and airports, and many other public or commercial locations.
On technical trading and social media indicators for cryptocurrency price classification through deep learning

Predicting the prices of cryptocurrencies is a notoriously challenging task due to high volatility and new mechanisms characterising the crypto markets. In this work, we focus on the two major cryptocurrencies for market capitalisation at the time of the study, Ethereum and Bitcoin, for the period 2017–2020. We present a comprehensive analysis of the predictability of price movements comparing four different deep learning algorithms (Multi Layers Perceptron (MLP), Convolutional Neural Network (CNN), Long Short Term Memory (LSTM) neural network and Attention Long Short Term Memory (ALSTM)). We use three classes of features, considering a combination of technical (e.g. opening and closing prices), trading (e.g. moving averages) and social (e.g. users’ sentiment) indicators as input to our classification algorithm. We compare a restricted model composed of technical indicators only, and an unrestricted model including technical, trading and social media indicators. We found an increase in accuracy for the daily classification task from a range of 51%–55% for the restricted model to 67%–84% for the unrestricted one. This study demonstrates that including both trading and social media indicators yields a significant improvement in the prediction and accuracy consistently across all algorithms.
Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network

Research on optical computing has recently attracted significant attention due to the transformative advances in machine learning. Among different approaches, diffractive optical networks composed of spatially-engineered transmissive surfaces have been demonstrated for all-optical statistical inference and performing arbitrary linear transformations using passive, free-space optical layers. Here, we introduce a polarization multiplexed diffractive processor to all-optically perform multiple, arbitrarily-selected linear transformations through a single diffractive network trained using deep learning. In this framework, an array of pre-selected linear polarizers is positioned between trainable transmissive diffractive materials that are isotropic, and different target linear transformations (complex-valued) are uniquely assigned to different combinations of input/output polarization states. The transmission layers of this polarization multiplexed diffractive network are trained and optimized via deep learning and error-backpropagation by using thousands of examples of the input/output fields corresponding to each one of the complex-valued linear transformations assigned to different input/output polarization combinations. Our results and analysis reveal that a single diffractive network can successfully approximate and all-optically implement a group of arbitrarily-selected target transformations with a negligible error when the number of trainable diffractive features/neurons (N) approaches N_p x N_i x N_o, where N_i and N_o represent the number of pixels at the input and output fields-of-view, respectively, and N_p refers to the number of unique linear transformations assigned to different input/output polarization combinations. This polarization-multiplexed all-optical diffractive processor can find various applications in optical computing and polarization-based machine vision tasks.
SGUIE-Net: Semantic Attention Guided Underwater Image Enhancement With Multi-Scale Perception

Due to the wavelength-dependent light attenuation, refraction and scattering, underwater images usually suffer from color distortion and blurred details. However, due to the limited number of paired underwater images with undistorted images as reference, training deep enhancement models for diverse degradation types is quite difficult. To boost the performance of data-driven approaches, it is essential to establish more effective learning mechanisms that mine richer supervised information from limited training sample resources. In this paper, we propose a novel underwater image enhancement network, called SGUIE-Net, in which we introduce semantic information as high-level guidance via region-wise enhancement feature learning. Accordingly, we propose semantic region-wise enhancement module to better learn local enhancement features for semantic regions with multi-scale perception. After using them as complementary features and feeding them to the main branch, which extracts the global enhancement features on the original image scale, the fused features bring semantically consistent and visually superior enhancements. Extensive experiments on the publicly available datasets and our proposed dataset demonstrate the impressive performance of SGUIE-Net. The code and proposed dataset are available at <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><uri>https://trentqq.github.io/SGUIE-Net.html</uri></monospace> .
Digital Transformation and SMART-The Analytics factor

E-government has been a growing trend today, reflecting the increased adoption of technological innovation by several countries. For instance, a report from The United Nations Department of Economics and Social Affairs shows that the United Arab Emirates (UAE) is at the vanguard among nations that have a reliable e-government system. Under a newly defined SMART in e-government, this study discusses the essence of implementation of Big Data Analytics (BDA) in most of the cities in the UAE, coupled with a few ‘developed’ nations. Furthermore, this study also examines the relationship between success and innovation of BDA, along with the technical expertise of employees, and access to high-quality infrastructure put together. The recommendations offered in relation to the results may be applied within Dubai and other cities. Notably, BDA has been used interchangeably with Analytics within SMART in this study.
Supersingular Isogeny Diffie-Hellman Key Exchange on 64-Bit ARM

We present an efficient implementation of the supersingular isogeny Diffie-Hellman (SIDH) key exchange protocol on 64-bit ARMv8 processors for 125and 160-bit post-quantum security levels. We analyze the use of both affine and projective SIDH formulas and provide a comprehensive analysis of both approaches based on the inversion-to-multiplication ratio. Implementation results show that regardless of security concerns, affine SIDH is competitive with the projective coordinates implementation, and even outperforms projective implementation in the final round of SIDH; however, projective SIDH shows better overall performance for the whole key exchange protocol. Notably, over larger finite fields, using optimized field multiplication leads to the much better performance of projective compared to affine formulas. We integrate our optimized software into the open quantum-safe OpenSSL library and compare our software with other available post-quantum primitives. The benchmark results on ARMv8 demonstrate speedup of up to 5X over the generic version of SIDH implementation which is available inside the OQS library for the same quantum security level. We observe that our highly-optimized implementation still suffers from a large number of operations for computing isogenies of elliptic curves. However, in terms of communication overhead, supersingular isogeny-based cryptosystem provides significantly smaller key size compared to its counterparts.
Image Dynamics-Based Visual Servoing for Quadrotors Tracking a Target With a Nonlinear Trajectory Observer

In this correspondence paper, an image dynamics-based visual servoing for quadrotors is proposed to realize stable hovering and tracking. Four perspective image moments are adopted as visual features to control all the independent degrees of freedom of a quadrotor. The complicated interaction matrix is simplified by projecting original image to virtual image plane. On this basis, the dynamics of the system is determined by considering the dynamics of image features and the quadrotor simultaneously. Backstepping controllers are then designed to stabilize the visual servoing system of the quadrotor. In reality, it is unrealistic to have exact prior knowledge about the trajectory parameters of an unpredictable moving target. To solve this problem, a trajectory observer based on nonlinear tracking-differentiator to estimate trajectory parameters of the target is firstly integrated into the quadrotor with image dynamics, which guarantees a satisfactory performance. The effectiveness of the proposed approach is verified by simulations.
Designated-verifier proof of assets for bitcoin exchange using elliptic curve cryptography

Based on the core technique of blockchain, bitcoin is designed for the first time. Bitcoin is a digital currency and a payment system. The blockchain is a digital ledger that records every bitcoin transaction that has ever occurred. The users’ privacy is preserved in the bitcoin exchange by using the blockchain. In some application scenarios, it is important to show the buyer’s assets strength in order to avoid the troublemakers. At the same time, it is also necessary to preserve the buyer’s assets privacy. In this paper, we propose the novel concept of DV-PoA (designated-verifier proof of assets) for bitcoin exchange. Since bitcoin exchange’s signature takes use of the elliptic curve cryptography, we design the first concrete DV-PoA scheme by using elliptic curve cryptography in order to be consistent with it. Then, we prove the security of the proposed DV-PoA scheme. After that, we analyze its efficiency from the two cases: theory and implementation. Our analysis shows that the designed DV-PoA scheme is provably secure and efficient.