Reviewerly/exHarmony huggingface dataset:
Dual Self-Guided Attention with Sparse Question Networks for Visual Question Answering

Visual Question Answering (VQA) is multi-task research that requires simultaneous processing of vision and text. Recent research on the VQA models employ a co-attention mechanism to build a model between the context and the image. However, the features of questions and the modeling of the image region force irrelevant information to be calculated in the model, thus affecting the performance. This paper proposes a novel dual self-guided attention with sparse question networks (DSSQN) to address this issue. The aim is to avoid having irrelevant information calculated into the model when modeling the internal dependencies on both the question and image. Simultaneously, it overcomes the coarse interaction between sparse question features and image features. First, the sparse question self-attention (SQSA) unit in the encoder calculates the feature with the highest weight. From the self-attention learning of question words, the question features of larger weights are reserved. Secondly, sparse question features are utilized to guide the focus on image features to obtain fine-grained image features, and to also prevent irrelevant information from being calculated into the model. A dual self-guided attention (DSGA) unit is designed to improve modal interaction between questions and images. Third, the sparse question self-attention of the parameter δ is optimized to select these question-related object regions. Our experiments with VQA 2.0 benchmark datasets demonstrate that DSSQN outperforms the state-of-the-art methods. For example, the accuracy of our proposed model on the test-dev and test-std is 71.03% and 71.37%, respectively. In addition, we show through visualization results that our model can pay more attention to important features than other advanced models. At the same time, we also hope that it can promote the development of VQA in the field of artificial intelligence (AI).
A Quantitative Comparison between Shannon and Tsallis–Havrda–Charvat Entropies Applied to Cancer Outcome Prediction

In this paper, we propose to quantitatively compare loss functions based on parameterized Tsallis-Havrda-Charvat entropy and classical Shannon entropy for the training of a deep network in the case of small datasets which are usually encountered in medical applications. Shannon cross-entropy is widely used as a loss function for most neural networks applied to the segmentation, classification and detection of images. Shannon entropy is a particular case of Tsallis-Havrda-Charvat entropy. In this work, we compare these two entropies through a medical application for predicting recurrence in patients with head-neck and lung cancers after treatment. Based on both CT images and patient information, a multitask deep neural network is proposed to perform a recurrence prediction task using cross-entropy as a loss function and an image reconstruction task. Tsallis-Havrda-Charvat cross-entropy is a parameterized cross-entropy with the parameter α. Shannon entropy is a particular case of Tsallis-Havrda-Charvat entropy for α=1. The influence of this parameter on the final prediction results is studied. In this paper, the experiments are conducted on two datasets including in total 580 patients, of whom 434 suffered from head-neck cancers and 146 from lung cancers. The results show that Tsallis-Havrda-Charvat entropy can achieve better performance in terms of prediction accuracy with some values of α.
Reversible Data Hiding with a New Local Contrast Enhancement Approach

Reversible data hiding schemes hide information into a digital image and simultaneously increase its contrast. The improvements of the different approaches aim to increase the capacity, contrast, and quality of the image. However, recent proposals contrast the image globally and lose local details since they use two common methodologies that may not contribute to obtaining better results. Firstly, to generate vacancies for hiding information, most schemes start with a preprocessing applied to the histogram that may introduce visual distortions and set the maximum hiding rate in advance. Secondly, just a few hiding ranges are selected in the histogram, which means that just limited contrast and capacity may be achieved. To solve these problems, in this paper, a novel approach without preprocessing performs an automatic selection of multiple hiding ranges into the histograms. The selection stage is based on an optimization process, and the iterative-based algorithm increases capacity at embedding execution. Results show that quality and capacity values overcome previous approaches. Additionally, visual results show how greyscale values are better differentiated in the image, revealing details globally and locally.
Passnumbers: An Approach of Graphical Password Authentication Based on Grid Selection

The authentication textual passwords are the most widely used technique. However, this type of legacy authentication is vulnerable to various attacks, such as shoulder-surfing attacks. Hence, graphical password authentication is one of these approaches which has been suggested to overcome the issues related to textual passwords. Nevertheless, the hackers have also developed new techniques that can be finally broken the graphical password, for instance, listening to the transmitted information between the client and the server. In this paper, Passnumbers graphical authentication password is proposed. Passnumbers approach involves two new stages, which are first, using the coordinates of a graphical grid cells-based numbers for entering the password. The second stage is represented by deploying a new technique to encrypt the password based on the image pixels. The performance evaluation reveals that the proposed Passnumbers can provide high resistance against several graphical password attacks including shoulder surfing and eavesdropping attacks. Passnumbers is evaluated using several metrics including security and usability.
Simple and loss-tolerant free-space quantum key distribution using a squeezed laser

We consider a continuous-variable (CV) quantum key distribution (QKD) protocol over free-space channels, which is simpler and more robust than typical CV QKD protocols. It uses a bright laser, squeezed and modulated in the amplitude quadrature, and self-homodyne detection. We consider a scenario, where the line of sight is classically monitored to detect active eavesdroppers, so that we can assume a passive eavesdropper. Under this assumption, we analyze security of the QKD protocol in the composable finite-size regime. Proper modulation of the squeezed laser to the shot-noise level can completely eliminate information leakage to the eavesdropper and also eliminate the turbulence-induced noise of the channel in the amplitude quadrature. Under these conditions, estimation of the eavesdropper's information is no longer required. The protocol is extremely robust to modulation noise and highly loss-tolerant, which makes it a promising candidate for satellite-based continuous-variable quantum communication.
Authentication of NIfTI Neuroimages Using Lifting Wavelet Transform, Arnold Cat Map, Z-Transform, and Hessenberg Decomposition

The technological progress in digital medical imaging has enabled the diagnosis of various ailments, and thus upgraded the global healthcare system. In the era of coronavirus 2019 (COVID-19), telemedicine plays the crucial role of supporting remote medical consultation in rural locations. During the remote consultation, numerous medical images are sent to each radiologist via the Internet. There has been a surge in the number of attacks on digital medical images worldwide, which severely threatens authenticity and ownership. To mitigate the threat, this paper proposes a robust and secure watermarking approach for NIfTI images. Our approach painstakingly incorporates a watermark into the chosen NIfTI image slice, aiming to accurately fit the watermark, while preserving the medical information contained in the slice. Specifically, the original image was converted through the lifting wavelet transform (LWT), realizing excellent modification during insertion. Next, Z-transform was applied over the low-low (LL) band, and the Hessenberg decomposition (HD) was performed on the transformed band, which contains the maximum energy of the image. Afterwards, Arnold Cat map was employed to scramble the watermark, before inserting it into the slice. Simulation results show that our approach strikes a perfect balance between security, imperceptibility, and robustness against various attacks, as suggested by metrics like peak signal-to-noise ratio (PSNR), normalized correlation (NC), structural similarity index measure (SSIM), and universal image quality index Q.
Harmonizing Pathological and Normal Pixels for Pseudo-Healthy Synthesis

Synthesizing a subject-specific pathology-free image from a pathological image is valuable for algorithm development and clinical practice. In recent years, several approaches based on the Generative Adversarial Network (GAN) have achieved promising results in pseudo-healthy synthesis. However, the discriminator (i.e., a classifier) in the GAN cannot accurately identify lesions and further hampers from generating admirable pseudo-healthy images. To address this problem, we present a new type of discriminator, the segmentor, to accurately locate the lesions and improve the visual quality of pseudo-healthy images. Then, we apply the generated images into medical image enhancement and utilize the enhanced results to cope with the low contrast problem existing in medical image segmentation. Furthermore, a reliable metric is proposed by utilizing two attributes of label noise to measure the health of synthetic images. Comprehensive experiments on the T2 modality of BraTS demonstrate that the proposed method substantially outperforms the state-of-the-art methods. The method achieves better performance than the existing methods with only 30% of the training data. The effectiveness of the proposed method is also demonstrated on the LiTS and the T1 modality of BraTS. The code and the pre-trained model of this study are publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Au3C2/Generator-Versus-Segmentor</uri> .
Hierarchical Memory Learning for Fine-Grained Scene Graph Generation

As far as Scene Graph Generation (SGG), coarse and fine predicates mix in the dataset due to the crowd-sourced labeling, and the long-tail problem is also pronounced. Given this tricky situation, many existing SGG methods treat the predicates equally and learn the model under the supervision of mixed-granularity predicates in one stage, leading to relatively coarse predictions. In order to alleviate the negative impact of the suboptimum mixed-granularity annotation and long-tail effect problems, this paper proposes a novel Hierarchical Memory Learning (HML) framework to learn the model from simple to complex, which is similar to the human beings' hierarchical memory learning process. After the autonomous partition of coarse and fine predicates, the model is first trained on the coarse predicates and then learns the fine predicates. In order to realize this hierarchical learning pattern, this paper, for the first time, formulates the HML framework using the new Concept Reconstruction (CR) and Model Reconstruction (MR) constraints. It is worth noticing that the HML framework can be taken as one general optimization strategy to improve various SGG models, and significant improvement can be achieved on the SGG benchmark (i.e., Visual Genome).
Integrating Vectorized Lexical Constraints for Neural Machine Translation

Lexically constrained neural machine translation (NMT), which controls the generation of NMT models with pre-specified constraints, is important in many practical scenarios.Due to the representation gap between discrete constraints and continuous vectors in NMT models, most existing works choose to construct synthetic data or modify the decoding algorithm to impose lexical constraints, treating the NMT model as a black box.In this work, we propose to open this black box by directly integrating the constraints into NMT models.Specifically, we vectorize source and target constraints into continuous keys and values, which can be utilized by the attention modules of NMT models.The proposed integration method is based on the assumption that the correspondence between keys and values in attention modules is naturally suitable for modeling constraint pairs.Experimental results show that our method consistently outperforms several representative baselines on four language pairs, demonstrating the superiority of integrating vectorized lexical constraints.
An effective modular approach for crowd counting in an image using convolutional neural networks

Abstract Abrupt and continuous nature of scale variation in a crowded scene is a challenging task to enhance crowd counting accuracy in an image. Existing crowd counting techniques generally used multi-column or single-column dilated convolution to tackle scale variation due to perspective distortion. However, due to multi-column nature, they obtain identical features, whereas, the standard dilated convolution (SDC) with expanded receptive field size has sparse pixel sampling rate. Due to sparse nature of SDC, it is highly challenging to obtain relevant contextual information. Further, features at multiple scale are not extracted despite some inception-based model is not used (which is cost effective). To mitigate theses drawbacks in SDC, we therefore, propose a hierarchical dense dilated deep pyramid feature extraction through convolution neural network (CNN) for single image crowd counting (HDPF). It comprises of three modules: general feature extraction module (GFEM), deep pyramid feature extraction module (PFEM) and fusion module (FM). The GFEM is responsible to obtain task independent general features. Whereas, PFEM plays a vital role to obtain the relevant contextual information due to dense pixel sampling rate caused by densely connected dense stacked dilated convolutional modules (DSDCs). Further, due to dense connections among DSDCs, the final feature map acquires multi-scale information with expanded receptive field as compared to SDC. Due to dense pyramid nature, it is very effective to propagate the extracted feature from lower dilated convolutional layers (DCLs) to middle and higher DCLs, which result in better estimation accuracy. The FM is used to fuse the incoming features extracted by other modules. The proposed technique is tested through simulations on three well known datasets: Shanghaitech (Part-A), Shanghaitech (Part-B) and Venice. Results justify its relative effectiveness in terms of selected performance.
Intelligent Video Ingestion for Real-time Traffic Monitoring

As an indispensable part of modern critical infrastructures, cameras deployed at strategic places and prime junctions in an intelligent transportation system can help operators in observing traffic flow, identifying any emergency situation, or making decisions regarding road congestion without arriving on the scene. However, these cameras are usually equipped with heterogeneous and turbulent networks, making the real-time smooth playback of traffic monitoring videos with high quality a grand challenge. In this article, we propose a lightweight Deep Reinforcement Learning-based approach, namely, sRC-C (smart bitRate Control with a Continuous action space) , to enhance the quality of real-time traffic monitoring by adjusting the video bitrate adaptively. Distinguished from the existing bitrate adjusting approaches, sRC-C can overcome the bias incurred by deterministic discretization of candidate bitrates by adjusting the video bitrate with more fine-grained control from a continuous action space, thus significantly improving the Quality-of-Service (QoS). With carefully designed state space and neural network model, sRC-C can be implemented on cameras with scarce resources to support real-time live video streaming with low inference time. Extensive experiments show that sRC-C can reduce the frame loss counts and hold time by 24% and 15.5%, respectively, even with comparable bandwidth utilization. Meanwhile, compared to the-state-of-art approaches, sRC-C can improve the QoS by 30.4%.
Fairness Audit of Machine Learning Models with Confidential Computing

Algorithmic discrimination is one of the significant concerns in applying machine learning models to a real-world system. Many researchers have focused on developing fair machine learning algorithms without discrimination based on legally protected attributes. However, the existing research has barely explored various security issues that can occur while evaluating model fairness and verifying fair models. In this study, we propose a fairness audit framework that assesses the fairness of ML algorithms while addressing potential security issues such as data privacy, model secrecy, and trustworthiness. To this end, our proposed framework utilizes confidential computing and builds a chain of trust through enclave attestation primitives combined with public scrutiny and state-of-the-art software-based security techniques, enabling fair ML models to be securely certified and clients to verify a certified one. Our micro-benchmarks on various ML models and real-world datasets show the feasibility of the fairness certification implemented with Intel SGX in practice. In addition, we analyze the impact of data poisoning, which is an additional threat during data collection for fairness auditing. Based on the analysis, we illustrate the theoretical curves of fairness gap and minimal group size and the empirical results of fairness certification on poisoned datasets.
Similarity learning networks for animal individual re-identification: an ecological perspective

The ability for researchers to re-identify animal individuals upon re-encounter is fundamental for the study of population dynamics, community, and behavioural ecology. Animal re-identification is traditionally performed using tagging or DNA sampling, which is laborious, invasive to an animal, and expensive. An alternative approach to re-identify is the use of computer vision in combination with pattern recognition algorithms. Deep learning has accelerated the success when solving pattern recognition in the field of computer vision when high data volume is available; however, conventional deep learning approaches require ample training data for a fixed number of classes. An alternative deep learning paradigm is similarity comparison networks which are trained to identify if two inputs are the same or different. This principle can be applied to images of animal individuals and allows for the re-identification of individuals beyond the original training data. Here, we test the potential and generality of similarity comparison networks for animal re-identification considering five datasets of different species: humans, chimpanzees, humpback whales, fruit flies, and Siberian tigers, each with their own unique set of challenges. We compare 10 similarity comparison networks by testing five well-established network architectures (AlexNet, VGG19, DenseNet201, ResNet152, and InceptionV3) and two different methods to train each of them: contrastive and triplet loss. Models were trained to re-identify individuals and those trained using the triplet loss outperformed contrastive loss for all species. Our work shows that without any species-specific modifications, similarity comparison networks can act as a general purpose animal re-identification system considering individuals from images. Our expectation is that similarity comparison networks are the beginning of a major trend that has the potential to revolutionize the study of population dynamics, community, and behavioural ecology. This work is an extension of a technical report presented at WACV 2020 (IEEE/CVF winter conference on applications of computer vision workshops) catered for an ecological audience.
Virtual-real combination Ritchey-Common interferometry

Large optical flats play a remarkable role in advanced large-aperture optical systems and the testing of the surface shape error is indispensable for the fabrication. The widely adopted Ritchey-Common test for large optical flats will fail without the rigorous test configurations including a large F/# prerequisition and a flat-to-interferometer distance invariance. A virtual-real combination Ritchey-Common interferometry is proposed to avoid the large F/# prerequisition by accurately modelling the optical path in a virtual interferometer. Furthermore, a virtual-real combination iterative algorithm is proposed in this method to break the flat-to-interferometer distance invariance. Measurement experiments for 100 mm and 422 mm aperture flats were performed to demonstrate the feasibility of this method. Compared with a direct testing in a standard Zygo interferometer, the peak to valley (PV) and root mean square (RMS) errors were less than 0.1 λ and 0.01 λ (λ=632.8 nm), respectively, in different Ritchey angles and flat-to-interferometer distances. Further numerical simulations demonstrate that RMS errors for various Zernike aberrations in arbitrary F/# are less than 0.01 λ. This method can break the distance invariance restriction and achieve high accuracy with an arbitrary F/#, thus providing substantial freedom in the design of test configurations to accommodate various test scenarios.
Closed-form model and analysis for the enhancement effect of a rectangular plate in the scattering characteristics of multiphoton quantum radar

A closed-form model of multiphoton quantum radar cross-section (QRCS) in the monostatic scenes is constructed for rectangular flat plates based on quantum interference and uncertainty. The model is justified by the comprehensive analysis of the model parameters in the model building process. Then, we use the model to quantitatively analyze the main lobe enhancement effect of multiphoton QRCS, which means that the more incident photons will enhance the main lobe magnitude of QRCS with other factors being the same. Moreover, we predict that enhancement effects might also exist for the side lobe close to the main lobe. In addition, we present the specific conditions for side lobe enhancement. On this basis, the enhancement angle range is defined to unify the description of the main lobe and side lobe enhancement effects. The influencing factors of the enhancement angle range are clarified. The results exhibit that the angle range of enhancement in multiphoton QRCS fluctuates with the change of target size and incident wavelength. All enhancement effects are exponentially related to the incident photon number. This work brings the description of multiphoton QRCS into the closed-form model analysis stage, which will provide prior information for research in many fields, such as photonic technology, radar technology, and precision metrology.