Reviewerly/exHarmony huggingface dataset:
The Best Protection is Attack: Fooling Scene Text Recognition With Minimal Pixels

Scene text recognition (STR) has witnessed tremendous progress in the era of deep learning, but it also raises concerns about privacy infringement as scene texts usually contain valuable or sensitive information. Previous works in privacy protection of scene texts mainly focus on masking out the texts from the image/video. In this work, we learn from the idea of adversarial examples and use minimal pixel perturbation to protect the privacy of text information. Although there are well-established attacking methods on non-sequential vision tasks (e.g., classification), the attack on sequential tasks (e.g., scene text recognition) has not received sufficient attention yet. Moreover, existing works mainly focus on the white-box setting, which requires complete knowledge of the target model (e.g., architecture, parameters, or gradients). These requirements limit the scope of applications for the white-box adversarial attack. Therefore, we propose a novel black-box attacking approach for the STR models, only requiring prior knowledge of the model output. Besides, instead of disturbing most pixels as in existing STR attack methods, our proposed approach only manipulates a few pixels, meaning the perturbation is more inconspicuous. To determine the location and value of the manipulated pixels, we also provide an efficient Adaptive-Discrete Differential Evolution (AD <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^{2}\text{E}$ </tex-math></inline-formula> ) by narrowing down the continuous searching space to a discrete space. It can greatly reduce the queries to the target model. Experiments on several real-world benchmarks show the effectiveness of our proposed approach. Especially, when attacking the commercial STR engine, Baidu-OCR, our method achieves higher attack success rates by a large margin than existing approaches. Our work establishes an important step towards using the black-box adversarial attack with minimal pixels to protect the privacy of text information from being easily obtained by STR models.
Mashup-Oriented Web API Recommendation Via Full-Text Semantic Mining of Developer Requirements

API recommendation differs from conventional recommendation systems in that it integrates the characteristics of development requirements, mashup, and API, where there are two primary challenges, i.e., how to effectively mine the personalized development requirements of developers and the sparseness of interaction data. This research proposes a novel model framework, called Bayesian Probabilistic Matrix Factorization Model with Text Similarity and Adversarial Training (SAMF), to address these two issues. Utilizing natural language processing technology to extract the full-text semantics of requirements documents, mashup descriptions, and API descriptions, fully mining personalized development requirements, and contextual information of mashups and APIs, and calculating text similarity, is our fundamental idea. Simultaneously, the collaborative filtering method is used to mine user needs, mashup, and API information from historical data, and the obtained text similarity is added to the training as auxiliary information. Furthermore, adversarial training is further incorporated to supplement data in order to minimize data sparsity and enhance model robustness and generalization. To assess the performance of SAMF, we run thorough experiments that illustrate the efficacy of each module of the model and explain the influence of hyperparameter settings. Specifically, compared to the baseline, the experimental findings demonstrate that SAMF can basically achieve better performance.
Joint Sparse Representation-based Single Image Super-Resolution for Remote Sensing Applications

Sparse representation-based single image super-resolution (SISR) methods use a coupled overcomplete dictionary trained from high-resolution (HR) images/ image patches. Since, remote sensing (RS) satellites capture images of large areas, these images usually have poor spatial resolution and obtaining an effective dictionary as such would be very challenging. Moreover, traditional patch-based sparse representation (PSR) models for reconstruction tend to give unstable sparse solution and produce visual artefact in the recovered images. To mitigate these problems, in this paper, we have proposed an adaptive joint sparse representation (JSR)-based SISR method that is dependent only on the input low-resolution (LR) image for dictionary training and sparse reconstruction. The new model combines patch-based local sparsity and group sparse representation (GSR)-based non-local sparsity in a single framework, which helps in stabilizing the sparse solution and improve the SISR results. Experimental results are evaluated both visually and quantitatively for several RGB and multispectral RS datasets, where the proposed method shows improvements in PSNR by 1–4 dB and 2–3 dB over the state-of-the-art sparse representation- and deep learning-based SR methods, respectively. Land cover classification applied on the super-resolved images further validate the advantages of the proposed method. Finally, for practical RS applications, we have performed parallel implementation in general purpose graphics processing units (GPGPU) and achieved significant speed-ups (30-40×) in the execution time.
Permissionless Blockchain Systems as Pseudo-Random Number Generators for Decentralized Consensus

Consensus algorithms that function in permissionless blockchain systems must randomly select new block proposers in a decentralised environment. Our contribution is a new blockchain consensus algorithm called Proof-of-Publicly Verifiable Randomness (PoPVR). It may be used in blockchain design to make permissionless blockchain systems function as pseudo-random number generators and to use the results for decentralised consensus. The method employs verifiable random functions to embed pseudo-random number seeds in the blockchain that are confidential, tamper-resistant, unpredictable, collision-resistant, and publicly verifiable. PoPVR does not require large-scale computation, as is the case with Proof-of-Work and is not vulnerable to the exclusion of less wealthy stakeholders from the consensus process inherent in stake-based alternatives. It aims to promote fairness of participation in the consensus process by all participants and functions transparently using only open-source algorithms. PoPVR may also be useful in blockchain systems where asset values cannot be directly compared, for example, logistical systems, intellectual property records and the direct trading of commodities and services. PoPVR scales well with complexity linear in the number of transactions per block.
OPEMI: Online Performance Evaluation Metrics Index for Deep Learning-Based Autonomous Vehicles

Vision-based autonomous driving is rapidly growing.There are, however, presently no agreed-upon metrics for assessing how well deep neural network (DNN) models perform in driving.To compare novel approaches and architectures to existing ones, some researchers employed a mean error between labeled and predicted values in a test dataset and others presented a new metric that is designed to match their requirements.The discrepancy in the usage of various performance metrics and lack of objective metrics to judge the driving performance were our primary motives for developing a feasible solution.In this study, we propose online performance evaluation metrics index (OPEMI), an integrated metric that can evaluate the driving capabilities of autonomous driving models in various driving scenarios.To evaluate driving performance precisely and objectively, OPEMI incorporates several variables, including driving control stability, driving trajectory stability, journey duration, travel distance, success rate, and speed.To demonstrate the validity of OPEMI, we first confirmed that the prediction accuracy has a weak correlation with driving performance.Then, we have discussed the constraints in the existing driving performance metrics in certain circumstances, and their failure to assess the driving models.Finally, we conducted experiments with four popular DNN models and two in-house models under three different driving scenarios (generic, urban, and racing).The results show that the proposed evaluation metric, OPEMI, realistically displays driving performance and demonstrates its validity in various driving scenarios.
ADQ—Anomaly Detection and Quantification From Delayed Neutron Monitoring Data of Nuclear Power Plants

Identifying anomalies and quantifying their severity from delayed neutron monitoring data are of great significance to ascertain the reason of anomalous behavior and for predictive maintenance of nuclear power plants (NPPs). An existing anomaly detection technique is primarily aimed to identify anomalies and does not consider quantification of anomalies. In this work, an anomaly detection and quantification (ADQ) technique has been proposed. It detects anomalies based on the fact that abnormal behavior occurs rarely in the real world as opposed to normal behavior. Thereby, the dissimilarity of an anomalous data instance happens to be large from a normal data instance in comparison with dissimilarity between normal data instances and anomalies in itself. This has been exploited by the proposed technique by employing the concept of candidate anomaly pair. It computes a quantified value for severity of given anomaly as intensity in reference to underlying normal behavior, representative of which is extracted from data itself. The proposed technique has been applied on delayed neutron monitoring data of 28 sensors for an NPP, which was provided by Nuclear Power Corporation of India Ltd. (NPCIL), Mumbai, India. The detected anomalies are categorized as point and interval anomalies, and sensors are ranked based on them. The performance of the proposed technique has been found highly promising on validating results with domain experts. The proposed technique has outperformed other three state-of-the-art techniques in terms of anomaly detection rate (ADR), false alarm rate (FAR), and precision.
Unsupervised Maritime Vessel Re-Identification With Multi-Level Contrastive Learning

Re-identification (re-ID) of maritime vessels plays an important role in marine surveillance, but remains highly unexplored due to the lack of large-scale annotated datasets. In vessel re-ID, contrastive methods are supposed to learn discriminative representation from unlabeled vessel images in an unsupervised manner. However, directly introducing classical instance-level contrastive methods to maritime vessel re-ID suffers from the difficulty of finding vessel images with the same pseudo label as positive images, which potentially leads to inefficient training and unsatisfactory performance. This paper proposes a simple but effective method to solve such a hard positive problem. Our method takes all images in an intra-batch cluster as positives and excludes them from the set of negative samples when computing instance-level contrastive loss. Based on this strategy, we construct a multi-level contrastive learning (MCL) framework for vessel re-ID trained with the specifically designed intra-batch cluster-level contrastive loss along with the instance-level one. Experiments on a newly proposed dataset consisting of 1,248 vessel identities show that MCL achieves the state-of-the-art performance compared with other unsupervised methods.
Intermediary-Guided Bidirectional Spatial–Temporal Aggregation Network for Video-Based Visible-Infrared Person Re-Identification

This work focuses on the task of Video-based Visible-Infrared Person Re-Identification, a promising technique for achieving 24-hour surveillance systems. Two main issues in this field are modality discrepancy mitigating and spatial–temporal information mining. In this work, we propose a novel method, named Intermediary-guided Bidirectional spatial–temporal Aggregation Network (IBAN), to address both issues at once. Specifically, IBAN is designed to learn modality-irrelevant features by leveraging the anaglyph data of pedestrian images to serve as the intermediary. Furthermore, a bidirectional spatial–temporal aggregation module is introduced to exploit the spatial–temporal information of video data, while mitigating the impact of noisy image frames. Finally, we design an Easy-sample-based loss to guide the final embedding space and further improve the model’s generalization performance. Extensive experiments on Video-based Visible-Infrared benchmarks show that IBAN achieves promising results and outperforms the state-of-the-art ReID methods by a large margin, improving the rank-1/mAP by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.29\%/3.46\%$ </tex-math></inline-formula> at the Infrared to Visible situation, and by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5.04\%/3.27\%$ </tex-math></inline-formula> at the Visible to Infrared situation. The source code of the proposed method will be released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/lhf12278/IBAN</uri> .
Virtual Reality for Software Engineering Presentations

Due to the impact of the pandemic situation, applying online learning methods become an immediate response to tackle the difficulties in teaching and learning, including software engineering courses. Online video meeting platforms (e.g., MS Teams, Webex) are popularly adopted as a medium between instructors and students to conduct online learning classes and they have been modified to provide functions supporting remote teaching and learning activities such as the breakout rooms for conducting group activities. However, maintaining student engagement is still a challenging problem in online learning. Especially, drawing students’ attention and enhancing their experience during in-class activities (e.g., project presentations, group discussions) is critical to achieving of activities’ objective. Virtual Reality (VR) has been considered to be a potential answer to this online teaching and learning enhancement. This study evaluates the benefit of adopting VR in software engineering class presentation activities. The evaluation result from 3 courses shows that VR improves the online learning and presentation experience by offering visual attractions and presence to students.
Efficient Reinforcement Learning with Generalized-Reactivity Specifications

Reinforcement learning has been used to solve sequential decision-making problems in intelligent systems. However, current RL approaches suffer from slow convergence and reward sparsity, and its reward mechanism is challenging to deal with complex task specifications. As one of the software engineering practices, temporal logic can describe nonMarkovian task specifications, the synthesized strategy of which could be used as a priori knowledge to train the agents to interact with the environment efficiently. This paper considers the intelligent agent reacts to the environment with a high-level reactive temporal logic specification called Generalized Reactivity of rank 1 (GR(1)). We first use the synthesized strategy of GR(1) to construct the Markov Decision Process with a potentialbased reward machine, which integrates the environment with high-level reactive temporal specifications. Then we developed a topological-sort-based reward shaping approach to calculate the potential functions of the reward machine, based on which we used Q-learning to train the agents. Experiments on multitask learning show that the proposed approach outperforms the state-of-art algorithms in learning rate and optimal rewards. Also, compared with the value-iteration-based reward shaping approaches, our topological-sort-based reward shaping approach could handle the cases where the synthesized strategies are in the form of directed cyclic graphs.
Covert Communication on IPFS using Secured Filecoin-Slate Blockchain

Interplanetary File System (IPFS) is a widely used distributed file system on a peer to peer network to store, share data and handle content management. There is often lack of availability, security and concealment on IPFS, resulting in compromise of confidential information during covert communication. The data in IPFS could be dropped and nodes compressed which results in unavailability of data. IPFS does not encrypt files and users who uploaded the sensitive data can be traced. To overcome these issues a Filecoin-Slate Blockchain (FSB) is adopted in this work. FSB is built on top of content addresser IPFS, a persistent storage ensuring data availability and transaction cost free for users who are uploading data into the FSB. This work proposes a robust and efficient implementation of FSB integrating plain text cryptography and Least Significant Bit (LSB) image steganography. Cryptography along with steganography ensures a strong two-tiered data protection, and adapting FSB guarantees file availability and integrity using inverse embedding technique. The proposed work is tested with nine high quality images and the results validate to maintain confidentiality, availability and integrity of data, for covert communication over FSB.
Artificial Bee Colony Optimization based Optimal Convolutional Neural Network Architecture Design

Convolutional neural networks (CNNs) are broadly used to solve various computer vision tasks. However, designing optimal CNN architecture for solving a particular task is a trial- and-error process and thus requires domain-specific expertise. Recently, several meta-heuristic strategies have been developed to solve the design challenges of CNN models. Here, we propose an approach to designing optimal CNN architecture using the artificial bee colony (ABC) algorithm and named as ABC-CNN. A variable-length encoding scheme and refinement strategy are proposed to evolve CNN architectures with the ABC algorithm. The proposed approach is evaluated on three well-known image classification datasets. Experimental results demonstrate the effectiveness of the proposed ABC-CNN algorithm compared with 11 state-of-the-art models, including manual and meta-heuristic based CNN models.
An Intelligent System for Face Mask Recognition using Open Computer Vision and Modified YOLO Algorithm

Face Mask detection is not just one of the most researched problems in Computer Vision, but it’s also a critical task in a wide range of applications. Face mask recognition is a difficult problem with many applications. Most automated face mask detection analysis algorithms on the market today are designed to recognize whether a human has weared mask or not. As a result of deep learning, there is now face mask recognition has made significant progress. Three algorithms will be used in the method to recognize face mask. Few algorithms are used like You Only Look Once (YOLO), Convolution Neural Networks (CNN), and Recurrent Neural Networks (RNN). A brief description on above said algorithm is covered in this research paper. These algorithms assistances in public places to recognize the face mask or not wearing a face mask and then provide the results of it.
An Analysis of Performance Evaluation Metrics for Voice Conversion Models

The process of transforming a source speaker’s vocal style or vocal feature to that of a target speaker while keeping the linguistic information of the source speaker unchanged is known as voice conversion (VC). Deep Learning algorithms have become widely used in VC processes in recent years. As a result of advances in deep learning algorithms, especially in deep generative models, they have been used for a number of applications such as automatic movie dubbing, singing voice conversion, military applications, etc. Despite of the immense improvements of deep learning based VC models, the objective analysis of the synthesized speech samples still exists as a major challenge. Previous studies in this domain have primarily focused on the developments of VC models. However, there is scope for the analysis of objective evaluation metrics utilised for performance analysis of VC models. As a result in this work, the extensive study of the VC processes, the speech features used in VC, and the objective evaluation metrics of VC models are explored in detail.
Mitigating Cold-Start Delay using Warm-Start Containers in Serverless Platform

Serverless computing is an emerging cloud technology which is preferred due to its quick response time and scale-to-zero features, where the code can run closer to the users to minimize the latency, and users are charged only for the actual amount of resource consumption. The unit of execution in serverless computing is the function that can independently run on a separate container. A container is required to configure and set up for each time a request is issued for function invocation. This requires some time before the actual execution start is known as a cold start. Cold-start delay and the frequency of cold-start highly affect the performance of serverless execution which needs to be addressed. Most of the researchers are trying to address the problem by keeping the container warm for a fixed period of time known as an idle container window, which may not be the right solution for mitigating cold-start delay. If the idle container window is too long it may lead to more resource consumption, which will contradict the scale to zero feature of serverless computing. On the other hand, if the window length is too short, it may not able to handle a large number of invocation requests. In this paper, an adaptive model is proposed to predict the length of idle container window and the requirement for a number of pre-warmed containers in advance. We have used a Deep neural network and LSTM model to capture the previous function invocation patterns, which can be used to pre-determine the length of the idle-container window. The effectiveness of the proposed model is extensively compared with other cold-start mitigation strategies followed in AWS Lambda, Microsoft Azure, Openfaas, and Openwhisk by using four real-world serverless applications.
Analyzing The Effect of Segmentation on PCG Based Biometric System

A new comparative approach to analyzing the effect of segmentation on PCG-based biometrics is proposed here. In this HSCT-11, a dataset is used which contains data that was collected from 206 people. First, PCG signals were pre-processed to remove noise and other artifacts using Discrete wavelet transform (DWT) up to the Fifth level, and after reconstruction of the signal nine-time and frequency domain features were selected to do various experiments using three different types of classifiers and found out that DNN gives a good response. A data set of 40 different person samples that are split into 11,468 samples is used here and an accuracy of 85.87% is achieved using DNN.
A Blockchain Based Credential Verification System using IPFS

Blockchain is one of the most emerging technology in the present IT world. It powers some highly monetized cryptocurrencies like Bitcoin, Ethereum, and many more. The uses of blockchain are not just limited to cryptocurrencies; researchers are working to implement real-world applications using blockchain. Blockchain has shown potential use cases in many domains like IoT, Logistics, Healthcare, E-commerce, Finance, and Security. Blockchain’s fundamental properties, like its decentralized nature, immutable architecture, transparent behaviour, and security, are the main reason behind its immense popularity. In this paper, a blockchain-based educational credential verification system is proposed, independent of credentials format and size, that uses the power of blockchain to provide a secure, decentralized interface to process, store and verify students’ academic credentials. The proposed solution can also deal with any archived credential with any well-known digital format like PDF, DOCX, JPG, PNG, etc.