Reviewerly/exHarmony huggingface dataset:
Robust Adversarial Defence: Use of Auto-inpainting

Adversarial patch attacks have become a primary concern in recent years as they pose a significant threat to the security and reliability of deep neural networks. Modifying benign images by introducing adversarial patches comprising localized adversarial pixels alters the salient features of the image resulting in misclassification. The novelty of our approach is in the use of image inpainting technique as an adversarial defence for rectifying the patch region. Adversarial patch is automatically localized using Fast Score Class Activation Map and superseded by inpainting using Fast Marching Method which efficiently propagates pixel information from the surrounding areas into the patch region. This approach ensures original image’s structural integrity while simultaneously inpainting the adversarial pixels. Moreover, at the time of the attack it is not expected to have prior knowledge about the patch. Therefore, we propose our novel adversarial defence technique in a black-box setting assuming no knowledge about the patch location, shape or its size. Furthermore, we do not rely on re-training our victim model on adversarial examples, indicating its potential usefulness for real-world applications. Our experimental results show that the proposed approach achieves accuracy up to 76.37% on ImageNet100 despite the adversarial patch attack amounting to a considerable improvement of 76.28% points. Moreover, on benign images our approach gives decent accuracy of $$81.11\%$$ thereby suggesting that our defence pipeline is applicable irrespective of whether the input image is adversarial or clean.
A Comparison of Neural Network-Based Super-Resolution Models on 3D Rendered Images

Super-resolution is an area of Computer Vision comprising various techniques to recover a high-resolution image from a low-resolution counterpart. These techniques can also be used to enhance a low-resolution input image without a native high-resolution original. Single Image Super-Resolution (SISR) techniques aim to do this in a picture-by-picture fashion. In recent years, deep learning models have achieved the best performance, using neural networks to find a mapping between an input low-resolution image and its high-resolution analogous. This work will compare three approaches based on some of the most notable works in neural-network based super-resolution: SRCNN, FSRCNN, and ESRGAN. These methods will be used to enhance 3D computer-generated low-resolution pictures obtained from popular video games and will be evaluated with respect to the quality of the enhanced picture and the required computation time. From our study, we can attest to the superiority of neural network-based methods on the SISR problem and the benefits of taking a perceptual approach when the quality of the resulting images.
The Interaction and Convergence of the Internet of Things and Artificial Intelligence

For any development in the present digital era, artificial intelligence (AI) and the Internet of Things (IoT) are two prominent technologies. Even though both platforms are strong in their domain—AI is a source of knowledge and the IoT is the unique feature that enables cross-communication—this combination becomes a trigger for various intelligent connections. Wearable technology uses for the continuous monitoring and reporting of user behaviors and preferences has resulted in many important uses in the healthcare business. While the IoT facilitates the connection of many hardware platforms, AI transforms the gathered data into actionable insights that can contribute significantly to making people healthy.
Augmented Reality for Indoor Localization and Navigation: The Case of UNIPI AR Experience

Indoor localization and navigation is a common problem mostly in large buildings where multiple floors, rooms and corridors may generate a struggling experience for the visitor. The complex internal environment, the composite architectural designs and the interference of objects and people in crowded areas, make the adoption of generic solutions hard to implement and apply, while their performance and the provided user experience do not meet the typical operational requirements. Different ways to achieve indoor localization are examined, but all require either static interventions (QR codes) or installing IoT sensors. In this work we present an AR Navigation System solution which utilizes a mobile device’s ability to exploit Augmented Reality (AR) for indoor localization and mapping. At the core of the system is a hybrid platform (cloud/edge), which enables the generate immersive AR navigation experiences. Key contribution of this work is the use of the aforementioned platform for introducing an AR “checkpoint” navigation system which integrates our algorithms for indoor localization, path planning, point of interest visualization and device interoperability. A prototype of the overall solution has already been implemented and it is deployed at the University of Piraeus for evaluation from students, personnel and visitors.
Interpolation Kernel Machines: Reducing Multiclass to Binary

Interpolating classifiers interpolate all the training data and thus have zero training error. Recent research shows their fundamental importance for high-performance ensemble techniques and other advantages. Interpolation kernel machines belong to the class of interpolating classifiers and do generalize well. They have been demonstrated to be a good alternative to support vector machines. In this work we further improve their performance. We propose not to use their inherent multiclass classification capacity, but instead apply them for solving binary classification instances based on a mutliclass-to-binary reduction. We experimentally study this ensemble approach in combination with six reducing multiple-to-binary methods. The experimental results show that the one-versus-one scheme consistently demonstrates superior performance.
A Benchmark and Investigation of Deep-Learning-Based Techniques for Detecting Natural Disasters in Aerial Images

Rapid emergency response and early detection of hazards caused by natural disasters are critical to preserving the lives of those in danger. Deep learning can aid emergency response authorities by automating UAV-based real-time disaster recognition. In this work, we provide an extended dataset for aerial disaster recognition and present a comprehensive investigation of popular Convolutional Neural Network models using transfer learning. In addition, we propose a new lightweight model, referred to as DiRecNet, that provides the best trade-off between accuracy and inference speed. We introduce a tunable metric that combines speed and accuracy to choose the best model based on application requirements. Lastly, we used the Grad-CAM explainability algorithm to investigate which models focus on human-aligned features. The experimental results show that the proposed model achieves a weighted F1-Score of 96.15% on four classes in the test set. When utilizing metrics that consider both inference time and accuracy, our model surpasses other pre-trained CNNs, offering a more efficient and precise solution for disaster recognition. This research provides a foundation for developing more specialized models within the computer vision community.
True Rank Guided Efficient Neural Architecture Search for End to End Low-Complexity Network Discovery

Neural architecture search (NAS) aims to automate neural network design process and has shown promising results for image classification tasks. Owing to combinatorially huge neural network design spaces coupled with training cost of candidates, NAS is computationally demanding. Hence, many NAS works focus on efficiency by constraining the search to only network building blocks (modular search) instead of searching for the entire architectures (global search), and by approximating candidates’ performance instead of expensive training. Modular search, however, offers only partial network discovery and final architecture configuration such as network’s depth or width requires manual trial and error. Further, approximating candidates’ performance incur misleading search directions due to their inaccurate relative rankings. In this work, we revisit NAS for end to end network discovery and guide the search using true rankings of candidates by training each from scratch. However, it is computationally infeasible for existing search strategies to navigate huge search spaces and determine accurate rankings at the same time. Therefore, we propose a novel search space and an efficient search algorithm that offers high accuracy low complexity network discovery with competitive search cost. Our proposed approach is evaluated on the CIFAR-10, yielding an error rate of 4% while the search cost is just 4.5 GPU days. Moreover, our model is 7.3 $$\times $$ , 3.7 $$\times $$ and 5.5 $$\times $$ smaller than the smallest models discovered by RL, evolutionary and gradient-based NAS methods respectively (Code and results available at: https://github.com/siddikui/TRG-NAS ).
Downsampling GAN for Small Object Data Augmentation

The limited visual information provided by small objects—under 32 $$\times $$ 32 pixels—makes small object detection a particularly challenging problem for current detectors. Moreover, standard datasets are biased towards large objects, limiting the variability of the training set for the small objects subset. Although new datasets specifically designed for small object detection have been recently released, the detection precision is still significantly lower than that of standard object detection. We propose a data augmentation method based on a Generative Adversarial Network (GAN) to increase the availability of small object samples at training time, boosting the performance of standard object detectors in this highly demanding subset. Our Downsampling GAN (DS-GAN) generates new small objects from larger ones, avoiding the unrealistic artifacts created by traditional resizing methods. The synthetically generated objects are inserted in the original dataset images in plausible positions without causing mismatches between foreground and background. The proposed method improves the AP $$^{@[.5,.95]}_{s}$$ and AP $$^{@.5}_{s}$$ of a standard object detector in the UVDT small subset by more than 4 and 10 points, respectively.
EMBiL: An English-Manipuri Bi-lingual Benchmark for Scene Text Detection and Language Identification

Detection and language identification of texts in an unconstrained scene image are quintessential processes in the multimedia information retrieval domain. Over the years, various approaches have investigated them by considering detection and language identification as separate problem statements. To the best of our knowledge, scene text datasets with minority Indic languages are not yet available. To this end, we created a scene image dataset called EMBiL containing a combination of English and Manipuri text. It contains 720 scene images with a total of over 28500 text instances. The Manipuri language is one of the official languages of India. To benchmark the performance of EMBiL, we proposed a single-stage simultaneous detection and language identification network called SceneTextYOLO-Net based on YOLOv5. We specifically included the shallow layer characteristics and applied a multi-scale detection head to improve small target text detection. We also inserted an attention mechanism between the neck and head structures to concentrate on the image’s essential regions. We performed extensive experiments on the proposed dataset using various state-of-the-art techniques. Furthermore, we performed experimental analysis on ICDAR2015 using SceneTextYOLO-Net and state-of-the-art methods. EMBiL is available at: https://github.com/Naosekpam/EMBiL-Dataset .
Low-Dimensionality Information Extraction Model for Semi-structured Documents

Most recent systems of information extraction (IE) from documents are regarded as complex models due to the large number of parameters they need and their resulting high memory footprint. In this paper, we propose a non complex model that extracts information from semi-structured documents (SSDs). We focus on the improvement of the model’s input modelling that provide a low memory consumption and better performance. The SSD is modelled using graphs to benefit from its content and layout properties. A Multi-layer Graph Attention network (Multi-GAT) classifier built on the SSD graph is then used to predict the text entities. To get rid of the unknown word embeddings in this kind of document, we provide a simple and efficient method of pre-trained subword embeddings fusion that doesn’t require any additional parameters. Our strategy for combining the multi-modal features of text, layout, and image entails concatenating the results of two Dense layers applied to the word embedding, position encoding and image embedding. Additionally, the graph adjacency matrix is built in a way to limit the graph dimension and enhance the classifier performance. All of these techniques improve the performance of our model while reducing its complexity and input dimensionality. Our model is evaluated on two artificial invoices datasets as well as one real dataset (SROIE). For the latter, we obtained a F1 score of 98.22%.
Stranger Trust Architecture: An Advancement to Zero Trust Architecture

Security in ever changing environment is the need of the hour and no business or establishment can survive without a proper and suitable security mechanism. There have been several mechanisms for attending the needs of different sectors. In recent years Zero Trust Architecture has emerged as one of the promising security mechanisms to ensure the utmost privacy and setback resistant authentication mechanism. In this research paper we have tried to propose an advanced level of authentication mechanism known as the stranger trust mechanism.
Cloud-Based Occlusion Aware Intrusion Detection System

This paper proposes a cloud-based occlusion-aware intrusion detection system that detects the faces of intruders and matches them to a database for recognition. A novel face detection or recognition system is proposed that enables robust face detection and recognition even in the case of occlusion, blurred images, face masks, side views, or even partial views of faces. It does so by focusing on all the visible features like eyes, ears, nose, etc., and using those for facial recognition. The proposed recognition system can match the side view of a face with the front view using the learned embeddings. The user gets an email notification as soon as an intruder is detected whose face does not match any person in the database. The entire system is deployed as a service on Google Cloud. Cloud deployment helps in removing any local computation requirement. The model is constantly updated via the Vertex AI pipelines feature of Google Cloud.
Brand Sentiment Analytics Using Flume

Applications that collect data in various forms can add data to the Hadoop stack by partnering with Name Node through an API capability. Using the entirety of the stockpiling and handling force of bunch servers and running dispersed processes on colossal volumes of information are simplified by Hadoop. The Hadoop building blocks can be utilized as an establishment for the making of a few administrations and applications.
A Novel Framework for VM Selection and Placement in Cloud Environment

In spite of various research that has been conducted in the past but there are some challenges that are still into existence related to balancing of workload in cloud applications. There has been a great need for efficient allocation of resources that is handling all the data center, servers & various virtual machines connected with cloud applications. It is the responsibility of cloud facility providers to confirm high facility delivery in an unavoidable situation. All such type of hosts is overloaded or underloaded based on their execution time and throughput. Task scheduling helps in balancing the load of resources and on the other hand task scheduling adheres to the requirement of service level agreement. SLA parameters such as deadlines are concentrated on the Load Balancing algorithm. This paper proposes algorithm which optimizes cloud resources and improves the balancing of load based on migration, SLA and energy efficiency. Proposed load-balancing algorithm discourses all states and focuses on existing research gaps by focusing on the literature gaps. Task scheduling is mainly concentrating on balancing the load and task scheduling mainly adheres to SLA. SLA is one of the documents offered by the service provider to the user. There are various parameters of load balancing such as deadlines which are discussed in the load balancing algorithm. The key focus of proposed process identifies optimize method of resources and improved Load Balancing based on QoS, priority migration of VMs and resource allocations. This proposed algorithm addressed these issues based on the literature review findings.
Revisiting k-NN for Fine-Tuning Pre-trained Language Models

Pre-trained Language Models (PLMs), as parametric-based eager learners, have become the de-facto choice for current paradigms of Natural Language Processing (NLP). In contrast, k-Nearest-Neighbor ( $$k$$ -NN) classifiers, as the lazy learning paradigm, tend to mitigate over-fitting and isolated noise. In this paper, we revisit $$k$$ -NN classifiers for augmenting the PLMs-based classifiers. From the methodological level, we propose to adopt $$k$$ -NN with textual representations of PLMs in two steps: (1) Utilize $$k$$ -NN as prior knowledge to calibrate the training process. (2) Linearly interpolate the probability distribution predicted by $$k$$ -NN with that of the PLMs’ classifier. At the heart of our approach is the implementation of $$k$$ -NN-calibrated training, which treats predicted results as indicators for easy versus hard examples during the training process. From the perspective of the diversity of application scenarios, we conduct extensive experiments on fine-tuning, prompt-tuning paradigms and zero-shot, few-shot and fully-supervised settings, respectively, across eight diverse end-tasks. We hope our exploration will encourage the community to revisit the power of classical methods for efficient NLP (Code and datasets are available in https://github.com/zjunlp/Revisit-KNN ).
MCLS: A Large-Scale Multimodal Cross-Lingual Summarization Dataset

Multimodal summarization which aims to generate summaries with multimodal inputs, e.g., text and visual features, has attracted much attention in the research community. However, previous studies only focus on monolingual multimodal summarization and neglect the non-native reader to understand the cross-lingual news in practical applications. It inspires us to present a new task, named Multimodal Cross-Lingual Summarization for news (MCLS), which generates cross-lingual summaries from multi-source information. To this end, we present a large-scale multimodal cross-lingual summarization dataset, which consists of 1.1 million article-summary pairs with 3.4 million images in 44 * 43 language pairs. To generate a summary in any language, we propose a unified framework that jointly trains the multimodal monolingual and cross-lingual summarization tasks, where a bi-directional knowledge distillation approach is designed to transfer knowledge between both tasks. Extensive experiments on many-to-many settings show the effectiveness of the proposed model.
Adder Encoder for Pre-trained Language Model

BERT, a pre-trained language model entirely based on attention, has proven to be highly performant for natural language understanding tasks. However, pre-trained language models (PLMs) are often computationally expensive and can hardly be implemented with limited resources. To reduce energy burden, we introduce adder operations into the Transformer encoder and propose a novel AdderBERT with powerful representation capability. Then, we adopt mapping-based distillation to further improve its energy efficiency with an assured performance. Empirical results demonstrate that AddderBERT $$_{6}$$ achieves highly competitive performance against that of its teacher BERT $$_\textrm{BASE}$$ on the GLUE benchmark while obtaining a 4.9x reduction in energy consumption.
Learnable Conjunction Enhanced Model for Chinese Sentiment Analysis

Sentiment analysis is a crucial text classification task that aims to extract, process, and analyze opinions, sentiments, and subjectivity within texts. In current research on Chinese text, sentence and aspect-based sentiment analysis is mainly tackled through well-designed models. However, despite the importance of word order and function words as essential means of semantic expression in Chinese, they are often underutilized. This paper presents a new Chinese sentiment analysis method that utilizes a Learnable Conjunctions Enhanced Model (LCEM). The LCEM adjusts the general structure of the pre-trained language model and incorporates conjunctions location information into the model’s fine-tuning process. Additionally, we discuss a variant structure of residual connections to construct a residual structure that can learn critical information in the text and optimize it during training. We perform experiments on the public datasets and demonstrate that our approach enhances performance on both sentence and aspect-based sentiment analysis datasets compared to the baseline pre-trained language models. These results confirm the effectiveness of our proposed method.
Training NLI Models Through Universal Adversarial Attack

Pre-trained language models are sensitive to adversarial attacks, and recent works have demonstrated universal adversarial attacks that can apply input-agnostic perturbations to mislead models. Here, we demonstrate that universal adversarial attacks can also be used to harden NLP models. Based on NLI task, we propose a simple universal adversarial attack that can mislead models to produce the same output for all premises by replacing the original hypothesis with an irrelevant string of words. To defend against this attack, we propose Training with UNiversal Adversarial Samples (TUNAS), which iteratively generates universal adversarial samples and utilizes them for fine-tuning. The method is tested on two datasets, i.e., MNLI and SNLI. It is demonstrated that, TUNAS can reduce the mean success rate of the universal adversarial attack from above 79% to below 5%, while maintaining similar performance on the original datasets. Furthermore, TUNAS models are also more robust to the attack targeting at individual samples: When search for hypotheses that are best entailed by a premise, the hypotheses found by TUNAS models are more compatible with the premise than those found by baseline models. In sum, we use universal adversarial attack to yield more robust models.
Performance of Cryptocurrencies Under a Sentiment Analysis Approach in the Time of COVID-19

This chapter presents a sentiment analysis model that examines the performance of the most popular and highly capitalized cryptocurrencies during the COVID-19 era. Specifically, we look at the performance of Bitcoin (BTC), Ethereum (ETH), Binance (BNB), and Cardano (ADA) during the period 1/1/2020–30/9/2021. We do not use unique and expensive data and we do not apply complex models for our analysis. Instead, we construct behavioral indices based on Google trends data and we test the impact of these indices on the performance of BTC, ETH, BNB, and ADA. We used open source software, such as Python, so similar analysis can be carried out by anyone who is interested in financial markets. The results of this study support the following conclusions: (i) when the health risk increases, the BTC, BNB, and ADA prices fall (and vice versa), (ii) the Google trends indices, which reveal the users’ intention to buy a cryptocurrency, could be useful indicators of crypto performance, but (iii) the sell sentiment, at least in our case, does not have any statistically significant effect. Thus, sentiment analysis based on Google searches could be useful for practitioners, investors, analysts, and scholars, and further study on this direction should be done.