Reviewerly/exHarmony huggingface dataset:
Combining Vision and Language Representations for Patch-based Identification of Lexico-Semantic Relations

Although a wide range of applications have been proposed in the field of multimodal natural language processing, very few works have been tackling multimodal relational lexical semantics. In this paper, we propose the first attempt to identify lexico-semantic relations with visual clues, which embody linguistic phenomena such as synonymy, co-hyponymy or hypernymy. While traditional methods take advantage of the paradigmatic approach or/and the distributional hypothesis, we hypothesize that visual information can supplement the textual information, relying on the apperceptum subcomponent of the semiotic textology linguistic theory. For that purpose, we automatically extend two gold-standard datasets with visual information, and develop different fusion techniques to combine textual and visual modalities following the patch-based strategy. Experimental results over the multimodal datasets show that the visual information can supplement the missing semantics of textual encodings with reliable performance improvements.
Detecting impersonators in examination halls using AI

Detecting impersonators in examination halls is very significant to conduct the examination fairly. Recently there were occasions of potential impersonators taking tests instead of the intended person. To overcome this issue, we require an efficient method with less manpower. With the advancement of machine learning and AI technology, we can overcome this issue. In this project, we are developing an AI system where images of students are saved and the model is developed using transfer learning process to get accurate results. If the student is an allowed one, it shows the hall ticket number and name of the student otherwise it appears unknown tag.
Examining YOLO for real-time face-mask detection

COVID-19 pandemic has been affecting human lives since it has been announced by the World Health Organization (WHO). SARS-CoV-2 virus is transferred be means of close contact. Wearing a face mask reduces the infection rate. Several works have been used to employ technology to alleviate its rapid and widespread. Currently, Artificial Intelligence (AI) plays an important role in classifying and detection objects. Several detection algorithms have been recently proposed. In this paper we investigate the performance of the latest versions of You Only Look Once (YOLO) real-time AI-based detector in detecting the wearing of masks.
Afterpulse effect in measurement-device-independent quantum key distribution

There is no doubt that measurement-device-independent quantum key distribution (MDI-QKD) is a crucial protocol that is immune to all possible detector side channel attacks. In the preparation phase, a simulation model is usually employed to get a set of optimized parameters, which is utilized for getting a higher secure key rate in reality. With the implementation of high-speed QKD, the afterpulse effect which is an intrinsic characteristic of the single-photon avalanche photodiode is no longer ignorable, this will lead to a great deviation compared with the existing analytical model. Here we develop an afterpulse-compatible MDI-QKD model to get the optimized parameters. Our results indicate that by using our afterpulse-compatible model, we can get a much higher key rate than the prior afterpulse-omitted model. It is significant to take the afterpulse effect into consideration because of the improvement of the system working frequency.
Optimal Strategic Mining Against Cryptographic Self-Selection in Proof-of-Stake

Cryptographic Self-Selection is a subroutine used to select a leader for modern proof-of-stake consensus protocols, such as Algorand. In cryptographic self-selection, each round $r$ has a seed $Q_r$. In round $r$, each account owner is asked to digitally sign $Q_r$, hash their digital signature to produce a credential, and then broadcast this credential to the entire network. A publicly-known function scores each credential in a manner so that the distribution of the lowest scoring credential is identical to the distribution of stake owned by each account. The user who broadcasts the lowest-scoring credential is the leader for round $r$, and their credential becomes the seed $Q_{r+1}$. Such protocols leave open the possibility of a selfish-mining style attack: a user who owns multiple accounts that each produce low-scoring credentials in round $r$ can selectively choose which ones to broadcast in order to influence the seed for round $r+1$. Indeed, the user can pre-compute their credentials for round $r+1$ for each potential seed, and broadcast only the credential (among those with a low enough score to be the leader) that produces the most favorable seed. We consider an adversary who wishes to maximize the expected fraction of rounds in which an account they own is the leader. We show such an adversary always benefits from deviating from the intended protocol, regardless of the fraction of the stake controlled. We characterize the optimal strategy; first by proving the existence of optimal positive recurrent strategies whenever the adversary owns last than $38\%$ of the stake. Then, we provide a Markov Decision Process formulation to compute the optimal strategy.
NeuralSympCheck: A Symptom Checking and Disease Diagnostic Neural Model with Logic Regularization

The symptom checking systems inquire users for their symptoms and perform a rapid and affordable medical assessment of their condition. The basic symptom checking systems based on Bayesian methods, decision trees, or information gain methods are easy to train and do not require significant computational resources. However, their drawbacks are low relevance of proposed symptoms and insufficient quality of diagnostics. The best results on these tasks are achieved by reinforcement learning models. Their weaknesses are the difficulty of developing and training such systems and limited applicability to cases with large and sparse decision spaces. We propose a new approach based on the supervised learning of neural models with logic regularization that combines the advantages of the different methods. Our experiments on real and synthetic data show that the proposed approach outperforms the best existing methods in the accuracy of diagnosis when the number of diagnoses and symptoms is large. The models and the code are freely available online ( https://github.com/SympCheck/NeuralSymptomChecker ).
Evolutionary Computation in Action: Hyperdimensional Deep Embedding Spaces of Gigapixel Pathology Images

One of the main obstacles of adopting digital pathology is the challenge of efficient processing of hyperdimensional digitized biopsy samples, called whole slide images (WSIs). Exploiting deep learning and introducing compact WSI representations are urgently needed to accelerate image analysis and facilitate the visualization and interpretability of pathology results in a postpandemic world. In this paper, we introduce a new evolutionary approach for WSI representation based on large-scale multi-objective optimization (LSMOP) of deep embeddings. We start with patch-based sampling to feed KimiaNet , a histopathology-specialized deep network, and to extract a multitude of feature vectors. Coarse multi-objective feature selection uses the reduced search space strategy guided by the classification accuracy and the number of features. In the second stage, the frequent features histogram (FFH), a novel WSI representation, is constructed by multiple runs of coarse LSMOP. Fine evolutionary feature selection is then applied to find a compact (short-length) feature vector based on the FFH and contributes to a more robust deep-learning approach to digital pathology supported by the stochastic power of evolutionary algorithms. We validate the proposed schemes using The Cancer Genome Atlas (TCGA) images in terms of WSI representation, classification accuracy, and feature quality. Furthermore, a novel decision space for multicriteria decision making in the LSMOP field is introduced. Finally, a patch-level visualization approach is proposed to increase the interpretability of deep features. The proposed evolutionary algorithm finds a very compact feature vector to represent a WSI (almost 14,000 times smaller than the original feature vectors) with 8% higher accuracy compared to the codes provided by the state-of-the-art methods.
Fine- and Coarse-Granularity Hybrid Self-Attention for Efficient BERT

Transformer-based pre-trained models, such as BERT, have shown extraordinary success in achieving state-of-the-art results in many natural language processing applications.However, deploying these models can be prohibitively costly, as the standard self-attention mechanism of the Transformer suffers from quadratic computational cost in the input sequence length.To confront this, we propose FCA, a fine-and coarse-granularity hybrid self-attention that reduces the computation cost through progressively shortening the computational sequence length in self-attention.Specifically, FCA conducts an attention-based scoring strategy to determine the informativeness of tokens at each layer.Then, the informative tokens serve as the fine-granularity computing units in selfattention and the uninformative tokens are replaced with one or several clusters as the coarsegranularity computing units in self-attention.Experiments on GLUE and RACE datasets show that BERT with FCA achieves 2x reduction in FLOPs over original BERT with <1% loss in accuracy.We show that FCA offers significantly better trade-off between accuracy and FLOPs compared to prior methods 1 .
Artificial Intelligence (AI) Powered Customer Care

This paper provides details on how Utilities can ensure elevated customer experience by reinventing the customer care experience leveraging Artificial Intelligence (AI). The paper introduces AI Powered Care—which as a concept is not just limited to only chatbots but extends seamless customer interaction both internally in the organization and externally through different channels. AI can provide Utilities with avenues to service customers in their preferred channels. AI can predict customer behavior and identify and nurture the most valuable leads, through in image and voice recognition are driving the growth of the regional market. Artificial Intelligence market will surpass $100 billion by 2025 and expected to trim business cost by more than $8 billion per year by 2022.
MRDFPD: Metadata Driven RDF Based Product Discovery Framework

In recent years, the number of people who use e-commerce has risen rapidly. Users can utilise a feature on shopping sites to search for items based on their descriptions. The inputs can be thought of as product discovery and recommendation queries. Product recommendation was previously made using syntactic methods, which were ineffective in categorising and discovering appropriate products solely based on keyword matching. Each online platform has a large number of products that must be organised into taxonomies in order to improve product discovery accuracy. A metadata-driven RDF-driven framework for product discovery and classification is proposed in this paper. Preprocessing and query upper ontology matching are included in the queries. The knowledge base that stores the common categories from e-commerce sites is held by the category upper ontology, a semantic model. The proposed framework takes into account the input queries as well as the number of user clicks. It uses logistic regression for product classification on the dataset and the Harris’ Hawks optimisation algorithm to compute semantic similarities to produce accurate recommendations and rankings.
Life after BERT: What do Other Muppets Understand about Language?

Existing pre-trained transformer analysis works usually focus only on one or two model families at a time, overlooking the variability of the architecture and pre-training objectives.In our work, we utilize the oLMpics benchmark and psycholinguistic probing datasets for a diverse set of 29 models including T5, BART, and ALBERT.Additionally, we adapt the oLMpics zero-shot setup for autoregressive models and evaluate GPT networks of different sizes.Our findings show that none of these models can resolve compositional questions in a zero-shot fashion, suggesting that this skill is not learnable using existing pre-training objectives.Furthermore, we find that global model decisions such as architecture, directionality, size of the dataset, and pre-training objective are not predictive of a model's linguistic capabilities.The code for this study is available on GitHub 1 .
We Don’t Need No Real Users?! Surveying the Adoption of User-less Automation Tools by UI Design Practitioners

The main principles for designing successful UIs in a perfect world have long been known—considering many possible solutions for a problem and involving representative users in the process. In practice, however, reasons for violating those principles can be plentiful: the infamous tight budgets and schedules, lack of management buy-in, restrictions for face-to-face meetings, etc. Yet, design tools that do not require real users, such as AI-/ML-powered solutions, which could mitigate these issues seem to experience a rather low adoption rate in industry. In this paper, we present a survey with 34 professional digital designers and user researchers intended to investigate the above hypotheses. We inquire into awareness and usage of 61 such tools and platforms, as well as participants’ design and research processes and general design tool adoption in industry. From the results we identify three particular challenges and three opportunities. Finding and recruiting relevant participants for user studies seems to be indeed problematic, and professional designers and researchers often lack the time and resources to follow a textbook process. They are, however, open to novel tools addressing these shortcomings—particular for ideation and evaluation—but at the same time seem to be largely unfamiliar with AI-/ML-based approaches or do not (yet) see added value in them. With these findings as a starting point, the Web Engineering community can work towards a deeper understanding of designers’ and researchers’ needs that could be met with AI-/ML-based support tools.
Garbled Circuits with Sublinear Evaluator

A recent line of work, Stacked Garbled Circuit (SGC), showed that Garbled Circuit (GC) can be improved for functions that include conditional behavior. SGC relieves the communication bottleneck of 2PC by only sending enough garbled material for a single branch out of the b total branches. Hence, communication is sublinear in the circuit size. However, both the evaluator and the generator pay in computation and perform at least factor $$\log b$$ extra work as compared to standard GC. We extend the sublinearity of SGC to also include the work performed by the GC evaluator $$E$$ ; thus we achieve a fully sublinear $$E$$ , which is essential when optimizing for the online phase. We formalize our approach as a garbling scheme called $$\mathsf {GCWise}$$ : GC WIth Sublinear Evaluator. We show one attractive and immediate application, Garbled PIR, a primitive that marries GC with Private Information Retrieval. Garbled PIR allows the GC to non-interactively and sublinearly access a privately indexed element from a publicly known database, and then use this element in continued GC evaluation.
Secure Multiparty Computation with Sublinear Preprocessing

A common technique for enhancing the efficiency of secure multiparty computation (MPC) with dishonest majority is via preprocessing: In an offline phase, parties engage in an input-independent protocol to securely generate correlated randomness. Once inputs are known, the correlated randomness is consumed by a “non-cryptographic” and highly efficient online protocol. The correlated randomness in such protocols traditionally comes in two flavors: multiplication triples (Beaver, Crypto ’91), which suffice for security against semi-honest parties, and authenticated multiplication triples (Bendlin et al., Eurocrypt ’11, Damgård et al., Crypto ’12) that yield efficient protocols against malicious parties. Recent constructions of pseudorandom correlation generators (Boyle et al., Crypto ’19, ’20) enable concretely efficient secure generation of multiplication triples with sublinear communication complexity. However, these techniques do not efficiently apply to authenticated triples, except in the case of secure two-party computation of arithmetic circuits over large fields. In this work, we propose the first concretely efficient approach for (malicious) MPC with preprocessing in which the offline communication is sublinear in the circuit size. More specifically, the offline communication scales with the square root of the circuit size. From a feasibility point of view, our protocols can make use of any secure protocol for generating (unauthenticated) multiplication triples together with any additive homomorphic encryption. We propose concretely efficient instantiations (based on strong but plausible “linear-only” assumptions) from existing homomorphic encryption schemes and pseudorandom correlation generators. Our technique is based on a variant of a recent protocol of Boyle et al. (Crypto ’21) for MPC with preprocessing. As a result, our protocols inherit the succinct correlated randomness feature of the latter protocol.
ESTIMATING TIME-VARYING REPRODUCTION NUMBER BY DEEP LEARNING TECHNIQUES

<abstract> Estimating time-varying reproduction number $ \mathcal{R}_{t} $ is important for quantifying the transmission ability, capturing the trend of infectious disease and assessing the effectiveness of public health intervention measures. However, accurate estimation of $ \mathcal{R}_{t} $ remains a challenging work. Deep neural networks are uniform approximators and have an unreasonable and counterintuitive effectiveness in learning unknown functions, thus can be applied to represent $ \mathcal{R}_{t} $. In this paper, we will estimate $ \mathcal{R}_{t} $ by universal differential equation method which embeds neural network $ \mathcal{R}_{t} $ into a differential equation. Compared with other methods such as state space, EpiEstim and EpiNow2 methods, deep learning method can achieve better performance with fewer data sources. </abstract>
Self-supervised Prototype Conditional Few-Shot Object Detection

Traditional deep learning-based object detection methods require a large amount of annotation for training, and creating such a dataset is expensive. Few-shot object detection which detects a new category of objects with a small amount of data is a difficult task. Many previous methods have been applied to detect new categories with few data by fine-tuning. However, fine-tuning methods cannot be sufficiently adapted quickly to new environments because of time-consuming training. In this study, we develop methods that can be adapted to new categories without fine-tuning. A prototype conditional detection module built on prototypes for each category is intended to be conditioned and detected by prototypes. Moreover, a self-supervised conditional detection module improves the representational capability for detecting new categories by a self-supervised task that uses embeddings outside the annotation region to detect the same regions before and after image transformation. The results of extensive experiments on the PASCAL VOC and MS COCO show the effectiveness of the proposed method.
Improved Facial Keypoint Regression Using Attention Modules

Recently, keypoint detection or pose estimation has gained a lot of applications and scope of research. However, the facial keypoint localization gives a poor prediction for low-quality and low-resolution images. Our goal is to overcome this and regress keypoint positions more accurately, the model needs to learn representations that focus more on the regions around the keypoints. In this paper, we are interested in finding the facial keypoints by regressing the gaussian heat mapped labels with distance loss and pixel-wise labels with segmentation loss functions like cross-entropy using Fully Convolution Network (FCN) and UNET segmentation models. We find the best version of the segmentation models by further refining the output features of the segmentation models by utilizing the popular channel and attention mechanisms like Convolution Block Attention Module (CBAM), Squeeze-and-Excitation Network (SENET) and Efficient Channel Attention Network (ECANET). Various experiments are performed to find the optimal position to place the attention module inside the segmentation models by monitoring the evaluation metrics. The refined features focus more on the relevant parts of the face, thereby improving the accuracy of the model. The intermediate channel activations and the final output are also visualized to observe the differences between various attention modules. Based on the quantitative and qualitative validation results, we rank the models best suited for our future applications such as facial tracking and emotion detection.
Data Variety Modeling: A Case of Contextual Diversity Identification from a Bottom-up Perspective

Variety is a property related to data diversity in Big Data Systems (BDS) that comprises several cases, such as structural diversity (variety in data types), source diversity (variety in the way data are produced), etc. Recently, adding contextual information allows more complex analyses, which open the possibility of modeling variety thinking of reuse. In this article, we introduce a proposal for modeling variety by following a dual data processing perspective. We exemplify the case of one of these perspectives by identifying and modeling contextual variations in a particular domain problem.
Bonafide Satellite Landslide Image Detection Using Deep Learning

The tremendous growth of technology in the modern era significantly generates fake images from the original images. Even the real-time data obtained from the satellites are being modified for their purposes. Due to this, the researchers and public face many hurdles in making predictions and interpretations about the geological conditions. Detecting the fake images and preserving the original images is crucial in our day-to-day life. Though there are solutions like fake image detection using machine learning techniques, they are not efficient. A real-time solution is proposed to overcome the loopholes in the existing solutions. The proposed solution comprises three modules. In module1, data pre-processing is performed for the image dataset to eliminate anomalies in the data. In module2, the pre-processed image data detection is performed to identify them as fake or bona fide using a deep learning technique which consists of Error Level Analysis (ELA). ELA produces brighter regions for fake images to differentiate fake and bona fide images. In module3, Artificial Neural Networks (ANN) is used for building the model efficiently. The proposed approach helps detect the bona fide satellite landslide images that researchers use to forecast geological conditions.