Reviewerly/exHarmony huggingface dataset:
Joint Learning for Multitasking Models

Artificial intelligence using neural networks has made tremendous progress in the field of computer vision. State-of-the-art models have been developed for various computer vision tasks such as image classification, object detection, image segmentation and keypoint estimation. Many of these models are tuned to get highest benchmark scores on curated datasets for any single specific task. However, deploying them to industrial use cases often requires multiple models to be used sequentially or as an ensemble to address different business use cases. Better hardware, model optimisation techniques such as quantisation and pruning have been widely used to improve the performance of individual models. In most cases, the features extracted by a model is not effectively used by the subsequent models. Each model will have its own pre- and post-processing which is a considerable overhead when scaled to industry requirements. We explore the concept of multitasking architectures and propose a joint learning approach to train a multitasking model that can do object detection, keypoint estimation and instance segmentation together using a single forward pass through it. Learning to predict multiple closely related tasks should help the model to learn better representations of the trained data and become robust to overfitting. Our best performing model achieved 32.26 frames per second (fps) with 41.2 AP on object detection, 38.2 AP on instance segmentation and 53.0 AP on keypoint estimation tasks when evaluated on COCO validation dataset. A lighter version of the model was able to process at 41.66 fps, enabling real-time computations for most use cases.
Exploring the Design Space of Social Physics Engines in Games

Social simulation in video games approximates believable social behavior between characters. Game franchises like Crusader Kings, The Sims, and Dwarf Fortress became famous for using social simulation for emergent storytelling. Despite the success of using social simulation as a core aspect of gameplay, there is a seeming lack of publicly available tools for helping game developers create these types of experiences. To help encourage the development of open-source social simulation tools, we further explore the concept of social physics engines—self-contained solutions for modeling dynamic social relationships between non-player characters and players. We propose a design space for constructing social physics systems. It is inspired by rigid-body physics engines and is informed by a design space analysis using commercial and academic social simulation games and systems.
AI Music Generator

Artificial intelligence is the study and creation of machines that can perform tasks that would ordinarily require human intelligence such as face detection, speech recognition, and decision-making.We will be using deep learning as a source to generate music through a computer using a database of music tunes without any expertise of any trained music artist.The music is generated through sentimental analysis, which helps in mood detection through a sample image of a human self.The music is stored as a piano-roll which is divided in several types of emotions that is generated through the testing process.The use of deep learning is that the machine trains itself through iteration of inputs that is provided in the database which will help in generating music from the input of a face of the user and will make music generation easier.The various models of deep learning are LSTM, CNN, GAN and many more that help in music generation and simultaneously help in enhancing the exposure to sentimental analysis.This project will be using some of these models to demonstrate music generation through sentimental analysis.
Book of Abstracts of the 21th European Conference on Eye Movements in Leicester 2022

Contents Keynotes: Iain Gilchrist: Integrative Active Vision p 5 Ziad Hafed: A Vision for orienting in Primate Oculomotor Control Circuitry p 6 Fatema Ghasia: Miniscule Eye Movements Play a Major Role in Binocular Vision Disorders p.7 Miriam Spering: Eye Movements as a Window into Human Decision-Making p.8 Monica S. Castelhano: Explorations of how Scene Context and Previous Experience Dynamically Influence Attention and Eye Movement Guidance p.9 Symposia: Eye Tracking and the Visual Arts p.19 Eye Movements during Text Processing and Multiline Reading p.23 Unstable Fixation and Nystagmus with a Focus on the Next Generation of Researchers p.84 Eye Movements as a measure of Higher-Level Text Processing p.97 Eye Movements in Memory Processes Between Working Memory and Long-Term Memory p.178 Symposium to Honour Alexander Pollatsek's Legacy to Eye Movement Research p.204 Talks: Reading p.30 Parafoveal Processing p.36 Cinical and Applied p.39 Visual Search p.92 Eye Movement Control in Reading I & II p.104 & 116 & 225 Reading Development p.110 Decision-Making p.122 Eye-tracking Methods p.128 Real World and Virtual Reality p.134 Chinese Reading p.185 Special Populations p.191 Visuo-motor p.195 Bilingual Reading p.201 & 217 Reading Comprehension p.219 Pupillometry p.235 Poster sessions: Attention p.44 & 139 Cognition p. 49 Visuo-Motor p.62 Memory p.145 Methods p.150 Reading p. 57 & 155 Real World p.169 Social Cognition p.173.
Improving Few-Shot Part Segmentation Using Coarse Supervision

A significant bottleneck in training deep networks for part segmentation is the cost of obtaining detailed annotations. We propose a framework to exploit coarse labels such as figure-ground masks and keypoint locations that are readily available for some categories to improve part segmentation models. A key challenge is that these annotations were collected for different tasks and with different labeling styles and cannot be readily mapped to the part labels. To this end, we propose to jointly learn the dependencies between labeling styles and the part segmentation model, allowing us to utilize supervision from diverse labels. To evaluate our approach we develop a benchmark on the Caltech-UCSD birds and OID Aircraft dataset. Our approach outperforms baselines based on multi-task learning, semi-supervised learning, and competitive methods relying on loss functions manually designed to exploit coarse supervision.
Personalized Education: Blind Knowledge Distillation

Knowledge distillation compresses a large model (teacher) to a smaller one by letting the student imitate the outputs of the teacher. An interesting question is why the student still typically underperforms the teacher after the imitation. The existing literature usually attributes this to model capacity differences between them. However, capacity differences are unavoidable in model compression, and even large capacity differences are desired for achieving high compression rates. By designing exploratory experiments with theoretical analysis, we find that model capacity differences are not necessarily the root reason; instead the distillation data matter when the student capacity is greater than a threshold. In light of this, we propose personalized education (PE) to first help each student adaptively find its own blind knowledge region (BKR) where the student has not captured the knowledge from the teacher, and then teach the student on this region. Extensive experiments on several benchmark datasets demonstrate that PE substantially reduces the performance gap between students and teachers, even enables small students to outperform large teachers, and also beats the state-of-the-art approaches. Code link: https://github.com/Xiang-Deng-DL/PEBKD .
Mel Spectrogram Based Automatic Speaker Verification Using GMM-UBM

Speech recognition refers to the technology that enables machines to recognize persons using their speech utterances. An automatic speaker verification (ASV) is included in one of the challenging task in speech community. The ASV system works based on the speaker recognition claimed against the model. In this paper, the system works as a text-independent speaker verification (TISV) and is outlined to verify the speaker using his/her voice samples. We followed two approaches, first approach is Gaussian Mixture Model (GMM) method is used to create speaker modeling and the second approach are GMMs created from training dataset, with Universal Background Model (UBM) used for adaptation of the dataset, well known approach for speaker verification (SV). GMM-UBMs are designed as well classifier for decision making. In both the approaches, the training is performed by the Expectation Maximization (EM) and Maximum A Posteriori (MAP) adaptation for better models respectively. The NIST 2003 database is evaluated using adapted GMM-UBM following NIST 2003 speaker recognition evaluation protocol and the relative performance improvement in the SV system using GMM and GMM-UBM in terms of EER are 9.43% and 8.88%.
Publisher of Intellectual Literature as a Maker of Culture

How do we cognize the intellectual culture of different eras? We turn to books, magazines, newspapers, in which ideas, signs, symbols, emotions and other signs of the times are imprinted in various forms. We discover authors whose reasoning makes us look at our modernity in a new way. We even pay attention to the illustrators who created the unique face of paper products. But often we do not pay attention to the one who made possible our acquaintance with this historical world of life and thought. The figure of the publisher has a very modest place in our perception of culture. Only “stars”-authors remain in the public mind (who today will remember the publisher Dostoevsky or Jules Verne), and historians of book publishing, as a rule, limit themselves to retelling published books and biographies of authors, and yet it is the figure of the publisher that is especially relevant today. They not only “cultivated” the intellectual space of the era, but, as entrepreneurs, they programmed new forms of activity. If we recall the term of J. Schumpeter, they created by destroying... This feature of the publisher of intellectual literature manifests especially in times of crisis. To comprehend the historical experience of the publisher of intellectual literature, we turn to the epistolary heritage of the Russian emigres in the 1930s, when Nazism was just beginning its march across Europe. The archive of the philosopher, musician Yevsey Davidovich Schor preserved correspondence with the founder of publishing house Vita Nova (Vita Nova Verlag), German cultural figure Rudolf Rössler. Their letters reveal to us both the dynamics of the publishing process and the role of the Publisher as a cultural maker.
Deep Bregman Divergence for Self-Supervised Representations Learning

Neural Bregman divergence measures the divergence of data points using convex neural networks, which is beyond Euclidean distance and capable of capturing divergence over distributions. The non-Euclidean geometry is not well explored in deep representation learning and remains a challenging endeavor for self-supervised representation learning. In this paper, we propose deep Bregman divergences for self-supervised pretext task learning, where we aim to enhance self-supervised embedding representation by training additional networks based on functional Bregman divergences. Our framework can capture the divergence of embedding distributions and improve the quality of learned representation using an arbitrary Bregman divergence over data embedding. Specifically, we develop a novel self-supervised architecture and a new divergence loss that measures the asymmetric distance of arbitrary Bergman divergences of neural networks. We show that the combination of self-supervised contrastive learning and our proposed method outperforms the baseline as well as most established methods for self-supervised and semi-supervised learning on multiple classifications and object detection tasks and datasets. Moreover, the learned representations generalize well when transferred to other datasets and tasks.
“Ah That is Cute, I Will Stay Away!” Wearer Experiences of a Backpack with Shape-Changing Accessory for Keeping Social Distance to Others in Public Context

Smart wearable solutions for social distancing have appeared during the Covid-19 pandemic with various types of outputs often directed only to the wearer or if directed to others, the visual output is usually displayed on the front side of the wearers body. This paper presents instead a design, implementation, and in-the-wild Wizard of Oz evaluation of a backpack with shape-changing accessory which output is directed to others approaching the wearer from behind. Qualitative findings from six participants suggest that there is potential in shape-changing accessory for keeping social distance in public context, however, the social acceptability issues caused by noisy motors and inclusiveness of the design limit the willingness to wear it. Even though the study was inspired by the Covid-19, other non-pandemic use cases for personal space respecting wearables are discussed also for users with special needs.
Enhanced Electronic Voting Machine Performance with an E-Voting Website

This paper presents the enhanced electronic voting machine performance with an E-Voting website. The Election Commission of India (ECI) plays a vital role in our country framing the government by conducting elections according to the by-law of the Constitution of India. To conduct the elections effectively the ECI has been introducing advanced technologies that reduce time consumption, cost, and manpower. At present, the ECI adopted a Voter-Verified Paper Audit Trail (VVPAT) with an Electronic Voting Machine (EVM) to increase its reliability. Due to manual verification of voters' details with the electoral lists malpractice continues during the elections. To overcome these problems, a face recognition device and a fingerprint sensor is embedded with an EVM. However, the average polling rate is unable to cross seventy percent in our democratic country. Why because nearly twenty percent of people are migrated to other states and abroad for employment or higher studies and are unable to cast their vote. In this approach, a secured online voting system has been introduced along with an existing EVM to cast their vote virtually. These votes are stored in the server and EVM with a secured interlocking system. The EVM hardware setup employs a web camera and a fingerprint sensor to ensure voters as per the record. Test results with an enhanced EVM are presented to validate performance with an E-Voting website.
Two-step calibration method of the extrinsic parameters with high accuracy for a bistatic non-orthogonal shafting laser theodolite system

Motivated by the increasing demands on the precision of 3D large-scale measurement, the extrinsic parameters calibration with high accuracy of the bistatic non-orthogonal shafting laser theodolite (N-theodolite) system is required. A two-step method is proposed to achieve the extrinsic parameters calibration with high accuracy in this paper. In the first step, by analyzing and setting the approximate emitted point during the motion of the laser axis in local space, the calculation of the initial extrinsic parameters can be simplified. In the second step, the above results are taken as the initial values of optimization, and the distances between the spatial laser points provided by PSD sensors with high accuracy in global space are used to construct the unconstrained optimal objective function. The proposed method is validated with the measurement experiment of the bistatic N-theodolite system, the average error of 3D coordinate measurement is less than 0.4 mm, and the average error of distance measurement is less than 0.3 mm within 5 m.
Unsupervised Content Based Image Retrieval Using Pre-Trained CNN and PCNN Features Extractors

Content-based image retrieval (CBIR) is a popular approach for searching and retrieving digital images from labeled or unlabeled image collections using image content features.The most important parts of the CBIR system are the computational complexity and the retrieval accuracy.Many studies have been conducted to increase the accuracy of image retrieval systems.In this paper, we propose a new fully unsupervised content-based image retrieval (CBIR) technique to increase the efficacy of image retrieval using a clustering approach on unlabeled image dataset.The proposed method combines features extracted from both pre-trained AlexNet model and pulse-coupled neural networks (PCNN) to extract high and low level features.Then principal component analysis (PCA) is performed on AlexNet's features and these combinations are fed to the K-means algorithm after normalization process.Then Euclidean distance is used to measure the similarity between query and stored images within the same cluster.Finally top similar images are ranked and retrieved.Experimental results on the benchmark Corel-1k and Corel-10k datasets show that the proposed method achieves high precision values of 92.8% and 64.8% respectively, on the top 20 retrieval levels compared to other methods.
Cyberthreats Facing High School Students and Methods of Addressing them

In this work, we provide an overview of the most common risks and threats related to information technology faced by high school students in Saudi Arabia. In this context, this work starts from the concept that using information technology is the basis of contemporary life once it has penetrated and become part of our daily activities, which also makes us challenged by the threats and risks that result from these technologies. Thus, we have written and applied a questionnaire to groups of high school students in all provinces of Saudi Arabia. The study sample of high school students who participated and completed the questionnaire was 2,312, divided into 1,128 male and 1,184 female students. In addition, these students were randomly selected from all 13 administrative regions in KSA. Specifically, we have examined the risks and threats of secondary school students using information technology, along with methods for securing their use of it in terms of awareness, envisioning, and response. Our results indicate a necessity to raise the awareness level regarding information security and the risks that come along with the use of technology through school guidance and counseling programs. Finally, we suggest that holding informational meetings and strengthening the partnership between the school and the family can be an effective way of mitigating the risks directly caused by the unsupervised use of information technology.
NecklaceFIR: A Large Volume Benchmarked Necklace Dataset for Fashion Image Retrieval

In this paper, we present a new necklace dataset related to the fashion domain, namely, NecklaceFIR. In recent years, the retrieval of clothing items has attracted a lot of attention. However, fashion products that are complex in nature, such as jewellery, have not received much attention due to the absence of a proper dataset. To address this problem, we have collected the dataset. The dataset contains over $${\sim }$$ 4.8K high-definition necklace images collected from various online retailers. The dataset is structurally labeled in 49 categories. To benchmark the dataset, we have used various classical methods and state-of-art deep models. The dataset will be publicly available at: https://github.com/skarifahmed/NecklaceFIR .
Decentralized Energy Management in Smart Cities Using Blockchain Technology

As electricity is a high demand technology in any society and city, the technology of distribution and billing must evolve within a certain time.There are lots of limitations present in current techniques, like we can’t access live information of previous unit usage, users’ privacy is also not maintained here, and one more important thing is that there is no transparency in the payment system. To solve such types of problems, we present a blockchain-based model for usage in smart cities. This will not only maintain transparency but will also maintain the anonymity of users. This paper determines how we can implement blockchain technology in smart cities to facilitate the development of smart cities and delivers a smart city ecosystem based model which depends on smart metres using blockchain technology, which will also build a smart contract between citizens and administrations. This paper will show how the reading of electric metres can be stored in the blockchain and how we can protect the privacy of users using blockchain technology. At the end, citizens will make payments without revealing their privacy. At the end of this paper, we will also conclude how we can do energy management using the data stored in the blockchain.
Online Shopping Fake Reviews Detection Using Machine Learning

Online shopping has drastically reduced the tiresome job of reaching out to offline stores and selecting goods in a limited product range. Almost everything is available online, right from the basic essential goods to costlier electrical appliances in today's world. The sellers increasingly misuse these massive online platforms for increasing their product sales by posting false reviews. Consumer engagement reports suggest that around 82% of customers read online reviews before purchasing a product online. So these reviews are crucial for them to decide if the product suits them and is reliable. So, in this paper, we propose various machine learning models for detecting fake reviews and delineate and do a comparative analysis of each model to determine the best algorithm. This work plays a vital role in reducing and checking fake reviews.
Towards an Accessibility Evaluation of eLearning Tools in Emerging 3D Virtual Environments Like Metaverse: Taking Advantage of Acquired Knowledge in Moodle and Second Life

Second Life (SL), a widely used Three-Dimensional Virtual World (3DVW), is being used as a tool for eLearning because of its interaction capabilities. On the other hand, accessibility is growing as a requirement for eLearning software tools, mainly because it can be an important factor for the learning process success by the impaired. In this paper we evaluate the accessibility of SL under WCAG 2.0/2.1 principles, and compare it with Moodle, which is a widely used web-based eLearning tool. We found that neither Moodle nor SL complies with accessibility under WCAG 2.0/2.1 rules, but SL does present itself as more accessible than Moodle. Moreover, WCAG 2.0/2.1 has several requirements specific for web pages, although they do not apply to 3DVWs. In this study, we intent to give a step forward towards evaluating emerging 3D virtual environments, like Metaverse, in the context of accessibility in the eLearning tools.
Towards a Secure Peer-to-Peer Federated Learning Framework

Machine learning techniques are used in numerous applications to identify complex patterns and relationships in data. However, data of a single actor is often insufficient, as large amounts of data are required to train powerful machine learning models. One approach to tackle this problem is the federated training of models by multiple cooperating entities. However, the cooperation raises security and privacy concerns, especially if competitors are involved that do not want to share business critical training data with each other. We present a secure, privacy preserving, decentralized P2P Federated Learning framework to address these issues. The framework eliminates the need to establish a central trusted server for model training, which often represents a communication bottleneck or single point of failure. Various steps of data preprocessing, as well as the aggregation of individual models are carried out by means of Secure Multi-Party Computation to protect the data of training participants. We describe our experiments to demonstrate the basic feasibility of the working prototype and highlight open technical and methodological issues that we aim to address in the future.