Reviewerly/exHarmony huggingface dataset:
Ontology-Based Method for Semantic Association Rules

Finding interesting association rules is a popular and current topic in data mining. The Apriori family of algorithms is built around two rule extraction measures: support and confidence. Even though these two measures are easy to compute, they yield many rules, the majority of which are redundant and may not be of interest to the user. Also, by themselves, support and confidence do not generate strong rules. Additional measures are required to mine interesting facts from data. Ontologies have become the fundamental building blocks for structuring and formalizing data. With the semantic structuring of information, the implicit relationship between data elements makes the analyst get important facts from the data. Our study proposes a novel framework for interestingness in data by combining domain ontology with semantic interestingness measures. The ontology-based method infers rules that are semantically enriched and strong. We analyze the quality of the rule considering the factors defined by the domain experts. It is observed that our methodology generates semantically enriched rules that are more acceptable to domain experts.
Robust CNN-based Camera Model Identification against Fixed L<sub>∞</sub>-norm Adversarial Examples

Various machine-learning and deep-learning models misclassify adversarial examples, which are formed by adding small perturbations to the original image. Image forensics is concerned with either the source camera identification or the identification of an image’s editing history. Convolution neural network (CNN) based forensic methods are constantly being used as they can learn the traces left in an image, that relates to both the camera model and the manipulation. However, these CNNs are vulnerable to adversarial examples. One of the most effective defenses against adversarial images is to train the classifier with adversarial perturbed images. But adversarial training takes a very long time due to many gradient computations to the pixel values. Also, steganalysis, which aims to unearth an image’s hidden message, is a natural choice for adversarial detection since both detect manipulations on the input. This paper proposes a robust method using a weighted steganalysis algorithm before passing it to a free adversarial trained camera- model classifier network. Five camera models have been used from the VISION dataset with DenseNet as the CNN classifier. The robust accuracy against the original images was 91.6 %, and for adversarial examples, it is 43.8 %. With the proposed filter, the robust accuracy against adversarial examples increases to 58.1%; however, the accuracy for the original images drops to 89.6 %
Speech to Speech Based Effortless Malayalam Dictionary Using Kaldi and Effect of CVR Modification on Isolated Word Recognition

Speech is one of the natural and easiest ways of communication among human beings. The applications for speech recognition are rapidly growing day by day. Malayalam is a Dravidian language spoken by 34 million people in India by the people in Kerala, the union territories of Lakshadweep and Puducherry, and it is a low resource language. The objective of this paper is to develop a speech to speech Malayalam dictionary based on Automatic Speech Recognition (ASR) using the Kaldi ASR toolkit. The dictionary consists of hundred isolated utterances in Malayalam. The meaning of these complex words can be obtained as an output speech utterance from the system when the system is prompted with the corresponding speech input. Kaldi toolkit is highly flexible and adaptive. We have used Mel Frequency Cepstral Coefficients (MFCC) as features in the ASR Module. The recognizer is tested with two sets of test data. The first set obtained a WER of 29% for monophone training, 21% for triphone training, 17% for triphone LDA training and 16% for triphone SAT training respectively. The second set obtained a WER of 19% for speaker 1, and 10% for speaker 2, for triphone SAT training. Using the same setup, it is possible to increase the vocabulary size so as to handle all the unfamiliar words in Malayalam. This will help users to use it as a fully functional speech activated Malayalam dictionary. The paper also examines the effect of Consonant Vowel Ratio (CVR) modification on isolated Malayalam word recognition when the speech input is corrupted by additive noise.
Automated Analysis and Behavioural Prediction of Interview Performance using Computer Vision

The behavioral interview is an interviewing technique that employers use to evaluate a potential employee. To accelerate the recruiting process and make recruiting on a simple scale while improving the applicant experience. This proposed model automatically analyses and predicts the non-verbal characteristics of an interviewee and also provides relevant feedback. This video-based analysis includes facial expressions and head gestures. To implement such a model, various regression, classification, and deep learning algorithms are generally used. By using the Random Forest algorithm and deep-learning frameworks we are predicting the consequences and prescribing feedback based on the behavior of the candidates. Furthermore, we will test the hypothesis "The first impression is the best impression," as well as output the attributes that the candidate lacked throughout the interview and provide essential suggestions on how to enhance performance.
An Efficient Floor Plan Classification with Optimized Image Features using Machine Learning

Adding pictures is a necessary part of advertising a property for sale. Agents typically do not label images and even if they do they are not standards for such labeling. Floor plan is one of the essential categories of listing images that real estate portals would like to highlight and attract attention to. When volumes are small, manual annotation is not a problem, but there is a point where this becomes too burdensome and ultimately infeasible. Here, we propose an approach to radically increase the efficiency of such tasks. We present a novel algorithm to classify floor plan images of any type with the help of unique intrinsic image features such as mean saturation, dominant color extraction etc. and machine learning. The overall pipeline can accept any listing image, extract its features and predict whether the given image is a floor plan or not in a single shot. The overall experimentation shows that there is a performance improvement when comparing with a deep learning pipeline for the same and nearly 100% accuracy with the test data. We are also showing some additional experimentation results with other ML models apart from the best one.
TinyML Based Physical Computing for Education

Neural networks, a type of machine learning, are generally big in size, from tens of MBs to a few GBs because they have millions of parameters such as weights and biases. They also need huge number of computations leading to the requirement of high memory and computational capacity in systems running them. However, edge devices have limited resources such as memory and computing power, have limited internet connectivity and are powered by limited battery capacity. These big models need high end processors, graphics processing units and hardware accelerators to run their training and inference phases. So, these models cannot fit in the resource-constrained edge devices. However, edge devices are excellent for Physical Computing which offers an interactive paradigm for learning. At the same time, using design paradigms like TinyML, the size of the models can be reduced enabling their deployment for inference on edge devices. This paper first discusses TinyML, with Raspberry Pi (RPi) as an example edge device, and then focuses on using such devices running tiny models to study opportunities for application to education, primarily tertiary education. We demonstrate such an application that uses You Look Only Once (YOLOv3) model to provide on-demand informative content for images captured through RPi camera thus demonstrating learning through physical computing.
Analysis of the key components of segmentation-free bilingual OCR for mobile phones

The recognition of text from camera-captured images using mobile phones has tremendous applications. The research work to drive mobile phone-based applications is much needed and ubiquitous. One of the applications is the transliteration of text in the image from one language into another. We need to recognize the text using an OCR engine and then perform transliteration. Here, we focus on the problems encountered while developing an OCR engine to recognize bilingual (Kannada and English) text from camera-captured images. A number of components are involved in building a bilingual OCR engine. We need a large corpus of real-world images to evaluate the OCR engine on camera-captured images. We need a neural network model that can handle text in two different languages without hassle. We need the model to run on mobile phones and recognize the text in the image. In this work, we analyze the challenges involved in achieving high performance. Still, there is scope for improvement in recognizing out-of-vocabulary words, which were not part of training the model.
Integration of Discriminate Features and Similarity Preserving for Unsupervised Domain Adaptation

Visual domain adaptations utilize source domain knowledge to train robust classifiers for the target domain. Reducing the gap between domain feature distributions could benefit from unsupervised domain adaptation. We have sufficient labeled source and unlabeled target domains in this scenario, but they come from distinct distributions while utilizing the same label space. As a result, applying information directly from the source to the target domain gives poor performance. To solve the issues, we propose the method IDFSP (Integrated Discriminative Features and Similarity Preserving) for unsupervised domain adaptation. In this paper, we apply entropy regularization for clustering and adapting the neighbors of unlabeled target domains to solve the problem. The discriminative and domaininvariant features for the source and target domains are combined. We consider retaining the discrimination and likeness of domains by viewing the linear and non-linearity data. We consider retaining the discrimination and likeness of domains by viewing the linear and non-linearity data. We carried out extensive experiments and confirmed our approach outperforms numerous state-of-the-art domain adaptation methods on four cross-domain visual tasks and the Amazon review sentiment analysis task.
Time Series prediction of Solar Radiation using Transformer Neural Networks

Global Solar Radiation (GSR) at any location is an essential requirement for the design of solar energy systems to be installed at that location. However, unlike conventional sources of energy the unpredictability of Solar Radiation is a major challenge in estimating and planning the usage of Solar energy systems. This paper presents the study, modelling and estimation of Global Solar Radiation (GSR) by considering solar radiation data as a time series function and using Transformer Neural Networks and Recurrent Neural Networks (RNNs). A Transformer Neural Network is the latest architecture in Machine Learning (ML) for handling sequenced data using attention mechanism. The results are also compared to other Machine learning algorithms handling time series data like Long Short Term Memory (LSTM) of RNNs. The results are compared using Root Mean Squared Error (RMSE) and Coefficient of Determinant (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) values. It was found that Transformer model provided comparable accuracy to LSTM model but was highly time efficient and stable while handling larger input and output data.
Factors Influencing Cost and Performance of Federated and Centralized Machine Learning

With the growing concern of the customer data privacy, serving them with personalized engaging experiences through Machine Learning (ML) models built on centralized servers is becoming a challenge. Recent developments such as Federated Learning(FL), which is a privacy preserving ML scheme are gaining much attention. In FL model training happens with data federated across devices and not leaving them to sustain user privacy. This is ensured by making the untrained or partially trained models to reach directly the individual devices and getting locally trained "on-device" using the device owned data, and the server aggregating all the partially trained model learnings to update a global model. Although almost all the model learning schemes in the federated learning setup use gradient descent, there are certain characteristic differences brought about by the non-IID nature of the data availability, that affects the training in comparison to the centralized schemes. In this paper, we discuss the various factors that affect the federated learning training, because of the non-IID distributed nature of the data, as well as the inherent differences in the federating learning approach as against the centralized gradient descent techniques. We empirically demonstrate the effect of number of samples per device and the distribution of labels on federated learning. In addition to the privacy advantage we seek through federated learning, we also study if there is a cost advantage while using federated learning frameworks. The cost includes the cloud infrastructure cost for training and deployment, including communication costs, and download and upload cost(of models). We show that federated learning does have an advantage in cost when the model sizes to be trained are not reasonably large. All in all, we present the need for careful design of model for both performance and cost advantages of FL system adoption across services.
Real-Time Privacy-Preserving Fall Detection using Dynamic Vision Sensors

In this fast-paced world, the only people who have to stay at home for the longest time are the sick or the elderly. Indoor surveillance for monitoring the safety of these individuals in real-time could help in the detection and prevention of fatal accidents. Achieving this task while preserving privacy brings us to the new research trend of privacy-preserving human fall detection. In this paper, we attempt to solve the problem of privacy-preserving fall detection, a subtask of human action recognition, using the Dynamic Vision Sensor (DVS). We demonstrate the effectiveness of this approach by performing real-time fall detection with a 3D-Convolutional Neural Network (3D-CNN). Our proposed methods achieved average sensitivity and specificity of 99.34% and 100% respectively. Due to the small memory footprint, number parameters, and operation count of the model, we could achieve real-time performance on an edge device.
Joint Cross-Domain Preserving and Distribution Adaptation for Heterogeneous Domain Adaptation

Domain adaptation (DA) is an efficient approach that transfers the knowledge gained from the source domain to learn an accurate classifier for the target domain. But the majority of the current domain adaptation algorithms assume that both the training and testing data have similar feature space with unaligned data distribution. Such algorithms fail when the feature representations vary. Heterogeneous Domain Adaptation (HDA) effectively addresses multi-modal data analysis where the features and distribution of source and target domain data are different. But, some of the existing heterogeneous domain adaptation approaches utilize either statistical or geometrical data alignment. Hence, we propose Joint Cross-Domain Preserving and Distribution Adaptation (JCDDA) for semi-supervised HDA, which learns a new latent subspace where the data samples from the source and target domain are projected. JCDDA preserves the cross-domain structure of the data and minimizes the discrimination differences between the domains. As our approach is semi-supervised HDA, we exploit the selective pseudo-labeling method to utilize unlabelled target data. We did extensive experiments using two benchmark object recognition datasets, and from our results, it is evident that our algorithm outperforms all of them.
Smartphone Based Human Activity Recognition Using RNN Variants

The availability of sensors in smart devices and the readiness to collect vital data have paved the way to applications that recognize human activities. The patterns formed by these data can be efficiently recognized by the use of deep learning networks. This study examines the two variants of recurrent neural network, namely, long short-term memory and gated recurrent units, in recognizing human activities using raw accelerometer sensor data as well as feature matrix computed from axis data. It is interesting that better accuracy at a faster rate can be obtained by training networks using feature vectors. The public datasets UCI-HAR and WISDM and data collected using smartphone application gave highest accuracy with the LSTM and GRU combination network.
Agent-based resource scheduling in a cloud environment using the intelligent technique

Regarding trading in cloud computing services, Infrastructure As a Service(IaaS) and Software As a Service(SaaS), suppliers and clients of cloud-based software applications have access to an increasingly diverse trading market. How SaaS providers implement application auto-scaling becomes increasingly essential in varied and dynamic environments. SaaS companies employ cloud resources provided by IaaS providers to deliver various SaaS services to their customers. In order to maximize their utility, the best behavioral strategy is to minimize renting costs while providing enough processing capacity to meet customer needs. Different virtual machine (VM) instances are offered by public IaaS providers such as Amazon, each with a unique set of price options. In addition, customer service requests change over time. For SaaS providers, this challenge is first formulated in this research. Then we present a Q-learning-based cloud resource allocation technique that may reduce the waiting time required for allocating resources and thereby minimize the renting costs under dynamic IaaS facility adjustments.
Wasserstein Distance for Attention based cross modality Person Re-Identification

Cross-modality based person re-identification is a challenging task because of the high inter modality gap present between the RGB and the IR data. These systems have to learn how to discriminate between different identities as well as how to match two modalities. This has been handled using various distance metrics and custom loss functions. But the already used distance metrics might not perform well in certain cases.Hence Optimal Transport Theory, which uses the Wasserstein distance is used for the person Re-ID problem. This distance measures the effort of aligning different distributions. Attention mechanism is used to enhance the model and also to deal with noisy data. Multiple loss functions are used in this Wasserstein distance based cross-modality person Re-Id with attention. This combination of loss functions also helps with dealing the low resolution images. A model is proposed with attention mechanism where the Earth Mover’s distance is used as the distance metric instead of the usual euclidean distance. This has showed great improvement to the euclidean distance based models. The proposed model is able to achieve a rank 10 accuracy of 92.3%
Physical Activity Classification with Smartphone based Accelerometer, Gyroscope and Device Motion for Personal Diabetes Healthcare Management

Physical inactivity has a substantial negative influence on one’s health, lowers quality of life, and frequently causes cardiovascular disease, diabetes, and mobility problems. Both diabetes and the patient’s lifestyle have a significant impact on each other. Although we shouldn’t overburden most diabetes patients with technology since they can manage their condition without it, lifestyle-monitoring technology can nevertheless be helpful for both patients and their doctors. As a result, we created a method of lifestyle monitoring that makes use of smartphones, which the majority of patients already have. In this study, we demonstrate our smartphone-based system that uses the accelerometer, gyroscope and GPS incorporated into smartphones as sensors to identify, categorise, and rate running, walking, laying and standing activities. On two publicly accessible data sets, namely the UCI HAR data set and the motion sense data set for a physical activity sensor, several classification analysis approaches are explored. In comparison to previous efforts, our classification model technique significantly improves the classification of various activities. The proposed gated recurrent unit (GRU) architecture have an average accuracy of 94.91% in classifying activities.
Voice Assisted Image Captioning and VQA For Visually Challenged Individuals

Vision is one of the most vital senses for a person’s well being. More than 285 million people suffer from the problem of visual impairment. This count is predicted to be three fold in coming 30 years. Independent navigation and safe travel are difficult for these individuals, as they have difficulty perceiving information from their surrounding and communicating. Rather than relying on visual cues to guide blind people, the proposed project will help them navigate the world through the use of audio means. It will empower visually impaired individuals to explore independently by using the system to detect objects in their vicinity and without any outside assistance. Within our proposed research, we have built a mobile application which leverages the power of image processing, and deep learning techniques to identify and describe the current scene through the camera and inform it to the user by audio cues. As a result of not being able to distinguish between different objects, the already existing approaches have been limited, resulting in low performance and accuracy. With this we attempt to provide enhanced performance, better accuracy and hence a more practicable and reliable alternative.
Air Writing Recognition of Geometric Shapes using CNN

Air Writing can be considered as virtually drawing or writing characters in air using hand gestures. This paper proposes a system which is a combination of Computer Vision and Convolutional Neural Network (CNN). Computer Vision is used for detecting and tracking the trajectory of the marker tip gestures performed in front of the webcam, whereas CNN is used for the recognition of geometric shapes. The proposed model recognises the geometric shapes (Square, Triangle and Circle) drawn in air using a marker of fixed color (here blue). The recognition accuracy of the model is improved further by implementing Transfer Learning in MobileNet. The final model achieved an accuracy of 98%. The system proposed in this paper does require only a laptop camera.
A user-oriented multi-input Sentiment Analysis and Feedback System for mental state awareness necessitated due to COVID-19

Development in technology has led to a spike in sharing of opinions about different subjects on social media, for instance, movie or product reviews. Unprecedented COVID-19 led to forced isolation and affected mental health negatively. This paper introduces a system to detect users’ emotions and mental states based on provided input. Among the different data sources available on social media, real-time Twitter data is used in this analysis. Sentiment analysis can be used as a tool at various levels, right from individual to organizational development. Deep learning algorithms like LSTM and CNN lay the foundation of this system. Python libraries and Google APIs are used to add functionalities. Earlier studies only focused on detecting emotions, whereas the proposed system provides the user with a graphical analysis of detected emotions and apt suggestions like motivational quotes or videos. The system accepts multilingual text input, speech, or video input. The scope of this system is not restricted to COVID-19 related texts. This research will assist individuals and businesses and aid future development.