Reviewerly/exHarmony huggingface dataset:
Self Attention based Bi-directional Long Short-Term Memory Auto Encoder for Video Anomaly Detection

In view of the existing algorithm ignoring global features and focusing only on local features, a Self Attention based Bi-directional long short-term memory Auto Encoder (SABiAE) is proposed. In this method, in order to solve problem of insufficient attention to some region in existing methods, the self attention based encoder is used to extract the global appearance features; in order to reduce the loss of relevant target features, the self attention based bi-directional long short-term memory network is extract the global temporal features. By using SABiAE to capture the global information between video frames, the decoder is used to reconstruct the global appearance and temporal features into a reconstructed frame. Thus, the reconstruction error between the reconstructed frame and the actual frame is calculated to judge the anomaly. Experimental results on two public datasets, UCSD ped2 and CUHK Avenue, show that the proposed method can accurately detect abnormal events in video, with frame level AUC of 95.6% and 84.7% respectively.
A Novel Framework for Sanskrit-Gujarati Symbolic Machine Translation System

Sanskrit falls under the Indo-European language family category.Gujarati, which has descended from the Sanskrit language, is a widely spoken language particularly in the Indian state of Gujarat.The proposed and realized Machine Translation framework uses a grammatical transfer approach to translate the written Sanskrit language to Gujarati.Because both languages are morphologically rich, studying the morphology of each item is difficult but necessary to incorporate into implementation.To improve the implementation accuracy and translation clarity, an in-depth research of the creation of Nouns, Verbs, Pronouns, and Indeclinables, as well as their mappings, has been carried out.Tokenization, lemmatization, morphological analysis, Sanskrit-Gujarati bilingual synonym-based dictionary, language synthesis, and transliteration are the proposed framework's primary components.The implementation outcome was tested on 1,000 phrases, using the automated Bilingual Evaluation Understudy (BLEU) scale which yielded a value of 58.04It was also tested on the ALPAC scale, yielding the Intelligibility score of 69.16 and the Fidelity score of 68.11.The results are encouraging and prove that the proposed system is promising and robust for the implementation in the real world applications.
UGent-T2K at the 2nd DialDoc Shared Task: A Retrieval-Focused Dialog System Grounded in Multiple Documents

This work presents the contribution from the Text-to-Knowledge team of Ghent University (UGent-T2K) 1 to the MultiDoc2Dial shared task on modeling dialogs grounded in multiple documents.We propose a pipeline system, comprising (1) document retrieval, (2) passage retrieval, and (3) response generation.We engineered these individual components mainly by, for (1)-( 2), combining multiple ranking models and adding a final LambdaMART reranker, and, for (3), by adopting a Fusion-in-Decoder (FiD) model.We thus significantly boost the baseline system's performance (over +10 points for both F1 and SacreBLEU).Further, error analysis reveals two major failure cases, to be addressed in future work: (i) in case of topic shift within the dialog, retrieval often fails to select the correct grounding document(s), and (ii) generation sometimes fails to use the correctly retrieved grounding passage.
Towards Establishing Consistent Proposal Binning Methods for Unimodal and Multimodal Interaction Elicitation Studies

More than two hundred papers on elicitation studies have been published in the last ten years. These works are mainly focused on generating user-defined gesture sets and discovering natural feeling multimodal interaction techniques with virtual objects. Few papers have discussed binning the elicited interaction proposals after data collection. Binning is a process of grouping the entire set of user-generated interaction proposals based on similarity criteria. The binned set of proposals is then analyzed to produce a consensus set, which results in the user-defined interaction set. This paper presents a formula to use when deciding how to bin interaction proposals, thus helping to establish a more consistent binning procedure. This work can provide human-computer interaction (HCI) researchers with the guidance they need for interaction elicitation data processing, which is largely missing from current elicitation study literature. Using this approach will improve the efficiency and effectiveness of the binning process, increase the reliability of us er-defined interaction sets, and most importantly, improve the replicability of elicitation studies.
An automated surface water cleaning robot for ponds and water reservoirs

Surface Water Cleaning robot will be used in surface water bodies like rivers, lakes, ponds where there is accumulation of wastes like plastics, etc. It is designed to collect and remove wastes from the water surface. Surface water is not only essential to humans for their everyday usage but also is a home to many living beings. This cost effective, efficient method of cleaning the water surface will collect the wastes through Conveyor Belt mechanism and deposit them in a container. Algae and Duckweed being difficult to collect are pushed to the edges of the water bodies by the help of the same robot. Constant monitoring of the wastes is done with the help of the camera module which also helps in navigation. Water propellers help the robot to drive and hence collect wastes. Monitoring of the camera and movement of the robot to collect wastes is done using an Android application with the help of Bluetooth Module. Three DC motors are used in the project two for the propellers and one for the conveyor belt. The entire structure of the robot is made with the help of recyclable plastics and waste materials. Human beings are being involved to remove the wastes from the water bodies which is adversely affecting their health. Hence the designed robot is able to constantly monitor, collect and remove wastes in an efficient way from the water surface. This Robot will not only benefit humans by cleaning water for their everyday usage but will also protect the life of aquatic animals.
A double scrambling-DNA row and column closed loop image encryption algorithm based on chaotic system

In this paper, a dynamic update algorithm of double scrambling-DNA row and column closed loop based on chaotic system is proposed. The classical scrambling and diffusion structure are used in the whole process. In the scrambling stage, a new pixel reconstruction method is proposed by combining the Hilbert curve with Knuth-Durstenfeld shuffle algorithm to overcome the shortcoming of nearby storage of Hilbert curve. This method reconstructs the pixel matrix of one-dimensional vector according to the Hilbert curve coding method, and achieves good scrambling effect, while reducing its time complexity and space complexity. In the diffusion stage, combining the plaintext row, the ciphertext row and the key row, and taking advantage of the parallel computing power and high storage density of the DNA encoding, the existing block diffusion operation is improved, and the two-round diffusion of the DNA encoding is proposed. When the last line of ciphertext is generated, the first line of ciphertext is updated and the closed-loop dynamic update of the encryption system is realized. Finally, SHA-256 is used to give the secret key and calculate the initial value of the chaotic system. The simulation results show that the “double scrambling-DNA row and column closed loop dynamic” update algorithm proposed in this paper can effectively improve the efficiency of information transmission and have high security.
Tied Bilateral learning for Aquaculture Image Enhancement

Aquaculture plays a critical role in food security and nutrition strategies. The application of intelligent aquaculture technology has shown promising performance in improving aquaculture productivity and increasing economic benefits with its rapid advancement and good prospects. However, degraded underwater images have hampered the existing computer vision applications in intelligent aquaculture. To this end, a novel Tied Bilateral learning network is proposed for Aquaculture Image Enhancement (TBAIE), which improves the degraded aquaculture images to meet the requirements of various computer vision applications in aquaculture. Concretely, a novel multiple tied guidance module is designed to generate a multi-channel feature map and capture long-range features based on input. Then, a feature fusion module is introduced with a novel tied attention block to blend the feature and suppress noise with a low computational resource. Experimental results demonstrate that the proposed TBAIE can improve the quality of aquaculture images and remove color distortion. Moreover, TBAIE can achieve state-of-the-art in quantitative and qualitative metrics and meet the practical requirements of different aquaculture vision tasks.
Personalizing Care Through Robotic Assistance and Clinical Supervision

By 2030, the World Health Organization (WHO) foresees a worldwide workforce shortfall of healthcare professionals, with dramatic consequences for patients, economies, and communities. Research in assistive robotics has experienced an increasing attention during the last decade demonstrating its utility in the realization of intelligent robotic solutions for healthcare and social assistance, also to compensate for such workforce shortages. Nevertheless, a challenge for effective assistive robots is dealing with a high variety of situations and contextualizing their interactions according to living contexts and habits (or preferences) of assisted people. This study presents a novel cognitive system for assistive robots that rely on artificial intelligence (AI) representation and reasoning features/services to support decision-making processes of healthcare assistants. We proposed an original integration of AI-based features, that is, knowledge representation and reasoning and automated planning to 1) define a human-in-the-loop continuous assistance procedure that helps clinicians in evaluating and managing patients and; 2) to dynamically adapt robot behaviors to the specific needs and interaction abilities of patients. The system is deployed in a realistic assistive scenario to demonstrate its feasibility to support a clinician taking care of several patients with different conditions and needs.
Knowledge Graph Embedding by Adaptive Limit Scoring Loss Using Dynamic Weighting Strategy

Knowledge graph embedding aims to represent entities and relations as low-dimensional vectors, which is an effective way for predicting missing links in knowledge graphs.Designing a strong and effective loss framework is essential for knowledge graph embedding models to distinguish between correct and incorrect triplets.The classic margin-based ranking loss limits the scores of positive and negative triplets to have a suitable margin.The recently proposed Limit-based Scoring Loss independently limits the range of positive and negative triplet scores.However, these loss frameworks use equal or fixed penalty terms to reduce the scores of positive and negative sample pairs, which is inflexible in optimization.Our intuition is that if a triplet score deviates far from the optimum, it should be emphasized.To this end, we propose Adaptive Limit Scoring Loss, which simply re-weights each triplet to highlight the less-optimized triplet scores.We apply this loss framework to several knowledge graph embedding models such as TransE, TransH and ComplEx.The experimental results on link prediction and triplet classification show that our proposed method has achieved performance on par with the state of the art.
Inductive Light Graph Convolution Network for Text Classification Based on Word-Label Graph

Nowadays, Graph Convolution Networks (GCNs) have flourished in the field of text classification, such as Text Graph Convolution Network (TextGCN). But good performance of those methods is based on building a graph whose nodes consist of an entire corpus, making their models transductive. Meanwhile rich label information has not been utilized in the graph structure. In this paper, we propose a new model named Inductive Light Graph Convolution Networks (ILGCN) with a new construction of graph. This approach uses labels and words to build the graph which removes the dependence between an individual text and entire corpus, and let ILGCN inductive. Besides, we simplify the model structure and only remain the neighborhood aggregation, which is the most important part of GCNs. Experiments on multiple benchmark show that our model outperforms existing state-of-the-art models on several text classification datasets.
Findings of the Shared Task on Multi-task Learning in Dravidian Languages

We present our findings from the first shared task on Multi-task Learning in Dravidian Languages at the second Workshop on Speech and Language Technologies for Dravidian Languages.In this task, a sentence in any of three Dravidian Languages is required to be classified into two closely related tasks namely Sentiment Analyis (SA) and Offensive Language Identification (OLI).The task spans over three Dravidian Languages, namely, Kannada, Malayalam, and Tamil.It is one of the first shared tasks that focuses on Multi-task Learning for closely related tasks, especially for a very low-resourced language family such as the Dravidian language family.In total, 55 people signed up to participate in the task, and due to the intricate nature of the task, especially in its first iteration, 3 submissions have been received.
SentiStory: A Multi-Layered Sentiment-Aware Generative Model for Visual Storytelling

The visual storytelling (VIST) task aims at generating reasonable, human-like and coherent stories with the image streams as input. Although many deep learning models have achieved promising results, most of them do not directly leverage the sentiment information of stories. In this paper, we propose a sentiment-aware generative model for VIST called SentiStory. The key of SentiStory is a multi-layered sentiment extraction module (MLSEM). For a given image stream, the higher layer gives coarse-grained but accurate sentiments, while the lower layer of the MLSEM extracts fine-grained but usually unreliable ones. The two layers are combined strategically to generate coherent and rich visual sentiment concepts for the VIST task. Results from both automatic and human evaluations demonstrate that with the help of the MLSEM, SentiStory achieves improvement in generating more coherent and human-like stories.
Deep Convolution Neural Networks for Image Classification

Deep learning is a highly active area of research in machine learning community.Deep Convolutional Neural Networks (DCNNs) present a machine learning tool that enables the computer to learn from image samples and extract internal representations or properties underlying grouping or categories of the images.DCNNs have been used successfully for image classification, object recognition, image segmentation, and image retrieval tasks.DCNN models such as Alex Net, VGG Net, and Google Net have been used to classify large dataset having millions of images into thousand classes.In this paper, we present a brief review of DCNNs and results of our experiment.We have implemented Alex Net on Dell Pentium processor using MATLAB deep learning toolbox.We have classified three image datasets.The first dataset contains four hundred images of two types of animals that was classified with 99.1 percent accuracy.The second dataset contains four thousand images of five types of flowers that was classified with 86.64 percent accuracy.In the first and second dataset seventy percent randomly chosen samples from each class were used for training.The third dataset contains forty images of stained pleura tissues from rat-lungs are classified into two classes with 75 percent accuracy.In this data set eighty percent randomly chosen samples were used in training the model.
Employing Co-Learning to Evaluate the Explainability of Multimodal Sentiment Analysis

Deep neural nets are opaque black-box models with little to no understanding of underlying model dynamics. This issue is more prevalent in the case of multimodal artificial intelligence (AI) systems, where model explainability and interpretability are prime concerns due to data integration from heterogeneous data streams and complex inter and intramodal interactions. However, the traditional explainable models are challenging to apply in the multimodal scenario. We propose a co-learning-based solution for fostering model explainability for the natural language processing (NLP)-based multimodal sentiment analysis application to address this issue. The proposed approach employs explainability by obeying the co-learning principles of dealing with noisy and missing modality either at train or test time to find the modality dominance by extracting the local and global model explanations. The proposed approach is validated with post hoc explainability methods such as local interpretable model-agnostic explanations (LIME) and SHapley Additive exPlanations (SHAP) gradient-based explanations to model the modality contributions and interactions at the fusion level. The co-learning-based system ensures trust and robustness in the model by providing some degree of model explainability along with robustness. The kind of explanations provided is multifaceted and is obtained through a peek inside the black box, hence is specifically helpful for the system designers and model developers to understand the complex model dynamics that are far more challenging in the case of multimodal applications.
PANDAS@Abusive Comment Detection in Tamil Code-Mixed Data Using Custom Embeddings with LaBSE

Abusive language has lately been prevalent in comments on various social media platforms.The increasing hostility observed on the internet calls for the creation of a system that can identify and flag such acerbic content, to prevent conflict and mental distress.This task becomes more challenging when low-resource languages like Tamil, as well as the oftenobserved Tamil-English code-mixed text, are involved.The approach used in this paper for the classification model includes different methods of feature extraction and the use of traditional classifiers.We propose a novel method of combining language-agnostic sentence embeddings with the TF-IDF vector representation that uses a curated corpus of words as vocabulary, to create a custom embedding, which is then passed to an SVM classifier.Our experimentation yielded an accuracy of 52% and a macro F1-score of 0.54.
An Efficient and Optimal Deep Learning Architecture using Custom U-Net and Mask R-CNN Models for Kidney Tumor Semantic Segmentation

Today, kidney medical imaging has become the backbone for health professionals in diagnosing kidney disease and determining its severity.Physicians commonly use Computerized Tomography (CT) and Magnetic Resonance Imaging (MRI) scan models to obtain kidney disease information.The significance and impact of kidney tumor analysis drew researchers to semantic segmentation of kidney tumors.Traditional image processing methodologies, in general, require more computational power and manual assistance to analyze kidney medical images for tumor segmentation.Deep Learning advances are enabling less computational and automated models for kidney medical image analysis and tumor lineation.Blobs (regions of interest) detection from medical images is gaining popularity in kidney disease diagnosis and is used widely in detecting tumors, glomeruli, and cell nuclei, among other things.Kidney Tumor segmentation is challenging compared to other segmentation models due to morphological diversity, object overlapping, intensity variance, and integrated noise.In this paper, It have proposed a kidney tumor semantic segmentation model based on CU-Net and Mask R-CNN to extract kidney tumor information from abdominal MR images.Initially, It trained the Custom U-Net architecture on abdominal MR images with kidney masks for kidney image segmentation.The Mask R-CNN model is then used to lineate tumors from kidney images.Experiments on abdominal MR images using Python image processing libraries revealed that the proposed deep learning architecture segmented the kidney images and lined up the tumors with high accuracy.
Textomics: A Dataset for Genomics Data Summary Generation

Summarizing biomedical discovery from genomics data using natural languages is an essential step in biomedical research but is mostly done manually.Here, we introduce Textomics, a novel dataset of genomics data description, which contains 22,273 pairs of genomics data matrices and their summaries.Each summary is written by the researchers who generated the data and associated with a scientific paper.Based on this dataset, we study two novel tasks: generating textual summary from a genomics data matrix and vice versa.Inspired by the successful applications of k nearest neighbors in modeling genomics data, we propose a kNN-Vec2Text model to address these tasks and observe substantial improvement on our dataset.We further illustrate how Textomics can be used to advance other applications, including evaluating scientific paper embeddings and generating masked templates for scientific paper understanding.Textomics serves as the first benchmark for generating textual summaries for genomics data and we envision it will be broadly applied to other biomedical and natural language processing applications.
Human Language Modeling

Natural language is generated by people, yet traditional language modeling views words or documents as if generated independently.Here, we propose human language modeling (HuLM), a hierarchical extension to the language modeling problem whereby a humanlevel exists to connect sequences of documents (e.g.social media messages) and capture the notion that human language is moderated by changing human states.We introduce, HaRT, a large-scale transformer model for the HULM task, pre-trained on approximately 100,000 social media users, and demonstrate it's effectiveness in terms of both language modeling (perplexity) for social media and fine-tuning for 4 downstream tasks spanning documentand user-levels: stance detection, sentiment classification, age estimation, and personality assessment.1 Results on all tasks meet or surpass the current state-of-the-art.
Learning Adaptive Segmentation Policy for End-to-End Simultaneous Translation

End-to-end simultaneous speech-to-text translation aims to directly perform translation from streaming source speech to target text with high translation quality and low latency.A typical simultaneous translation (ST) system consists of a speech translation model and a policy module, which determines when to wait and when to translate.Thus the policy is crucial to balance translation quality and latency.Conventional methods usually adopt fixed policies, e.g.segmenting the source speech with a fixed length and generating translation.However, this method ignores contextual information and suffers from low translation quality.This paper proposes an adaptive segmentation policy for end-toend ST.Inspired by human interpreters, the policy learns to segment the source streaming speech into meaningful units by considering both acoustic features and translation history, maintaining consistency between the segmentation and translation.Experimental results on English-German and Chinese-English show that our method achieves a good accuracylatency trade-off over recently proposed stateof-the-art methods.