Reviewerly/exHarmony huggingface dataset:
The LAM Dataset: A Novel Benchmark for Line-Level Handwritten Text Recognition

Handwritten Text Recognition (HTR) is an open problem at the intersection of Computer Vision and Natural Language Processing. The main challenges, when dealing with historical manuscripts, are due to the preservation of the paper support, the variability of the handwriting – even of the same author over a wide time-span – and the scarcity of data from ancient, poorly represented languages. With the aim of fostering the research on this topic, in this paper we present the Ludovico Antonio Muratori (LAM) dataset, a large line-level HTR dataset of Italian ancient manuscripts edited by a single author over 60 years. The dataset comes in two configurations: a basic splitting and a date-based splitting which takes into account the age of the author. The first setting is intended to study HTR on ancient documents in Italian, while the second focuses on the ability of HTR systems to recognize text written by the same writer in time periods for which training data are not available. For both configurations, we analyze quantitative and qualitative characteristics, also with respect to other line-level HTR benchmarks, and present the recognition performance of state-of-the-art HTR architectures. The dataset is available for download at https://aimagelab.ing.unimore.it/go/lam.
Improvements to a temporal filter for video coding

This paper presents improvements to the original motion compensated temporal pre-filter implemented in the VVC reference model VTM and the HEVC reference model HM. The improvements involve motion estimation modifications, increasing the number of neighboring pictures to use, and filtering strength weighting depending on block error metrics. Results are presented on VTM-14.0. Tested content includes JVET common conditions test set, sequences with introduced fading and scene cuts, and additional 4:2:2 and 4:4:4 chroma format test sequences.
An Adversarial Meta-Training Framework for Cross-Domain Few-Shot Learning

Meta-learning provides a promising way for deep learning models to efficiently learn in few-shot learning. With this capacity, many deep learning systems can be applied in many real applications. However, many existing meta-learning based few-shot learning systems suffer from vulnerable generalization when new tasks are from unseen domains (a.k.a, cross-domain few-shot learning). In this work, we consider this problem from the perspective of designing a model-agnostic meta-training framework to improve the generalization of existing meta-learning methods in cross-domain few-shot learning. In this way, compared with focusing on elaborately designing modules for a specific meta-learning model, our method is endowed with the ability to be compatible with different meta-learning models in various few-shot problems. To achieve this goal, a novel adversarial meta-training framework is proposed. The proposed framework utilizes max-min episodic iteration. In the episode of maximization, our framework focuses on how to dynamically generate appropriate pseudo tasks which benefit learning cross-domain knowledge. In the episode of minimization, our method aims to solve how to help meta-learning model learn cross-task and robust meta-knowledge. To comprehensively evaluate our framework, experiments are conducted on two few-shot learning settings, three meta-learning models, and eight datasets. These results demonstrate that our method is applicable to various meta-learning models in different few-shot learning problems. The superiority of our method is verified compared with existing state-of-the-art methods.
Computer-Aided Diagnostic Tool for Classification of Colonic Polyp Assessment

Colorectal cancer is the third most common malignancy and the fourth leading cause of cancer-related deaths worldwide. This paper presents a combination of techniques (e.g., pre-processing, transfer learning, principal component analysis, and support vector machine) to detect the polyp during colonoscopy. In particular, we carefully choose the pre-trained deep convolutional neural networks (i.e., AlexNet, GoogLeNet, ResNet-50, and VGG-19) according to their performance extracting features. A publicly available database, Kvasir, is used to train and test the detection model. The result indicates that to use ResNet-50 as a pre-trained network provides the best results among the rest. Our proposed model achieves an accuracy of 99.39%, and its sensitivity and specificity are 99.39, 99.41, and 99.38%, respectively.
Evaluating Human-like Explanations for Robot Actions in Reinforcement Learning Scenarios

Explainable artificial intelligence is a research field that tries to provide more transparency for autonomous intelligent systems. Explainability has been used, particularly in reinforcement learning and robotic scenarios, to better understand the robot decision-making process. Previous work, however, has been widely focused on providing technical explanations that can be better understood by AI practitioners than non-expert end-users. In this work, we make use of human-like explanations built from the probability of success to complete the goal that an autonomous robot shows after performing an action. These explanations are intended to be understood by people who have no or very little experience with artificial intelligence methods. This paper presents a user trial to study whether these explanations that focus on the probability an action has of succeeding in its goal constitute a suitable explanation for non-expert end-users. The results obtained show that non-expert participants rate robot explanations that focus on the probability of success higher and with less variance than technical explanations generated from Q-values, and also favor counterfactual explanations over standalone explanations.
Mutation Models: Learning to Generate Levels by Imitating Evolution

Search-based procedural content generation (PCG) is a well-known method for level generation in games. Its key advantage is that it is generic and able to satisfy functional constraints. However, due to the heavy computational costs to run these algorithms online, search-based PCG is rarely utilized for real-time generation. In this paper, we introduce mutation models, a new type of iterative level generator based on machine learning. We train a model to imitate the evolutionary process and use the trained model to generate levels. This trained model is able to modify noisy levels sequentially to create better levels without the need for a fitness function during inference. We evaluate our trained models on a 2D maze generation task. We compare several different versions of the method: training the models either at the end of evolution (normal evolution) or every 100 generations (assisted evolution) and using the model as a mutation function during evolution. Using the assisted evolution process, the final trained models are able to generate mazes with a success rate of and high diversity of . The trained model is many times faster than the evolutionary process it was trained on. This work opens the door to a new way of learning level generators guided by an evolutionary process, meaning automatic creation of generators with specifiable constraints and objectives that are fast enough for runtime deployment in games.
An Effective 3D Text Recurrent Voting Generator for Metaverse

Metaverse is a novel innovative platform that connects users worldwide in the distributed virtual environment. People share their interests, opinions, and resources on this virtual reality platform. With this, we come to know that besides other fundamental techniques, the language generation method is also a necessity to regulate the VR environment. There are several types of language generation methods in 3D, including neural learning, such as GRU, RNN, and GPT-3, and transfer learning. This paper proposes a recurrent voting generator (RVG) system that understands the 3D text of a book and performs emotional analytics within a metaverse space. The proposed model RVG evaluates emotions through three algorithms such as the first module is a recurrent sentiment generator (RSG) that analyzes emotions and calculates and generates the distributions. The second module is the sentiment decomposition (SD) that optimizes higher dimensions in Big Data. And, the third module is the compound voting learning (CVL) module that performs calculations with an emphasis on optimal performance. The dataset used to evaluate RVG is based on the content of movie reviews and books. The performance evaluation shows that the proposed approach outperforms better compared to the existing 2D RNN models in the metaverse.
One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones

We study the problem of developing autonomous agents that can follow human instructions to infer and perform a sequence of actions to complete the underlying task. Significant progress has been made in recent years, especially for tasks with short horizons. However, when it comes to long-horizon tasks with extended sequences of actions, an agent can easily ignore some instructions or get stuck in the middle of the long instructions and eventually fail the task. To address this challenge, we propose a modelagnostic milestone-based task tracker (M-TRACK) to guide the agent and monitor its progress. Specifically, we propose a milestone builder that tags the instructions with navigation and interaction milestones which the agent needs to complete step by step, and a milestone checker that systemically checks the agent's progress in its current milestone and determines when to proceed to the next. On the challenging ALFRED dataset, our M-Track leads to a notable 33% and 52% relative improvement in unseen success rate over two competitive base models.
Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation

Text-based video segmentation aims to segment the target object in a video based on a describing sentence. Incorporating motion information from optical flow maps with appearance and linguistic modalities is crucial yet has been largely ignored by previous work. In this paper, we design a method to fuse and align appearance, motion, and linguistic features to achieve accurate segmentation. Specifically, we propose a multi-modal video transformer, which can fuse and aggregate multi-modal and temporal features between frames. Furthermore, we design a language-guided feature fusion module to progressively fuse appearance and motion features in each feature level with guidance from linguistic features. Finally, a multi-modal alignment loss is proposed to alleviate the semantic gap between features from different modalities. Extensive experiments on A2D Sentences and J-HMDB Sentences verify the performance and the generalization ability of our method compared to the state-of-the-art methods.
Intra-Class Adaptive Augmentation With Neighbor Correction for Deep Metric Learning

Deep metric learning aims to learn an embedding space, where semantically similar samples are close together and dissimilar ones are repelled against. To explore more hard and informative training signals for augmentation and generalization, recent methods focus on generating synthetic samples to boost metric learning losses. However, these methods just use the deterministic and class-independent generations (e.g., simple linear interpolation), which only can cover the limited part of distribution spaces around original samples. They have overlooked the wide characteristic changes of different classes and can not model abundant intra-class variations for generations. Therefore, generated samples not only lack rich semantics within the certain class, but also might be noisy signals to disturb training. In this paper, we propose a novel intra-class adaptive augmentation (IAA) framework for deep metric learning. We reasonably estimate intra-class variations for every class and generate adaptive synthetic samples to support hard samples mining and boost metric learning losses. Further, for most datasets that have a few samples within the class, we propose the neighbor correction to revise the inaccurate estimations, according to our correlation discovery where similar classes generally have similar variation distributions. Extensive experiments on five benchmarks show our method significantly improves and outperforms the state-of-the-art methods on retrieval performances by 3%-6%.
White Blood Cells type Detection using YOLOv5

White blood cells (WBCs) is the main defense system for human health. It tells about the person health and genetic information, which are able to describe any abnormality occur. A healthy white blood cells are able to identify by its visual characteristic. A common white blood cells disease is the leukemia, which it affect the production of healthy cells. It is crucial to diagnose as soon as possible in order to perform further treatment. Current blood smear diagnoses are done manually by pathologist and it can be very time consuming. This problem can be resolve by developing a computer aided detection system. This paper studied the effect of the latest state-of-art detection model, YOLOv5 on detecting each type of white blood cells (eosinophil, lymphocyte, monocyte, and neutrophil). It is then trained with 2800 images from Kaggle.com public dataset. YOLOv5 have four different model sizes, which are YOLOv5s, YOLOv5m, YOLOv51 and the YOLOv5x. The YOLOv5m achieved the highest mean average precision (mAP) value of 99.42% with 90 minutes of training time.
Personalization of a Mid-Air Gesture Keyboard using Multi-Objective Bayesian Optimization

We present AdaptiKeyboard, a mid-air gesture keyboard that uses multi-objective Bayesian optimization to adaptively change layout size to simultaneously optimize speed and accuracy. Gesture keyboards are well suited for enabling mid-air text entry in augmented reality (AR) due to their relative robustness to articulation inaccuracy. However, transplanting gesture keyboards to AR involves a larger design and operational space compared to touchscreen interactions. One potential advantage of this larger design and operational space is that mid-air keyboards presented in AR can be more versatile than their touchscreen equivalents. A key component of a mid-air gesture keyboard is the layout size, which can be made adaptive in order to optimize text entry speed and accuracy at the individual user level. This adaptive personalization can refine the keyboard design to reflect the differences users exhibit in motor behaviors and personal preferences. In this paper, we propose a multi-objective Bayesian optimization approach for adapting the layout size of a mid-air gesture keyboard to individual users. We show that this process can deliver a 14.4% improvement in speed and a 13.8% improvement in accuracy relative to a baseline design with a constant size derived from the default system keyboard on the HoloLens 2.
Identity-Aware Hand Mesh Estimation and Personalization from RGB Images

Reconstructing 3D hand meshes from monocular RGB images has attracted increasing amount of attention due to its enormous potential applications in the field of AR/VR. Most state-of-the-art methods attempt to tackle this task in an anonymous manner. Specifically, the identity of the subject is ignored even though it is practically available in real applications where the user is unchanged in a continuous recording session. In this paper, we propose an identity-aware hand mesh estimation model, which can incorporate the identity information represented by the intrinsic shape parameters of the subject. We demonstrate the importance of the identity information by comparing the proposed identity-aware model to a baseline which treats subject anonymously. Furthermore, to handle the use case where the test subject is unseen, we propose a novel personalization pipeline to calibrate the intrinsic shape parameters using only a few unlabeled RGB images of the subject. Experiments on two large scale public datasets validate the state-of-the-art performance of our proposed method.
Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data

Transfer learning for GANs successfully improves generation performance under low-shot regimes. However, existing studies show that the pretrained model using a single benchmark dataset is not generalized to various target datasets. More importantly, the pretrained model can be vulnerable to copyright or privacy risks as membership inference attack advances. To resolve both issues, we propose an effective and unbiased data synthesizer, namely Primitives - PS, inspired by the generic characteristics of natural images. Specifically, we utilize 1) the generic statistics on the frequency magnitude spectrum, 2) the elementary shape (i.e., image composition via elementary shapes) for representing the structure information, and 3) the existence of saliency as prior. Since our synthesizer only considers the generic properties of natural images, the single model pretrained on our dataset can be consistently transferred to various target datasets, and even outperforms the previous methods pretrained with the natural images in terms of Fréchet inception distance. Extensive analysis, ablation study, and evaluations demonstrate that each component of our data synthesizer is effective, and provide insights on the desirable nature of the pretrained model for the transferability of GANs.
Reward Once, Penalize Once: Rectifying Time Series Anomaly Detection

While anomaly detection in time series has been an active area of research for several years, most recent approaches employ an inadequate evaluation criterion leading to an inflated F1 score. We show that a rudimentary Random Guess method can outperform state-of-the-art detectors in terms of this popular but faulty evaluation criterion. In this work, we propose a proper evaluation metric that measures the timeliness and precision of detecting sequential anomalies. Moreover, most existing approaches are unable to capture temporal features from long sequences. Self-attention based approaches, such as transformers, have been demonstrated to be particularly efficient in capturing long-range dependencies while being computationally efficient during training and inference. We also propose an efficient transformer approach for anomaly detection in time series and extensively evaluate our proposed approach on several popular benchmark datasets.
FH-Net: A Fast Hierarchical Network for Scene Flow Estimation on Real-World Point Clouds

Estimating scene flow from real-world point clouds is a fundamental task for practical 3D vision. Previous methods often rely on deep models to first extract expensive per-point features at full resolution, and then get the flow either from complex matching mechanism or feature decoding, suffering high computational cost and latency. In this work, we propose a fast hierarchical network, FH-Net, which directly gets the key points flow through a lightweight Trans-flow layer utilizing the reliable local geometry prior, and optionally back-propagates the computed sparse flows through an inverse Trans-up layer to obtain hierarchical flows at different resolutions. To focus more on challenging dynamic objects, we also provide a new copy-and-paste data augmentation technique based on dynamic object pairs generation. Moreover, to alleviate the chronic shortage of real-world training data, we establish two new large-scale datasets to this field by collecting lidar-scanned point clouds from public autonomous driving datasets and annotating the collected data through novel pseudo-labeling. Extensive experiments on both public and proposed datasets show that our method outperforms prior state-of-the-arts while running at least $$\boldsymbol{7 \times }$$ faster at $$\boldsymbol{113}$$ FPS. Code and data are released at https://github.com/pigtigger/FH-Net .
OraB - A Community of Donors

In today’s time healthcare is a sensitive subject for anyone living in any part of the world. Medical science works on enhancing the quality and period of life. To provide healthy life at times people might have to undergo surgeries or transplantation as well. The world has improved a lot in technologies and with the advancement; security in the process has been major concern. In times of need for organs or blood, it’s crucial to have a secure and transparent system. People tend to panic in the situation and information about organ or blood donors is not easily accessible when needed. Thus, in this paper a web application is proposed for both organ and blood donors in one place. Or(organ) and B(blood) together is combined to make OraB - a platform for building a community of donors. The aim is to make a decentralized web application that will be a reliable platform for connecting donee with the donors. As it’s a decentralized web app, it will draw out the involvement of a third-party networks completely. Vanilla JS has been used for the front-end, Web3.js has been used for the back-end, and its smart contract is made using javascript similar language solidity. This is a blockchain-based project which assures complete security and transparency in tracking the transaction of organ donation and is quite economical.
A Nobel Approach to Recommend Online Courses by using Hybrid Recommendation System

The Recommender systems, also known as recommendation engines are a relatively new but extremely indemand machine learning application being extensively used by websites. They achieve the function of a highly intelligent and responsive virtual salesman on the website by recommending or pitching an idea to the customers in which he/she might be potentially interested. The primary purpose of recommender systems is broadening the profit of the enterprise by simulating sale of maximum products to a customer and to enhance the user experience of the website. Such recommender systems are being widely developed using machine learning concepts, implemented by the Python language, by online enterprises big and small all across the globe. This work exhibits a recommender scheme for an e-learning website which bids various online courses to its users for learning new skills on an online platform. A composite method is functional to implement the same which in turn uses the two basic methods namely content based and collaborative filtering in a non-inclusive manner.
GigaDepth: Learning Depth from Structured Light with Branching Neural Networks

Structured light-based depth sensors provide accurate depth information independently of the scene appearance by extracting pattern positions from the captured pixel intensities. Spatial neighborhood encoding, in particular, is a popular structured light approach for off-the-shelf hardware. However, it suffers from the distortion and fragmentation of the projected pattern by the scene’s geometry in the vicinity of a pixel. This forces algorithms to find a delicate balance between depth prediction accuracy and robustness to pattern fragmentation or appearance change. While stereo matching provides more robustness at the expense of accuracy, we show that learning to regress a pixel’s position within the projected pattern is not only more accurate when combined with classification but can be made equally robust. We propose to split the regression problem into smaller classification sub-problems in a coarse-to-fine manner with the use of a weight-adaptive layer that efficiently implements branching per-pixel Multilayer Perceptrons applied to features extracted by a Convolutional Neural Network. As our approach requires full supervision, we train our algorithm on a rendered dataset sufficiently close to the real-world domain. On a separately captured real-world dataset, we show that our network outperforms state-of-the-art and is significantly more robust than other regression-based approaches.