Reviewerly/exHarmony huggingface dataset:
The Future of Artificial Intelligence in Education 4.0

In recent years, artificial intelligence (AI) has grown exponentially. Technologies using artificial intelligence can sort through and decipher vast volumes of data from several sources to carry out a range of activities. The expansion of social production is facilitated by the development of artificial intelligence, but it also poses significant challenges to conventional wisdom in human education in the post-COVID-19 context. This chapter discusses how various educational techniques may have been influenced by the emergence of artificial intelligence technologies. The authors concentrate on the significance of using artificial intelligence in education 4.0 with a view to the green idea in light of the study. The study also looked at the kinds of green artificial intelligence technologies that are most commonly employed in the education sector. The study in this field discusses the value of technology, including artificial intelligence, and how instructors and students use them in the teaching and learning process.
Recognition and 3D Visualization of Human Body Parts and Bone Areas Using CT Images

Abstract The advent of medical imaging significantly assisted in disease diagnosis and treatment. This study introduces to a framework for detecting several human body parts in Computerised Tomography (CT) images formatted in DICOM files. In addition, the method can highlight the bone areas inside CT images and transform 2D slices into a visual 3D model to illustrate the structure of human body parts. Firstly, we leveraged shallow convolutional Neural Networks to classify body parts and detect bone areas in each part. Then, Grad-CAM was applied to highlight the bone areas. Finally, Insight and Visualization libraries were utilized to visualize slides in 3D of a body part. As a result, the classifiers achieved 98 % in F1-score in the classification of human body parts on a CT image dataset, including 1234 slides capturing body parts from a woman for the training phase and 1245 images from a male for testing. In addition, distinguishing between bone and non-bone images can reach 97 % in F1-score on the dataset generated by setting a threshold value to reveal bone areas in CT images. Moreover, the Grad-CAM-based approach can provide clear, accurate visualizations with segmented bones in the image. Also, we successfully converted 2D slice images of a body part into a lively 3D model that provided a more intuitive view from any angle. The proposed approach is expected to provide an interesting visual tool for supporting doctors in medical image-based disease diagnosis.
LEARNING THE EFFICIENT ESTIMATION TECHNIQUES FOR SUCCESSFUL SOFTWARE PROJECT MANAGEMENT

The process of software project management involves planning and supervising the development of software projects to deliver a quality product within the customer’s budget and schedule. This process begins with project planning, which includes estimating the work to be done, required resources, and project schedule. Once these activities are accomplished, a project schedule is established that defines software engineering tasks, identifies responsible parties, and specifies inter-task dependencies. The paper aims to explore project management activities and techniques for estimating project size. Overall, software project management involves managing, allocating, and timing resources to develop software that meets requirements and is delivered within budget and schedule. This paper highlights the significance of employing efficient estimation methods to achieve successful software project management. Estimation plays a critical role in the software development process, as it helps project managers to determine the resources and time required for completing the project. The paper starts by introducing the concept of software project management and highlighting its importance in delivering successful projects. The advantages and disadvantages of each technique are discussed, and guidelines for selecting the appropriate technique for a specific project are provided. The paper also explores the importance of accurate estimation in agile software development and the use of estimation tools to simplify the process. Finally, the paper concludes by summarizing the key takeaways from the discussion and emphasizing the significance of efficient estimation techniques in ensuring successful software project management.
Scene Text Recognition with Image-Text Matching-Guided Dictionary

Employing a dictionary can efficiently rectify the deviation between the visual prediction and the ground truth in scene text recognition methods. However, the independence of the dictionary on the visual features may lead to incorrect rectification of accurate visual predictions. In this paper, we propose a new dictionary language model leveraging the Scene Image-Text Matching(SITM) network, which avoids the drawbacks of the explicit dictionary language model: 1) the independence of the visual features; 2) noisy choice in candidates etc. The SITM network accomplishes this by using Image-Text Contrastive (ITC) Learning to match an image with its corresponding text among candidates in the inference stage. ITC is widely used in vision-language learning to pull the positive image-text pair closer in feature space. Inspired by ITC, the SITM network combines the visual features and the text features of all candidates to identify the candidate with the minimum distance in the feature space. Our lexicon method achieves better results(93.8% accuracy) than the ordinary method results(92.1% accuracy) on six mainstream benchmarks. Additionally, we integrate our method with ABINet and establish new state-of-the-art results on several benchmarks.
TPFNet: A Novel Text In-painting Transformer for Text Removal

Text erasure from an image is helpful for various tasks such as image editing and privacy preservation. We present TPFNet, a novel one-stage network for text removal from images. TPFNet has two parts: feature synthesis and image generation. Since noise can be more effectively removed from low-resolution images, part 1 operates on low-resolution images. Part 1 uses PVT or EfficientNet-B6 as the encoder. Further, we use a novel multi-headed decoder that generates a high-pass filtered image and a segmentation map, along with a text-free image. The segmentation branch helps locate the text precisely, and the high-pass branch helps in learning the image structure. Part 2 uses the features learned in part 1 to predict a high-resolution text-free image. To precisely locate the text, TPFNet employs an adversarial loss that is conditional on the segmentation map rather than the input image. On Oxford, SCUT, SCUT-EnsText and ICDAR2013 datasets, TPFNet outperforms recent networks on nearly all the metrics. E.g., on Oxford dataset, TPFNet has a PSNR (higher is better) of 44.2 and a text-detection precision (lower is better) of 39.0, compared to MTRNet++’s PSNR of 33.7 and precision of 50.4. The source code can be obtained from https://github.com/CandleLabAI/TPFNet .
Reading Between the Lanes: Text VideoQA on the Road

Text and signs around roads provide crucial information for drivers, vital for safe navigation and situational awareness. Scene text recognition in motion is a challenging problem, while textual cues typically appear for a short time span, and early detection at a distance is necessary. Systems that exploit such information to assist the driver should not only extract and incorporate visual and textual cues from the video stream but also reason over time. To address this issue, we introduce RoadTextVQA, a new dataset for the task of video question answering (VideoQA) in the context of driver assistance. RoadTextVQA consists of 3, 222 driving videos collected from multiple countries, annotated with 10, 500 questions, all based on text or road signs present in the driving videos. We assess the performance of state-of-the-art video question answering models on our RoadTextVQA dataset, highlighting the significant potential for improvement in this domain and the usefulness of the dataset in advancing research on in-vehicle support systems and text-aware multimodal question answering. The dataset is available at http://cvit.iiit.ac.in/research/projects/cvit-projects/roadtextvqa .
Semantic Triple-Assisted Learning for Question Answering Passage Re-ranking

Passage re-ranking in question answering (QA) systems is a method to reorder a set of retrieved passages, related to a given question so that answer-containing passages are ranked higher than non-answer-containing passages. With recent advances in language models, passage ranking has become more effective due to improved natural language understanding of the relationship between questions and answer passages. With neural network models, question-passage pairs are used to train a cross-encoder that predicts the semantic relevance score of the pairs and is subsequently used to rank retrieved passages. This paper reports on the use of open information extraction (OpenIE) triples in the form $${<subject, verb, object>}$$ for questions and passages to enhance answer passage ranking in neural network models. Coverage and overlap scores of question-passage triples are studied and a novel loss function is developed using the proposed triple-based features to better learn a cross-encoder model to rerank passages. Experiments on three benchmark datasets are compared to the baseline BERT and ERNIE models using the proposed loss function demonstrating improved passage re-ranking performance.
Consistent Nested Named Entity Recognition in Handwritten Documents via Lattice Rescoring

Named Entity Recognition (NER) on handwritten documents is often approached as a decoupled process where Handwritten Text Recognition (HTR) is used to obtain a transcription and classic NER techniques are applied to this transcription. This approach is error-prone due to the noisy nature of the HTR output. On the other hand, coupled approaches, where HTR and NER are simultaneously performed, have been proposed. These coupled approaches still have difficulties to obtain consistent tagging in certain tasks because of long-term dependencies between opening and closing tags. In this paper we propose the usage of a Finite State Automata (FSA), which acts as a tagging template during the decoding stage of a coupled HTR-NER system to syntactically constrain the output. This ensures consistent tagging and improves the performance of the base system with a substantially smaller computational cost compared to other output-constraining methods.
ESTER-Pt: An Evaluation Suite for TExt Recognition in Portuguese

Optical Character Recognition (OCR) is a technology that enables machines to read and interpret printed or handwritten texts from scanned images or photographs. However, the accuracy of OCR systems can vary depending on several factors, such as the quality of the input image, the font used, and the language of the document. As a general tendency, OCR algorithms perform better in resource-rich languages as they have more annotated data to train the recognition process. In this work, we propose ESTER-Pt, an Evaluation Suite for TExt Recognition in Portuguese. Despite being one of the largest languages in terms of speakers, OCR in Portuguese remains largely unexplored. Our evaluation suite comprises four types of resources: synthetic text-based documents, synthetic image-based documents, real scanned documents, and a hybrid set with real image-based documents that were synthetically degraded. Additionally, we provide results of OCR engines and post-OCR correction tools on ESTER-Pt, which can serve as a baseline for future work.
Diffusion Models for Document Image Generation

Image generation has got wide attention in recent times; however, despite advances in image generation techniques, document image generation having wide industry application has remained largely neglected. The previous research on structured document image generation uses adversarial training, which is prone to mode collapse and over-fitting, resulting in lower sample diversity. Since then, diffusion models have surpassed previous models on conditional and unconditional image generation. In this work, we propose diffusion models for unconditional and layout-controlled document image generation. The unconditional model achieves state-of-the-art FID 14.82 in document image generation on DocLayNet. Furthermore, our layout-controlled document image generation models beat previous state-of-the-art in image fidelity and diversity. On the PubLayNet dataset, we get an FID score of 15.02. On the complicated DocLayNet dataset, we obtained an FID score of 20.58 with $$256 \times 256$$ resolution for conditional image generation.
An End-to-End Local Attention Based Model for Table Recognition

Recently, due to the rapid development of deep learning, especially Transformer, many Transformer-based methods have been studied and proven to be very powerful for table recognition. However, Transformer-based models usually struggle to process big tables due to the limitation of their global attention mechanism. In this paper, we propose a local attention mechanism to address the limitation of the global attention mechanism. We also present an end-to-end local attention-based model for recognizing both table structure and table cell content from a table image. The proposed model consists of four main components: 1) an encoder for feature extraction; 2) the three decoders for the three sub-tasks of the table recognition problem. In the experiments, we evaluate the performance of the proposed model and the effectiveness of the local attention mechanism on the two large-scale datasets: PubTabNet and FinTabNet. The experiment results show that the proposed model outperforms the state-of-the-art methods on all benchmark datasets. Furthermore, we demonstrate the effectiveness of the local attention mechanism for table recognition, especially for big table recognition.
Multi-teacher Knowledge Distillation for End-to-End Text Image Machine Translation

Text image machine translation (TIMT) has been widely used in various real-world applications, which translates source language texts in images into another target language sentence. Existing methods on TIMT are mainly divided into two categories: the recognition-then-translation pipeline model and the end-to-end model. However, how to transfer knowledge from the pipeline model into the end-to-end model remains an unsolved problem. In this paper, we propose a novel Multi-Teacher Knowledge Distillation (MTKD) method to effectively distillate knowledge into the end-to-end TIMT model from the pipeline model. Specifically, three teachers are utilized to improve the performance of the end-to-end TIMT model. The image encoder in the end-to-end TIMT model is optimized with the knowledge distillation guidance from the recognition teacher encoder, while the sequential encoder and decoder are improved by transferring knowledge from the translation sequential and decoder teacher models. Furthermore, both token and sentence-level knowledge distillations are incorporated to better boost the translation performance. Extensive experimental results show that our proposed MTKD effectively improves the text image translation performance and outperforms existing end-to-end and pipeline models with fewer parameters and less decoding time, illustrating that MTKD can take advantage of both pipeline and end-to-end models. Our codes are available at: https://github.com/EriCongMa/MTKD_TIMT .
Analyzing the Impact of Tokenization on Multilingual Epidemic Surveillance in Low-Resource Languages

Pre-trained language models have been widely successful, particularly in settings with sufficient training data. However, achieving similar results in low-resource multilingual settings and specialized domains, such as epidemic surveillance, remains challenging. In this paper, we propose hypotheses regarding the factors that could impact the performance of an epidemic event extraction system in a multilingual low-resource scenario: the type of pre-trained language model, the quality of the pre-trained tokenizer, and the characteristics of the entities to be extracted. We perform an exhaustive analysis of these factors and observe a strong correlation between them and the observed model performance on a low-resource multilingual epidemic surveillance task. Consequently, we believe that providing language-specific adaptation and extension of multilingual tokenizers with domain-specific entities is beneficial to multilingual epidemic event extraction in low-resource settings.
Relative Position Embedding Asymmetric Siamese Network for Offline Handwritten Mathematical Expression recognition

Currently, Recurrent Neural Network(RNN)-based encoder-decoder models are widely used in handwritten mathematical expression recognition (HMER). Due to its recursive pattern, the problem of gradient disappearance or gradient explosion also exists for RNN, which makes them inefficient in processing long HME sequences. In order to solve above problems, this paper proposes a Transformer-based encoder-decoder model consisting of an asymmetric siamese network, relative position embedding Transformer (ASNRT). With the assistance of printed images, the asymmetric siamese network further narrows the difference betweeen feature maps of similar formula images and increases the encoding gap between dissimilar formula images. We insert coordinate attention into the encoder, additionally we replace RNN with Transformer as the decoder. Moreover, rotary position embedding is used, incorporating relative position information through absolute embedding ways. Given the symmetry of MEs, we adopt the bidirectional decoding strategy. Extensive experiments show that our model improves the ExpRate of state-of-the-art methods on CROHME 2014, CROHME 2016, and CROHME 2019 by 0.94 $$\%$$ , 2.18 $$\%$$ and 2.12 $$\%$$ , respectively.
Keyword Spotting Simplified: A Segmentation-Free Approach Using Character Counting and CTC Re-scoring

Recent advances in segmentation-free keyword spotting borrow from state-of-the-art object detection systems to simultaneously propose a word bounding box proposal mechanism and compute a corresponding representation. Contrary to the norm of such methods that rely on complex and large DNN models, we propose a novel segmentation-free system that efficiently scans a document image to find rectangular areas that include the query information. The underlying model is simple and compact, predicting character occurrences over rectangular areas through an implicitly learned scale map, trained on word-level annotated images. The proposed document scanning is then performed using this character counting in a cost-effective manner via integral images and binary search. Finally, the retrieval similarity by character counting is refined by a pyramidal representation and a CTC-based re-scoring algorithm, fully utilizing the trained CNN model. Experimental validation on two widely-used datasets shows that our method achieves state-of-the-art results outperforming the more complex alternatives, despite the simplicity of the underlying model.
SpaDen: Sparse and Dense Keypoint Estimation for Real-World Chart Understanding

We introduce a novel bottom-up approach for the extraction of chart data. Our model utilizes images of charts as inputs and learns to detect keypoints (KP), which are used to reconstruct the components within the plot area. Our novelty lies in detecting a fusion of continuous and discrete KP as predicted heatmaps. A combination of sparse and dense per-pixel objectives coupled with a uni-modal self-attention-based feature-fusion layer is applied to learn KP embeddings. Further leveraging deep metric learning for unsupervised clustering, allows us to segment the chart plot area into various objects. By further matching the chart components to the legend, we are able to obtain the data series names. A post-processing threshold is applied to the KP embeddings to refine the object reconstructions and improve accuracy. Our extensive experiments include an evaluation of different modules for KP estimation and the combination of deep layer aggregation and corner pooling approaches. The results of our experiments provide extensive evaluation for the task of real-world chart data extraction. Our Code is publicly available ( https://github.com/cse-ai-lab/SpaDen ).
Towards End-to-End Semi-Supervised Table Detection with Deformable Transformer

Table detection is the task of classifying and localizing table objects within document images. With the recent development in deep learning methods, we observe remarkable success in table detection. However, a significant amount of labeled data is required to train these models effectively. Many semi-supervised approaches are introduced to mitigate the need for a substantial amount of label data. These approaches use CNN-based detectors that rely on anchor proposals and post-processing stages such as NMS. To tackle these limitations, this paper presents a novel end-to-end semi-supervised table detection method that employs the deformable transformer for detecting table objects. We evaluate our semi-supervised method on PubLayNet, DocBank, ICADR-19 and TableBank datasets, and it achieves superior performance compared to previous methods. It outperforms the fully supervised method (Deformable transformer) by +3.4 points on 10% labels of TableBank-both dataset and the previous CNN-based semi-supervised approach (Soft Teacher) by +1.8 points on 10% labels of PubLayNet dataset. We hope this work opens new possibilities towards semi-supervised and unsupervised table detection methods.
Analyzing Font Style Usage and Contextual Factors in Real Images

There are various font styles in the world. Different styles give different impressions and readability. This paper analyzes the relationship between font styles and contextual factors that might affect font style selection with large-scale datasets. For example, we will analyze the relationship between font style and its surrounding object (such as “bus”) by using about 800,000 words in the Open Images dataset. We also use a book cover dataset to analyze the relationship between font styles with book genres. Moreover, the meaning of the word is assumed as another contextual factor. For these numeric analyses, we utilize our own font-style feature extraction model and word2vec. As a result of co-occurrence-based relationship analysis, we found several instances of specific font styles being used for specific contextual factors.