bluuebunny/crossref_metadata_2025 huggingface dataset:
Detection and Prediction of Nutrient Deficiency Stress using Longitudinal Aerial Imagery
Authors: Dadsetan Saba , Rose Gisele , Hovakimyan Naira , Hobbs Jennifer
Published: September 2022

Early, precise detection of nutrient deficiency stress (NDS) has key economic as well as environmental impact; precision application of chemicals in place of blanket application reduces operational costs for the growers while reducing the amount of chemicals which may enter the environment unnecessarily. Furthermore, earlier treatment reduces the amount of yield loss and therefore boosts crop production during a given season. With this in mind, we collect sequences of high-resolution aerial imagery and construct semantic segmentation models to detect and predict NDS across the field; our work sits at the intersection of agriculture, remote sensing, and deep learning. First, we establish a baseline for full-field detection of NDS and quantify the impact of pretraining, backbone architecture, input representation, and sampling strategy. We then quantify the amount of information available at different points in the season by building a single-timestamp model based on a U-Net.
Next, we construct our proposed spatiotemporal architecture, which combines a U-Net with a convolutional LSTM to accurately detect regions of the field showing NDS; this approach has an impressive IOU score of 0.53. Finally, we show that this architecture can be trained to predict regions of the field which are expected to show NDS in a later flight- potentially more than three weeks in the future- maintaining an IOU score of 0.47-0.51 depending on how far in advance the prediction is made. We will also release a dataset which we believe will benefit the computer vision, remote sensing, and agriculture fields. This work contributes to the recent developments in deep learning for remote sensing and agriculture while addressing a key social challenge with implications for economics and sustainability.
Prediction of Landfall Intensity, Location, and Time of a Tropical Cyclone
Authors: Kumar Sandeep , Biswas Koushik , Pandey Ashish Kumar
Published: September 2022

The prediction of the intensity, location and time of the landfall of a tropical cyclone well advance in time and with high accuracy can reduce human and material loss immensely. In this article, we develop a Long Short-Term memory based Recurrent Neural network model to predict intensity (in terms of maximum sustained surface wind speed), location (latitude and longitude), and time (in hours after the observation period) of the landfall of a tropical cyclone which originates in the North Indian ocean. The model takes as input the best track data of cyclone consisting of its location, pressure, sea surface temperature, and intensity for certain hours (from 12 to 36 hours) anytime during the course of the cyclone as a time series and then provide predictions with high accuracy. For example, using 24 hours data of a cyclone anytime during its course, the model provides state-of-the-art results by predicting landfall intensity, time, latitude, and longitude with a mean absolute error of 4.24 knots, 4.5 hours, 0.24 degree, and 0.37 degree respectively, which resulted in a distance error of 51.7 kilometers from the landfall location. We further check the efficacy of the model on three recent devastating cyclones Bulbul, Fani, and Gaja, and achieved better results than the test dataset.
Court Opinion Generation from Case Fact Description with Legal Basis
Authors: Li Quanzhi , Zhang Qiong
Published: September 2022

In this study, we proposed an approach to automatically generating court view from the fact description of a legal case. This is a text-to-text natural language generation problem, and it can help the automatic legal document generation. Due to the specialty of the legal domain, our model exploits the charge and law article information in the generation process, instead of utilizing just the fact description text. The BERT model is used as the encoder and a Transformer architecture is used as decoder. To smoothly integrate these two parts together, we employ two separate optimizers for the two components during the training process. The experiments on two data sets of Chinese legal cases show that our approach outperforms other methods.
Unifying Principles and Metrics for Safe and Assistive AI
Authors: Srivastava Siddharth
Published: September 2022

The prevalence and success of AI applications have been tempered by concerns about the controllability of AI systems about AI's impact on the future of work. These concerns reflect two aspects of a central question: how would humans work with AI systems? While research on AI safety focuses on designing AI systems that allow humans to safely instruct and control AI systems, research on AI and the future of work focuses on the impact of AI on humans who may be unable to do so. This Blue Sky Ideas paper proposes a unifying set of declarative principles that enable a more uniform evaluation of arbitrary AI systems along multiple dimensions of the extent to which they are suitable for use by specific classes of human operators. It leverages recent AI research and the unique strengths of the field to develop human-centric principles for AI systems that address the concerns noted above.
Thou Shalt Love Thy Neighbor as Thyself When Thou Playest: Altruism in Game Theory
Authors: Rothe Jörg
Published: September 2022

Game theory is typically used to model the interaction among (software) agents in multiagent systems and, therefore, is a key topic at leading AI conferences. Game-theoretic models, however, are often based on the assumption that agents are perfectly rational and narrowly selfish and are interested only in maximizing their own gains, no matter what the costs to the other agents are. This summary paper presents various ways of introducing certain notions of altruism into existing game-theoretic models in both noncooperative and cooperative games, in the hope that simulating altruistic behavior in AI systems will make AI better suit real-world applications—and thus may make the real world a better place.
Author Homepage Discovery in CiteSeerX
Authors: Patel Krutarth , Caragea Cornelia , Caragea Doina , Giles C. Lee
Published: September 2022

Scholarly digital libraries provide access to scientific publications and comprise useful resources for researchers. CiteSeerX is one such digital library search engine that provides access to more than 10 million academic documents. We propose a novel search-driven approach to build and maintain a large collection of homepages that can be used as seed URLs in any digital library including CiteSeerX to crawl scientific documents. Precisely, we integrate Web search and classification in a unified approach to discover new homepages: first, we use publicly-available author names and research paper titles as queries to a Web search engine to find relevant content, and then we identify the correct homepages from the search results using a powerful deep learning classifier based on Convolutional Neural Networks. Moreover, we use Self-Training in order to reduce the labeling effort and to utilize the unlabeled data to train the efficient researcher homepage classifier. Our experiments on a large scale dataset highlight the effectiveness of our approach, and position Web search as an effective method for acquiring authors' homepages. We show the development and deployment of the proposed approach in CiteSeerX and the maintenance requirements.
An Automated Engineering Assistant: Learning Parsers for Technical Drawings
Authors: Van Daele Dries , Decleyre Nicholas , Dubois Herman , Meert Wannes
Published: September 2022

Manufacturing companies rely on technical drawings to develop new designs or adapt designs to customer preferences. The database of historical and novel technical drawings thus represents the knowledge that is core to their operations. With current methods, however, utilizing these drawings is mostly a manual and time consuming effort. In this work, we present a software tool that knows how to interpret various parts of the drawing and can translate this information to allow for automatic reasoning and machine learning on top of such a large database of technical drawings. For example, to find erroneous designs, to learn about patterns present in successful designs, etc. To achieve this, we propose a method that automatically learns a parser capable of interpreting technical drawings, using only limited expert interaction. The proposed method makes use of both neural methods and symbolic methods. Neural methods to interpret visual images and recognize parts of two-dimensional drawings. Symbolic methods to deal with the relational structure and understand the data encapsulated in complex tables present in the technical drawing. Furthermore, the output can be used, for example, to build a similarity based search algorithm. We showcase one deployed tool that is used to help engineers find relevant, previous designs more easily as they can now query the database using a partial design instead of through limited and tedious keyword searches. A partial design can be a part of the two-dimensional drawing, part of a table, part of the contained textual information, or combinations thereof.
Ontology-Enriched Query Answering on Relational Databases
Authors: Ahmetaj Shqiponja , Efthymiou Vasilis , Fagin Ronald , Kolaitis Phokion G. , Lei Chuan , Özcan Fatma , Popa Lucian
Published: September 2022

We develop a flexible, open-source framework for query answering on relational databases by adopting methods and techniques from the Semantic Web community and the data exchange community, and we apply this framework to a medical use case. We first deploy module-extraction techniques to derive a concise and relevant sub-ontology from an external reference ontology. We then use the chase procedure from the data exchange community to materialize a universal solution that can be subsequently used to answer queries on an enterprise medical database. Along the way, we identify a new class of well-behaved acyclic EL-ontologies extended with role hierarchies, suitably restricted functional roles, and domain/range restrictions, which cover our use case. We show that such ontologies are C-stratified, which implies that the chase procedure terminates in polynomial time. We provide a detailed overview of our real-life application in the medical domain and demonstrate the benefits of this approach, such as discovering additional answers and formulating new queries.
Topological Machine Learning Methods for Power System Responses to Contingencies
Authors: Bush Brian , Chen Yuzhou , Ofori-Boateng Dorcas , Gel Yulia R.
Published: September 2022

While deep learning tools, coupled with the emerging machinery of topological data analysis, are proven to deliver various performance gains in a broad range of applications, from image classification to biosurveillance to blockchain fraud detection, their utility in areas of high societal importance such as power system modeling and, particularly, resilience quantification in the energy sector yet remains untapped. To provide fast acting synthetic regulation and contingency reserve services to the grid while having minimal disruptions on customer quality of service, we propose a new topology-based system that depends on a neural network architecture for impact metric classification and prediction in power systems. This novel topology-based system allows one to evaluate the impact of three power system contingency types, in conjunction with transmission lines, transformers, and transmission lines combined with transformers. We show that the proposed new neural network architecture equipped with local topological measures facilitates more accurate classification of unserved load as well as the amount of unserved load. In addition, we are able to learn more about the complex relationships between electrical properties and local topological measurements on their simulated response to contingencies for the NREL-SIIP power system.
DeepCOVID: An Operational Deep Learning-driven Framework for Explainable Real-time COVID-19 Forecasting
Authors: Rodríguez Alexander , Tabassum Anika , Cui Jiaming , Xie Jiajia , Ho Javen , Agarwal Pulak , Adhikari Bijaya , Prakash B. Aditya
Published: September 2022

How do we forecast an emerging pandemic in real time in a purely data-driven manner? How to leverage rich heterogeneous data based on various signals such as mobility, testing, and/or disease exposure for forecasting? How to handle noisy data and generate uncertainties in the forecast? In this paper, we present DeepCOVID, an operational deep learning framework designed for real-time COVID-19 forecasting. DeepCOVID works well with sparse data and can handle noisy heterogeneous data signals by propagating the uncertainty from the data in a principled manner resulting in meaningful uncertainties in the forecast. The deployed framework also consists of modules for both real-time and retrospective exploratory analysis to enable interpretation of the forecasts. Results from real-time predictions (featured on the CDC website and FiveThirtyEight.com) since April 2020 indicates that our approach is competitive among the methods in the COVID-19 Forecast Hub, especially for short-term predictions.
Carbon to Diamond: An Incident Remediation Assistant System From Site Reliability Engineers’ Conversations in Hybrid Cloud Operations
Authors: Samanta Suranjana , Gupta Ajay , Mohapatra Prateeti , Azad Amar Prakash
Published: September 2022

Conversational channels are changing the landscape of hybrid cloud service management. These channels are becoming important avenues for Site Reliability Engineers (SREs) %Subject Matter Experts (SME)
to collaboratively work together to resolve an incident or issue. Identifying segmented conversations and extracting key insights or artefacts from them can help engineers to improve the efficiency of the incident remediation process by using information retrieval mechanisms for similar incidents. However, it has been empirically observed that due to the semi-formal behavior of such conversations (human language) the conversations are very unique in nature and also contain domain-specific terms. %It is important to identify the correct keywords and artefacts like symptoms, issue etc., present in the conversation chats.
In this paper, we build a framework that taps into the conversational channels and uses various learning methods to (1) understand and extract key artefacts from conversations like diagnostic steps and resolution actions taken and (2) present an approach to identify past conversations about similar issues. Experimental results on our dataset show the efficacy of the methods used in our proposed system.
AI-Infused Collaborative Inquiry in Upper Elementary School: A Game-Based Learning Approach
Authors: Lee Seung , Mott Bradford , Ottenbreit-Leftwich Anne , Scribner Adam , Taylor Sandra , Park Kyungjin , Rowe Jonathan , Glazewski Krista , Hmelo-Silver Cindy E. , Lester James
Published: September 2022

Artificial intelligence has emerged as a technology that is profoundly reshaping society and enabling rapid improvements in science, engineering, and mathematics, as well as information technology itself. This has generated increased demand for fostering an AI-literate populace as well as a growing recognition of the importance of promoting K-12 students’ awareness and interest in AI. Although efforts are be-ginning to incorporate AI learning within K-12 education, there is little research exploring how to introduce students to AI and how to support teachers to integrate AI learning experiences in their classrooms. This is especially true at the elementary school level. A particularly promising approach for providing effective and engaging AI learning experiences for elementary students is game-based learning. In this paper, we explore how to introduce AI-infused collaborative inquiry learning into upper elementary school (student ages 8 to 11) using game-based learning. To ground the work in the realities of elementary school classrooms, we present insights from interviews with elementary school teachers to under-stand how best to support them in integrating AI into their classrooms. We then present the design of PrimaryAI, a game-based learning environment that supports rich problem-based learning activities within upper elementary classrooms centered on AI applied toward solving life-science problems. Finally, we discuss some of the challenges we face in bringing AI-infused collaborative inquiry learning to upper elementary students.
Verification and Repair of Neural Networks
Authors: Guidotti Dario
Published: September 2022

Neural Networks (NNs) are popular machine learning models which have found successful application in many different domains across computer science.
However, it is hard to provide any formal guarantee on the behaviour of neural networks and therefore their reliability is still in doubt, especially concerning their deployment in safety and security-critical applications.
Verification emerged as a promising solution to address some of these problems. In the following, I will present some of my recent efforts in verifying NNs.
On Learning Deep Models with Imbalanced Data Distribution
Authors: Majumdar Puspita , Singh Richa , Vatsa Mayank
Published: September 2022

The availability of large training data has led to the development of sophisticated deep learning algorithms to achieve state-of-the-art performance on various tasks and several applications have been benefited immensely. Despite the unparalleled success, the performance of deep learning algorithms depends significantly on the training data distribution. An imbalance in training data distribution affects the performance of deep models. Our research focuses on designing and developing solutions for different real-world problems, specifically related to facial analytic tasks, with imbalanced data distribution. These problems include injured face recognition, fake image detection, and estimation and mitigation of bias in model prediction.
An Unfair Affinity Toward Fairness: Characterizing 70 Years of Social Biases in BHollywood (Student Abstract)
Authors: Khadilkar Kunal , R. KhudaBukhsh Ashiqur
Published: September 2022

Bollywood, aka the Mumbai film industry, is one of the biggest movie industries in the world with a current movie market share of worth 2.1 billion dollars and a target audience base of 1.2 billion people. While the entertainment impact in terms of lives that Bollywood can potentially touch is mammoth, no NLP study on social biases in Bollywood content exists. We thus seek to understand social biases in a developing country through the lens of popular movies. Our argument is simple -- popular movie content reflects social norms and beliefs in some form or shape. We present our preliminary findings on a longitudinal corpus of English subtitles of popular Bollywood movies focusing on (1) social bias toward a fair skin color (2) gender biases, and (3) gender representation. We contrast our findings with a similar corpus of Hollywood movies. Surprisingly, we observe that much of the biases we report in our preliminary experiments on the Bollywood corpus, also gets reflected in the Hollywood corpus.
Robustness to Missing Features using Hierarchical Clustering with Split Neural Networks (Student Abstract)
Authors: Khincha Rishab , Sarawgi Utkarsh , Zulfikar Wazeer , Maes Pattie
Published: September 2022

The problem of missing data has been persistent for a long time and poses a major obstacle in machine learning and statistical data analysis. Past works in this field have tried using various data imputation techniques to fill in the missing data, or training neural networks (NNs) with the missing data. In this work, we propose a simple yet effective approach that clusters similar input features together using hierarchical clustering and then trains proportionately split neural networks with a joint loss. We evaluate this approach on a series of benchmark datasets and show promising improvements even with simple imputation techniques. We attribute this to learning through clusters of similar features in our model architecture.
Attention Beam: An Image Captioning Approach (Student Abstract)
Authors: Shrimal Anubhav , Chakraborty Tanmoy
Published: September 2022

The aim of image captioning is to generate textual description of a given image. Though seemingly an easy task for humans, it is challenging for machines as it requires the ability to comprehend the image (computer vision) and consequently generate a human-like description for the image (natural language understanding). In recent times, encoder-decoder based architectures have achieved state-of-the-art results for image captioning. Here, we present a heuristic of beam search on top of the encoder-decoder based architecture that gives better quality captions on three benchmark datasets: Flickr8k, Flickr30k and MS COCO.
WildfireNet: Predicting Wildfire Profiles (Student Abstract)
Authors: Sung Samuel , Li Yuping , Ortolano Leonard
Published: September 2022

Forecasting an accurate wildfire profile is an essential tool for firefighters when planning an evacuation strategy. Therefore, we propose a WildfireNet that can predict the shape of the wildfire of the next day, when given historical wildfire profiles, elevation, and weather data. The motivation behind WildfireNet is to locate fires in a precise manner and be able to accurately predict upcoming fires. The model’s architecture was built in the inspiration of U-Net, which is a Convolutional Neural Network (CNN) commonly used in a biomedical image segmentation. Intersection over Union (IoU) and recall were calculated to measure the performance of the model. The model achieved an IoU score of 0.997 in the test set. Since the objective of the model is to predict upcoming fires, pixels that were labeled as fire but not on the previous days were extracted to calculate recall. In the test set, Wild-fireNet scored a recall of around 0.75 for fires that grew slowly. Overall, WildfireNet is a novel wildfire spread model and has the potential to be a tool to aid firefighters in their decision making.
Knowledge-aware Dialogue Generation with Hybrid Attention (Student Abstract)
Authors: Zhao Yaru , Cheng Bo , Zhang Yingying
Published: September 2022

Using commonsense knowledge to assist dialogue generation is a big step forward for dialogue generation task. However, how to fully utilize commonsense information is always a challenge. Furthermore, the entities generated in the response do not match the information in the post most often. In this paper, we propose a dialogue generation model which uses hybrid attention to better generate rational entities. When a user post is given, the model encodes relevant knowledge graphs from a knowledge base with a graph attention mechanism. Then it will encode the user post and graphs with a co-attention mechanism, which effectively encodes complex related data. Through the above mechanism, we can get a better mutual understanding of post and knowledge. The experimental results show that our model is more effective than the current state-of-the-art model (CCM).
Exploration of Unknown Environments Using Deep Reinforcement Learning
Authors: McCalmon Joseph
Published: September 2022

My research presents a method for efficient exploration of an outdoor, unknown area, which aims to achieve precise coverage of regions of interest within that area. While this method for autonomous exploration was designed for autonomous controllers in unmanned aerial vehicles (UAVs), the concepts apply to any vehicle which uses autonomous navigation. We consider an environment with areas of interest of various sizes littered throughout, and a reinforcement learning agent which is tasked with discovering and mapping these areas in an efficient manner.
Efficient Robust Music Genre Classification with Depthwise Separable Convolutions and Source Separation
Authors: Mersy Gabriel
Published: September 2022

Given recent advances in deep music source separation, a feature representation method is proposed that combines source separation with a state-of-the-art representation learning technique that is suitably repurposed for computer audition (i.e. machine listening). A depthwise separable convolutional neural network is trained on a challenging electronic dance music (EDM) data set and its performance is compared to convolutional neural networks operating on both source separated and standard spectrograms. It is shown that source separation improves classification performance in a limited-data setting compared to the standard single spectrogram approach.
MOTIF-Driven Contrastive Learning of Graph Representations
Authors: Subramonian Arjun
Published: September 2022

We propose a MOTIF-driven contrastive framework to pretrain a graph neural network in a self-supervised manner so that it can automatically mine motifs from large graph datasets. Our framework achieves state-of-the-art results on various graph-level downstream tasks with few labels, like molecular property prediction.
VEGA: a Virtual Environment for Exploring Gender Bias vs. Accuracy Trade-offs in AI Translation Services
Authors: Bernagozzi Mariana , Srivastava Biplav , Rossi Francesca , Usmani Sheema
Published: September 2022

Machine translation services are a very popular class of Artificial Intelligence (AI) services nowadays but public's trust in these services is not guaranteed since they have been shown to have issues like bias. In this work, we focus on the behavior of machine translators with respect to gender bias as well as their accuracy. We have created the first-of-its-kind virtual environment, called VEGA, where the user can interactively explore translation services and compare their trust ratings using different visuals.
AutoText: An End-to-End AutoAI Framework for Text
Authors: Chaudhary Arunima , Issak Alayt , Kate Kiran , Katsis Yannis , Valente Abel , Wang Dakuo , Evfimievski Alexandre , Gurajada Sairam , Kawas Ban , Malossi Cristiano , Popa Lucian , Pedapati Tejaswini , Samulowitz Horst , Wistuba Martin , Li Yunyao
Published: September 2022

Building models for natural language processing (NLP) tasks remains a daunting task for many, requiring significant technical expertise, efforts, and resources. In this demonstration, we present AutoText, an end-to-end AutoAI framework for text, to lower the barrier of entry in building NLP models. AutoText combines state-of-the-art AutoAI optimization techniques and learning algorithms for NLP tasks into a single extensible framework. Through its simple, yet powerful UI, non-AI experts (e.g., domain experts) can quickly generate performant NLP models with support to both control (e.g., via specifying constraints) and understand learned models.
ESO-MAPF: Bridging Discrete Planning and Continuous Execution in Multi-Agent Pathfinding
Authors: Chudý Ján , Surynek Pavel
Published: September 2022

We present ESO-MAPF, a research and educational platform for experimenting with multi-agent path finding (MAPF). ESO-MAPF focuses on demonstrating the planning-acting chain in the MAPF domain. MAPF is the task of finding collision free paths for agents from their starting positions to given individual goals. The standard MAPF uses the abstraction where agents move in an undirected graph via traversing its edges in discrete steps. The discrete abstraction simplifies the planning phase however resulting discrete plans often need to be executed in the real continuous environment. ESO-MAPF shows how to bridge discrete planning and the acting phase in which the resulting plans are executed on physical robots. We simulate centralized plans on a group of OZOBOT Evo robots using their reflex functionalities and outputs on the surface of the screen that serves as the environment. Various problems arising along the planning-acting chain are illustrated to emphasize the educational point of view.
TODS: An Automated Time Series Outlier Detection System
Authors: Lai Kwei-Herng , Zha Daochen , Wang Guanchu , Xu Junjie , Zhao Yue , Kumar Devesh , Chen Yile , Zumkhawaka Purav , Wan Minyang , Martinez Diego , Hu Xia
Published: September 2022

We present TODS, an automated Time Series Outlier Detection System for research and industrial applications. TODS is a highly modular system that supports easy pipeline construction. The basic building block of TODS is primitive, which is an implementation of a function with hyperparameters. TODS currently supports 70 primitives, including data processing, time series processing, feature analysis, detection algorithms, and a reinforcement module. Users can freely construct a pipeline using these primitives and perform end- to-end outlier detection with the constructed pipeline. TODS provides a Graphical User Interface (GUI), where users can flexibly design a pipeline with drag-and-drop. Moreover, a data-driven searcher is provided to automatically discover the most suitable pipelines given a dataset. TODS is released under Apache 2.0 license at https://github.com/datamllab/tods. A video is available on YouTube (https://youtu.be/JOtYxTclZgQ)
Proof of Learning (PoLe): Empowering Machine Learning with Consensus Building on Blockchains (Demo)
Authors: Lan Yixiao , Liu Yuan , Li Boyang , Miao Chunyan
Published: September 2022

The consensus algorithm is the core component of a blockchain system, which determines the efficiency, security, and scalability of the blockchain network. The representative consensus algorithm is the proof of work (PoW) proposed in Bitcoin, where the consensus process consumes large amount of compute in solving meaningless Hash puzzel. Meanwhile, the deep learning (DL) has brought unprecedented performance gains at heavy computate cost. In this demo, we channels the otherwise wasted computational power to the practical purpose of training neural network models, through the proposed proof of learning (PoL) consensus algorithm. In PoLe, the training/testing data are released to the entire blockchain network (BCN) and the consensus nodes train NN models on the data, which serves as the proof of learning. When the consensus on the BCN considers a NN model to be valid, a new block is appended to the blockchain. Through our system, we investigate the potential of enpowering machine learning with consensus building on blockchains.
Interactive Video Object Mask Annotation
Authors: Le Trung-Nghia , Nguyen Tam V. , Tran Quoc-Cuong , Nguyen Lam , Hoang Trung-Hieu , Le Minh-Quan , Tran Minh-Triet
Published: September 2022

In this paper, we introduce a practical system for interactive video object mask annotation, which can support multiple back-end methods. To demonstrate the generalization of our system, we introduce a novel approach for video object annotation. Our proposed system takes scribbles at a chosen key-frame from the end-users via a user-friendly interface and produces masks of corresponding objects at the key-frame via the Control-Point-based Scribbles-to-Mask (CPSM) module. The object masks at the key-frame are then propagated to other frames and refined through the Multi-Referenced Guided Segmentation (MRGS) module. Last but not least, the user can correct wrong segmentation at some frames, and the corrected mask is continuously propagated to other frames in the video via the MRGS to produce the object masks at all video frames.
CNN Profiler on Polar Coordinate Images for Tropical Cyclone Structure Analysis
Authors: Chen Boyo , Chen Buo-Fu , Hsiao Chun Min
Published: September 2022

Convolutional neural networks (CNN) have achieved great success in analyzing tropical cyclones (TC) with satellite images in several tasks, such as TC intensity estimation. In contrast, TC structure, which is conventionally described by a few parameters estimated subjectively by meteorology specialists, is still hard to be profiled objectively and routinely. This study applies CNN on satellite images to create the entire TC structure profiles, covering all the structural parameters. By utilizing the meteorological domain knowledge to construct TC wind profiles based on historical structure parameters, we provide valuable labels for training in our newly released benchmark dataset. With such a dataset, we hope to attract more attention to this crucial issue among data scientists. Meanwhile, a baseline is established based on a specialized convolutional model operating on polar-coordinates. We discovered that it is more feasible and physically reasonable to extract structural information on polar-coordinates, instead of Cartesian coordinates, according to a TC’s rotational and spiral natures. Experimental results on the released benchmark dataset verified the robustness of the proposed model and demonstrated the potential for applying deep learning techniques for this barely developed yet important topic.
For codes and implementation details, please visit https://github.com/BoyoChen/TCSA-CNN-profiler.
DramaQA: Character-Centered Video Story Understanding with Hierarchical QA
Authors: Choi Seongho , On Kyoung-Woon , Heo Yu-Jung , Seo Ahjeong , Jang Youwon , Lee Minsu , Zhang Byoung-Tak
Published: September 2022

Despite recent progress on computer vision and natural language processing, developing a machine that can understand video story is still hard to achieve due to the intrinsic difficulty of video story. Moreover, researches on how to evaluate the degree of video understanding based on human cognitive process have not progressed as yet. In this paper, we propose a novel video question answering (Video QA) task, DramaQA, for a comprehensive understanding of the video story. The DramaQA focuses on two perspectives: 1) Hierarchical QAs as an evaluation metric based on the cognitive developmental stages of human intelligence. 2) Character-centered video annotations to model local coherence of the story. Our dataset is built upon the TV drama "Another Miss Oh" and it contains 17,983 QA pairs from 23,928 various length video clips, with each QA pair belonging to one of four difficulty levels. We provide 217,308 annotated images with rich character-centered annotations, including visual bounding boxes, behaviors and emotions of main characters, and coreference resolved scripts. Additionally, we suggest Multi-level Context Matching model which hierarchically understands character-centered representations of video to answer questions. We release our dataset and model publicly for research purposes, and we expect our work to provide a new perspective on video story understanding research.
DecAug: Augmenting HOI Detection via Decomposition
Authors: Fang Hao-Shu , Xie Yichen , Shao Dian , Li Yong-Lu , Lu Cewu
Published: September 2022

Human-object interaction (HOI) detection requires a large amount of annotated data. Current algorithms suffer from insufficient training samples and category imbalance within datasets. To increase data efficiency, in this paper, we propose an efficient and effective data augmentation method called DecAug for HOI detection. Based on our proposed object state similarity metric, object patterns across different HOIs are shared to augment local object appearance features without changing their states. Further, we shift spatial correlation between humans and objects to other feasible configurations with the aid of a pose-guided Gaussian Mixture Model while preserving their interactions. Experiments show that our method brings up to 3.3 mAP and 1.6 mAP improvements on V-COCO and HICO-DET dataset for two advanced models. Specifically, interactions with fewer samples enjoy more notable improvement. Our method can be easily integrated into various HOI detection models with negligible extra computational consumption.
A Systematic Evaluation of Object Detection Networks for Scientific Plots
Authors: Ganguly Pritha , Methani Nitesh S , Khapra Mitesh M. , Kumar Pratyush
Published: September 2022

Are existing object detection methods adequate for detecting text and visual elements in scientific plots which are arguably different than the objects found in natural images? To answer this question, we train and compare the accuracy of Fast/Faster R-CNN, SSD, YOLO and RetinaNet on the PlotQA dataset with over 220,000 scientific plots. At the standard IOU setting of 0.5, most networks perform well with mAP scores greater than 80% in detecting the relatively simple objects in plots. However, the performance drops drastically when evaluated at a stricter IOU of 0.9 with the best model giving a mAP of 35.70%. Note that such a stricter evaluation is essential when dealing with scientific plots where even minor localisation errors can lead to large errors in downstream numerical inferences. Given this poor performance, we propose minor modifications to existing models by combining ideas from different object detection networks. While this significantly improves the performance, there are still two main issues: (i) performance on text objects which are essential for reasoning is very poor, and (ii) inference time is unacceptably large considering the simplicity of plots. To solve this open problem, we make a series of contributions: (a) an efficient region proposal method based on Laplacian edge detectors, (b) a feature representation of region proposals that includes neighbouring information, (c) a linking component to join multiple region proposals for detecting longer textual objects, and (d) a custom loss function that combines a smooth L1-loss with an IOU-based loss. Combining these ideas, our final model is very accurate at extreme IOU values achieving a mAP of 93.44%@0.9 IOU. Simultaneously, our model is very efficient with an inference time 16x lesser than the current models, including one-stage detectors. Our model also achieves a high accuracy on an extrinsic plot-to-table conversion task with an F1 score of 0.77. With these contributions, we make a definitive progress in object detection for plots and enable further exploration on automated reasoning of plots.
DropLoss for Long-Tail Instance Segmentation
Authors: Hsieh Ting-I , Robb Esther , Chen Hwann-Tzong , Huang Jia-Bin
Published: September 2022

Long-tailed class distributions are prevalent among the practical applications of object detection and instance segmentation. Prior work in long-tail instance segmentation addresses the imbalance of losses between rare and frequent categories by reducing the penalty for a model incorrectly predicting a rare class label. We demonstrate that the rare categories are heavily suppressed by correct background predictions, which reduces the probability for all foreground categories with equal weight. Due to the relative infrequency of rare categories, this leads to an imbalance that biases towards predicting more frequent categories. Based on this insight, we develop DropLoss -- a novel adaptive loss to compensate for this imbalance without a trade-off between rare and frequent categories. With this loss, we show state-of-the-art mAP across rare, common, and frequent categories on the LVIS dataset. Codes are available at https://github.com/timy90022/DropLoss.
Asynchronous Teacher Guided Bit-wise Hard Mining for Online Hashing
Authors: Jin Sheng , Zhou Qin , Yao Hongxun , Liu Yao , Hua Xian-Sheng
Published: September 2022

Online hashing for streaming data has attracted increasing attention recently. However, most existing algorithms focus on batch inputs and instance-balanced optimization, which is limited in the single datum input case and does not match the dynamic training in online hashing. Furthermore, constantly updating the online model with new-coming samples will inevitably lead to the catastrophic forgetting problem. In this paper, we propose a novel online hashing method to handle the above-mentioned issues jointly, termed Asynchronus Teacher-Guided Bit-wise Hard Mining for Online Hashing. Firstly, to meet the needs of datum-wise online hashing, we design a novel binary codebook that is discriminative to separate different classes. Secondly, we propose a novel semantic loss (termed bit-wise attention loss) to dynamically focus on hard samples of each bit during training. Last but not least, we design a asynchronous knowledge distillation scheme to alleviate the catastrophic forgetting problem, where the teacher model is delaying updated to maintain the old knowledge, guiding the student model learning. Extensive experiments conducted on two public benchmarks demonstrate the favorable performance of our method over the state-of-the-arts.
Deep Low-Contrast Image Enhancement using Structure Tensor Representation
Authors: Jung Hyungjoo , Jang Hyunsung , Ha Namkoo , Sohn Kwanghoon
Published: September 2022

We present a new deep learning framework for low-contrast image enhancement, which trains the network using the multi-exposure sequences rather than explicit ground-truth images. The purpose of our method is to enhance a low-contrast image so as to contain abundant details in various exposure levels. To realize this, we propose to design the loss function using the structure tensor representation, which has been widely used as high-dimensional image contrast. Our loss function penalizes the difference of the structure tensor between the network output and the multi-exposure images in a multi-scale manner. Eventually, the network trained by the loss function produces a high-quality image approximating the overall contrast of the sequence. We provide in-depth analysis on our method and comparison with conventional loss functions. Quantitative and qualitative evaluations demonstrate that the proposed method outperforms the existing state-of-the-art approaches in various benchmarks.
Visual Comfort Aware-Reinforcement Learning for Depth Adjustment of Stereoscopic 3D Images
Authors: Kim Hak Gu , Park Minho , Lee Sangmin , Kim Seongyeop , Ro Yong Man
Published: September 2022

Depth adjustment aims to enhance the visual experience of stereoscopic 3D (S3D) images, which accompanied with improving visual comfort and depth perception. For a human expert, the depth adjustment procedure is a sequence of iterative decision making. The human expert iteratively adjusted the depth until he is satisfied with the both levels of visual comfort and the perceived depth. In this work, we present a novel deep reinforcement learning (DRL)-based approach for depth adjustment named VCA-RL (Visual Comfort Aware Reinforcement Learning) to explicitly model human sequential decision making in depth editing operations. We formulate the depth adjustment process as a Markov decision process where actions are defined as camera movement operations to control the distance between the left and right cameras. Our agent is trained based on the guidance of an objective visual comfort assessment metric to learn the optimal sequence of camera movement actions in terms of perceptual aspects in stereoscopic viewing. With extensive experiments and user studies, we show the effectiveness of our VCA-RL model on three different S3D databases.
Cross-Domain Grouping and Alignment for Domain Adaptive Semantic Segmentation
Authors: Kim Minsu , Joung Sunghun , Kim Seungryong , Park JungIn , Kim Ig-Jae , Sohn Kwanghoon
Published: September 2022

Existing techniques to adapt semantic segmentation networks across source and target domains within deep convolutional neural networks (CNNs) deal with all the samples from the two domains in a global or category-aware manner. They do not consider an inter-class variation within the target domain itself or estimated category, providing the limitation to encode the domains having a multi-modal data distribution. To overcome this limitation, we introduce a learnable clustering module, and a novel domain adaptation framework, called cross-domain grouping and alignment. To cluster the samples across domains with an aim to maximize the domain alignment without forgetting precise segmentation ability on the source domain, we present two loss functions, in particular, for encouraging semantic consistency and orthogonality among the clusters. We also present a loss so as to solve a class imbalance problem, which is the other limitation of the previous methods. Our experiments show that our method consistently boosts the adaptation performance in semantic segmentation, outperforming the state-of-the-arts on various domain adaptation settings.
Patch-Wise Attention Network for Monocular Depth Estimation
Authors: Lee Sihaeng , Lee Janghyeon , Kim Byungju , Yi Eojindl , Kim Junmo
Published: September 2022

In computer vision, monocular depth estimation is the problem of obtaining a high-quality depth map from a two-dimensional image. This map provides information on three-dimensional scene geometry, which is necessary for various applications in academia and industry, such as robotics and autonomous driving. Recent studies based on convolutional neural networks achieved impressive results for this task. However, most previous studies did not consider the relationships between the neighboring pixels in a local area of the scene. To overcome the drawbacks of existing methods, we propose a patch-wise attention method for focusing on each local area. After extracting patches from an input feature map, our module generates attention maps for each local patch, using two attention modules for each patch along the channel and spatial dimensions. Subsequently, the attention maps return to their initial positions and merge into one attention feature. Our method is straightforward but effective. The experimental results on two challenging datasets, KITTI and NYU Depth V2, demonstrate that the proposed method achieves significant performance. Furthermore, our method outperforms other state-of-the-art methods on the KITTI depth estimation benchmark.
Semi-Supervised Learning for Multi-Task Scene Understanding by Neural Graph Consensus
Authors: Leordeanu Marius , Pîrvu Mihai Cristian , Costea Dragos , Marcu Alina E , Slusanschi Emil , Sukthankar Rahul
Published: September 2022

We address the challenging problem of semi-supervised learning in the context of multiple visual interpretations of the world by finding consensus in a graph of neural networks. Each graph node is a scene interpretation layer, while each edge is a deep net that transforms one layer at one node into another from a different node. During the supervised phase edge networks are trained independently. During the next unsupervised stage edge nets are trained on the pseudo-ground truth provided by consensus among multiple paths that reach the nets' start and end nodes. These paths act as ensemble teachers for any given edge and strong consensus is used for high-confidence supervisory signal. The unsupervised learning process is repeated over several generations, in which each edge becomes a "student" and also part of different ensemble "teachers" for training other students. By optimizing such consensus between different paths, the graph reaches consistency and robustness over multiple interpretations and generations, in the face of unknown labels. We give theoretical justifications of the proposed idea and validate it on a large dataset. We show how prediction of different representations such as depth, semantic segmentation, surface normals and pose from RGB input could be effectively learned through self-supervised consensus in our graph. We also compare to state-of-the-art methods for multi-task and semi-supervised learning and show superior performance.