OpenRaiser/Intern-Atlas huggingface dataset:
Learning TFIDF Enhanced Joint Embedding for Recipe-Image Cross-Modal Retrieval Service
Venue: arXiv cs.CV 2021
Year: 2021

It is widely acknowledged that learning joint embeddings of recipes with images is challenging due to the diverse composition and deformation of ingredients in cooking procedures. We present a Multi-modal Semantics enhanced Joint Embedding approach (MSJE) for learning a common feature space between the two modalities (text and image), with the ultimate goal of providing high-performance cross-modal retrieval services. Our MSJE approach has three unique features. First, we extract the TFIDF feature from the title, ingredients and cooking instructions of recipes. By determining the significance of word sequences through combining LSTM learned features with their TFIDF features, we encode a recipe into a TFIDF weighted vector for capturing significant key terms and how such key terms are used in the corresponding cooking instructions. Second, we combine the recipe TFIDF feature with the recipe sequence feature extracted through two-stage LSTM networks, which is effective in capturing the unique relationship between a recipe and its associated image(s). Third, we further incorporate TFIDF enhanced category semantics to improve the mapping of image modality and to regulate the similarity loss function during the iterative learning of cross-modal joint embedding. Experiments on the benchmark dataset Recipe1M show the proposed approach outperforms the state-of-the-art approaches.
AI Techniques for Software Requirements Prioritization
Venue: arXiv cs.SE 2021
Year: 2021

Aspects such as limited resources, frequently changing market demands, and different technical restrictions regarding the implementation of software requirements (features) often demand for the prioritization of requirements. The task of prioritization is the ranking and selection of requirements that should be included in future software releases. In this context, an intelligent prioritization decision support is extremely important. The prioritization approaches discussed in this paper are based on different Artificial Intelligence (AI) techniques that can help to improve the overall quality of requirements prioritization processes
Resilience of Well-structured Graph Transformation Systems
Venue: arXiv cs.SE 2021
Year: 2021

Resilience is a concept of rising interest in computer science and software engineering. For systems in which correctness w.r.t. a safety condition is unachievable, fast recovery is demanded. We investigate resilience problems of graph transformation systems. Our main contribution is the decidability of two resilience problems for well-structured graph transformation systems (with strong compatibility). We prove our results in the abstract framework of well-structured transition systems and apply them to graph transformation systems, incorporating also the concept of adverse conditions.
An Applied Deep Learning Approach for Estimating Soybean Relative Maturity from UAV Imagery to Aid Plant Breeding Decisions
Venue: arXiv cs.CV 2021
Year: 2021

For a global breeding organization, identifying the next generation of superior crops is vital for its success. Recognizing new genetic varieties requires years of in-field testing to gather data about the crop's yield, pest resistance, heat resistance, etc. At the conclusion of the growing season, organizations need to determine which varieties will be advanced to the next growing season (or sold to farmers) and which ones will be discarded from the candidate pool. Specifically for soybeans, identifying their relative maturity is a vital piece of information used for advancement decisions. However, this trait needs to be physically observed, and there are resource limitations (time, money, etc.) that bottleneck the data collection process. To combat this, breeding organizations are moving toward advanced image capturing devices. In this paper, we develop a robust and automatic approach for estimating the relative maturity of soybeans using a time series of UAV images. An end-to-end hybrid model combining Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) is proposed to extract features and capture the sequential behavior of time series data. The proposed deep learning model was tested on six different environments across the United States. Results suggest the effectiveness of our proposed CNN-LSTM model compared to the local regression method. Furthermore, we demonstrate how this newfound information can be used to aid in plant breeding advancement decisions.
My Eyes Are Up Here: Promoting Focus on Uncovered Regions in Masked Face Recognition
Venue: arXiv cs.CV 2021
Year: 2021

The recent Covid-19 pandemic and the fact that wearing masks in public is now mandatory in several countries, created challenges in the use of face recognition systems (FRS). In this work, we address the challenge of masked face recognition (MFR) and focus on evaluating the verification performance in FRS when verifying masked vs unmasked faces compared to verifying only unmasked faces. We propose a methodology that combines the traditional triplet loss and the mean squared error (MSE) intending to improve the robustness of an MFR system in the masked-unmasked comparison mode. The results obtained by our proposed method show improvements in a detailed step-wise ablation study. The conducted study showed significant performance gains induced by our proposed training paradigm and modified triplet loss on two evaluation databases.
Rapidly-Exploring Random Graph Next-Best View Exploration for Ground Vehicles
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, a novel approach is introduced which utilizes a Rapidly-exploring Random Graph to improve sampling-based autonomous exploration of unknown environments with unmanned ground vehicles compared to the current state of the art. Its intended usage is in rescue scenarios in large indoor and underground environments with limited teleoperation ability. Local and global sampling are used to improve the exploration efficiency for large environments. Nodes are selected as the next exploration goal based on a gain-cost ratio derived from the assumed 3D map coverage at the particular node and the distance to it. The proposed approach features a continuously-built graph with a decoupled calculation of node gains using a computationally efficient ray tracing method. The Next-Best View is evaluated while the robot is pursuing a goal, which eliminates the need to wait for gain calculation after reaching the previous goal and significantly speeds up the exploration. Furthermore, a grid map is used to determine the traversability between the nodes in the graph while also providing a global plan for navigating towards selected goals. Simulations compare the proposed approach to state-of-the-art exploration algorithms and demonstrate its superior performance.
A Framework for Multi-View Classification of Features
Venue: arXiv cs.LG 2021
Year: 2021

One of the most important problems in the field of pattern recognition is data classification. Due to the increasing development of technologies introduced in the field of data classification, some of the solutions are still open and need more research. One of the challenging problems in this area is the curse of dimensionality of the feature set of the data classification problem. In solving the data classification problems, when the feature set is too large, typical approaches will not be able to solve the problem. In this case, an approach can be used to partition the feature set into multiple feature sub-sets so that the data classification problem is solved for each of the feature subsets and finally using the ensemble classification, the classification is applied to the entire feature set. In the above-mentioned approach, the partitioning of feature set into feature sub-sets is still an interesting area in the literature of this field. In this research, an innovative framework for multi-view ensemble classification, inspired by the problem of object recognition in the multiple views theory of humans, is proposed. In this method, at first, the collaboration values between the features is calculated using a criterion called the features collaboration criterion. Then, the collaboration graph is formed based on the calculated collaboration values. In the next step, using the community detection method, graph communities are found. The communities are considered as the problem views and the different base classifiers are trained for different views using the views corresponding training data. The multi-view ensemble classifier is then formed by a combination of base classifiers based on the AdaBoost algorithm. The simulation results of the proposed method on the real and synthetic datasets show that the proposed method increases the classification accuracy.
Distributed Attention for Grounded Image Captioning
Venue: arXiv cs.CV 2021
Year: 2021

We study the problem of weakly supervised grounded image captioning. That is, given an image, the goal is to automatically generate a sentence describing the context of the image with each noun word grounded to the corresponding region in the image. This task is challenging due to the lack of explicit fine-grained region word alignments as supervision. Previous weakly supervised methods mainly explore various kinds of regularization schemes to improve attention accuracy. However, their performances are still far from the fully supervised ones. One main issue that has been ignored is that the attention for generating visually groundable words may only focus on the most discriminate parts and can not cover the whole object. To this end, we propose a simple yet effective method to alleviate the issue, termed as partial grounding problem in our paper. Specifically, we design a distributed attention mechanism to enforce the network to aggregate information from multiple spatially different regions with consistent semantics while generating the words. Therefore, the union of the focused region proposals should form a visual region that encloses the object of interest completely. Extensive experiments have demonstrated the superiority of our proposed method compared with the state-of-the-arts.
S$^2$-MLPv2: Improved Spatial-Shift MLP Architecture for Vision
Venue: arXiv cs.CV 2021
Year: 2021

Recently, MLP-based vision backbones emerge. MLP-based vision architectures with less inductive bias achieve competitive performance in image recognition compared with CNNs and vision Transformers. Among them, spatial-shift MLP (S$^2$-MLP), adopting the straightforward spatial-shift operation, achieves better performance than the pioneering works including MLP-mixer and ResMLP. More recently, using smaller patches with a pyramid structure, Vision Permutator (ViP) and Global Filter Network (GFNet) achieve better performance than S$^2$-MLP. In this paper, we improve the S$^2$-MLP vision backbone. We expand the feature map along the channel dimension and split the expanded feature map into several parts. We conduct different spatial-shift operations on split parts. Meanwhile, we exploit the split-attention operation to fuse these split parts. Moreover, like the counterparts, we adopt smaller-scale patches and use a pyramid structure for boosting the image recognition accuracy. We term the improved spatial-shift MLP vision backbone as S$^2$-MLPv2. Using 55M parameters, our medium-scale model, S$^2$-MLPv2-Medium achieves an $83.6\%$ top-1 accuracy on the ImageNet-1K benchmark using $224\times 224$ images without self-attention and external training data.
Hybrid Quantum-Classical Neural Network for Incident Detection
Venue: arXiv cs.LG 2021
Year: 2021

The efficiency and reliability of real-time incident detection models directly impact the affected corridors' traffic safety and operational conditions. The recent emergence of cloud-based quantum computing infrastructure and innovations in noisy intermediate-scale quantum devices have revealed a new era of quantum-enhanced algorithms that can be leveraged to improve real-time incident detection accuracy. In this research, a hybrid machine learning model, which includes classical and quantum machine learning (ML) models, is developed to identify incidents using the connected vehicle (CV) data. The incident detection performance of the hybrid model is evaluated against baseline classical ML models. The framework is evaluated using data from a microsimulation tool for different incident scenarios. The results indicate that a hybrid neural network containing a 4-qubit quantum layer outperforms all other baseline models when there is a lack of training data. We have created three datasets; DS-1 with sufficient training data, and DS-2 and DS-3 with insufficient training data. The hybrid model achieves a recall of 98.9%, 98.3%, and 96.6% for DS-1, DS-2, and DS-3, respectively. For DS-2 and DS-3, the average improvement in F2-score (measures model's performance to correctly identify incidents) achieved by the hybrid model is 1.9% and 7.8%, respectively, compared to the classical models. It shows that with insufficient data, which may be common for CVs, the hybrid ML model will perform better than the classical models. With the continuing improvements of quantum computing infrastructure, the quantum ML models could be a promising alternative for CV-related applications when the available data is insufficient.
A computational geometry approach for modeling neuronal fiber pathways
Venue: arXiv cs.CV 2021
Year: 2021

We propose a novel and efficient algorithm to model high-level topological structures of neuronal fibers. Tractography constructs complex neuronal fibers in three dimensions that exhibit the geometry of white matter pathways in the brain. However, most tractography analysis methods are time consuming and intractable. We develop a computational geometry-based tractography representation that aims to simplify the connectivity of white matter fibers. Given the trajectories of neuronal fiber pathways, we model the evolution of trajectories that encodes geometrically significant events and calculate their point correspondence in the 3D brain space. Trajectory inter-distance is used as a parameter to control the granularity of the model that allows local or global representation of the tractogram. Using diffusion MRI data from Alzheimer's patient study, we extract tractography features from our model for distinguishing the Alzheimer's subject from the normal control. Software implementation of our algorithm is available on GitHub.
AGAR a microbial colony dataset for deep learning detection
Venue: arXiv cs.CV 2021
Year: 2021

The Annotated Germs for Automated Recognition (AGAR) dataset is an image database of microbial colonies cultured on agar plates. It contains 18000 photos of five different microorganisms as single or mixed cultures, taken under diverse lighting conditions with two different cameras. All the images are classified into "countable", "uncountable", and "empty", with the "countable" class labeled by microbiologists with colony location and species identification (336442 colonies in total). This study describes the dataset itself and the process of its development. In the second part, the performance of selected deep neural network architectures for object detection, namely Faster R-CNN and Cascade R-CNN, was evaluated on the AGAR dataset. The results confirmed the great potential of deep learning methods to automate the process of microbe localization and classification based on Petri dish photos. Moreover, AGAR is the first publicly available dataset of this kind and size and will facilitate the future development of machine learning models. The data used in these studies can be found at https://agar.neurosys.com/.
fauci-email: a json digest of Anthony Fauci's released emails
Venue: arXiv cs.SI 2021
Year: 2021

A collection of over 3000 pages of emails sent by Anthony Fauci and his staff were released in an effort to understand the United States government response to the COVID-19 pandemic. We describe how this email data was translated into a resource consisting of json files that make many future studies easy. Findings from our processed data include (i) successful organizational partitions using the simple mincut techniques in Zachary's karate club methodology, (ii) a natural example where the normalized cut and minimum conductance set are extremely different, and (iii) organizational groups identified by optimum modularity clusters that illustrate a working hierarchy. These example uses suggest the data will be useful for future research and pedagogical uses in terms of human and system behavioral interactions. We explain a number of ways to turn email information into a network, a hypergraph, a temporal sequence, and a tensor for subsequent analysis as well as a few examples of such analysis.
The Performance Evaluation of Attention-Based Neural ASR under Mixed Speech Input
Venue: arXiv cs.SD 2021
Year: 2021

In order to evaluate the performance of the attention based neural ASR under noisy conditions, the current trend is to present hours of various noisy speech data to the model and measure the overall word/phoneme error rate (W/PER). In general, it is unclear how these models perform when exposed to a cocktail party setup in which two or more speakers are active. In this paper, we present the mixtures of speech signals to a popular attention-based neural ASR, known as Listen, Attend, and Spell (LAS), at different target-to-interference ratio (TIR) and measure the phoneme error rate. In particular, we investigate in details when two phonemes are mixed what will be the predicted phoneme; in this fashion we build a model in which the most probable predictions for a phoneme are given. We found a 65% relative increase in PER when LAS was presented with mixed speech signals at TIR = 0 dB and the performance approaches the unmixed scenario at TIR = 30 dB. Our results show the model, when presented with mixed phonemes signals, tend to predict those that have higher accuracies during evaluation of original phoneme signals.
Visualizing Data using GTSNE
Venue: arXiv cs.LG 2021
Year: 2021

We present a new method GTSNE to visualize high-dimensional data points in the two dimensional map. The technique is a variation of t-SNE that produces better visualizations by capturing both the local neighborhood structure and the macro structure in the data. This is particularly important for high-dimensional data that lie on continuous low-dimensional manifolds. We illustrate the performance of GTSNE on a wide variety of datasets and compare it the state of art methods, including t-SNE and UMAP. The visualizations produced by GTSNE are better than those produced by the other techniques on almost all of the datasets on the macro structure preservation.
Accountability and Forensics in Blockchains: XDC Consensus Engine DPoS 2.0
Venue: arXiv cs.CR 2021
Year: 2021

This document introduces XinFin DPoS 2.0, the proposed next generation decentralized consensus engine for the XinFin XDC Network. Built upon the most advanced BFT consensus protocol, this upgrade will empower the XDC Network with military-grade security and performance while consuming extremely low resources, and will be fully backwards-compatible in terms of APIs. It will also pave the road to the future evolution of the XDC Network. The core invention is the holistic integration of accountability and forensics in blockchains: the ability to identify malicious actors with cryptographic integrity directly from the blockchain records, incorporating the latest peer-reviewed academic research with state of the art engineering designs and implementation plans.
Continuous Non-Invasive Eye Tracking In Intensive Care
Venue: arXiv cs.CV 2021
Year: 2021

Delirium, an acute confusional state, is a common occurrence in Intensive Care Units (ICUs). Patients who develop delirium have globally worse outcomes than those who do not and thus the diagnosis of delirium is of importance. Current diagnostic methods have several limitations leading to the suggestion of eye-tracking for its diagnosis through in-attention. To ascertain the requirements for an eye-tracking system in an adult ICU, measurements were carried out at Chelsea & Westminster Hospital NHS Foundation Trust. Clinical criteria guided empirical requirements of invasiveness and calibration methods while accuracy and precision were measured. A non-invasive system was then developed utilising a patient-facing RGB-camera and a scene-facing RGBD-camera. The system's performance was measured in a replicated laboratory environment with healthy volunteers revealing an accuracy and precision that outperforms what is required while simultaneously being non-invasive and calibration-free The system was then deployed as part CONfuSED, a clinical feasibility study where we report aggregated data from 5 patients as well as the acceptability of the system to bedside nursing staff. The system is the first eye-tracking system to be deployed in an ICU.
Adaptively Optimize Content Recommendation Using Multi Armed Bandit Algorithms in E-commerce
Venue: arXiv cs.IR 2021
Year: 2021

E-commerce sites strive to provide users the most timely relevant information in order to reduce shopping frictions and increase customer satisfaction. Multi armed bandit models (MAB) as a type of adaptive optimization algorithms provide possible approaches for such purposes. In this paper, we analyze using three classic MAB algorithms, epsilon-greedy, Thompson sampling (TS), and upper confidence bound 1 (UCB1) for dynamic content recommendations, and walk through the process of developing these algorithms internally to solve a real world e-commerce use case. First, we analyze the three MAB algorithms using simulated purchasing datasets with non-stationary reward distributions to simulate the possible time-varying customer preferences, where the traffic allocation dynamics and the accumulative rewards of different algorithms are studied. Second, we compare the accumulative rewards of the three MAB algorithms with more than 1,000 trials using actual historical A/B test datasets. We find that the larger difference between the success rates of competing recommendations the more accumulative rewards the MAB algorithms can achieve. In addition, we find that TS shows the highest average accumulative rewards under different testing scenarios. Third, we develop a batch-updated MAB algorithm to overcome the delayed reward issue in e-commerce and enable an online content optimization on our App homepage. For a state-of-the-art comparison, a real A/B test among our batch-updated MAB algorithm, a third-party MAB solution, and the default business logic are conducted. The result shows that our batch-updated MAB algorithm outperforms the counterparts and achieves 6.13% relative click-through rate (CTR) increase and 16.1% relative conversion rate (CVR) increase compared to the default experience, and 2.9% relative CTR increase and 1.4% relative CVR increase compared to the external MAB service.