Reviewerly/exHarmony huggingface dataset:
Towards Simultaneous Image Compression and Indexing for Scalable Content-Based Retrieval in Remote Sensing

Due to the rapidly growing remote-sensing (RS) image archives, images are usually stored in a compressed format for reducing their storage sizes. Thus, most of the existing content-based RS image retrieval systems require fully decoding images (i.e., decompression) that is computationally demanding for large-scale archives. To address this issue, we introduce a novel approach devoted to simultaneous RS image compression and indexing for scalable content-based image retrieval (denoted as SCI-CBIR). The proposed SCI-CBIR prevents the requirement of decoding RS images before image search and retrieval. To this end, it includes two main steps: 1) deep-learning-based compression and 2) deep-hashing-based indexing. The first step effectively compresses RS images by employing a pair of deep encoder and decoder neural networks and an entropy model. The second step produces hash codes with a high discrimination capability for RS images by employing pairwise, bit-balancing, and classification loss functions. For the training of the SCI-CBIR approach, we also introduce a novel multistage learning procedure with automatic loss weighting techniques to characterize RS image representations that are appropriate for both RS image indexing and compression. The proposed learning procedure enables automatically weighting different loss functions considered for the proposed approach instead of computationally demanding grid search. Experimental results show the effectiveness of the proposed approach when compared to widely used approaches in RS. The code of the proposed approach is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://git.tu-berlin.de/rsim/SCI-CBIR</uri> .
Transfer Learning Approach Towards a Smarter Recycling

The increasing consumption of electrical and electronic devices is alarming. Therefore, the transition from linear to circular economy becomes essential. The key solution to support this transformation is artificial intelligence. This work presents a transfer learning approach to support the recycling of electrical and electronic waste (Ewaste). We emphasize the use of transfer learning technique, particularly, to classify Ewaste. In this approach, we design a hybrid model of residual nets and inception modules that can classify features of a source domain (smartphones in our case) and leverage this knowledge to another device (electric screwdrivers, as an example). Using our model, we achieve an overall accuracy of $$94.27\%$$ and $$97.22\%$$ , respectively. These are comparable to the popular pre-trained models, which use similar network topologies. We use a web crawler program for collecting images from search engines to build the datasets with less efforts. We show that transfer learning is more robust and performs better than training from scratch. It avoids duplication and waste of computational resources. As a result, with the benefits of transfer learning, we can provide detailed information about the devices that need to be recycled. Ultimately, this would greatly support the overall recycling process.
Object Detection with Probabilistic Guarantees: A Conformal Prediction Approach

Providing reliable uncertainty quantification for complex visual tasks such as object detection is of utmost importance for safety-critical applications such as autonomous driving, tumor detection, etc. Conformal prediction methods offer simple yet practical means to build uncertainty estimations that come with probabilistic guarantees. In this paper we apply such methods to the task of object localization and illustrate our analysis on a pedestrian detection use-case. We highlight both theoretical and practical implications of our analysis.
Towards tacit knowledge mining within context: Visual cognitive graph model and eye movement image interpretation

Visual attention is one of the most important brain cognitive functions, which filters the rich information of the outside world to ensure the efficient operation of limited cognitive resources. The underlying knowledge, i.e., tacit knowledge, hidden in the human attention allocation performances, is context-related and is hard to be expressed by experts, but it is essential for novice operator training and interaction system design. Traditional models of visual attention allocation and corresponding analysis methods seldomly involve task contextual information or present the tacit knowledge in an explicit and quantified way. Thus, it is challenging to pass on the expert's tacit knowledge to the novice or utilize it to construct an interaction system by employing traditional methods. Therefore, this paper first proposes a new model called the visual cognitive graph model based on graph theory to model the visual attention allocation associated with the task context. Then, based on this graph model, utilize the data mining method to reveal attention patterns within context to quantitatively analyze the operator's tacit knowledge during operation tasks. We introduced three physical quantities derived from graph theory to describe the tacit knowledge, which can be used directly to construct an interaction system or operator training. For example, discover the essential information within the task context, the relevant information affecting critical information, and the bridge information revealing the decision-making process. We tested the proposed method in the example of flight operation, the comparison results with the traditional eye movement graph model demonstrate that the proposed visual cognitive model can compromise the task context. The comparison results with the statistical analysis method demonstrate that our tacit knowledge mining method can reveal the underlying knowledge hidden in the visual information. Finally, we give practical applications in the examples of operator training guidance and adaptive interaction system. Our proposed method can explore more in-depth knowledge of visual information, such as the correlations of different obtained information and the way operator obtains information, most of which are even not noticed by operators themselves.
Introducing the Diagrammatic Semiotic Mode

Abstract As the use and diversity of diagrams across many disciplines grows, there is an increasing interest in the diagrams research community concerning how such diversity might be documented and explained. In this article, we argue that one way of achieving increased reliability, coverage, and utility for a general classification of diagrams is to draw on recently developed semiotic principles developed within the field of multimodality. To this end, we sketch out the internal details of what may tentatively be termed the diagrammatic semiotic mode . This provides a natural account of how diagrammatic representations organising lines of research integrate natural language, various forms of graphics, diagrammatic elements such as arrows, lines and other expressive resources into coherent organisations, while still respecting the crucial diagrammatic contributions of visual organisation. We illustrate the proposed approach using two recent diagram corpora and show how a multimodal approach supports the empirical analysis of diagrammatic representations, especially in identifying diagrammatic constituents and describing their interrelations in a manner that may be generalised across diagram types and be used to characterise distinct kinds of functionality.
A Framework for Project Delivery Systems via Hybrid Fuzzy Risk Analysis: Application and Extension in ICT

One of the essential factors of project success is selecting the proper delivery method. This study aimed to provide a new hybrid decision-making framework to assist project stakeholders in evaluating and selecting the most appropriate Project Delivery System (PDS) and documenting the decision process. For this purpose, the selection factors of PDSs were obtained from a literature review, and critical selection factors were screened based on the fuzzy Delphi method, whereby expert feedback was on Information and Communication Technology (ICT) projects was obtained. Subsequently, the ICT project risks were identified and categorized into six competitive constraints, including time, cost, quality, reputation, value, and scope, and the risk factors were prioritized in each area. Then, the effect of project risks on the decision criteria was investigated using a fuzzy cognitive map (FCM). Finally, the PDSs were ranked through Fuzzy Technique for Order of Preference by Similarity to Ideal Solution (FTOPSIS). This article researched a novel multi-layer decision system combining the FCM and FTOPSIS techniques. The decision criteria received their weights from the evaluation of the causal relationships between PDS selection factors and project risks. Thus, PDSs were ranked based on different project characteristics, the opinions of stakeholders, and the effect of project risks on the decision-making process; this increased the likelihood of project success. The results showed that the impact of the most critical project risks on the selection factors was so severe that they changed the weight of the criteria in the decision matrix and, subsequently, changed the ranking of decision options.
Study of selected methods for balancing independent data sets in <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="d1e7421" altimg="si3.svg"><mml:mi>k</mml:mi></mml:math>-nearest neighbors classifiers with Pawlak conflict analysis

The article is devoted to the issue of classification based on independent data sets. More specifically, the impact of using different methods for balancing data sets on the classification quality in an approach that uses Pawlak conflict analysis is investigated. The newly proposed method for classification based on independent data sets assumes applying algorithms for imbalanced classification separately to all fragmented sets. The following algorithms are considered: SMOTE, random over-sampling, TOMEK links, Near Miss, random under-sampling and a combination of SMOTE and TOMEK links. For balanced data sets — decision tables, conflict analysis is used, and coalitions of tables are created. Then the aggregated table for each coalition is defined, and a modified k -nearest neighbors algorithm is used to determine the decision vectors. The majority voting method is used to fuse decision vectors. Experimental results showed that the proposed approach, in most cases, gives much better results than without using methods for imbalanced data. In addition, the proposed approach achieves better results than other methods known from the literature applied to dispersed data. It was noticed that for dispersed and independent data, the best results are generated by the over-sampling approach, especially by the SMOTE, the random over-sampling and the SMOTE and TOMEK methods. • Modification of methods for imbalanced and dispersed data was proposed. • Six different methods were compared. • System with advanced conflict analysis for independent data sets was used. • It was proven that it is better to use the method to balance the fragmented data. • It was shown that, in general, the SMOTE method is most effective.
tylr: a tiny tile-based structure editor

Structure editors designed for keyboard input often struggle to resolve the tension between maintaining hierarchical term structure and offering efficient linear editing affordances. Contemporary designs either compromise structure by deferring to text near the leaves or else maintain structure by permitting only edits that transform the selected term. However, visually adjacent sequences (e.g. of operators, operands, and individual delimiters) do not always cleave cleanly to term boundaries, so even experienced users report difficulties with selection and code restructuring tasks. We propose a novel approach to structure editing, tile-based editing, that maintains term structure while offering linear selection and modification affordances. The idea is to allow disassembly of terms into linearly sequenced tiles and shards around user selections, while guiding the user through restructuring actions and automatically inserting holes in a manner that ensures reassembly into a term.
How to safely use extensionality in Liquid Haskell

Refinement type checkers are a powerful way to reason about functional programs. For example, one can prove properties of a slow, specification implementation and port the proofs to an optimized pure implementation that behaves the same. But to reason about higher-order programs, we must reason about equalities between functions: we need a consistent encoding of functional extensionality.
OW-TAL: Learning Unknown Human Activities for Open-World Temporal Action Localization

Current temporal action localization methods work well on a closed-world assumption, in which all action categories to be localized are known as a priori. However, this assumption doesn't apply to open-world scenarios, as novel categories that never appeared in the training stage will be encountered without explicit supervision. Distinct from the closed-world setting, localizing actions under the open-world setup poses two significant challenges: 1) identifying unknown actions from diverse knowns and localizing their temporal boundaries. 2) defying forgetting of previous actions when incrementally updating knowledge of identified unknown actions. To address the aforementioned challenges, we develop a two-branch framework with Unknown and Known action modeling Networks, a.k.a. UK-Net, for the problem of Open-World Temporal Action Localization (OW-TAL). The potential patterns underlying unknown and known actions, as well as their dynamic transformation, are modeled in a unified pipeline. Specifically, a self-attention based position-sensitive module is designed to produce actionness scores for unknown actions in a class-agnostic way. Besides, an iterative optimization strategy is developed to enable knowledge derived from known categories to be shared with the unknowns. In addition, a self-paced learning strategy is proposed to instructionally guide class-incremental learning while defying catastrophic forgetting. Benefiting from the above components, our UK-Net yields superior performance on three challenging datasets, i.e., THUMOS14, ActivityNet1.2, and MUSES. Experimental results also demonstrate the competitive performance of our method when compared with traditional closed-world counterparts.
Adaptive Broad Echo State Network for Nonstationary Time Series Forecasting

Time series forecasting provides a vital basis for the control and management of various systems. The time series data in the real world are usually strongly nonstationary and nonlinear, which increases the difficulty of reliable forecasting. To fully utilize the learning capability of machine learning in time series forecasting, an adaptive broad echo state network (ABESN) is proposed in this paper. Firstly, the broad learning system (BLS) is used as a framework, and the reservoir pools in the echo state network (ESN) are introduced to form the broad echo state network (BESN). Secondly, for the problem of information redundancy in the reservoir structure in BESN, an adaptive optimization algorithm for the BESN structure based on the pruning algorithm is proposed. Thirdly, an adaptive optimization algorithm of hyperparameters based on the nonstationary test index is proposed. In brief, the structure and hyperparameter optimization algorithms are studied to form the ABESN based on the proposed BESN model in this paper. The ABESN is applied to the data forecasting of air humidity and electric load. The experiments show that the proposed ABESN has a better learning ability for nonstationary time series data and can achieve higher forecasting accuracy.
Early detection of failures from vehicle equipment data using K-means clustering design

Through data mining from vehicle construction equipment, the paper presents a unique approach to the early diagnosis of machinery failures. We used an effective K-means clustering algorithm to discover inconsistencies in field data. Data is stored in a warehouse using the Structural Query Language (SQL). The vehicle equipment data is processed using the K-means algorithm. During K-means clustering, we applied the Waikato Environment for Knowledge Analysis (WEKA) application dataset to represent the correct information, which enables risk detection based on the stored warehouse. The K-means clustering performs early defects identification in a dataset of construction equipment supplied by a Libyan organization. The clustering mechanism is based on the Euclidean distance, which is used to calculate the correctness of files in a large dataset. The tests make use of real-world equipment datasheets spanning five years. The results show that the K-means algorithm produces more accurate results than other methods.
A weak approach to suffix automata simulation for exact and approximate string matching

String matching is one of the most extensively studied problems in computer science, mainly due to its direct applications to such diverse areas as text, image and signal processing, speech analysis and recognition, information retrieval, data compression, computational biology and chemistry. In the last few decades a myriad of alternative solutions have been proposed, based on very different techniques. However, automata have always played a very important role in the design of efficient string matching algorithms. In this paper we present the Range Automaton, a weak yet efficient variant of the non-deterministic suffix automaton of a string whose configuration can be encoded in a very simple form and which is particularly suitable to be used for solving a multitude of text-searching problems. We will firstly develop our approach in the case of exact string matching and present an efficient algorithm, named Backward Range Automaton Matcher, which turns out to be very fast in many practical cases. Later, we will show how the Range Automaton can be adapted in an effective way also to non-standard string matching problems such as swap matching and multiple string matching. Experimental results suggest that our approach is flexible and effective for all three search problems addressed, especially in the case of long patterns.
Online transfer learning with partial feedback

Online learning for multi-class classification is a well-studied topic in machine learning. The standard multi-class classification online learning setting assumes continuous availability of the ground-truth class labels. However, in many real-life applications, only partial feedback of the predicted label can be obtained and only the correctness of the prediction is available. Hence, knowledge of the correct label is missing in the case of erroneous predictions. In this case, learning may be slower and classifiers less accurate than in the full feedback scenario. Although several online learning algorithms with partial feedback have been proposed, real-world applications would still benefit from further performance improvement. In this paper, we exploit transfer learning to improve learning in the case of erroneous predictions. We propose the Partial Feedback Online Transfer Learning (PFOTL) algorithm, which uses learned knowledge from the source domain in addition to received partial feedback, and present analysis and a mistake bound for the algorithm. In our experimental results on four benchmark datasets, the proposed algorithm achieves higher online cumulative accuracy than the comparable state-of-the-art algorithms. Two potential applications of our work would be online recommender systems and privacy protection.
A code-mixed task-oriented dialog dataset for medical domain

In the healthcare domain, medical and patient interactions form a crucial part of the diagnosis. Initially, the AI models developed for healthcare centered only on monolingual data. However, such models do not cater to the multilingual regions, where most conversations are Code-Mixed. We present the Code-Mixed Medical Task-Oriented Dialog Dataset to facilitate the research and development of Code-Mixed medical dialog systems. We analyzed the dataset using medical, conversational, and linguistic theories. The dataset contains 3005 Telugu–English Code-Mixed dialogs between patients and doctors with 29 k utterances covering ten specializations with an average code-mixing index (CMI) of 33.3%. We manually annotated the conversational dataset with intents and slot labels. We also present baselines to establish benchmarks on the dataset using existing state-of-the-art Natural Language Understanding (NLU) models. We improved the existing baselines using contextual ground truth intent labels and processing the slots as chunks. The data is made publically available.1
Gradient-Guided Single Image Super-Resolution Based on Joint Trilateral Feature Filtering

The state of the arts (SOTAs) of single image super-resolution always exploit guidance from gradient prior. The fusion of gradient guidance is implemented by channel-wise concatenation followed by a convolutional layer. However, the kernels sharing in spatial positions cannot adaptively tune the effect of gradient guidance for all feature positions. To resolve this problem, a novel network module is proposed to simulate the traditional Joint Trilateral Filter (JTF) by extending the definition domain from pixels to features. Moreover, to improve the efficiency and flexibility, the functions of JTF kernel generation for image features and gradient features are explicitly learned instead of individual kernel weights, e.g., the exponential functions in the traditional JTF. Based on the proposed JTF modules, this paper follows the gradient-guided framework which simultaneously infers high-resolution (HR) image features and HR gradient features within two parallel branches, respectively. Specifically, by treating image features and gradient features as cross guidance to each other, the proposed JTF modules adaptively adjust the fusion patterns for local features via a bi-directional way. By doing so, the quality of image features and gradient features is alternatively enhanced. Compared with SOTAs, the proposed JTF-SISR shows improvement which is evaluated for multiple upsampling scales and degradation modes on 5 synthetic datasets, i.e., Set5, Set14, B100, Urban100 and Manga109, and 1 real dataset, i.e., RealSRSet. The code is public in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/a239xjc/JTF-SISR</uri> .