Reviewerly/exHarmony huggingface dataset:
AI2: a novel explainable machine learning framework using an NLP interface

This paper proposes a novel machine learning framework that encapsulates recent concerns of the data scientists community: accessibility and explainability. This framework, called AI2, proposes a natural language interface, making the framework accessible even to a non-expert. Traditionally, machine learning frameworks are accessible using a programming language. Python is one of the most common programming language for coding different machine learning methods. The AI2 framework, although made with Python scripts, is made to be accessed in a natural language, namely, English. Hence, the first contribution is about accessibility, allowing a non-data scientist to exploit a machine learning framework without knowing how to code. For decades, the data scientists community has known that one of the drawbacks in the machine learning field is the black-box problem. Data scientists have to create different methods to explain their results. The second contribution of this paper is to encapsulate the principle of explainability in the framework, systematically proposing not only the results but also the explanations of the results for every included machine learning algorithm.
BridgeTower: Building Bridges between Encoders in Vision-Language Representation Learning

Vision-Language (VL) models with the Two-Tower architecture have dominated visual-language representation learning in recent years. Current VL models either use lightweight uni-modal encoders and learn to extract, align and fuse both modalities simultaneously in a deep cross-modal encoder, or feed the last-layer uni-modal representations from the deep pre-trained uni-modal encoders into the top cross-modal encoder. Both approaches potentially restrict vision-language representation learning and limit model performance. In this paper, we propose BridgeTower, which introduces multiple bridge layers that build a connection between the top layers of uni-modal encoders and each layer of the cross-modal encoder. This enables effective bottom-up cross-modal alignment and fusion between visual and textual representations of different semantic levels of pre-trained uni-modal encoders in the cross-modal encoder. Pre-trained with only 4M images, BridgeTower achieves state-of-the-art performance on various downstream vision-language tasks. In particular, on the VQAv2 test-std set, BridgeTower achieves an accuracy of 78.73%, outperforming the previous state-of-the-art model METER by 1.09% with the same pre-training data and almost negligible additional parameters and computational costs. Notably, when further scaling the model, BridgeTower achieves an accuracy of 81.15%, surpassing models that are pre-trained on orders-of-magnitude larger datasets. Code and checkpoints are available at https://github.com/microsoft/BridgeTower.
VA + Embeddings STAR: A State‐of‐the‐Art Report on the Use of Embeddings in Visual Analytics

Abstract Over the past years, an increasing number of publications in information visualization, especially within the field of visual analytics, have mentioned the term “embedding” when describing the computational approach. Within this context, embeddings are usually (relatively) low‐dimensional, distributed representations of various data types (such as texts or graphs), and since they have proven to be extremely useful for a variety of data analysis tasks across various disciplines and fields, they have become widely used. Existing visualization approaches aim to either support exploration and interpretation of the embedding space through visual representation and interaction, or aim to use embeddings as part of the computational pipeline for addressing downstream analytical tasks. To the best of our knowledge, this is the first survey that takes a detailed look at embedding methods through the lens of visual analytics, and the purpose of our survey article is to provide a systematic overview of the state of the art within the emerging field of embedding visualization. We design a categorization scheme for our approach, analyze the current research frontier based on peer‐reviewed publications, and discuss existing trends, challenges, and potential research directions for using embeddings in the context of visual analytics. Furthermore, we provide an interactive survey browser for the collected and categorized survey data, which currently includes 122 entries that appeared between 2007 and 2023.
A Cost-Sensitive Approach applied on Shallow and Deep Neural Networks for Classification of Imbalanced Data

In this study, we propose a cost-sensitive learning approach applied on neural networks to deal with classification under imbalanced domains.Our approach is able to automatically learn robust features for both frequent and rare classes by automatically assigning misclassification penalties to each class based the frequency of occurrence of that class.This approach is investigated in the context of shallow networks (multi-layer perceptrons) and deep networks (convolutional neural networks).Moreover, it offers not only a better convergence but also a faster convergence since it can boost optimization by increasing weight gradients which are getting small due to their fitting to the frequent classes.Extensive experiments were carried out on one-and two-dimensional datasets.Running experiments using several loss functions showed the efficacity of our approach on loss functions which do not have probability estimates.Additionally, our approach achieved a good performance compared to common undersampling and oversampling methods as well as models based on generative adversarial networks.
KnowGL: Knowledge Generation and Linking from Text

We propose KnowGL, a tool that allows converting text into structured relational data represented as a set of ABox assertions compliant with the TBox of a given Knowledge Graph (KG), such as Wikidata. We address this problem as a sequence generation task by leveraging pre-trained sequence-to-sequence language models, e.g. BART. Given a sentence, we fine-tune such models to detect pairs of entity mentions and jointly generate a set of facts consisting of the full set of semantic annotations for a KG, such as entity labels, entity types, and their relationships. To showcase the capabilities of our tool, we build a web application consisting of a set of UI widgets that help users to navigate through the semantic data extracted from a given input text. We make the KnowGL model available at https://huggingface.co/ibm/knowgl-large.
Effect of information literacy on lifelong learning, creativity, and work performance among journalists

Purpose This research examined the effects of information literacy on lifelong learning, creativity, and work performance among journalists in Pakistan. Design/methodology/approach Survey research design was applied to conduct this research. The participants were recruited through a stratified convenient sampling process from the press clubs of four provinces (e.g. Punjab, Sindh, Khyber Pakhtunkhwa, and Baluchistan) and the federal capital Islamabad with the consent of relevant authorities for data collection. An online questionnaire was distributed among these journalists and a total of 1,089 responses were received. The data were analyzed by applying descriptive and inferential statistics in SPSS. Findings The results revealed that these journalists perceived themselves as information literate. The information literacy (IL) skills of journalists appeared to have a direct and positive effect on their lifelong learning, creativity, and work performance. In other words, the lifelong learning, creativity, and work performance of journalists increase as their levels of IL skills increase. Practical implications These results generated useful insights for academicians and organizations about the importance of IL in the workplace and its influence on organizational effectiveness and performance in gaining a sustainable competitive advantage. This knowledge might be crucial for media employers to initiate training programs for journalists to impart IL education. Originality/value This research would be a worthwhile contribution to the existing research on workplace IL, particularly in the context of journalists' workplace as no such comprehensive study using these variables appeared so far. Peer review The peer review history for this article is available at: https://publons.com/publon/10.1108/OIR-06-2022-0345 .
Localizing in-domain adaptation of transformer-based biomedical language models

In the era of digital healthcare, the huge volumes of textual information generated every day in hospitals constitute an essential but underused asset that could be exploited with task-specific, fine-tuned biomedical language representation models, improving patient care and management. For such specialized domains, previous research has shown that fine-tuning models stemming from broad-coverage checkpoints can largely benefit additional training rounds over large-scale in-domain resources. However, these resources are often unreachable for less-resourced languages like Italian, preventing local medical institutions to employ in-domain adaptation. In order to reduce this gap, our work investigates two accessible approaches to derive biomedical language models in languages other than English, taking Italian as a concrete use-case: one based on neural machine translation of English resources, favoring quantity over quality; the other based on a high-grade, narrow-scoped corpus natively written in Italian, thus preferring quality over quantity. Our study shows that data quantity is a harder constraint than data quality for biomedical adaptation, but the concatenation of high-quality data can improve model performance even when dealing with relatively size-limited corpora. The models published from our investigations have the potential to unlock important research opportunities for Italian hospitals and academia. Finally, the set of lessons learned from the study constitutes valuable insights towards a solution to build biomedical language models that are generalizable to other less-resourced languages and different domain settings.
Learning by Small Loss Approach Multi-label to Deal with Noisy Labels

Noisy data samples is a common problem for deep learning models applied to real-world applications. In this context, noisy samples refer to samples with incorrect labels, which can potentially degenerate the robustness of a model. Several works account for this issue in multi-class scenarios. However, despite a number of possible applications, multi-label noise remains an under-explored research field. In this work, two novel approaches to handle noise in this scenario are presented. First, we propose a new multi-label version of the Small Loss Approach (SLA), formerly multi-class, to handle multi-label noise. Second, we apply the multi-label SLA to a novel model, Learning by SLA Multi-label, based on Co-teaching. The proposed model achieves a performance gain of $$15\%$$ in the benchmark UcMerced when compared to its baseline Co-teaching and a standard model (without any noise-handling technique). In addition, the model is also evaluated in a real-world scenario of underwater equipment imagery classification, yielding a relative improvement of $$9\%$$ in F1-Score.
Performance analysis of multi-shot shadow estimation

Shadow estimation is an efficient method for predicting many observables of a quantum state with a statistical guarantee. In the multi-shot scenario, one performs projective measurement on the sequentially prepared state for <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>K</mml:mi></mml:math> times after the same unitary evolution, and repeats this procedure for <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>M</mml:mi></mml:math> rounds of random sampled unitary. As a result, there are <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>M</mml:mi><mml:mi>K</mml:mi></mml:math> times measurements in total. Here we analyze the performance of shadow estimation in this multi-shot scenario, which is characterized by the variance of estimating the expectation value of some observable <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>O</mml:mi></mml:math>. We find that in addition to the shadow-norm <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo fence="false" stretchy="false">&amp;#x2016;</mml:mo><mml:mi>O</mml:mi><mml:msub><mml:mo fence="false" stretchy="false">&amp;#x2016;</mml:mo><mml:mrow class="MJX-TeXAtom-ORD"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mi mathvariant="normal">s</mml:mi><mml:mi mathvariant="normal">h</mml:mi><mml:mi mathvariant="normal">a</mml:mi><mml:mi mathvariant="normal">d</mml:mi><mml:mi mathvariant="normal">o</mml:mi><mml:mi mathvariant="normal">w</mml:mi></mml:mrow></mml:mrow></mml:msub></mml:math> introduced in \cite{huang2020predicting}, the variance is also related to another norm, and we denote it as the cross-shadow-norm <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo fence="false" stretchy="false">&amp;#x2016;</mml:mo><mml:mi>O</mml:mi><mml:msub><mml:mo fence="false" stretchy="false">&amp;#x2016;</mml:mo><mml:mrow class="MJX-TeXAtom-ORD"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mi mathvariant="normal">X</mml:mi><mml:mi mathvariant="normal">s</mml:mi><mml:mi mathvariant="normal">h</mml:mi><mml:mi mathvariant="normal">a</mml:mi><mml:mi mathvariant="normal">d</mml:mi><mml:mi mathvariant="normal">o</mml:mi><mml:mi mathvariant="normal">w</mml:mi></mml:mrow></mml:mrow></mml:msub></mml:math>. For both random Pauli and Clifford measurements, we analyze and show the upper bounds of <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mo fence="false" stretchy="false">&amp;#x2016;</mml:mo><mml:mi>O</mml:mi><mml:msub><mml:mo fence="false" stretchy="false">&amp;#x2016;</mml:mo><mml:mrow class="MJX-TeXAtom-ORD"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mi mathvariant="normal">X</mml:mi><mml:mi mathvariant="normal">s</mml:mi><mml:mi mathvariant="normal">h</mml:mi><mml:mi mathvariant="normal">a</mml:mi><mml:mi mathvariant="normal">d</mml:mi><mml:mi mathvariant="normal">o</mml:mi><mml:mi mathvariant="normal">w</mml:mi></mml:mrow></mml:mrow></mml:msub></mml:math>. In particular, we figure out the exact variance formula for Pauli observable under random Pauli measurements. Our work gives theoretical guidance for the application of multi-shot shadow estimation.
Real-Time Segmentation of Unstructured Environments by Combining Domain Generalization and Attention Mechanisms

This paper presents a focused investigation into real-time segmentation in unstructured environments, a crucial aspect for enabling autonomous navigation in off-road robots. To address this challenge, an improved variant of the DDRNet23-slim model is proposed, which includes a lightweight network architecture and reclassifies ten different categories, including drivable roads, trees, high vegetation, obstacles, and buildings, based on the RUGD dataset. The model’s design includes the integration of the semantic-aware normalization and semantic-aware whitening (SAN–SAW) module into the main network to improve generalization ability beyond the visible domain. The model’s segmentation accuracy is improved through the fusion of channel attention and spatial attention mechanisms in the low-resolution branch to enhance its ability to capture fine details in complex scenes. Additionally, to tackle the issue of category imbalance in unstructured scene datasets, a rare class sampling strategy (RCS) is employed to mitigate the negative impact of low segmentation accuracy for rare classes on the overall performance of the model. Experimental results demonstrate that the improved model achieves a significant 14% increase mIoU in the invisible domain, indicating its strong generalization ability. With a parameter count of only 5.79M, the model achieves mAcc of 85.21% and mIoU of 77.75%. The model has been successfully deployed on a a Jetson Xavier NX ROS robot and tested in both real and simulated orchard environments. Speed optimization using TensorRT increased the segmentation speed to 30.17 FPS. The proposed model strikes a desirable balance between inference speed and accuracy and has good domain migration ability, making it applicable in various domains such as forestry rescue and intelligent agricultural orchard harvesting.
Reinforcement Learning Applied to AI Bots in First-Person Shooters: A Systematic Review

Reinforcement Learning is one of the many machine learning paradigms. With no labelled data, it is concerned with balancing the exploration and exploitation of an environment with one or more agents present in it. Recently, many breakthroughs have been made in the creation of these agents for video game machine learning development, especially in first-person shooters with platforms such as ViZDoom, DeepMind Lab, and Unity’s ML-Agents. In this paper, we review the state-of-the-art of creation of Reinforcement Learning agents for use in multiplayer deathmatch first-person shooters. We selected various platforms, frameworks, and training architectures from various papers and examined each of them, analysing their uses. We compared each platform and training architecture, and then concluded whether machine learning agents can now face off against humans and whether they make for better gameplay than traditional Artificial Intelligence. In the end, we thought about future research and what researchers should keep in mind when exploring and testing this area.
Trustworthy Medical Image Segmentation with improved performance for in-distribution samples

Despite the enormous achievements of Deep Learning (DL) based models, their non-transparent nature led to restricted applicability and distrusted predictions. Such predictions emerge from erroneous In-Distribution (ID) and Out-Of-Distribution (OOD) samples, which results in disastrous effects in the medical domain, specifically in Medical Image Segmentation (MIS). To mitigate such effects, several existing works accomplish OOD sample detection; however, the trustworthiness issues from ID samples still require thorough investigation. To this end, a novel method TrustMIS (Trustworthy Medical Image Segmentation) is proposed in this paper, which provides the trustworthiness and improved performance of ID samples for DL-based MIS models. TrustMIS works in three folds: IT (Investigating Trustworthiness), INT (Improving Non-Trustworthy prediction) and CSO (Classifier Switching Operation). Initially, the IT method investigates the trustworthiness of MIS by leveraging similar characteristics and consistency analysis of input and its variants. Subsequently, the INT method employs the IT method to improve the performance of the MIS model. It leverages the observation that an input providing erroneous segmentation can provide correct segmentation with rotated input. Eventually, the CSO method employs the INT method to scrutinise several MIS models and selects the model that delivers the most trustworthy prediction. The experiments conducted on publicly available datasets using well-known MIS models reveal that TrustMIS has successfully provided a trustworthiness measure, outperformed the existing methods, and improved the performance of state-of-the-art MIS models. Our implementation is available at https://github.com/SnehaShukla937/TrustMIS.
Advances in validation of remotely sensed land surface evapotranspiration

北京 100875; 2. 重庆市勘测院, 重庆 400020;
Advancing human action recognition: A hybrid approach using attention-based LSTM and 3D CNN

In this paper, we propose a novel approach to video action recognition that integrates a modified and optimized 3D Convolutional Neural Network, a Long Short-Term Memory network, and attention mechanisms. This synergy enhances the overall performance, offering an advantage over existing methods in managing the intricacies of real-world scenarios. The uniqueness of our approach lies in its capacity to capture both spatial and temporal information from video sequences and the incorporation of an attention mechanism that selectively emphasizes key areas within the sequences, thereby enhancing recognition accuracy. The model is particularly tailored to handle complex scenarios, such as those with multiple actors or objects, or instances of occlusion. It effectively addresses the subjectivity and variability inherent in action annotations within datasets. We also apply an array of preprocessing techniques to further optimize model performance. Through rigorous experimental evaluations on benchmark datasets, namely UCF101 and HMDB51, we demonstrate that our proposed approach significantly outperforms existing state-of-the-art methods in action recognition. These results underscore the potential of our approach for further advancements in video action recognition research.
Effective Software Effort Estimation Leveraging Machine Learning for Digital Transformation

Software effort estimation is a necessary component of software development projects that belong to industrial software systems and digital transformation initiatives. Digital transformation refers to the process of integrating digital technology into various components of a company or organization in order to improve operations, procedures, customer experiences, and overall performance. Industrial software systems are trained software packages designed for use in industrial and manufacturing processes. The paper deals with the machine learning based effort estimation in order to create an effective and robust model for predicting effort. The paper proposes an Omni-Ensemble Learning (OEL) approach, which is a combination of static ensemble selection along with genetic algorithm and dynamic ensemble selection. The paper identifies the impact of software effort estimation in industrial software system, and works on the these attributes to implement a robust ensemble model. The proposed Omni-Ensemble Selection (OES) provides better overall performance (in terms of evaluation metrics) and on comparing with multiple machine learning models over Finnish and Maxwell datasets.