Reviewerly/exHarmony huggingface dataset:
Corpus creation and language identification for code-mixed Indonesian-Javanese-English Tweets

With the massive use of social media today, mixing between languages in social media text is prevalent. In linguistics, the phenomenon of mixing languages is known as code-mixing. The prevalence of code-mixing exposes various concerns and challenges in natural language processing (NLP), including language identification (LID) tasks. This study presents a word-level language identification model for code-mixed Indonesian, Javanese, and English tweets. First, we introduce a code-mixed corpus for Indonesian-Javanese-English language identification (IJELID). To ensure reliable dataset annotation, we provide full details of the data collection and annotation standards construction procedures. Some challenges encountered during corpus creation are also discussed in this paper. Then, we investigate several strategies for developing code-mixed language identification models, such as fine-tuning BERT, BLSTM-based, and CRF. Our results show that fine-tuned IndoBERTweet models can identify languages better than the other techniques. This is the result of BERT’s ability to understand each word’s context from the given text sequence. Finally, we show that sub-word language representation in BERT models can provide a reliable model for identifying languages in code-mixed texts.
Asymmetry and tighter uncertainty relations for Rényi entropies via quantum-classical decompositions of resource measures

The author proposes a general method of constructing quantum--classical decompositions of resources such as uncertainty, with the quantum contribution specified by a measure of the noncommutativity of a given set of operators relative to the quantum state, and the classical contribution generated by the mixedness of the state. One result of this formalism is that entropic uncertainty relations, computed by employing the entropy instead of the variance as a measure of uncertainty about a measurement outcome, are stronger than previously known.
Digital skills and entrepreneurial intentions for final-year undergraduates: entrepreneurship education as a moderator and entrepreneurial alertness as a mediator

Purpose This study aims to test the impact of digital skills on the entrepreneurial intentions of last-year undergraduate students in Jordanian universities, especially after the Coronavirus disease pandemic and the digital transformation in education and business patterns. In addition, it aims to assess the role of entrepreneurial alertness as a mediator and entrepreneurship education as a moderator in the relationship between the independent and dependent variables. Design/methodology/approach The quantitative study used a questionnaire distributed to 401 students from different Jordanian universities. The data was collected over 2 months and two structural equation models were developed using AMOS 25 to examine the relationship. Findings A significant negative relationship was found between digital skills and entrepreneurial intentions of last-year undergraduate students in Jordanian universities post-coronavirus disease 2019 (COVID-19) pandemic. A fully mediating role of the alertness variable has appeared in addition to a significant moderating role of entrepreneurship education. Originality/value This is the first study that attempts to investigate the impact of digital skills on students' entrepreneurial intentions in Jordan after the COVID-19 pandemic, In addition, it is one of the few studies that assess the mediator's and moderator's effects on the same conditions. Finally, the study provided a review of the definitions and models used as part of the contribution to upcoming reviews.
Inexact iterative numerical linear algebra for neural network-based spectral estimation and rare-event prediction

Understanding dynamics in complex systems is challenging because there are many degrees of freedom, and those that are most important for describing events of interest are often not obvious. The leading eigenfunctions of the transition operator are useful for visualization, and they can provide an efficient basis for computing statistics, such as the likelihood and average time of events (predictions). Here, we develop inexact iterative linear algebra methods for computing these eigenfunctions (spectral estimation) and making predictions from a dataset of short trajectories sampled at finite intervals. We demonstrate the methods on a low-dimensional model that facilitates visualization and a high-dimensional model of a biomolecular system. Implications for the prediction problem in reinforcement learning are discussed.
Interactive Learning of Intrinsic and Extrinsic Properties for All-Day Semantic Segmentation

Scene appearance changes drastically throughout the day. Existing semantic segmentation methods mainly focus on well-lit daytime scenarios and are not well designed to cope with such great appearance changes. Naively using domain adaption does not solve this problem because it usually learns a fixed mapping between the source and target domain and thus have limited generalization capability on all-day scenarios ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., from dawn to night). In this paper, in contrast to existing methods, we tackle this challenge from the perspective of image formulation itself, where the image appearance is determined by both intrinsic ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., semantic category, structure) and extrinsic ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., lighting) properties. To this end, we propose a novel intrinsic-extrinsic interactive learning strategy. The key idea is to interact between intrinsic and extrinsic representations during the learning process under spatial-wise guidance. In this way, the intrinsic representation becomes more stable and, at the same time, the extrinsic representation gets better at depicting the changes. Consequently, the refined image representation is more robust to generate pixel-wise predictions for all-day scenarios. To achieve this, we propose an All-in-One Segmentation Network (AO-SegNet) in an end-to-end manner. Large scale experiments are conducted on three real datasets (Mapillary, BDD100K and ACDC) and our proposed synthetic All-day CityScapes dataset. The proposed AO-SegNet shows a significant performance gain against the state-of-the-art under a variety of CNN and ViT backbones on all the datasets.
Testing correlation of unlabeled random graphs

We study the problem of detecting the edge correlation between two random graphs with n unlabeled nodes. This is formalized as a hypothesis testing problem, where under the null hypothesis, the two graphs are independently generated; under the alternative, the two graphs are edge-correlated under some latent node correspondence, but have the same marginal distributions as the null. For both Gaussian-weighted complete graphs and dense Erdős–Rényi graphs (with edge probability n−o(1)), we determine the sharp threshold at which the optimal testing error probability exhibits a phase transition from zero to one as n→∞. For sparse Erdős–Rényi graphs with edge probability n−Ω(1), we determine the threshold within a constant factor. The proof of the impossibility results is an application of the conditional second-moment method, where we bound the truncated second moment of the likelihood ratio by carefully conditioning on the typical behavior of the intersection graph (consisting of edges in both observed graphs) and taking into account the cycle structure of the induced random permutation on the edges. Notably, in the sparse regime, this is accomplished by leveraging the pseudoforest structure of subcritical Erdős–Rényi graphs and a careful enumeration of subpseudoforests that can be assembled from short orbits of the edge permutation.
Hourly global horizontal irradiance over West Africa: A case study of one-year satellite- and reanalysis-derived estimates vs. in situ measurements

Estimates of global horizontal irradiance (GHI) from reanalysis and satellite-based data are the most important information for the design and monitoring of PV systems in Africa, but their quality is unknown due to the lack of in situ measurements. In this study, we evaluate the performance of hourly GHI from state-of-the-art reanalysis and satellite-based products (ERA5, CAMS, MERRA-2, and SARAH-2) with 37 quality-controlled in situ measurements from novel meteorological networks established in Burkina Faso and Ghana under different weather conditions for the year 2020. The effects of clouds and aerosols are also considered in the analysis by using common performance measures for the main quality attributes and a new overall performance value for the joint assessment. The results show that satellite data performs better than reanalysis data under different atmospheric conditions. Nevertheless, both data sources exhibit significant bias of more than 150 W/m2 in terms of RMSE under cloudy skies compared to clear skies. The new measure of overall performance clearly shows that the hourly GHI derived from CAMS and SARAH-2 could serve as viable alternative data for assessing solar energy in the different climatic zones of West Africa.
Secure Blockchain-Enabled Authentication Key Management Framework with Big Data Analytics for Drones in Networks Beyond 5G Applications

One of the most significant recent advances in technology is the advent of unmanned aerial vehicles (UAVs), i.e., drones. They have widened the scope of possible applications and provided a platform for a wide range of creative responses to a variety of challenges. The Internet of Drones (IoD) is a relatively new concept that has arisen as a consequence of the combination of drones and the Internet. The fifth-generation (5G) and beyond cellular networks (i.e., drones in networks beyond 5G) are promising solutions for achieving safe drone operations and applications. They may have many applications, like surveillance or urban areas, security, surveillance, retaliation, delivering items, smart farming, film production, capturing nature videos, and many more. Due to the fact that it is susceptible to a wide variety of cyber-attacks, there are certain concerns regarding the privacy and security of IoD communications. In this paper, a secure blockchain-enabled authentication key management framework with the big data analytics feature for drones in networks beyond 5G applications is proposed (in short, SBBDA-IoD). The security of SBBDA-IoD against multiple attacks is demonstrated through a detailed security analysis. The Scyther tool is used to perform a formal security verification test on the SBBDA-IoD’s security, confirming the system’s resistance to various potential attacks. A detailed comparative analysis has identified that SBBDA-IoD outperforms the other schemes by a significant margin. Finally, a real-world implementation of SBBDA-IoD is shown to evaluate its effect on several measures of performance.
Few-Shot Learning with Semi-Supervised Transformers for Electronic Health Records.

With the growing availability of Electronic Health Records (EHRs), many deep learning methods have been developed to leverage such datasets in medical prediction tasks. Notably, transformer-based architectures have proven to be highly effective for EHRs. Transformer-based architectures are generally very effective in "transferring" the acquired knowledge from very large datasets to smaller target datasets through their comprehensive "pre-training" process. However, to work efficiently, they still rely on the target datasets for the downstream tasks, and if the target dataset is (very) small, the performance of downstream models can degrade rapidly. In biomedical applications, it is common to only have access to small datasets, for instance, when studying rare diseases, invasive procedures, or using restrictive cohort selection processes. In this study, we present CEHR-GAN-BERT, a semi-supervised transformer-based architecture that leverages both in- and out-of-cohort patients to learn better patient representations in the context of few-shot learning. The proposed method opens new learning opportunities where only a few hundred samples are available. We extensively evaluate our method on four prediction tasks and three public datasets showing the ability of our model to achieve improvements upwards of 5% on all performance metrics (including AUROC and F1 Score) on the tasks that use less than 200 annotated patients during the training process.
A Simple and Strong Baseline for End-to-End Neural RST-style Discourse Parsing

To promote and further develop RST-style discourse parsing models, we need a strong baseline that can be regarded as a reference for reporting reliable experimental results.This paper explores a strong baseline by integrating existing simple parsing strategies, top-down and bottom-up, with various transformer-based pre-trained language models.The experimental results obtained from two benchmark datasets demonstrate that the parsing performance strongly relies on the pretrained language models rather than the parsing strategies.In particular, the bottom-up parser achieves large performance gains compared to the current best parser when employing DeBERTa.We further reveal that language models with a span-masking scheme especially boost the parsing performance through our analysis within intra-and multi-sentential parsing, and nuclearity prediction.
Exploiting Global and Local Hierarchies for Hierarchical Text Classification

Hierarchical text classification aims to leverage label hierarchy in multi-label text classification.Existing methods encode label hierarchy in a global view, where label hierarchy is treated as the static hierarchical structure containing all labels.Since global hierarchy is static and irrelevant to text samples, it makes these methods hard to exploit hierarchical information.Contrary to global hierarchy, local hierarchy as a structured labels hierarchy corresponding to each text sample.It is dynamic and relevant to text samples, which is ignored in previous methods.To exploit global and local hierarchies, we propose Hierarchy-guided BERT with Global and Local hierarchies (HBGL), which utilizes the large-scale parameters and prior language knowledge of BERT to model both global and local hierarchies.Moreover, HBGL avoids the intentional fusion of semantic and hierarchical modules by directly modeling semantic and hierarchical information with BERT.Compared with the state-of-the-art method HGCLR, our method achieves significant improvement on three benchmark datasets.
Counterfactual Active Learning for Out-of-Distribution Generalization

We study the out-of-distribution generalization of active learning that adaptively selects samples for annotation in learning the decision boundary of classification.Our empirical study finds that increasingly annotating seen samples may hardly benefit the generalization.To address the problem, we propose Counterfactual Active Learning (CounterAL) that empowers active learning with counterfactual thinking to bridge the seen samples with unseen cases.In addition to annotating factual samples, Coun-terAL requires annotators to answer counterfactual questions to construct counterfactual samples for training.To achieve CounterAL, we design a new acquisition strategy that selects the informative factual-counterfactual pairs for annotation; and a new training strategy that pushes the model update to focus on the discrepancy between factual and counterfactual samples.We evaluate CounterAL on multiple public datasets of sentiment analysis and natural language inference.The experiment results show that CounterAL requires fewer acquisition rounds and outperforms existing active learning methods by a large margin in OOD tests with comparable IID performance.
BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models

It is crucial to automatically construct knowledge graphs (KGs) of diverse new relations to support knowledge discovery and broad applications.Previous KG construction methods, based on either crowdsourcing or text mining, are often limited to a small predefined set of relations due to manual cost or restrictions in text corpus.Recent research proposed to use pretrained language models (LMs) as implicit knowledge bases that accept knowledge queries with prompts.Yet, the implicit knowledge lacks many desirable properties of a full-scale symbolic KG, such as easy access, navigation, editing, and quality assurance.In this paper, we propose a new approach of harvesting massive KGs of arbitrary relations from pretrained LMs.With minimal input of a relation definition (a prompt and a few shot of example entity pairs), the approach efficiently searches in the vast entity pair space to extract diverse accurate knowledge of the desired relation.We develop an effective search-and-rescore mechanism for improved efficiency and accuracy.We deploy the approach to harvest KGs of over 400 new relations from different LMs.Extensive human and automatic evaluations show our approach manages to extract diverse accurate knowledge, including tuples of complex relations (e.g., "A is capable of but not good at B").The resulting KGs as a symbolic interpretation of the source LMs also reveal new insights into the LMs' knowledge capacities.
Label Sleuth: From Unlabeled Text to a Classifier in a Few Hours

Text classification can be useful in many realworld scenarios, saving a lot of time for end users.However, building a custom classifier typically requires coding skills and ML knowledge, which poses a significant barrier for many potential users.To lift this barrier, we introduce Label Sleuth, a free open source system for labeling and creating text classifiers.This system is unique for (a) being a no-code system, making NLP accessible to non-experts, (b) guiding users through the entire labeling process until they obtain a custom classifier, making the process efficient -from cold start to classifier in a few hours, and (c) being open for configuration and extension by developers.By open sourcing Label Sleuth we hope to build a community of users and developers that will broaden the utilization of NLP models.
Just Fine-tune Twice: Selective Differential Privacy for Large Language Models

Protecting large language models from privacy leakage is becoming increasingly crucial with their wide adoption in real-world products.Yet applying differential privacy (DP), a canonical notion with provable privacy guarantees for machine learning models, to those models remains challenging due to the trade-off between model utility and privacy loss.Utilizing the fact that sensitive information in language data tends to be sparse, Shi et al. ( 2021) formalized a DP notion extension called Selective Differential Privacy (SDP) to protect only the sensitive tokens defined by a policy function.However, their algorithm only works for RNN-based models.In this paper, we develop a novel framework, Just Fine-tune Twice (JFT), that achieves SDP for state-of-the-art large transformer-based models.Our method is easy to implement: it first finetunes the model with redacted in-domain data, and then fine-tunes it again with the original in-domain data using a private training mechanism.Furthermore, we study the scenario of imperfect implementation of policy functions that misses sensitive tokens and develop systematic methods to handle it.Experiments show that our method achieves strong utility compared to previous baselines.We also analyze the SDP privacy guarantee empirically with the canary insertion attack 1 .
Transforming Sequence Tagging Into A Seq2Seq Task

Pretrained, large, generative language models (LMs) have had great success in a wide range of sequence tagging and structured prediction tasks.Casting a sequence tagging task as a Seq2Seq one requires deciding the formats of the input and output sequences.However, we lack a principled understanding of the tradeoffs associated with these formats (such as the effect on model accuracy, sequence length, multilingual generalization, hallucination).In this paper, we rigorously study different formats one could use for casting input text sentences and their output labels into the input and target (i.e., output) of a Seq2Seq model.Along the way, we introduce a new format, which we show to to be both simpler and more effective.Additionally the new format demonstrates significant gains in the multilingual settings -both zero-shot transfer learning and joint training.Lastly, we find that the new format is more robust and almost completely devoid of hallucination -an issue we find common in existing formats.With well over a 1000 experiments studying 14 different formats, over 7 diverse public benchmarksincluding 3 multilingual datasets spanning 7 languages -we believe our findings provide a strong empirical basis in understanding how we should tackle sequence tagging tasks.
A Fast Image Encryption Algorithm Based on Logistic Mapping and Hyperchaotic Lorenz System for Clear Text Correlation

Image encryption (IE) technology is vital to privacy, but the parameter range of the traditional image encryption technology is limited.So it is important to enhance the performance of IE methods.In this study, a plaintext associative IE based on the improved logistic map and hyperchaotic system is proposed.The improved logistic map scrambles the pixels of the image.Then the hyperchaotic system performs diffusion and confusion operations on the image.For parameters μ and r, the sequence generated by the improved logistic map can traverse and uniformly distribute in the space (0,1).The Lyapunov exponent of the improved logistic map is greater than 21.5.The correlation coefficients of the encrypted image pixels after encryption are all less than 0.05.The UACI and NPCR indices of the improved encryption method are close to their theoretical values of 98.6133 and 33.1287, respectively, which are higher than those of the reference methods.The improved encryption method overcomes the limitations in the parameter range of traditional image and text encryption techniques, slow encryption speed, and uneven chaotic sequences.This method has higher security and sensitivity, andit can be used to establish a plaintext associative IE method.
A Benchmark for Neural Readability Assessment of Texts in Spanish

Laura Vásquez-Rodríguez, Pedro-Manuel Cuenca-Jiménez, Sergio Morales-Esquivel, Fernando Alva-Manchego. Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022). 2022.