Reviewerly/exHarmony huggingface dataset:
Empirical Studies of Institutional Federated Learning For Natural Language Processing

Federated learning has sparkled new interests in the deep learning society to make use of isolated data sources from independent institutes.With the development of novel training tools, we have successfully deployed federated natural language processing networks on GPU-enabled server clusters.This paper demonstrates federated training of a popular NLP model, TextCNN, with applications in sentence intent classification.Furthermore, differential privacy is introduced to protect participants in the training process, in a manageable manner.Distinguished from previous client-level privacy protection schemes, the proposed differentially private federated learning procedure is defined in the dataset sample level, inherent with the applications among institutions instead of individual users.Optimal settings of hyper-parameters for the federated TextCNN model are studied through comprehensive experiments.We also evaluated the performance of federated TextCNN model under imbalanced data load configuration.Experiments show that, the sampling ratio has a large impact on the performance of the FL models, causing up to 38.4% decrease in the test accuracy, while they are robust to different noise multiplier levels, with less than 3% variance in the test accuracy.It is also found that the FL models are sensitive to data load balancedness among client datasets.When the data load is imbalanced, model performance dropped by up to 10%.
Data-Driven Fuzzy Modeling Using Restricted Boltzmann Machines and Probability Theory

Fuzzy modeling has many advantages over nonfuzzy methods, such as robustness with respect to uncertainties and less sensitivity to the varying dynamics of nonlinear systems. Data-driven fuzzy modeling needs to extract fuzzy rules from input and output data, and to train the fuzzy parameters of the fuzzy model. This paper takes advantages from deep learning, probability theory, fuzzy modeling, and extreme learning machines (ELMs). Restricted Boltzmann machine (RBM) and probability theory are used to overcome some common problems in data-driven modeling methods. The RBM is modified such that it can be trained with continuous values. A probability-based clustering method is proposed to partition the hidden features from the RBM. The obtained fuzzy rules have probability measurement. ELM and an optimization method are applied to train the fuzzy model. The proposed method is validated with two benchmark problems.
Noisy Text Data: Achilles’ Heel of BERT

Owing to the phenomenal success of BERT on various NLP tasks and benchmark datasets, industry practitioners are actively experimenting with fine-tuning BERT to build NLP applications for solving industry use cases. For most datasets that are used by practitioners to build industrial NLP applications, it is hard to guarantee absence of any noise in the data. While BERT has performed exceedingly well for transferring the learnings from one use case to another, it remains unclear how BERT performs when fine-tuned on noisy text. In this work, we explore the sensitivity of BERT to noise in the data. We work with most commonly occurring noise (spelling mistakes, typos) and show that this results in significant degradation in the performance of BERT. We present experimental results to show that BERT’s performance on fundamental NLP tasks like sentiment analysis and textual similarity drops significantly in the presence of (simulated) noise on benchmark datasets viz. IMDB Movie Review, STS-B, SST-2. Further, we identify shortcomings in the existing BERT pipeline that are responsible for this drop in performance. Our findings suggest that practitioners need to be vary of presence of noise in their datasets while fine-tuning BERT to solve industry use cases.
Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data

Neural text generation (data-or text-to-text) demonstrates remarkable performance when training data is abundant which for many applications is not the case.To collect a large corpus of parallel data, heuristic rules are often used but they inevitably let noise into the data, such as phrases in the output which cannot be explained by the input.Consequently, models pick up on the noise and may hallucinategenerate fluent but unsupported text.Our contribution is a simple but powerful technique to treat such hallucinations as a controllable aspect of the generated text, without dismissing any input and without modifying the model architecture.On the WikiBio corpus (Lebret et al., 2016), a particularly noisy dataset, we demonstrate the efficacy of the technique both in an automatic and in a human evaluation.
CRSM: An Effective Blockchain Consensus Resource Slicing Model for Real-Time Distributed Energy Trading

Distributed energy trading has become an essential part of the energy trading market and provides a useful supplement to traditional centralized energy trading, but there are still problems such as opaque trading information and asymmetric user data. The blockchain technology has the advantages of traceability, trade openness, and data transparency, which is naturally suitable for distributed energy transactions. The electricity information data transmission represented by distributed energy transaction has the characteristics of real-time, which has a high-efficiency requirement on the selected blockchain technology. The consensus algorithm is the core of blockchain technology and affects the efficiency of the blockchain system. The efficiency of the existing consensus algorithms for energy transaction-oriented blockchain still needs to be improved. In this paper, a consensus resource slicing model(CRSM) is designed to meet the requirements of consensus efficiency in energy trading scenarios. Specifically, CRSM divides consensus nodes into different consensus domains for concurrent consensus, and the storage domain only stores block information without consensus. By building an experimental platform, the efficiency of CRSM was verified, the communication pressure of the blockchain system was reduced, and the consensus speed was effectively improved.
Graph Kernels: State-of-the-Art and Future Challenges

Graph-structured data are an integral part of many application domains, including chemoinformatics, computational biology, neuroimaging, and social network analysis. Over the last two decades, numerous graph kernels, i.e. kernel functions between graphs, have been proposed to solve the problem of assessing the similarity between graphs, thereby making it possible to perform predictions in both classification and regression settings. This manuscript provides a review of existing graph kernels, their applications, software plus data resources, and an empirical comparison of state-of-the-art graph kernels.
PolicyQA: A Reading Comprehension Dataset for Privacy Policies

Privacy policy documents are long and verbose.A question answering (QA) system can assist users in finding the information that is relevant and important to them.Prior studies in this domain frame the QA task as retrieving the most relevant text segment or a list of sentences from the policy document given a question.On the contrary, we argue that providing users with a short text span from policy documents reduces the burden of searching the target information from a lengthy text segment.In this paper, we present PolicyQA, a dataset that contains 25,017 reading comprehension style examples curated from an existing corpus of 115 website privacy policies.PolicyQA provides 714 human-annotated questions written for a wide range of privacy practices.We evaluate two existing neural QA models and perform rigorous analysis to reveal the advantages and challenges offered by PolicyQA.
Difference-aware Knowledge Selection for Knowledge-grounded Conversation Generation

In a multi-turn knowledge-grounded dialog, the difference between the knowledge selected at different turns usually provides potential clues to knowledge selection, which has been largely neglected in previous research.In this paper, we propose a difference-aware knowledge selection method.It first computes the difference between the candidate knowledge sentences provided at the current turn and those chosen in the previous turns.Then, the differential information is fused with or disentangled from the contextual information to facilitate final knowledge selection.Automatic, human observational, and interactive evaluation shows that our method is able to select knowledge more accurately and generate more informative responses, significantly outperforming the state-of-the-art baselines.The codes are available at https://github.com/chujiezheng/DiffKS.
Entity Enhanced BERT Pre-training for Chinese NER

Character-level BERT pre-trained in Chinese suffers a limitation of lacking lexicon information, which shows effectiveness for Chinese NER.To integrate the lexicon into pre-trained LMs for Chinese NER, we investigate a semisupervised entity enhanced BERT pre-training method.In particular, we first extract an entity lexicon from the relevant raw text using a newword discovery method.We then integrate the entity information into BERT using Char-Entity-Transformer, which augments the selfattention using a combination of character and entity representations.In addition, an entity classification task helps inject the entity information into model parameters in pre-training.The pre-trained models are used for NER finetuning.Experiments on a news dataset and two datasets annotated by ourselves for NER in long-text show that our method is highly effective and achieves the best results.
Boundary-Aware Multitask Learning for Remote Sensing Imagery

Semantic segmentation and height estimation play fundamental roles in the scene understanding of remote sensing images with their wide variety of aerial applications. Recently, deep convolutional neural networks (DCNNs) have achieved state-of-the-art performance in both tasks. However, DCNN-based methods learn to accumulate contextual information over large receptive fields while lose the local detailed information, resulting in blurry object boundaries. The complicated ground object distribution and low interclass variance further aggravate the difficulty in generating accurate predictions. To address the above-mentioned issues, we propose a novel boundary-aware multitask learning (BAMTL) framework to perform three tasks, semantic segmentation, height estimation, and boundary detection, within a unified model. The boundary detection is employed as an auxiliary task to regularize the other two master tasks at both the feature space and output space. We present a boundary attentive module to build the cross-task interaction for master tasks, which enforce the networks to filter out the confident area and focus on learning the high-frequency details. We then introduce a boundary regularized loss term to further refine the prediction maps to be locally consistent while preserving boundary structures. With these formulations, our model improves the performance of both segmentation and height tasks, especially along the boundaries. Experimental results on two publicly available remote sensing datasets demonstrate that the proposed approach performs favorably against the state-of-the-art methods.
Reinforcement learning-based optimal complete water-blasting for autonomous ship hull corrosion cleaning system

Routine cleaning of the corroded ship hulls in dry dock maintenance guarantees the smooth operation of the shipping industry. Deploying the autonomous system to remove the corrosion by water-blasting is a feasible approach to ease the burden in manual operation and to reduce water, time, and energy consumption. In this paper, the water-blasting framework is proposed for a novel robot platform named Hornbill with the adhesion mechanism by permanent magnetic, self-localization by sensor fusion to navigate smoothly on a vertical surface. Hence, we propose a complete waypoint path planning (CWPP) to re-blast the self-synthesizing deep convolutional neural network (DCNN) based corrosion heatmap by initial-blasting. The optimal CWPP problem, including the shortest travel distance and shortest travel time to save water, power while ensuring visiting all predefined waypoints by benchmarking output, is modeled as the classic Travel Salesman Problem (TSP). Further, the Pareto-optimal trajectory for given TSP has been driven by the reinforcement learning (RL) technique with a proposed reward function based on the robot's operation during blasting. From the experimental results at the shipyard site, the proposed RL-based CWPP generates the Pareto-optimal trajectory that enables the water-blasting robot to spend about 10% of energy and 9% of water less than the second-best evolutionary-based optimization method in various workspaces.
Information and communication technologies (ICTs) usage among agricultural extension officers and its impact on extension delivery in Ghana

In recent years, the agricultural industry has been experiencing an ever-increasing application of information and communication technologies globally. This new revolution has been touted to impact efficiency and productivity in the agricultural extension services within the agriculture sector. Notwithstanding this, empirical research need to be carried out amongst its users in the sector to ascertain these assertions. Therefore, the main objective of this study is to assess the use of Information and Communication Technologies (ICT) among agricultural extension workers and its implications on extension service delivery. A simple random sampling technique was used to select 153 field extension workers, and a structured questionnaire was used to elicit information from the respondents. The data obtained were analysed using IBM SPSS Statistics software version 22. The study revealed that agricultural extension officers use ICT for personal communication, but not mainly for extension activities. It was recommended that the agricultural extension services provide intensive ICT training for the agricultural extension workers to enhance ICT incorporation into extension advisory service.
Post-processing numerical weather prediction ensembles for probabilistic solar irradiance forecasting

In order to enable the transition towards renewable energy sources, probabilistic energy forecasting is of critical importance for incorporating volatile power sources such as solar energy into the electrical grid. Solar energy forecasting methods often aim to provide probabilistic predictions of solar irradiance. In particular, many hybrid approaches combine physical information from numerical weather prediction models with statistical methods. Even though the physical models can provide useful information at intra-day and day-ahead forecast horizons, ensemble weather forecasts from multiple model runs are often not calibrated and show systematic biases. We propose a post-processing model for ensemble weather predictions of solar irradiance at temporal resolutions between 30 min and 6 h. The proposed models provide probabilistic forecasts in the form of a censored logistic probability distribution for lead times up to 5 days and are evaluated in two case studies covering distinct physical models, geographical regions, temporal resolutions, and types of solar irradiance. We find that post-processing consistently and significantly improves the forecast performance of the ensemble predictions for lead times up to at least 48 h and is well able to correct the systematic lack of calibration.
Generalized theory of pseudomodes for exact descriptions of non-Markovian quantum processes

We develop an exact framework for describing the non-Markovian dynamics of an open quantum system interacting with an environment modeled by a generalized spectral density function. The approach relies on mapping the initial system onto an auxiliary configuration, comprising the original open system coupled to a small number of discrete modes, which in turn are each coupled to an independent Markovian reservoir. Based on the connection between the discrete modes and the poles of the spectral density function, we show how expanding the system using the discrete modes allows for the full inclusion non-Markovian effects within an enlarged open system whose dynamics is governed by an exact Lindblad master equation. Initially we apply this result to obtain a generalization of the pseudomode method [B. M. Garraway, Phys. Rev. A. 55, 2290 (1997)] in cases where the spectral density function has a Lorentzian structure. For many other types of spectral density function, we extend our proof to show that an open system dynamics may be modeled physically using discrete modes which admit a non-Hermitian coupling to the system and for such cases determine the equivalent master equation to no longer be of Lindblad form. For applications involving two discrete modes, we demonstrate how to convert between pathological and Lindblad forms of the master equation using the techniques of the pseudomode method.Received 12 April 2020Accepted 23 September 2020DOI:https://doi.org/10.1103/PhysRevResearch.2.043058Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasOpen quantum systems & decoherenceTechniquesMaster equationNon-Markovian processesGeneral Physics
Forecasting adverse surgical events using self-supervised transfer learning for physiological signals

Hundreds of millions of surgical procedures take place annually across the world, which generate a prevalent type of electronic health record (EHR) data comprising time series physiological signals. Here, we present a transferable embedding method (i.e., a method to transform time series signals into input features for predictive machine learning models) named PHASE (PHysiologicAl Signal Embeddings) that enables us to more accurately forecast adverse surgical outcomes based on physiological signals. We evaluate PHASE on minute-by-minute EHR data of more than 50,000 surgeries from two operating room (OR) datasets and patient stays in an intensive care unit (ICU) dataset. PHASE outperforms other state-of-the-art approaches, such as long-short term memory networks trained on raw data and gradient boosted trees trained on handcrafted features, in predicting six distinct outcomes: hypoxemia, hypocapnia, hypotension, hypertension, phenylephrine, and epinephrine. In a transfer learning setting where we train embedding models in one dataset then embed signals and predict adverse events in unseen data, PHASE achieves significantly higher prediction accuracy at lower computational cost compared to conventional approaches. Finally, given the importance of understanding models in clinical applications we demonstrate that PHASE is explainable and validate our predictive models using local feature attribution methods.
A hybrid fuzzy rule-based multi-criteria framework for sustainable-security assessment of web application

The present day environment of cyber-attacks is a threat to even the most secure systems. With each passing day, the attackers are adapting new technologies to infiltrate systems and exploit even the minutest of the vulnerabilities. In such a scenario, it is imperative to design and develop web applications that are not only secure but sustainable as well. While security is the most defined feature of a web application, sustainability also must be prioritized because any web application will be more efficacious if it proffers both to the users. In this league, it is imperative to reckon in on mechanisms that can effectively evaluate the web applications’ sustainable–security during the development process. To achieve the high level of sustainable-security in web applications, this contribution has used the Fuzzy Analytic Hierarchy Process (Fuzzy AHP) for evaluating the efficacy of the overall sustainable-security and its characteristics. Furthermore, the study tests the Fuzzy AHP method on ten successive versions of an institutional web application to corroborate the efficiency of this approach. Despite the web application's sensitivity, this study has used various versions of three web applications of the university. Results have been conclusive and far more effective when compared to the existing processes for testing the sustainable–security of web applications. Besides saving upon the time and resource investments, the mechanism proposed in our study seeks to set a template which the web developers can adhere to for optimizing on the sustainable-security while developing the web applications.
Trends in Vehicle Re-Identification Past, Present, and Future: A Comprehensive Review

Vehicle Re-identification (re-id) over surveillance camera network with non-overlapping field of view is an exciting and challenging task in intelligent transportation systems (ITS). Due to its versatile applicability in metropolitan cities, it gained significant attention. Vehicle re-id matches targeted vehicle over non-overlapping views in multiple camera network. However, it becomes more difficult due to inter-class similarity, intra-class variability, viewpoint changes, and spatio-temporal uncertainty. In order to draw a detailed picture of vehicle re-id research, this paper gives a comprehensive description of the various vehicle re-id technologies, applicability, datasets, and a brief comparison of different methodologies. Our paper specifically focuses on vision-based vehicle re-id approaches, including vehicle appearance, license plate, and spatio-temporal characteristics. In addition, we explore the main challenges as well as a variety of applications in different domains. Lastly, a detailed comparison of current state-of-the-art methods performances over VeRi-776 and VehicleID datasets is summarized with future directions. We aim to facilitate future research by reviewing the work being done on vehicle re-id till to date.
Cryptocurrencies in Modern Finance: A Literature Review

The focus on cryptocurrencies in the finance and banking sectors is gaining momentum. In this paper, we investigate the role of cryptocurrencies in modern finance. We apply a narrative literature review method to synthesize prior research and draw insights into the opportunities and challenges of leveraging cryptocurrencies. The results indicate that cryptocurrencies offer businesses and individuals’ lower transaction costs, higher efficiencies, increased security and privacy, meaningful diversification benefits, alternative financing solutions, and financial inclusion.Challenges exist related to the integration of cryptocurrencies in modern finance. These include the lack of regulatory standards, the risk of criminal activity, high energy and environmental costs, regulatory bans and usage restrictions, security and privacy concerns, and the high volatility of cryptocurrencies.The current review is useful for scholars and managers, including those seeking to have a more balanced understanding of these emerging financial instruments.JEL Classification: E42, F30, F65, G21, G23How to Cite:Rejeb, A., Rejeb, K., &amp; Keogh, J. G. (2021). Cryptocurrencies in Modern Finance: a Literature Review. Etikonomi, 20(1), 93 – 118. https://doi.org/10.15408/etk.v20i1.16911.