Reviewerly/exHarmony huggingface dataset:
More Identifiable yet Equally Performant Transformers for Text Classification

Interpretability is an important aspect of the trustworthiness of a model's predictions.Transformer's predictions are widely explained by the attention weights, i.e., a probability distribution generated at its self-attention unit (head).Current empirical studies provide shreds of evidence that attention weights are not explanations by proving that they are not unique.A recent study showed theoretical justifications to this observation by proving the non-identifiability of attention weights.For a given input to a head and its output, if the attention weights generated in it are unique, we call the weights identifiable.In this work, we provide deeper theoretical analysis and empirical observations on the identifiability of attention weights.Ignored in the previous works, we find the attention weights are more identifiable than we currently perceive by uncovering the hidden role of the key vector.However, the weights are still prone to be non-unique attentions that make them unfit for interpretation.To tackle this issue, we provide a variant of the encoder layer that decouples the relationship between key and value vector and provides identifiable weights up to the desired length of the input.We prove the applicability of such variations by providing empirical justifications on varied text classification tasks.The implementations are available at https://github.com/declare-lab/identifiable-transformers.
Compositionality of Complex Graphemes in the Undeciphered Proto-Elamite Script using Image and Text Embedding Models

We introduce a language modeling architecture which operates over sequences of images, or over multimodal sequences of images with associated labels.We use this architecture alongside other embedding models to investigate a category of signs called complex graphemes (CGs) in the undeciphered proto-Elamite script.We argue that CGs have meanings which are at least partly compositional, and we discover novel rules governing the construction of CGs.We find that a language model over sign images produces more interpretable results than a model over text or over sign images and text, which suggests that the names given to signs may be obscuring signals in the corpus.Our results reveal previously unknown regularities in proto-Elamite sign use that can inform future decipherment efforts, and our image-aware language model provides a novel way to abstract away from biases introduced by human annotators.
MULTILABEL OVER-SAMPLING AND UNDER-SAMPLING WITH CLASS ALIGNMENT FOR IMBALANCED MULTILABEL TEXT CLASSIFICATION

Simultaneous multiple labelling of documents, also known as multilabel text classification, will not perform optimally if the class is highly imbalanced. Class imbalanced entails skewness in the fundamental data for distribution that leads to more difficulty in classification. Random over-sampling and under-sampling are common approaches to solve the class imbalanced problem. However, these approaches have several drawbacks; the under-sampling is likely to dispose of useful data, whereas the over-sampling can heighten the probability of overfitting. Therefore, a new method that can avoid discarding useful data and overfitting problems is needed. This study proposes a method to tackle the class imbalanced problem by combining multilabel over-sampling and under-sampling with class alignment (ML-OUSCA). In the proposed ML-OUSCA, instead of using all the training instances, it draws a new training set by over-sampling small size classes and under-sampling big size classes. To evaluate our proposed ML-OUSCA, evaluation metrics of average precision, average recall and average F-measure on three benchmark datasets, namely, Reuters-21578, Bibtex, and Enron datasets, were performed. Experimental results showed that the proposed ML-OUSCA outperformed the chosen baseline random resampling approaches; K-means SMOTE and KNN-US. Thus, based on the results, we can conclude that designing a resampling method based on the class imbalanced together with class alignment will improve multilabel classification even better than just the random resampling method.
Learning Syntactic Dense Embedding with Correlation Graph for Automatic Readability Assessment

Deep learning models for automatic readability assessment generally discard linguistic features traditionally used in machine learning models for the task.We propose to incorporate linguistic features into neural network models by learning syntactic dense embeddings based on linguistic features.To cope with the relationships between the features, we form a correlation graph among features and use it to learn their embeddings so that similar features will be represented by similar embeddings.Experiments with six data sets of two proficiency levels demonstrate that our proposed methodology can complement BERT-only model to achieve significantly better performances for automatic readability assessment.
An empirical study of derivative-free-optimization algorithms for targeted black-box attacks in deep neural networks

Abstract We perform a comprehensive study on the performance of derivative free optimization (DFO) algorithms for the generation of targeted black-box adversarial attacks on Deep Neural Network (DNN) classifiers assuming the perturbation energy is bounded by an $$\ell _\infty$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>ℓ</mml:mi> <mml:mi>∞</mml:mi> </mml:msub> </mml:math> constraint and the number of queries to the network is limited. This paper considers four pre-existing state-of-the-art DFO-based algorithms along with a further developed algorithm built on BOBYQA, a model-based DFO method. We compare these algorithms in a variety of settings according to the fraction of images that they successfully misclassify given a maximum number of queries to the DNN. The experiments disclose how the likelihood of finding an adversarial example depends on both the algorithm used and the setting of the attack; algorithms limiting the search of adversarial example to the vertices of the $$\ell ^\infty$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msup> <mml:mi>ℓ</mml:mi> <mml:mi>∞</mml:mi> </mml:msup> </mml:math> constraint work particularly well without structural defenses, while the presented BOBYQA based algorithm works better for especially small perturbation energies. This variance in performance highlights the importance of new algorithms being compared to the state-of-the-art in a variety of settings, and the effectiveness of adversarial defenses being tested using as wide a range of algorithms as possible.
Online Memory Leak Detection in the Cloud-Based Infrastructures

A memory leak in an application deployed on the cloud can affect the availability and reliability of the application. Therefore, to identify and ultimately resolve it quickly is highly important. However, in the production environment running on the cloud, memory leak detection is a challenge without the knowledge of the application or its internal object allocation details. This paper addresses this challenge of online detection of memory leaks in cloud-based infrastructure without having any internal application knowledge by introducing a novel machine learning based algorithm Precog. This algorithm solely uses one metric i.e the system's memory utilization on which the application is deployed for the detection of a memory leak. The developed algorithm's accuracy was tested on 60 virtual machines manually labeled memory utilization data provided by our industry partner Huawei Munich Research Center and it was found that the proposed algorithm achieves the accuracy score of 85\% with less than half a second prediction time per virtual machine.
Use of Formal Ethical Reviews in NLP Literature: Historical Trends and Current Practices

Ethical aspects of research in language technologies have received much attention recently.It is a standard practice to get a study involving human subjects reviewed and approved by a professional ethics committee/board of the institution.How commonly do we see mention of ethical approvals in NLP research?What types of research or aspects of studies are usually subject to such reviews?With the rising concerns and discourse around the ethics of NLP, do we also observe a rise in formal ethical reviews of NLP studies?And, if so, would this imply that there is a heightened awareness of ethical issues that was previously lacking?We aim to address these questions by conducting a detailed quantitative and qualitative analysis of the ACL Anthology, as well as comparing the trends in our field to those of other related disciplines, such as cognitive science, machine learning, data mining, and systems.
On Interactive Oracle Proofs for Boolean R1CS Statements

The framework of interactive oracle proofs (IOP) has been used with great success to construct a number of efficient transparent zk-SNARKs in recent years. However, these constructions are based on Reed-Solomon codes and can only be applied directly to statements given in the form of arithmetic circuits or R1CS over large enough fields $$\mathbb {F}$$ . This motivates the question: what is the best way to apply these IOPs to statements that are naturally written as R1CS over small fields, and more concretely, the binary field $$\mathbb {F}_2$$ ? While one can just see the system as one over an extension field $$\mathbb {F}_{2^e}$$ containing $$\mathbb {F}_2$$ , this seems wasteful, as it uses e bits to encode just one “information” bit. In fact, in FC21 the work BooLigero devised a way to apply the well-known Ligero while being able to encode $$\sqrt{e}$$ bits into one element of $$\mathbb {F}_{2^e}$$ . In this paper, we introduce a new protocol for $$\mathbb {F}_2$$ -R1CS which among other things relies on a more efficient embedding which (for practical parameters) allows to encode $$\ge e/4$$ bits into an element of $$\mathbb {F}_{2^e}$$ . Our protocol makes then black box use of lincheck and rowcheck protocols for the larger field. Using the lincheck and rowcheck introduced in Aurora and Ligero respectively we obtain $$1.31 - 1.65 \times $$ smaller proofs for Aurora and $$3.71 \times $$ for Ligero. We also estimate the reduction of prover time by a factor of $$24.7 \times $$ for Aurora and between $$6.9 - 32.5 \times $$ for Ligero without interactive repetitions. Our methodology uses the notion of reverse multiplication friendly embeddings introduced in the area of secure multiparty computation, combined with a new IOPP to test linear statements modulo a subspace $$V \le \mathbb {F}_{2^e}$$ which may be of independent interest.
LICHEE: Improving Language Model Pre-training with Multi-grained Tokenization

Language model pre-training based on large corpora has achieved tremendous success in terms of constructing enriched contextual representations and has led to significant performance gains on a diverse range of Natural Language Understanding (NLU) tasks.Despite the success, most current pre-trained language models, such as BERT, are trained based on single-grained tokenization, usually with fine-grained characters or sub-words, making it hard for them to learn the precise meaning of coarse-grained words and phrases.In this paper, we propose a simple yet effective pretraining method named LICHEE to efficiently incorporate multi-grained information of input text.Our method can be applied to various pretrained language models and improve their representation capability.Extensive experiments conducted on CLUE and SuperGLUE demonstrate that our method achieves comprehensive improvements on a wide variety of NLU tasks in both Chinese and English with little extra inference cost incurred, and that our best ensemble model achieves the state-of-the-art performance on CLUE benchmark competition.
Formation conditions of noble metal mineralization in sulfide cobalt-copper-nickel ores of Kamchatka (on the example of Annabergitovaya Schel ore occurrence)

The authors present research results, the purpose of which is to study the specifics of noble metal mineralization and its genesis in sulfide cobalt-copper-nickel ores of the Kamchatka nickel-bearing province. The paper is dedicated to one of its many ore occurrences called Annabergitovaya Schel (Annabergite Gap). The material composition of platinoid, silver, gold, bismuth and tellurium minerals, as well as sulfarsenides in the ores of this occurrence was investigated. Based on the data of mineral formation sequence and the use of geosensors, conclusions were drawn regarding the genesis of noble metal mineralization.&#x0D; Formation of platinoid minerals, silver and gold at the Annabergitovaya Schel ore occurrence is mainly associated with the epigenetic effect of post-ore granitoids on ore-bearing intrusion rocks of the Dukuk complex of the cortlandite-norite formation and on syngenetic ores. An early association of noble metal minerals is represented by sperrylite, irarsite, and rare unnamed phases of Pt + Ir + Te. Irarsite and Pt + Ir + Te phases were formed at the contact-metasomatic stage. Sperrylite can be assumed to be of magmatic origin. Silver sulfides and tellurides, silver and palladium bismuth tellurides, and native gold were formed at the late, hydrothermal-metasomatic, stage. The occurrence conditions of mineral parageneses, associated with noble metal mineralization, correspond to the formation of shallow-depth metasomatic rocks (5 km). Sub-developed quartz-feldspar metasomatites, associated with the formation of early platinoid arsenides and sulfarsenides, are in equilibrium with circumneutral solutions (pH of 4.5-6.5) at temperatures of 350-600 °C. Late hydrothermal association with Pd, Ag and Au minerals is close to propylites and was formed at pH values of 4.5-6.5 and temperature of 150-350 °C.
Scalable Surface Reconstruction with Delaunay‐Graph Neural Networks

We introduce a novel learning-based, visibility-aware, surface reconstruction method for large-scale, defect-laden point clouds. Our approach can cope with the scale and variety of point cloud defects encountered in real-life Multi-View Stereo (MVS) acquisitions. Our method relies on a 3D Delaunay tetrahedralization whose cells are classified as inside or outside the surface by a graph neural network and an energy model solvable with a graph cut. Our model, making use of both local geometric attributes and line-of-sight visibility information, is able to learn a visibility model from a small amount of synthetic training data and generalizes to real-life acquisitions. Combining the efficiency of deep learning methods and the scalability of energy based models, our approach outperforms both learning and non learning-based reconstruction algorithms on two publicly available reconstruction benchmarks. Our code and data is available at https://github.com/raphaelsulzer/dgnn.
Effectiveness and Adoption of NIST Managerial Practices for Cyber Resilience in Italy

In the last years the use of information and communication technology in organizations has become vital to the point that each menace to its continuous functioning is considered a noteworthy danger for each organization. Cybersecurity has the aim of protecting the organization from these events called cyber-attacks. The emergent cyber resilience management integrates cyber risk management (which is based on identifying, analyzing and mitigating risk of cyber-attacks) with the ability to front them, recover from them and adapt the organization to the new situation when unpredictable attacks occur, without regressing. Several guidelines have been developed to guide organizations in managing cyber resilience, the NIST framework suggested and organized IT and managerial practices among different reference cyber security standards offering a practical overview followed by companies all over the world. Practices and guidelines must be general and consequently they need to be adapted to the specific context in which the company is embedded. In order to identify the effectiveness of the suggested managerial practices and the way they are implemented in the Italian context, we conducted a multiple case study analysis interviewing 20 cybersecurity experts included in the official list realised by the Italian Ministry of Economic Development in 2019. Interesting insights emerged including the lack of disciplinary measures in case of any misconduct, the importance of investing in building a comprehensive awareness of people about cyber threats, the importance of log information for multiple reasons and the urgency for each organization of developing its own tailored policies.
Semi-supervised Meta-learning for Cross-domain Few-shot Intent Classification

Meta-learning aims to optimize the model's capability to generalize to new tasks and domains.Lacking a data-efficient way to create meta training tasks has prevented the application of meta-learning to the real-world few shot learning scenarios.Recent studies have proposed unsupervised approaches to create meta-training tasks from unlabeled data for free, e.g., the SMLMT method (Bansal et al., 2020a) constructs unsupervised multiclass classification tasks from the unlabeled text by randomly masking words in the sentence and let the meta learner choose which word to fill in the blank.This study proposes a semi-supervised meta-learning approach that incorporates both the representation power of large pre-trained language models and the generalization capability of prototypical networks enhanced by SMLMT.The semi-supervised meta training approach avoids overfitting prototypical networks on a small number of labeled training examples and quickly learns cross-domain task-specific representation only from a few supporting examples.By incorporating SMLMT with prototypical networks, the meta learner generalizes better to unseen domains and gains higher accuracy on out-ofscope examples without the heavy lifting of pre-training.We observe significant improvement in few-shot generalization after training only a few epochs on the intent classification tasks evaluated in a multi-domain setting.
Multi‐hop non‐destructive qudit teleportation via non‐maximally entangled GHZ channels

A scheme for the multihop non-destructive teleportation of an arbitrary single-qudit state based on various non-maximally entangled Greenberger–Horne–Zeilinger channels is proposed here. The parallel entanglement swap is used to form a direct d-dimensional ( d ≥ 2 ) entangled channel shared between the sender and the receiver. The authors’ propose a novel idea that the third-party Charlie with high quantum information processing technology executes the channel modulation, which reduces the technical requirements of the sender and the receiver. In addition, if the teleportation fails, the unknown state will be restored by the sender. This work provides a feasible method for teleportation and ensures the integrity of the unknown quantum information.
SwapCT: Swap Confidential Transactions for Privacy-Preserving Multi-Token Exchanges

Abstract Decentralized token exchanges allow for secure trading of tokens without a trusted third party. However, decentralization is mostly achieved at the expense of transaction privacy. For a fair exchange, transactions must remain private to hide the participants and volumes while maintaining the possibility for noninteractive execution of trades. In this paper we present a swap confidential transaction system (SwapCT) which is related to ring confidential transactions (e.g. used in Monero) but supports multiple token types to trade among and enables secure, partial transactions for noninteractive swaps. We prove that SwapCT is secure in a strict, formal model and present its efficient performance in a prototype implementation with logarithmic signature sizes for large anonymity sets. For our construction we design an aggregatable signature scheme which might be of independent interest. Our SwapCT system thereby enables a secure and private exchange for tokens without a trusted third party.
An Identity-Based Blind Signature Scheme with Message Recovery from Pairings

As a variant of digital signature schemes, the blind signature enables that the signer signs a message without knowing its content. In identity-based cryptography, many blind signature schemes have been proposed. Among them, Verma et al. designed an identity-based blind signature scheme with message recovery (IDBS-MR). Unfortunately, after our cryptanalysis, their scheme cannot satisfy untraceability, which enables the signer to break the anonymity of users by tracking the previous signature transmission scripts. To solve the problem, we construct a new IDBS-MR scheme using the bilinear pairing and demonstrates that the proposed scheme has blindness, untraceability, and unforgeability in the random oracle model. Performance results show that compared with Verma et al.’s scheme, the proposed scheme has the same computation costs in views of signers, twice computation costs in views of users, but reduces 36% computation costs in views of verifiers.
Mechanism Construction of Human Resource Management based on Blockchain Technology

Abstract The traditional human resource management system is faced with such problems as the quality of staff recruitment can not be guaranteed, the training performance is not consistent with the actual performance, the enterprise performance appraisal is unfair, and the salary distribution is unfair, which seriously affects the enthusiasm and loyalty of employees and threatens the survival and development of enterprises. In view of the current situation and problems of human resource management system, combined with the technical characteristics of blockchain, this paper proposes a human resource management mechanism based on blockchain technology, in order to achieve the company’s human resource management system accurate, efficient, open and transparent. It is found that there is a good coupling between blockchain technology and human resource management system. This paper constructs a human resource management mechanism based on blockchain, innovates the application scenarios of blockchain technology, and provides certain guiding significance for the establishment of human resource management system and process improvement in the future.
Analysis of Transformation Tools Applicable on NewSQL Databases

With the introduction of NoSQL databases the new opportunities in the field of cloud technology like scalability and performance were utilized. In fact not NoSQL databases could not satisfy all requirements towards some applications like consistency and ACID transactions. For this reason NewSQL databases were developed - a mixture of relational databases and NoSQL databases with the aim to get the advantages of both. Besides, presenting the most known NewSQL databases and the requirements of transformation tools, the most popular transformation tools in the field of NewSQL databases were introduced. An analysis shows how the comply with the requirements.