Reviewerly/exHarmony huggingface dataset:
Early-Bird GCNs: Graph-Network Co-optimization towards More Efficient GCN Training and Inference via Drawing Early-Bird Lottery Tickets

Graph Convolutional Networks (GCNs) have emerged as the state-of-the-art deep learning model for representation learning on graphs. However, it remains notoriously challenging to train and inference GCNs over large graph datasets, limiting their application to large real-world graphs and hindering the exploration of deeper and more sophisticated GCN graphs. This is because as the graph size grows, the sheer number of node features and the large adjacency matrix can easily explode the required memory and data movements. To tackle the aforementioned challenges, we explore the possibility of drawing lottery tickets when sparsifying GCN graphs, i.e., subgraphs that largely shrink the adjacency matrix yet are capable of achieving accuracy comparable to or even better than their full graphs. Specifically, we for the first time discover the existence of graph early-bird (GEB) tickets that emerge at the very early stage when sparsifying GCN graphs, and propose a simple yet effective detector to automatically identify the emergence of such GEB tickets. Furthermore, we advocate graph-model co-optimization and develop a generic efficient GCN early-bird training framework dubbed GEBT that can significantly boost the efficiency of GCN training by (1) drawing joint early-bird tickets between the GCN graphs and models and (2) enabling simultaneously sparsification of both the GCN graphs and models. Experiments on various GCN models and datasets consistently validate our GEB finding and the effectiveness of our GEBT, e.g., our GEBT achieves up to 80.2% ~ 85.6% and 84.6% ~ 87.5% savings of GCN training and inference costs while offering a comparable or even better accuracy as compared to state-of-the-art methods. Our source code and supplementary appendix are available at https://github.com/RICE-EIC/Early-Bird-GCN.
Fitting and recognition of geometric primitives in segmented 3D point clouds using a localized voting procedure

The automatic creation of geometric models from point clouds has numerous applications in CAD (e.g., reverse engineering, manufacturing, assembling) and, more in general, in shape modelling and processing. Given a segmented point cloud representing a man-made object, we propose a method for recognizing simple geometric primitives and their interrelationships. Our approach is based on the Hough transform (HT) for its ability to deal with noise, missing parts and outliers. In our method we introduce a novel technique for processing segmented point clouds that, through a voting procedure, is able to provide an initial estimate of the geometric parameters characterizing each primitive type. By using these estimates, we localize the search of the optimal solution in a dimensionally-reduced parameter space thus making it efficient to extend the HT to more primitives than those that are generally found in the literature, i.e. planes and spheres. Then, we extract a number of geometric descriptors that uniquely characterize a segment, and, on the basis of these descriptors, we show how to aggregate parts of primitives (segments). Experiments on both synthetic and industrial scans reveal the robustness of the primitive fitting method and its effectiveness for inferring relations among segments.
Amortized Generation of Sequential Algorithmic Recourses for Black-Box Models

Explainable machine learning (ML) has gained traction in recent years due to the increasing adoption of ML-based systems in many sectors. Algorithmic Recourses (ARs) provide "what if" feedback of the form "if an input datapoint were x' instead of x, then an ML-based system's output would be y' instead of y." Recourses are attractive due to their actionable feedback, amenability to existing legal frameworks, and fidelity to the underlying ML model. Yet, current recourse approaches are single shot that is, they assume x can change to x' in a single time period. We propose a novel stochastic-control-based approach that generates sequential recourses, that is, recourses that allow x to move stochastically and sequentially across intermediate states to a final state x'. Our approach is model agnostic and black box. Furthermore, the calculation of recourses is amortized such that once trained, it applies to multiple datapoints without the need for re-optimization. In addition to these primary characteristics, our approach admits optional desiderata such as adherence to the data manifold, respect for causal relations, and sparsity identified by past research as desirable properties of recourses. We evaluate our approach using three real-world datasets and show successful generation of sequential recourses that respect other recourse desiderata.
Go Wider Instead of Deeper

More transformer blocks with residual connections have recently achieved impressive results on various tasks. To achieve better performance with fewer trainable parameters, recent methods are proposed to go shallower by parameter sharing or model compressing along with the depth. However, weak modeling capacity limits their performance. Contrastively, going wider by inducing more trainable matrixes and parameters would produce a huge model requiring advanced parallelism to train and inference. In this paper, we propose a parameter-efficient framework, going wider instead of deeper. Specially, following existing works, we adapt parameter sharing to compress along depth. But, such deployment would limit the performance. To maximize modeling capacity, we scale along model width by replacing feed-forward network (FFN) with mixture-of-experts (MoE). Across transformer blocks, instead of sharing normalization layers, we propose to use individual layernorms to transform various semantic representations in a more parameter-efficient way. To evaluate our plug-and-run framework, we design WideNet and conduct comprehensive experiments on popular computer vision and natural language processing benchmarks. On ImageNet-1K, our best model outperforms Vision Transformer (ViT) by 1.5% with 0.72 times trainable parameters. Using 0.46 times and 0.13 times parameters, our WideNet can still surpass ViT and ViT-MoE by 0.8% and 2.1%, respectively. On four natural language processing datasets, WideNet outperforms ALBERT by 1.8% on average and surpass BERT using factorized embedding parameterization by 0.8% with fewer parameters.
Cross-Domain Few-Shot Graph Classification

We study the problem of few-shot graph classification across domains with nonequivalent feature spaces by introducing three new cross-domain benchmarks constructed from publicly available datasets. We also propose an attention-based graph encoder that uses three congruent views of graphs, one contextual and two topological views, to learn representations of task-specific information for fast adaptation, and task-agnostic information for knowledge transfer. We run exhaustive experiments to evaluate the performance of contrastive and meta-learning strategies. We show that when coupled with metric-based meta-learning frameworks, the proposed encoder achieves the best average meta-test classification accuracy across all benchmarks.
Double-Color-Image Compression-Encryption Algorithm Based on Quaternion Multiple Parameter DFrAT and Feature Fusion with Preferable Restoration Quality

To achieve multiple color images encryption, a secure double-color-image encryption algorithm is designed based on the quaternion multiple parameter discrete fractional angular transform (QMPDFrAT), a nonlinear operation and a plaintext-related joint permutation-diffusion mechanism. QMPDFrAT is first defined and then applied to encrypt multiple color images. In the designed algorithm, the low-frequency and high-frequency sub-bands of the three color components of each plaintext image are obtained by two-dimensional discrete wavelet transform. Then, the high-frequency sub-bands are further made sparse and the main features of these sub-bands are extracted by a Zigzag scan. Subsequently, all the low-frequency sub-bands and high-frequency fusion images are represented as three quaternion signals, which are modulated by the proposed QMPDFrAT with three quaternion random phase masks, respectively. The spherical transform, as a nonlinear operation, is followed to nonlinearly make the three transform results interact. For better security, a joint permutation-diffusion mechanism based on plaintext-related random pixel insertion is performed on the three intermediate outputs to yield the final encryption image. Compared with many similar color image compression-encryption schemes, the proposed algorithm can encrypt double-color-image with higher quality of image reconstruction. Numerical simulation results demonstrate that the proposed double-color-image encryption algorithm is feasibility and achieves high security.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control

Abstractive summarization systems leveraging pre-training language models have achieved superior results on benchmark datasets.However, such models have been shown to be more prone to hallucinate facts that are unfaithful to the input context.In this paper, we propose a method to remedy entity-level extrinsic hallucinations with Entity Coverage Control (ECC).We first compute entity coverage precision and prepend the corresponding control code for each training example, which implicitly guides the model to recognize faithfulness contents in the training phase.We further extend our method via intermediate fine-tuning on large but noisy data extracted from Wikipedia to unlock zero-shot summarization.We show that the proposed method leads to more faithful and salient abstractive summarization in supervised fine-tuning and zero-shot settings according to our experimental results on three benchmark datasets XSum, Pubmed, and SAM-Sum of very different domains and styles.
Blockchain for Electronic Vaccine Certificates: More Cons Than Pros?

Electronic vaccine certificates (EVC) for COVID-19 vaccination are likely to become widespread. Blockchain (BC) is an electronic immutable distributed ledger and is one of the more common proposed EVC platform options. However, the principles of blockchain are not widely understood by public health and medical professionals. We attempt to describe, in an accessible style, how BC works and the potential benefits and drawbacks in its use for EVCs. Our assessment is BC technology is not well suited to be used for EVCs. Overall, blockchain technology is based on two key principles: the use of cryptography, and a distributed immutable ledger in the format of blockchains. While the use of cryptography can provide ease of sharing vaccination records while maintaining privacy, EVCs require some amount of contribution from a centralized authority to confirm vaccine status; this is partly because these authorities are responsible for the distribution and often the administration of the vaccine. Having the data distributed makes the role of a centralized authority less effective. We concluded there are alternative ways to use cryptography outside of a BC that allow a centralized authority to better participate, which seems necessary for an EVC platform to be of practical use.
Tensor networks in machine learning

A tensor network is a type of decomposition used to express and approximate large arrays of data. A given data-set, quantum state or higher dimensional multi-linear map is factored and approximated by a composition of smaller multi-linear maps. This is reminiscent to how a Boolean function might be decomposed into a gate array: this represents a special case of tensor decomposition, in which the tensor entries are replaced by 0, 1 and the factorisation becomes exact. The collection of associated techniques are called, tensor network methods: the subject developed independently in several distinct fields of study, which have more recently become interrelated through the language of tensor networks. The tantamount questions in the field relate to expressability of tensor networks and the reduction of computational overheads. A merger of tensor networks with machine learning is natural. On the one hand, machine learning can aid in determining a factorization of a tensor network approximating a data set. On the other hand, a given tensor network structure can be viewed as a machine learning model. Herein the tensor network parameters are adjusted to learn or classify a data-set. In this survey we recover the basics of tensor networks and explain the ongoing effort to develop the theory of tensor networks in machine learning.
A Two-Phase Machine Learning Framework for Context-Aware Service Selection to Empower People with Disabilities

The use of software and IoT services is increasing significantly among people with special needs, who constitute 15% of the world's population. However, selecting appropriate services to create a composite assistive service based on the evolving needs and context of disabled user groups remains a challenging research endeavor. Our research applies a scenario-based design technique to contribute (1) an inclusive disability ontology for assistive service selection, (2) semi-synthetic generated disability service datasets, and (3) a machine learning (ML) framework to choose services adaptively to suit the dynamic requirements of people with special needs. The ML-based selection framework is applied in two complementary phases. In the first phase, all available atomic tasks are assessed to determine their appropriateness to the user goal and profiles, whereas in the subsequent phase, the list of service providers is narrowed by matching their quality-of-service factors against the context and characteristics of the disabled person. Our methodology is centered around a myriad of user characteristics, including their disability profile, preferences, environment, and available IT resources. To this end, we extended the widely used QWS V2.0 and WS-DREAM web services datasets with a fusion of selected accessibility features. To ascertain the validity of our approach, we compared its performance against common multi-criteria decision making (MCDM) models, namely AHP, SAW, PROMETHEE, and TOPSIS. The findings demonstrate superior service selection accuracy in contrast to the other methods while ensuring accessibility requirements are satisfied.
A Unifying Framework for Reinforcement Learning and Planning

Sequential decision making, commonly formalized as optimization of a Markov Decision Process, is a key challenge in artificial intelligence. Two successful approaches to MDP optimization are reinforcement learning and planning, which both largely have their own research communities. However, if both research fields solve the same problem, then we might be able to disentangle the common factors in their solution approaches. Therefore, this paper presents a unifying algorithmic framework for reinforcement learning and planning (FRAP), which identifies underlying dimensions on which MDP planning and learning algorithms have to decide. At the end of the paper, we compare a variety of well-known planning, model-free and model-based RL algorithms along these dimensions. Altogether, the framework may help provide deeper insight in the algorithmic design space of planning and reinforcement learning.
Fully-Semantic Parsing and Generation: the BabelNet Meaning Representation

A language-independent representation of meaning is one of the most coveted dreams in Natural Language Understanding.With this goal in mind, several formalisms have been proposed as frameworks for meaning representation in Semantic Parsing.And yet, the dependencies these formalisms share with respect to language-specific repositories of knowledge make the objective of closing the gap between high-and low-resourced languages hard to accomplish.In this paper, we present the Ba-belNet Meaning Representation (BMR), an interlingual formalism that abstracts away from language-specific constraints by taking advantage of the multilingual semantic resources of BabelNet and VerbAtlas.We describe the rationale behind the creation of BMR and put forward BMR 1.0, a dataset labeled entirely according to the new formalism.Moreover, we show how BMR is able to outperform previous formalisms thanks to its fully-semantic framing, which enables top-notch multilingual parsing and generation.We release the code at https: //github.com/SapienzaNLP/bmr.
DiBiMT: A Novel Benchmark for Measuring Word Sense Disambiguation Biases in Machine Translation

Lexical ambiguity poses one of the greatest challenges in the field of Machine Translation.Over the last few decades, multiple efforts have been undertaken to investigate incorrect translations caused by the polysemous nature of words.Within this body of research, some studies have posited that models pick up semantic biases existing in the training data, thus producing translation errors.In this paper, we present DIBIMT, the first entirely manuallycurated evaluation benchmark which enables an extensive study of semantic biases in Machine Translation of nominal and verbal words in five different language combinations, namely, English and one or other of the following languages: Chinese, German, Italian, Russian and Spanish.Furthermore, we test state-of-the-art Machine Translation systems, both commercial and non-commercial ones, against our new test bed and provide a thorough statistical and linguistic analysis of the results.We release DIBIMT at https:// nlp.uniroma1.it/dibimtas a closed benchmark with a public leaderboard.
AN ANALYSIS OF STUDENTS’ SPEAKING ANXIETY IN ACADEMIC SPEAKING CLASS

The study aimed to gain information about what made students felt anxious and what factors triggered students’ speaking anxiety in Academic Speaking class in English Language Education Program (ELEP) in a private university in Central Java, Indonesia. The participants of the study were 52 second-year ELEP students. A close-ended and open-ended questionnaire developed from Horwitz et al, (1986) and Angelia and Listyani (2019) were administered to the participants. The findings revealed that students’ speaking anxiety in English speaking class was derived from three main factors of anxiety, i.e. communication apprehension, test anxiety and fear of negative evaluation. Moreover, as the questionnaires answers and interview data revealed, lack of vocabulary, underestimate their ability, lack of preparation, being afraid of making mistakes, and worried of being embarassed at by his/her friends had also contributed to students’ anxiety in speaking. These findings suggest that the teachers’ role is very important in this situation. Students and teachers could build a positive classroom atmosphere in order to overcome students’ anxiety.
Toward Real-World Super-Resolution Technique for Fringe Projection Profilometry

Deep learning technique has exhibited promising performance in achieving high-resolution (HR) image from their low-resolution (LR) image in super-resolution (SR) field. However, most of the existing SR methods have two underlying problems. Firstly, degraded datasets (i.e., bicubic downsampling) are usually used to train and evaluate network model, which maybe lead to less effective in practical scenarios. Secondly, 2D-to-3D SR technique is lacking. In this paper, a real-world 2D-to-3D technique is developed to realize SR 3D shape from 2D fringe images in fringe projection profilometry (FPP). A FPP system consisting of one projector and a dual-camera is applied to obtain the real-world dataset where paired LR-HR images on same scene are captured. The 3D geometrical constraints solved from FPP system are employed to align the image pairs by pixel-to-pixel mapping so that more accurate dataset can be obtained. In addition, a flexible multiple-to-two network structure is introduced to achieve SR 3D point cloud from multiple phase-shifting patterns. Experiments demonstrate the comparing between traditional degraded training and our training.
Modality-specific Learning Rates for Effective Multimodal Additive Late-fusion

In multimodal machine learning, additive latefusion is a straightforward approach to combine the feature representations from different modalities, in which the final prediction can be formulated as the sum of unimodal predictions.While it has been found that certain late-fusion models can achieve competitive performance with lower computational costs compared to complex multimodal interactive models, how to effectively search for a good late-fusion model is still an open question.Moreover, for different modalities, the best unimodal models may work under significantly different learning rates due to the nature of the modality and the computational flow of the model; thus, selecting a global learning rate for late-fusion models can result in a vanishing gradient for some modalities.To help address these issues, we propose a Modality-Specific Learning Rate (MSLR) method to effectively build late-fusion multimodal models from fine-tuned unimodal models.We investigate three different strategies to assign learning rates to different modalities.Our experiments show that MSLR outperforms global learning rates on multiple tasks and settings, and enables the models to effectively learn each modality.
Breast Cancer Classification using Decision Tree Algorithms

Cancer is a major health issue that affects individuals all over the world.This disease has claimed the lives of many people, and will continue to do so in the future.Breast cancer has recently surpassed cervical cancer as the most frequent cancer among women in both industrialized and developing countries and it is now the second leading cause of cancer mortality among women.A high number of women die each year as a result of this disease.Breast cancer is significantly easier to treat if caught early.This paper introduces a decision tree-based data mining technique for breast cancer early detection with highest accuracy, which helps patients to recover.Breast cancers are classed as benign (unable to penetrate surrounding tissue) or malignant (able to infiltrate adjacent tissue) breast growths.Two tests were included in the review.The primary study uses 10 breast cancer samples from the Kaggle archive, whereas the follow-up study uses 286 breast cancer samples from the same pool.The Decision Tree's accuracy in the first trial was 100%, while it was 97.9% in the follow-up inquiry.These findings justify the use of the proposed machine learning-based Decision Tree classifier in pre-evaluating patients for triage and decision-making prior to the availability of data.
The YiTrans Speech Translation System for IWSLT 2022 Offline Shared Task

This paper describes the submission of our end-to-end YiTrans speech translation system for the IWSLT 2022 offline task, which translates from English audio to German, Chinese, and Japanese.The YiTrans system is built on large-scale pre-trained encoder-decoder models.More specifically, we first design a multistage pre-training strategy to build a multimodality model with a large amount of labeled and unlabeled data.We then fine-tune the corresponding components of the model for the downstream speech translation tasks.Moreover, we make various efforts to improve performance, such as data filtering, data augmentation, speech segmentation, model ensemble, and so on.Experimental results show that our YiTrans system obtains a significant improvement than the strong baseline on three translation directions, and it achieves +5.2 BLEU improvements over last year's optimal end-toend system on tst2021 English-German.