Reviewerly/exHarmony huggingface dataset:
Edge-competing Pathological Liver Vessel Segmentation with Limited Labels

The microvascular invasion (MVI) is a major prognostic factor in hepatocellular carcinoma, which is one of the malignant tumors with the highest mortality rate. The diagnosis of MVI needs discovering the vessels that contain hepatocellular carcinoma cells and counting their number in each vessel, which depends heavily on experiences of the doctor, is largely subjective and time-consuming. However, there is no algorithm as yet tailored for the MVI detection from pathological images. This paper collects the first pathological liver image dataset containing $522$ whole slide images with labels of vessels, MVI, and hepatocellular carcinoma grades. The first and essential step for the automatic diagnosis of MVI is the accurate segmentation of vessels. The unique characteristics of pathological liver images, such as super-large size, multi-scale vessel, and blurred vessel edges, make the accurate vessel segmentation challenging. Based on the collected dataset, we propose an Edge-competing Vessel Segmentation Network (EVS-Net), which contains a segmentation network and two edge segmentation discriminators. The segmentation network, combined with an edge-aware self-supervision mechanism, is devised to conduct vessel segmentation with limited labeled patches. Meanwhile, two discriminators are introduced to distinguish whether the segmented vessel and background contain residual features in an adversarial manner. In the training stage, two discriminators are devised to compete for the predicted position of edges. Exhaustive experiments demonstrate that, with only limited labeled patches, EVS-Net achieves a close performance of fully supervised methods, which provides a convenient tool for the pathological liver vessel segmentation. Code is publicly available at https://github.com/wang97zh/EVS-Net.
Tell Me, What Do You See?—Interpretable Classification of Wiring Harness Branches with Deep Neural Networks

In the context of the robotisation of industrial operations related to manipulating deformable linear objects, there is a need for sophisticated machine vision systems, which could classify the wiring harness branches and provide information on where to put them in the assembly process. However, industrial applications require the interpretability of the machine learning system predictions, as the user wants to know the underlying reason for the decision made by the system. We propose several different neural network architectures that are tested on our novel dataset to address this issue. We conducted various experiments to assess the influence of modality, data fusion type, and the impact of data augmentation and pretraining. The outcome of the network is evaluated in terms of the performance and is also equipped with saliency maps, which allow the user to gain in-depth insight into the classifier’s operation, including a way of explaining the responses of the deep neural network and making system predictions interpretable by humans.
Stylized Story Generation with Style-Guided Planning

Current storytelling systems focus more on generating stories with coherent plots regardless of the narration style, which is important for controllable text generation.Therefore, we propose a new task, stylized story generation, namely generating stories with specified style given a leading context.To tackle the problem, we propose a novel generation model that first plans the stylized keywords and then generates the whole story with the guidance of the keywords.Besides, we propose two automatic metrics to evaluate the consistency between the generated story and the specified style.Experiments demonstrates that our model can controllably generate emotion-driven or event-driven stories based on the ROCStories dataset (Mostafazadeh et al., 2016).Our study presents insights for stylized story generation in further research.
Augmented Reality Experience for Inaccessible Areas in Museums

In recent years, new innovations have been introduced into the cultural heritage sector, aimed at offering more engaging and accessible tours to the public.The article discusses the development process of the VirgilTell project, for the visit of inaccessible places in the Racconigi Castle, one of the Savoy residences in Piedmont (Italy), by the UXD Team of Politecnico di Torino.The places assigned to the project are being restored and therefore excluded from the tour.The VirgilTell experience aims to include and make them accessible to visitors, through a virtual tour including multimedia content for the entertainment and involvement of the users.The visit has been realised through mixed techniques between VR and AR for the narration of the spaces in which the user takes part in a journey made of narratives by characters from the past and objects that no longer exist.The VirgilTell visit was also created with the aim of anticipating, in promotional terms, the physical visit to the museum spaces undergoing restoration, so as to attract visitors back inside once the work is finished.Design for heritage.Augmented reality.
Capturing Relations between Scientific Papers: An Abstractive Model for Related Work Section Generation

Given a set of related publications, related work section generation aims to provide researchers with an overview of the specific research area by summarizing these works and introducing them in a logical order.Most of existing related work section generation models follow the inflexible extractive style, which directly extract sentences from multiple original papers to form a related work discussion.Hence, in this paper, we propose a Relationaware Related work Generator (RRG), which generates an abstractive related work section from multiple scientific papers in the same research area.Concretely, we propose a relationaware multi-document encoder that relates one document to another according to their content dependency in a relation graph.The relation graph and the document representation interact and are refined iteratively, complementing each other in the training process.We also contribute two public datasets composed of related work sections and their corresponding papers 1 .Extensive experiments on the two datasets show that the proposed model brings substantial improvements over several strong baselines.We hope that this work will promote advances in related work section generation task.
Learning to Solve NLP Tasks in an Incremental Number of Languages

In real scenarios, a multilingual model trained to solve NLP tasks on a set of languages can be required to support new languages over time.Unfortunately, the straightforward retraining on a dataset containing annotated examples for all the languages is both expensive and time-consuming, especially when the number of considered languages grows.Moreover, the original annotated material may no longer be available due to storage or business constraints.Re-training only with the new language data will inevitably result in Catastrophic Forgetting of previously acquired knowledge.We propose a Continual Learning strategy that updates a model to support new languages over time, while maintaining consistent results on previously learned languages.We define a Teacher-Student framework where the existing model "teaches" to a student model its knowledge about the languages it supports, while the student is also trained on a new language.We report an experimental evaluation in several tasks including Sentence Classification, Relational Learning and Sequence Labeling.
A Targeted Assessment of Incremental Processing in Neural Language Models and Humans

We present a targeted, scaled-up comparison of incremental processing in humans and neural language models by collecting by-word reaction time data for sixteen different syntactic test suites across a range of structural phenomena.Human reaction time data comes from a novel online experimental paradigm called the Interpolated Maze task.We compare human reaction times to by-word probabilities for four contemporary language models, with different architectures and trained on a range of data set sizes.We find that across many phenomena, both humans and language models show increased processing difficulty in ungrammatical sentence regions with human and model 'accuracy' scores (à la Marvin and Linzen (2018)) about equal.However, although language model outputs match humans in direction, we show that models systematically under-predict the difference in magnitude of incremental processing difficulty between grammatical and ungrammatical sentences.Specifically, when models encounter syntactic violations they fail to accurately predict the longer reaction times observed in the human data.These results call into question whether contemporary language models are approaching human-like performance for sensitivity to syntactic violations.1. S d (was impressed) < Sc(was impressed) 2. S d (was impressed) < S b (was impressed) 3. (S d (was impressed) -Sc(was impressed)) < (S b (was impressed) -Sa(was impressed))
Capturing Event Argument Interaction via A Bi-Directional Entity-Level Recurrent Decoder

Capturing interactions among event arguments is an essential step towards robust event argument extraction (EAE).However, existing efforts in this direction suffer from two limitations: 1) The argument role type information of contextual entities is mainly utilized as training signals, ignoring the potential merits of directly adopting it as semantically rich input features; 2) The argument-level sequential semantics, which implies the overall distribution pattern of argument roles over an event mention, is not well characterized.To tackle the above two bottlenecks, we formalize EAE as a Seq2Seq-like learning problem for the first time, where a sentence with a specific event trigger is mapped to a sequence of event argument roles.A neural architecture with a novel Bi-directional Entity-level Recurrent Decoder (BERD) is proposed to generate argument roles by incorporating contextual entities' argument role predictions, like a wordby-word text generation process, thereby distinguishing implicit argument distribution patterns within an event more accurately.
CitationIE: Leveraging the Citation Graph for Scientific Information Extraction

Automatically extracting key information from scientific documents has the potential to help scientists work more efficiently and accelerate the pace of scientific progress.Prior work has considered extracting documentlevel entity clusters and relations end-to-end from raw scientific text, which can improve literature search and help identify methods and materials for a given problem.Despite the importance of this task, most existing works on scientific information extraction (SciIE) consider extraction solely based on the content of an individual paper, without considering the paper's place in the broader literature.In contrast to prior work, we augment our text representations by leveraging a complementary source of document context: the citation graph of referential links between citing and cited papers.On a test set of English-language scientific documents, we show that simple ways of utilizing the structure and content of the citation graph can each lead to significant gains in different scientific information extraction tasks.When these tasks are combined, we observe a sizable improvement in end-to-end information extraction over the state-of-the-art, suggesting the potential for future work along this direction.We release software tools to facilitate citation-aware SciIE development.
An Online Network Intrusion Detection Model Based on Improved Regularized Extreme Learning Machine

Extreme learning machine (ELM) is a novel single-hidden layer feedforward neural network to obtain fast learning speed by randomly initializing weights and deviations. Due to its extremely fast learning speed, it has been widely used in training of massive data in recent years. In order to adapt to the real network environment, based on the ELM, we propose an improved particle swarm optimized online regularized extreme learning machine (IPSO-IRELM) intrusion detection algorithm model. First, the model replaces the traditional batch learning with sequential learning by dynamically adapting the new data obtained in the training network instead of training all collected samples in an offline manner; second, we improve the particle swarm optimization algorithm and compare it with typical improved algorithms to prove its effectiveness; finally, to solve the random initialization problem of IRELM, we use IPSO to optimize the initial weights and deviations of IRELM to improve the classification ability of IRELM. The experimental results show that IPSO-IRELM algorithm has better generalization ability, which not only improves the accuracy of intrusion detection, but also has certain recognition ability for minority class samples.
RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English)

Most words are ambiguous-they convey distinct meanings in different contexts-and even the meanings of unambiguous words are context-dependent.Both phenomena present a challenge for NLP.Recently, the advent of contextualized word embeddings has led to success on tasks involving lexical ambiguity, such as Word Sense Disambiguation.However, there are few tasks that directly evaluate how well these embeddings accommodate the continuous, dynamic nature of word meaningparticularly in a way that matches human intuitions.We introduce RAW-C, a dataset of graded, human relatedness judgments for 112 ambiguous words in context (with 672 sentence pairs total), as well as human estimates of sense dominance.The average inter-annotator agreement for the relatedness norms (assessed using a leave-one-annotatorout method) was 0.79.We then show that a measure of cosine distance, computed using contextualized embeddings from BERT and ELMo, correlates with human judgments, but that cosine distance also systematically underestimates how similar humans find uses of the same sense of a word to be, and systematically overestimates how similar humans find uses of different-sense homonyms.Finally, we propose a synthesis between psycholinguistic theories of the mental lexicon and computational models of lexical semantics.
Audio-Oriented Multimodal Machine Comprehension via Dynamic Inter- and Intra-modality Attention

While Machine Comprehension (MC) has attracted extensive research interests in recent years, existing approaches mainly belong to the category of Machine Reading Comprehension task which mines textual inputs (paragraphs and questions) to predict the answers (choices or text spans). However, there are a lot of MC tasks that accept audio input in addition to the textual input, e.g. English listening comprehension test. In this paper, we target the problem of Audio-Oriented Multimodal Machine Comprehension, and its goal is to answer questions based on the given audio and textual information. To solve this problem, we propose a Dynamic Inter- and Intra-modality Attention (DIIA) model to effectively fuse the two modalities (audio and textual). DIIA can work as an independent component and thus be easily integrated into existing MC models. Moreover, we further develop a Multimodal Knowledge Distillation (MKD) module to enable our multimodal MC model to accurately predict the answers based only on either the text or the audio. As a result, the proposed approach can handle various tasks including: Audio-Oriented Multimodal Machine Comprehension, Machine Reading Comprehension and Machine Listening Comprehension, in a single model, making fair comparisons possible between our model and the existing unimodal MC models. Experimental results and analysis prove the effectiveness of the proposed approaches. First, the proposed DIIA boosts the baseline models by up to 21.08% in terms of accuracy; Second, under the unimodal scenarios, the MKD module allows our multimodal MC model to significantly outperform the unimodal models by up to 18.87%, which are trained and tested with only audio or textual data.
Controllable Image Restoration for Under-Display Camera in Smartphones

Under-display camera (UDC) technology is essential for full-screen display in smartphones and is achieved by removing the concept of drilling holes on display. However, this causes inevitable image degradation in the form of spatially variant blur and noise because of the opaque display in front of the camera. To address spatially variant blur and noise in UDC images, we propose a novel controllable image restoration algorithm utilizing pixel-wise UDC-specific kernel representation and a noise estimator. The kernel representation is derived from an elaborate optical model that reflects the effect of both normal and oblique light incidence. Also, noise-adaptive learning is introduced to control noise levels, which can be utilized to provide optimal results depending on the user preferences. The experiments showed that the proposed method achieved superior quantitative performance as well as higher perceptual quality on both a real-world dataset and a monitor-based aligned dataset compared to conventional image restoration algorithms.
Presentation of Novel Hybrid Algorithm for Detection and Classification of Breast Cancer Using Growth Region Method and Probabilistic Neural Network

Mammography is a significant screening test for early detection of breast cancer, which increases the patient’s chances of complete recovery. In this paper, a clustering method is presented for the detection of breast cancer tumor locations and areas. To implement the clustering method, we used the growth region approach. This method detects similar pixels nearby. To find the best initial point for detection, it is essential to remove human interaction in clustering. Therefore, in this paper, the FCM-GA algorithm is used to find the best point for starting growth. Their results are compared with the manual selection method and Gaussian Mixture Model method for verification. The classification is performed to diagnose breast cancer type in two primary datasets of MIAS and BI-RADS using features of GLCM and probabilistic neural network (PNN). Results of clustering show that the presented FCM-GA method outperforms other methods. Moreover, the accuracy of the clustering method for FCM-GA is 94%, as the best approach used in this paper. Furthermore, the result shows that the PNN methods have high accuracy and sensitivity with the MIAS dataset.
Exploiting Behavioral Consistence for Universal User Representation

User modeling is critical for developing personalized services in industry. A common way for user modeling is to learn user representations that can be distinguished by their interests or preferences. In this work, we focus on developing universal user representation model. The obtained universal representations are expected to contain rich information, and be applicable to various downstream applications without further modifications (e.g., user preference prediction and user profiling). Accordingly, we can be free from the heavy work of training task-specific models for every downstream task as in previous works. In specific, we propose Self-supervised User Modeling Network (SUMN) to encode behavior data into the universal representation. It includes two key components. The first one is a new learning objective, which guides the model to fully identify and preserve valuable user information under a self-supervised learning framework. The other one is a multi-hop aggregation layer, which benefits the model capacity in aggregating diverse behaviors. Extensive experiments on benchmark datasets show that our approach can outperform state-of-the-art unsupervised representation methods, and even compete with supervised ones.
Converting Foreign Accent Speech Without a Reference

Foreign accent conversion (FAC) is the problem of generating a synthetic voice that has the voice identity of a second-language (L2) learner and the pronunciation patterns of a native (L1) speaker. This synthetic voice has been referred to as a “golden-speaker” in the pronunciation-training literature. FAC is generally achieved by building a voice-conversion model that maps utterances from a source (L1) speaker onto the target (L2) speaker. As such, FAC requires that a reference utterance from the L1 speaker be available at synthesis time. This greatly restricts the application scope of the FAC system. In this work, we propose a “reference-free” FAC system that eliminates the need for reference L1 utterances at synthesis time, and transforms L2 utterances directly. The system is trained in two steps. First, a conventional FAC procedure is used to create a golden-speaker using utterances from a reference L1 speaker (which are then discarded) and the L2 speaker. Second, a pronunciation-correction model is trained to convert L2 utterances to match the golden-speaker utterances obtained in the first step. At synthesis time, the pronunciation-correction model directly transforms a novel L2 utterance into its golden-speaker counterpart. Our results show that the system reduces foreign accents in novel L2 utterances, achieving a 20.5% relative reduction in word-error-rate of an American English automatic speech recognizer and a 19% reduction in perceptual ratings of foreign accentedness obtained through listening tests. Over 73% of the listeners also rated golden-speaker utterances as having the same voice identity as the original L2 utterances.
3D reconstruction of structured light fields based on point cloud adaptive repair for highly reflective surfaces

In this paper, a novel method, to the best of our knowledge, of structured light fields based on point cloud adaptive repair is proposed to realize 3D reconstruction for highly reflective surfaces. We have designed and built a focused light field camera whose spatial and angular resolution can be flexibly adjusted as required. Then the subaperture image extraction algorithm based on image mosaic is deduced and presented to obtain multidirectional images. After that, the 3D reconstruction of structured light field imaging based on point cloud adaptive repair is presented to accurately reconstruct for highly reflective surfaces. In addition, a method based on smoothness and repair rate is also proposed to objectively evaluate the performance of the 3D reconstruction. Experimental results demonstrate the validity of the proposed method to perform high-quality depth reconstruction for highly reflective surfaces. Generally, our method takes advantage of the multidirectional imaging of the light field camera and can ensure good modulation effect of structured light while avoiding hardware complexity, which makes it application more convenient.
AIoT-Enabled Rehabilitation Recognition System—Exemplified by Hybrid Lower-Limb Exercises

Ubiquitous health management (UHM) is vital in the aging society. The UHM services with artificial intelligence of things (AIoT) can assist home-isolated healthcare in tracking rehabilitation exercises for clinical diagnosis. This study combined a personalized rehabilitation recognition (PRR) system with the AIoT for the UHM of lower-limb rehabilitation exercises. The three-tier infrastructure integrated the recognition pattern bank with the sensor, network, and application layers. The wearable sensor collected and uploaded the rehab data to the network layer for AI-based modeling, including the data preprocessing, featuring, machine learning (ML), and evaluation, to build the recognition pattern. We employed the SVM and ANFIS methods in the ML process to evaluate 63 features in the time and frequency domains for multiclass recognition. The Hilbert-Huang transform (HHT) process was applied to derive the frequency-domain features. As a result, the patterns combining the time- and frequency-domain features, such as relative motion angles in y- and z-axis, and the HHT-based frequency and energy, could achieve successful recognition. Finally, the suggestive patterns stored in the AIoT-PRR system enabled the ML models for intelligent computation. The PRR system can incorporate the proposed modeling with the UHM service to track the rehabilitation program in the future.