Miracle-dz/UniFAR-FTUs huggingface dataset:
Mixed Multi-Head Self-Attention for Neural Machine Translation

Recently, the Transformer becomes a state-of-the-art architecture in the filed of neural machine translation (NMT). A key point of its high-performance is the multi-head self-attention which is supposed to allow the model to independently attend to information from different representation subspaces. However, there is no explicit mechanism to ensure that different attention heads indeed capture different features, and in practice, redundancy has occurred in multiple heads. In this paper, we argue that using the same global attention in multiple heads limits multi-head self-attention’s capacity for learning distinct features. In order to improve the expressiveness of multi-head self-attention, we propose a novel Mixed Multi-Head Self-Attention (MMA) which models not only global and local attention but also forward and backward attention in different attention heads. This enables the model to learn distinct representations explicitly among multiple heads. In our experiments on both WAT17 English-Japanese as well as IWSLT14 German-English translation task, we show that, without increasing the number of parameters, our models yield consistent and significant improvements (0.9 BLEU scores on average) over the strong Transformer baseline.
II. EMPATHY IN HUMANS AND IN AGENTS

Empathy is seen as the capacity to perceive, understand and experience others' emotions. This notion is often considered as one of the major elements in social interactions between humans. As such, when creating social agents, that are believable and able to engage users in social interactions, empathy needs to be addressed. Indeed, for the past few years, many researchers have been looking at this problem, not only in trying to find ways to perceive the user's emotions, but also to adapt to them, and react in an empathic way. This paper provides a small overview of this new challenging area of research, by analyzing empathy in the social relations established between humans and social agents, and providing a concrete model for the creation of empathic social agents.
Supplementary Material for ”IL2M: Class Incremental Learning With Dual Memory”

VGGFace2 [2] is a recent dataset focused on face recognition. It includes over 9000 unique identities. We selected the 1000 identities which have the largest number of associated images for the evaluation in order to have a dataset similar in size to ILSVRC. VGGFace2 is well balanced and includes a mean of 491.7 images per class, with 49.4 standard deviation. The dataset includes loosely cropped face images and, following the usual face recognition pipeline, we extracted tighter crops before training and testing. Face detection was done using the publicly available MTCNN [10] framework. The dataset ILSVRC VGGFace2 Landmarks
Physiological Responses During Hybrid BNCI Control of an Upper-Limb Exoskeleton

When combined with assistive robotic devices, such as wearable robotics, brain/neural-computer interfaces (BNCI) have the potential to restore the capabilities of handicapped people to carry out activities of daily living. To improve applicability of such systems, workload and stress should be reduced to a minimal level. Here, we investigated the user’s physiological reactions during the exhaustive use of the interfaces of a hybrid control interface. Eleven BNCI-naive healthy volunteers participated in the experiments. All participants sat in a comfortable chair in front of a desk and wore a whole-arm exoskeleton as well as wearable devices for monitoring physiological, electroencephalographic (EEG) and electrooculographic (EoG) signals. The experimental protocol consisted of three phases: (i) Set-up, calibration and BNCI training; (ii) Familiarization phase; and (iii) Experimental phase during which each subject had to perform EEG and EoG tasks. After completing each task, the NASA-TLX questionnaire and self-assessment manikin (SAM) were completed by the user. We found significant differences (p-value < 0.05) in heart rate variability (HRV) and skin conductance level (SCL) between participants during the use of the two different biosignal modalities (EEG, EoG) of the BNCI. This indicates that EEG control is associated with a higher level of stress (associated with a decrease in HRV) and mental work load (associated with a higher level of SCL) when compared to EoG control. In addition, HRV and SCL modulations correlated with the subject’s workload perception and emotional responses assessed through NASA-TLX questionnaires and SAM.
Joint Semantic and Distributional Word Representations with Multi-Graph Embeddings

Word embeddings continue to be of great use for NLP researchers and practitioners due to their training speed and easiness of use and distribution. Prior work has shown that the representation of those words can be improved by the use of semantic knowledge-bases. In this paper we propose a novel way of combining those knowledge-bases while the lexical information of co-occurrences of words remains. It is conceptually clear, as it consists in mapping both distributional and semantic information into a multi-graph and modifying existing node embeddings techniques to compute word representations. Our experiments show improved results compared to vanilla word embeddings, retrofitting and concatenation techniques using the same information, on a variety of data-sets of word similarities.
Evolving Epidemiologic Trends of Renal Cell Cancer by Histologic Subtype: An Updated Analysis of the California Cancer Registry

Background: While most renal cell carcinomas (RCC) are of the clear cell subtype, other histologic subtypes are well described and have distinct clinical behavior. This study seeks to evaluate survival of clear and non-clear cell RCC retrospectively from a large, population-based cancer registry. Objectives: The key objectives of this study were to determine cancer-specific survival (CSS) and overall survival (OS) of RCC by histologic subtype and to examine survival by histologic subtype since the advent of anti-angiogenesis therapy in
Automatic and Robust Skull Registration Based on Discrete Uniformization

Skull registration plays a fundamental role in forensic science and is crucial for craniofacial reconstruction. The complicated topology, lack of anatomical features, and low quality reconstructed mesh make skull registration challenging. In this work, we propose an automatic skull registration method based on the discrete uniformization theory, which can handle complicated topologies and is robust to low quality meshes. We apply dynamic Yamabe flow to realize discrete uniformization, which modifies the mesh combinatorial structure during the flow and conformally maps the multiply connected skull surface onto a planar disk with circular holes. The 3D surfaces can be registered by matching their planar images using harmonic maps. This method is rigorous with theoretic guarantee, automatic without user intervention, and robust to low mesh quality. Our experimental results demonstrate the efficiency and efficacy of the method.
On choosing a model for measuring

This paper describes an approach to the issue of selecting a measurement model that is based on a comprehensive framework for measurement consisting of four conceptual building blocks: The construct map, the items design, the outcome space, and the measurement model. Starting from this framework of building blocks, the measurement model selected must conform to the constraints imposed by the other three components. Specifically, to preserve the interpretability of the construct map, the models must preserve the order of items throughout the range of person locations, and must do so in a way that is consistent with the interpretational requirements of the map. In the case of item response modeling, this translates into selecting models that have a constant slope—i.e., they must be from the Rasch family. In the conclusion, some next steps in investigating these issues are discussed.
Effect of moxibustion preconditioning on autophagy-related proteins in rats with myocardial ischemia reperfusion injury.

Background Autophagy has increasingly been recognized as playing an essential role in the pathogenesis of myocardial ischemia reperfusion injury (MIRI). Moxibustion, a form of heat therapy commonly used in traditional Chinese medicine (TCM), has been shown to exhibit cardioprotective effects. However, whether the cardioprotective effect of moxibustion is related to the regulation of autophagy remains unknown. This study aimed to investigate the possible mechanism underlying the cardioprotective effect of moxibustion preconditioning at PC6 on MIRI by measuring the expressions of proteins involved in the regulation of autophagy. Methods Male Sprague-Dawley rats were randomly divided to receive moxibustion preconditioning or autophagy inhibitor 3-Methyladenine (3-MA) intervention. Then the MIRI model was established by ligating the left anterior descending (LAD) coronary artery for 30 minutes followed by reperfusion for 4 hours. After 4 hours of reperfusion, the myocardial infarction area was assessed using Evans blue and TTC staining, and cTnT and lactate dehydrogenase (LDH) levels in the serum were determined by ELISA. Hematoxylin and eosin (H&E) staining was performed for morphological evaluation of ventricular tissues. Expressions of autophagy components Beclin 1, Bcl-2, and Akt were assessed using quantitative real-time PCR, immunohistochemistry (IHC) and western blot. Results Moxibustion preconditioning significantly reduced the necrotic area and the levels of cTnT and LDH were similar to the 3-MA intervention, also attenuated morphological alterations were induced by MIRI. Simultaneously, the mRNA and protein expressions of Beclin 1 and Akt were up-regulated, while those of Bcl-2 were down-regulated by MIRI. Moxibustion preconditioning and 3-MA intervention reversed MIRI-induced changes in Beclin 1, Akt, and Bcl-2 expressions. Conclusions Moxibustion preconditioning at PC6 can attenuate myocardial injury for MIRI in a similar way to 3-MA intervention. This cardioprotective effect of moxibustion preconditioning may be mediated by modulating autophagy via regulation of Beclin 1, Bcl-2 and Akt.
The phenomenon of creative economy in Ukraine and defining its place in the architectural space

The article shows the strong and weak sides of the current situation in the area of creative economy in Ukraine as well as develops the list of proposals for supporting and activating its potential. The innovative approach and research significance is presented at the intersection of economy and architecture in as much as the study also sheds light on the typology and architectural functional issues of innovative creative spaces and structures, which are the centers of creative economy flourishing. The authors’ research material includes analysis of existing creative spaces of Ukrainian cities as well as gives architectural proposals of future creative hub designed by author’s supervision. Keywords—creative economy, creativity, creative spaces, hubs, architectural organization, design concept.
Comparison of multi-objective optimization methods applied to 1 urban drainage adaptation problems 2

This article compares three multi-objective evolutionary algorithms (MOEAs) with 12 application to the urban drainage system (UDS) adaptation of a capital city in North China. 13 Particularly, the well-known NSGA-II, the built-in solver in the MATLAB Global Optimization 14 Toolbox (MLOT), and a newly-developed hybrid MOEA called GALAXY are considered. A 15 variety of parameter combinations of each MOEA are systemically applied to examine their 16 impacts on optimization efficiency. Results suggest that traditional MOEAs suffer from severe 17 parameterization issues. For NSGA-II, the distribution indices of crossover and mutation 18 operators were found to have dominant impacts, while the probabilities of the two operators 19 played a secondary role. For MLOT, the two-point and scattered crossover accompanying the 20 adaptive feasible mutation gained the best Pareto fronts, provided that the crossover fraction is 21 set to lower values. In contrast, GALAXY was the most robust and easy-to-use tool among the 22
Microblog User Interest Mining Based on Improved TextRank Model

As microblogs have become one of the most important social platforms, it is considered to be extremely valuable to extract user interests hidden behind microblogs. In this paper, we introduce a framework, which is built on the improved TextRank model, to analyze the personal interest of microblog users. In the framework, we first create a catalog of user interests basing on hot tags of Sina Weibo, the largest microblog system in China. And then TFIDF factor is used in TextRank model to deal with pre-processed microblog contents. After ranking and mapping all extracted words into user interests catalog established previously, we get corresponding user interests tags and a user interests model. Experimental results on Sina Weibo data imply that the proposed framework outperforms other existing methods.
Title Sampling-based adaptive bounding evolutionary algorithm for continuous optimization problems

This paper proposes a novel sampling-based adaptive bounding evolutionary algorithm termed SABEA that is capable of dynamically updating the search space during the evolution process for continuous optimization problems. The proposed SABEA adopts two bounding strategies, namely fitness-based bounding and probabilistic sampling-based bounding, to select a set of individuals over multiple generations and leverage the value information from these individuals to update the search space of a given problem to improve the solution accuracy and search efficiency. To evaluate the performance of this method, SABEA is applied on top of the classic differential evolution (DE) algorithm and a DE variant, and SABEA is compared to a state-of-the-art Distribution-based Adaptive Bounding Genetic Algorithm (DABGA) on a set of 27 selected benchmark functions. The results show that SABEA can be used as a complementary strategy for further enhancing the performance of existing EA algorithms and it also outperforms the DABGA method. Moreover, a practical problem, namely the model calibration for an agent-based simulation, is used to further evaluate SABEA. The results show SABEA’s applicability to diverse problems and its advantages over the traditional genetic algorithm-based calibration method and the DABGA method.
Unsupervised Aspect-Based Multi-Document Abstractive Summarization

User-generated reviews of products or services provide valuable information to customers. However, it is often impossible to read each of the potentially thousands of reviews: it would therefore save valuable time to provide short summaries of their contents. We address opinion summarization, a multi-document summarization task, with an unsupervised abstractive summarization neural system. Our system is based on (i) a language model that is meant to encode reviews to a vector space, and to generate fluent sentences from the same vector space (ii) a clustering step that groups together reviews about the same aspects and allows the system to generate summary sentences focused on these aspects. Our experiments on the Oposum dataset empirically show the importance of the clustering step.
Deep Tensor ADMM-Net for Snapshot Compressive Imaging

Snapshot compressive imaging (SCI) systems have been developed to capture high-dimensional ($\ge$ 3) signals using low-dimensional off-the-shelf sensors, \ie, mapping multiple video frames into a single measurement frame. One key module of a SCI system is an accurate decoder that recovers the original video frames. However, existing model-based decoding algorithms require exhaustive parameter tuning with prior knowledge and cannot support practical applications due to the extremely long running time. In this paper, we propose a deep tensor ADMM-Net for video SCI systems that provides high-quality decoding in seconds. Firstly, we start with a standard tensor ADMM algorithm, unfold its inference iterations into a layer-wise structure, and design a deep neural network based on tensor operations. Secondly, instead of relying on a pre-specified sparse representation domain, the network learns the domain of low-rank tensor through stochastic gradient descent. It is worth noting that the proposed deep tensor ADMM-Net has potentially mathematical interpretations. On public video data, the simulation results show the proposed {method} achieves average $0.8 \sim 2.5$ dB improvement in PSNR and $0.07 \sim 0.1$ in SSIM, and $1500\times \sim 3600 \times$ speedups over the state-of-the-art methods. On real data captured by SCI cameras, the experimental results show comparable visual results with the state-of-the-art methods but in much shorter running time.
Supplementary Material Canonical Surface Mapping via Geometric Cycle Consistency

We use a 5 Layer UNet [4] with 4 ×4 convolution at each layer. The UNet takes input an image and learns to predict an unit-vector which parameterizes u, v. Along with that we also train the UNet to predict a segmentation of the object which is necessary for keypoint evaluations. We train our networks for over 200 epochs on all the datasets independently. We use Adam [2] for optimization of our neural network with a learning rate of 10−4.
VOAR : a configurable web-based environment for visualizing and manipulating multiple ontology alignments

Ontology matching is an active research area aiming at providing solutions to the ontology heterogeneity problem. While most efforts in the field are dedicated to the development of matching approaches, less has been done in terms of alignment visualization. However, providing ways for visualizing alignments is required in many tasks involving users in the process (alignment evaluation, validation, comparison, repairing, etc.). This paper presents a Web-based environment for visualizing, editing and evaluating multiple ontology alignments. Users can configure their environment choosing different visualization modes and creating different profiles. They can apply a set of operations on the alignments (filtering, merge, etc.) and evaluate them against a reference one, using classical compliance metrics. Ontology and alignment libraries allow for users storing and searching them in the system.
Integrated Assessment of Greenhouse Gas Stabilization Concentrations, Emission Pathways, and Impact Threshold Values for Control of Global Warming

To mitigate the dangerous impacts of global warming to the greatest extent possible, various greenhouse gas stabilization scenarios have been proposed. Integrated studies are proceeding on emission pathways and the costs of achieving climate stabilization, as well as on the impact risks of global warming. This paper summarizes the existing knowledge on temperature rise, mitigation measures and impact threshold values, with a particular emphasis on climate stabilization scenarios. It also gives an outline of the AIM/Impact[Policy] policy support tool developed for integrated assessment of global warming, and reports on the results of analyses of global mean temperature increase, emission pathways, and global warming impacts under GHG concentration stabilization constraints.
BIROn - Birkbeck Institutional Research Online

To better understand the volcanic phenomena acting on Montserrat, the SEA-CALIPSO seismic experiment (Seismic Experiment with Airgun-source – Caribbean Andesitic Lava Island Precision Seismo-geodetic Observatory) was conducted in 2007 December with the aim of imaging the upper crust and the magmatic system feeding the active Soufri´ere Hills Volcano. The 3-D survey covered an area of about 50 × 40 km and involved the deployment of 247 land stations and ocean-bottom seismometers (OBSs). A subset of the data, recorded by four OBSs and four land stations on a southeast to northwest line, has been analysed, and traveltimes have been inverted to obtain a 2-D seismic velocity model through the island. Inverted phases include crustal and sediment P waves and wide-angle reﬂections. The resulting velocity model reveals the presence of a high velocity body (3.5–5.5 km s − 1 ) beneath the island, with highest velocities beneath the Soufri´ere and Centre Hills, corresponding primarily to the cores of these volcanic ediﬁces, built of a pile of andesite lava domes and subsequent intrusions. In the offshore region, velocities in the surﬁcial sediment layer vary from 1.5 to 3.0 km s − 1 , consistent with a mainly calcareous and volcaniclastic composition. A wide-angle reﬂector is observed at a depth of ∼ 1200 m below the seabed, and appears to deepen beneath the island. The upper crust beneath this reﬂector has velocities of 4.0–6.0 km s − 1 and is inferred to correspond to plutonic and hypabyssal rocks and sedimentary material of the old arc. The high velocity region beneath the island, extends into the crust to a depth of at least 5 km, and is believed to be caused by an intrusive complex, possibly of intermediate composition. A low velocity zone, as would be expected in the presence of an active magma chamber, was not observed perhaps due to the limited resolution beneath ∼ 5 km depth. Our results so far provide the ﬁrst wide-angle seismic constraints on the upper crustal structure of the island to a depth of 10 km, and will help understanding the processes that drive volcanism at Montserrat and other island arc volcanoes.
Spondylolisthesis Use of a Reverse Bohlman Technique for Low-Grade

Background: Treatment of spondylolisthesis can be difficult with regard to patients with high sacral slopes that may prohibit placement of interbody grafts for fusions across that segment. Here, we describe placement of a reverse Bohlman technique from an anterior approach to obtain fusion across a low-grade spondylolisthesis with a high sacral slope to obtain anterior fusion. Methods: A chart review was conducted on this single patient regarding his clinical course and outcome. Results: A 54-year-old male presented with low-back pain associated with bilateral leg pain dating back several years. Plain films demonstrated a Grade II isthmic spondylolisthesis at L5–S1 with spinopelvic measurements of 738 sacral slope, 828 lumbar lordosis, 128 pelvic tilt, and 948 pelvic incidence. Magnetic resonance imaging showed bilateral L5 pars defects with diffuse degenerative disease from L4 through S1 and significant ligamentous and facet hypertrophy. He underwent an L4–5 anterior lumbar interbody fusion and an L5–S1 reverse Bohlman placement of a transvertebral transsacral titanium mesh cage. This was supplemented with a posterior decompression and instrumentation from L4ilium. He had resolution of his radiculopathy and has maintained a good clinical outcome at 3 years follow up. Conclusions: We present here a patient with low-grade spondylolisthesis and a steep sacral slope who underwent a successful reverse Bohlman approach with long-term follow up. This report highlights the potential utility of this method as a viable alternative for patients with low-grade spondylolisthesis. Level of Evidence: IV. Clinical Relevance: Technical description of surgical technique.