Reviewerly/exHarmony huggingface dataset:
Visual Parameter Space Exploration in Time and Space

Abstract Computational models, such as simulations, are central to a wide range of fields in science and industry. Those models take input parameters and produce some output. To fully exploit their utility, relations between parameters and outputs must be understood. These include, for example, which parameter setting produces the best result (optimization) or which ranges of parameter settings produce a wide variety of results (sensitivity). Such tasks are often difficult to achieve for various reasons, for example, the size of the parameter space, and supported with visual analytics. In this paper, we survey visual parameter space exploration (VPSE) systems involving spatial and temporal data. We focus on interactive visualizations and user interfaces. Through thematic analysis of the surveyed papers, we identify common workflow steps and approaches to support them. We also identify topics for future work that will help enable VPSE on a greater variety of computational models.
Parameterized Algebraic Protocols

We propose algebraic protocols that enable the definition of protocol templates and session types analogous to the definition of domain-specific types with algebraic datatypes. Parameterized algebraic protocols subsume all regular as well as most context-free and nested session types and, at the same time, replace the expensive superlinear algorithms for type checking by a nominal check that runs in linear time. Algebraic protocols in combination with polymorphism increase expressiveness and modularity by facilitating new ways of parameterizing and composing session types.
FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement

With the increasing data volume, there is a trend of using large-scale pre-trained models to store the knowledge into an enormous number of model parameters. The training of these models is composed of lots of dense algebras, requiring a huge amount of hardware resources. Recently, sparsely-gated Mixture-of-Experts (MoEs) are becoming more popular and have demonstrated impressive pretraining scalability in various downstream tasks. However, such a sparse conditional computation may not be effective as expected in practical systems due to the routing imbalance and fluctuation problems. Generally, MoEs are becoming a new data analytics paradigm in the data life cycle and suffering from unique challenges at scales, complexities, and granularities never before possible. In this paper, we propose a novel DNN training framework, FlexMoE, which systematically and transparently address the inefficiency caused by dynamic dataflow. We first present an empirical analysis on the problems and opportunities of training MoE models, which motivates us to overcome the routing imbalance and fluctuation problems by a dynamic expert management and device placement mechanism. Then we introduce a novel scheduling module over the existing DNN runtime to monitor the data flow, make the scheduling plans, and dynamically adjust the model-to-hardware mapping guided by the real-time data traffic. A simple but efficient heuristic algorithm is exploited to dynamically optimize the device placement during training. We have conducted experiments on both NLP models (e.g., BERT and GPT) and vision models (e.g., Swin). And results show FlexMoE can achieve superior performance compared with existing systems on real-world workloads -- FlexMoE outperforms DeepSpeed by 1.70x on average and up to 2.10x, and outperforms FasterMoE by 1.30x on average and up to 1.45x.
RELIANT: Fair Knowledge Distillation for Graph Neural Networks

Graph Neural Networks (GNNs) have shown satisfying performance on various graph learning tasks. To achieve better fitting capability, most GNNs are with a large number of parameters, which makes these GNNs computationally expensive. Therefore, it is difficult to deploy them onto edge devices with scarce computational resources, e.g., mobile phones and wearable smart devices. Knowledge Distillation (KD) is a common solution to compress GNNs, where a light-weighted model (i.e., the student model) is encouraged to mimic the behavior of a computationally expensive GNN (i.e., the teacher GNN model). Nevertheless, most existing GNN-based KD methods lack fairness consideration. As a consequence, the student model usually inherits and even exaggerates the bias from the teacher GNN. To handle such a problem, we take initial steps towards fair knowledge distillation for GNNs. Specifically, we first formulate a novel problem of fair knowledge distillation for GNN-based teacher-student frameworks. Then we propose a principled framework named RELIANT to mitigate the bias exhibited by the student model. Notably the design of RELIANT is decoupled from any specific teacher and student model structures, and thus can be easily adapted to various GNN-based KD frameworks. We perform extensive experiments on multiple real-world datasets, which corroborates that RELIANT achieves less biased GNN knowledge distillation while maintaining high prediction utility. Open-source code can be found at https://github.com/yushundong/RELIANT.KeywordsGraph Neural NetworksAlgorithmic FairnessKnowledge Distillation
Reparameterizable Multibranch Bottleneck Network for Lightweight Image Super-Resolution

Deployment of deep convolutional neural networks (CNNs) in single image super-resolution (SISR) for edge computing devices is mainly hampered by the huge computational cost. In this work, we propose a lightweight image super-resolution (SR) network based on a reparameterizable multibranch bottleneck module (RMBM). In the training phase, RMBM efficiently extracts high-frequency information by utilizing multibranch structures, including bottleneck residual block (BRB), inverted bottleneck residual block (IBRB), and expand–squeeze convolution block (ESB). In the inference phase, the multibranch structures can be combined into a single 3 × 3 convolution to reduce the number of parameters without incurring any additional computational cost. Furthermore, a novel peak-structure-edge (PSE) loss is proposed to resolve the problem of oversmoothed reconstructed images while significantly improving image structure similarity. Finally, we optimize and deploy the algorithm on the edge devices equipped with the rockchip neural processor unit (RKNPU) to achieve real-time SR reconstruction. Extensive experiments on natural image datasets and remote sensing image datasets show that our network outperforms advanced lightweight SR networks regarding objective evaluation metrics and subjective vision quality. The reconstruction results demonstrate that the proposed network can achieve higher SR performance with a 98.1 K model size, which can be effectively deployed to edge computing devices.
CAB: Empathetic Dialogue Generation with Cognition, Affection and Behavior

Empathy is an important characteristic to be considered when building a more intelligent and humanized dialogue agent. However, existing methods did not fully comprehend empathy as a complex process involving three aspects: cognition, affection and behavior. In this paper, we propose CAB, a novel framework that takes a comprehensive perspective of cognition, affection and behavior to generate empathetic responses. For cognition, we build paths between critical keywords in the dialogue by leveraging external knowledge. This is because keywords in a dialogue are the core of sentences. Building the logic relationship between keywords, which is overlooked by the majority of existing works, can improve the understanding of keywords and contextual logic, thus enhance the cognitive ability. For affection, we capture the emotional dependencies with dual latent variables that contain both interlocutors’ emotions. The reason is that considering both interlocutors’ emotions simultaneously helps to learn the emotional dependencies. For behavior, we use appropriate dialogue acts to guide the dialogue generation to enhance the empathy expression. Extensive experiments demonstrate that our multi-perspective model outperforms the state-of-the-art models.
“The Kids Nobel” Project as the platform for scientific knowledge popularization at modern scientific libraries

The National Library Association “Libraries of the Future” (NALF) supported through the RF Presidential Grant for Civil Society Development approved by the Presidential Grants Foundation, has been implementing the All-Russian Scientific and Educational Project “The Kids Nobel”. The Project main goals are to promote scientific knowledge and to engage 7–8 grade students in urban and rural settlements in research. At this stage, four pilot RF subjects are selected, namely Arkhangelsk, Kaluga, Ryazan and Saratov regions. Russian National Public Library for Science and Technology, today the country’s largest sci-tech library, and a NALF founder, is the main partner of “The Kids Nobel”. The All-Russian Scientific Educational Project “The Kids Nobel” is accomplished in two vectors: educational webinars for children in the main fields of the Nobel Prize, i. e. physics, chemistry, physiology, medicine, literature, politics (Nobel Peace Prize). Outstanding scientists, pedagogues, public officials, librarians deliver the lectures. The lecture course is followed by preparing and presenting students’ research works. The regional project curators, i. e. children’s school, universal, etc., libraries inform students on the events or use the course and workshop materials to organize their own events. The second vector is the contest of kids’ research projects in the fields of Nobel Prize. Under supervision of library curators, contest participants are to present their research work and findings through the Project website before July 1, 2023, one research work for a field, though several fields can be selected.
Fast reconstruction of programmable integrated interferometers

Programmable linear optical interferometers are important for classical and quantum information technologies, as well as for building hardware-accelerated artificial neural networks. Recent results showed the possibility of constructing optical interferometers that could implement arbitrary transformations of input fields even in the case of high manufacturing errors. The building of detailed models of such devices drastically increases the efficiency of their practical use. The integral design of interferometers complicates its reconstruction since the internal elements are hard to address. This problem can be approached by using optimization algorithms [Opt. Express 29, 38429 (2021)]. In this paper, we present a novel efficient algorithm based on linear algebra only, which does not use computationally expensive optimization procedures. We show that this approach makes it possible to perform fast and accurate characterization of high-dimensional programmable integrated interferometers. Moreover, the method provides access to the physical characteristics of individual interferometer layers.
Does IT Help? Information Technology in Banking and Entrepreneurship

We provide novel evidence on the importance of information technology (IT) in banking for entrepreneurship. We establish that job creation by young firms is stronger in US counties more exposed to IT-intensive banks. We present evidence consistent with banks' IT adoption facilitating entrepreneurship by improving banks' use of hard information, in particular collateral. At the county level, entrepreneurship increases by more in IT-exposed counties when house prices rise. Bank-level regressions confirm that banks' IT makes credit supply more responsive to changes in house prices, and that it reduces the importance of geographical distance between borrowers and lenders.
Context-Aware Coherent Speaking Style Prediction with Hierarchical Transformers for Audiobook Speech Synthesis

Recent advances in text-to-speech have significantly improved the expressiveness of synthesized speech. However, it is still challenging to generate speech with contextually appropriate and coherent speaking style for multi-sentence text in audiobooks. In this paper, we propose a context-aware coherent speaking style prediction method for audiobook speech synthesis. To predict the style embedding of the current utterance, a hierarchical transformer-based context-aware style predictor with a mixture attention mask is designed, considering both text-side context information and speech- side style information of previous speeches. Based on this, we can generate long-form speech with coherent style and prosody sentence by sentence. Objective and subjective evaluations on a Mandarin audiobook dataset demonstrate that our proposed model can generate speech with more expressive and coherent speaking style than baselines, for both single-sentence and multi-sentence test <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Characterization of Memory Access in Deep Learning and Its Implications in Memory Management

Due to the recent trend of software intelligence in the Fourth Industrial Revolution, deep learning has become a mainstream workload for modern computer systems. Since the data size of deep learning increasingly grows, managing the limited memory capacity efficiently for deep learning workloads becomes important. In this paper, we analyze memory accesses in deep learning workloads and find out some unique characteristics differentiated from traditional workloads. First, when comparing instruction and data accesses, data access accounts for 96%–99% of total memory accesses in deep learning workloads, which is quite different from traditional workloads. Second, when comparing read and write accesses, write access dominates, accounting for 64%–80% of total memory accesses. Third, although write access makes up the majority of memory accesses, it shows a low access bias of 0.3 in the Zipf parameter. Fourth, in predicting re-access, recency is important in read access, but frequency provides more accurate information in write access. Based on these observations, we introduce a Non-Volatile Random Access Memory (NVRAM)-accelerated memory architecture for deep learning workloads, and present a new memory management policy for this architecture. By considering the memory access characteristics of deep learning workloads, the proposed policy improves memory performance by 64.3% on average compared to the CLOCK policy.
Four-directional controlled quantum teleportation by using a nine-qubit entangled state as the quantum channel

In this paper, we propose a four-directional controlled quantum teleportation (CQT) by using a nine-qubit entangled state as the quantum channel. In 2020, a scheme for four-directional controlled quantum teleportation by using a 10-qubit state as the quantum resource had been proposed in X. W. Zha and K. Li [Mod. Phys. Lett. B 34, 2050412 (2020)] in which the controller has no control over the two teleportation processes. In this study, it is also shown that two receivers cannot reconstruct the original state sent from the respective senders and hence the teleportation processes cannot be completed in the previous scheme. However, these problems have been resolved in our proposed scheme in which the controller has control over all the teleportation processes and none of the receivers can reconstruct the original state sent from the respective senders without the permission of the controller. Thus, our proposed scheme reduces the consumption of quantum resource and is more secure and successful as compared to the previous scheme.
Stereoscopic Vision Recalling Memory for Monocular 3D Object Detection

Monocular 3D object detection has drawn increasing attention in various human-related applications, such as autonomous vehicles, due to its cost-effective property. On the other hand, a monocular image alone inherently contains insufficient information to infer the 3D information. In this paper, we propose a new monocular 3D object detector that can recall the stereoscopic visual information about an object, given a left-view monocular image. Here, we devise a location embedding module to handle each object by being aware of its location. Next, given the object appearance of the left-view monocular image, we devise Monocular-to-Stereoscopic (M2S) memory that can recall the object appearance of the right-view and depth information. For this purpose, we introduce a stereoscopic vision memorizing loss that guides the M2S memory to store the stereoscopic visual information. Furthermore, we propose a binocular vision association loss to guide the M2S memory that can associate the information of the left-right view about the object when estimating the depth. As a result, our monocular 3D object detector with the M2S memory can effectively exploit the recalled stereoscopic visual information in the inference phase. The comprehensive experimental results on two public datasets, KITTI 3D Object Detection Benchmark and Waymo Open Dataset, demonstrate the effectiveness of the proposed method. We claim that our method is a step-forward method that follows the behaviors of humans that can recall the stereoscopic visual information even when one eye is closed.
Using Machine Learning to Predict the In-Hospital Mortality in Women with ST-Segment Elevation Myocardial Infarction

Background: Several studies have shown that women have a higher mortality rate than do men from ST-segment elevation myocardial infarction (STEMI).The present study was aimed at developing a new risk-prediction model for all-cause in-hospital mortality in women with STEMI, using predictors that can be obtained at the time of initial evaluation.Methods: We enrolled 8158 patients who were admitted with STEMI to the Tianjin Chest Hospital and divided them into two groups according to hospital outcomes.The patient data were randomly split into a training set (75%) and a testing set (25%), and the training set was preprocessed by adaptive synthetic (ADASYN) sampling.Four commonly used machine-learning (ML) algorithms were selected for the development of models; the models were optimized by 10-fold cross-validation and grid search.The performance of all-population-derived models and female-specific models in predicting in-hospital mortality in women with STEMI was compared by several metrics, including accuracy, specificity, sensitivity, G-mean, and area under the curve (AUC).Finally, the SHapley Additive exPlanations (SHAP) value was applied to explain the models.Results: The performance of models was significantly improved by ADASYN.In the overall population, the support vector machine (SVM) combined with ADASYN achieved the best performance.However, it performed poorly in women with STEMI.Conversely, the proposed female-specific models performed well in women with STEMI, and the best performing model achieved 72.25% accuracy, 82.14% sensitivity, 71.69% specificity, 76.74% G-mean and 79.26% AUC.The accuracy and G-mean of the female-specific model were greater than the all-population-derived model by 34.64% and 9.07%, respectively.Conclusions: A machine-learning-based female-specific model can conveniently and effectively identify high-risk female STEMI patients who often suffer from an incorrect or delayed management.
Split-Level Evolutionary Neural Architecture Search With Elite Weight Inheritance

Neural architecture search (NAS) has recently gained extensive interest in the deep learning community because of its great potential in automating the construction process of deep models. Among a variety of NAS approaches, evolutionary computation (EC) plays a pivotal role with its merit of gradient-free search ability. However, a massive number of the current EC-based NAS approaches evolve neural architectures in an absolutely discrete manner, which makes it tough to flexibly handle the number of filters for each layer, since they often reduce it to a limit set rather than searching for all possible values. Moreover, EC-based NAS methods are often criticized for their inefficiency in performance evaluation, which usually requires laborious full training for hundreds of candidate architectures generated. To address the inflexible search issue on the number of filters, this work proposes a split-level particle swarm optimization (PSO) approach. Each dimension of the particle is subdivided into an integer part and a fractional part, encoding the configurations of the corresponding layer, and the number of filters within a large range, respectively. In addition, the evaluation time is greatly saved by a novel elite weight inheritance method based on an online updating weight pool, and a customized fitness function considering multiple objectives is developed to well control the complexity of the searched candidate architectures. The proposed method, termed split-level evolutionary NAS (SLE-NAS), is computationally efficient, and outperforms many state-of-the-art peer competitors at much lower complexity across three popular image classification benchmark datasets.
Dynamic Multi-View Fusion Mechanism for Chinese Relation Extraction

Abstract Recently, many studies incorporate external knowledge into character-level feature based models to improve the performance of Chinese relation extraction. However, these methods tend to ignore the internal information of the Chinese character and cannot filter out the noisy information of external knowledge. To address these issues, we propose a mixture-of-view-experts framework (MoVE) to dynamically learn multi-view features for Chinese relation extraction. With both the internal and external knowledge of Chinese characters, our framework can better capture the semantic information of Chinese characters. To demonstrate the effectiveness of the proposed framework, we conduct extensive experiments on three real-world datasets in distinct domains. Experimental results show consistent and significant superiority and robustness of our proposed framework. Our code and dataset will be released at: https://gitee.com/tmg-nudt/multi-view-of-expert-for-chinese-relation-extraction
Methodology for generating synthetic labeled datasets for visual container inspection

Nowadays, containerized freight transport is one of the most important transportation systems that is undergoing an automation process due to the Deep Learning success. However, it suffers from a lack of annotated data in order to incorporate state-of-the-art neural network models to its systems. In this paper we present an innovative methodology to generate a realistic, varied, balanced, and labelled dataset for visual inspection task of containers in a dock environment. In addition, we validate this methodology with multiple visual tasks recurrently found in the state of the art. We prove that the generated synthetic labelled dataset allows to train a deep neural network that can be used in a real world scenario. On the other side, using this methodology we provide the first open synthetic labelled dataset called SeaFront available in: https://datasets.vicomtech.org/di21-seafront/readme.txt.