Reviewerly/exHarmony huggingface dataset:
Learning De-biased Representations with Biased Representations

Many machine learning algorithms are trained and evaluated by splitting data from a single source into training and test sets. While such focus on in-distribution learning scenarios has led to interesting advancement, it has not been able to tell if models are relying on dataset biases as shortcuts for successful prediction (e.g., using snow cues for recognising snowmobiles), resulting in biased models that fail to generalise when the bias shifts to a different class. The cross-bias generalisation problem has been addressed by de-biasing training data through augmentation or re-sampling, which are often prohibitive due to the data collection cost (e.g., collecting images of a snowmobile on a desert) and the difficulty of quantifying or expressing biases in the first place. In this work, we propose a novel framework to train a de-biased representation by encouraging it to be different from a set of representations that are biased by design. This tactic is feasible in many scenarios where it is much easier to define a set of biased representations than to define and quantify bias. We demonstrate the efficacy of our method across a variety of synthetic and real-world biases; our experiments show that the method discourages models from taking bias shortcuts, resulting in improved generalisation. Source code is available at this https URL.
Dialogue State Tracking with Explicit Slot Connection Modeling

Recent proposed approaches have made promising progress in dialogue state tracking (DST). However, in multi-domain scenarios, ellipsis and reference are frequently adopted by users to express values that have been mentioned by slots from other domains. To handle these phenomena, we propose a Dialogue State Tracking with Slot Connections (DST-SC) model to explicitly consider slot correlations across different domains. Given a target slot, the slot connecting mechanism in DST-SC can infer its source slot and copy the source slot value directly, thus significantly reducing the difficulty of learning and reasoning. Experimental results verify the benefits of explicit slot connection modeling, and our model achieves state-of-the-art performance on MultiWOZ 2.0 and MultiWOZ 2.1 datasets.
Semi-supervised generative adversarial networks for the segmentation of the left ventricle in pediatric MRI

Segmentation of the left ventricle in magnetic resonance imaging (MRI) is important for assessing cardiac function. We present DT-GAN, a generative adversarial network (GAN) segmentation approach for the identification of the left ventricle in pediatric MRI. Segmentation of the left ventricle requires a large amount of annotated data; generating such data can be time-consuming and subject to observer variability. Additionally, it can be difficult to accomplish in a clinical setting. During the training of our GAN, we therefore introduce a semi-supervised semantic segmentation to reduce the number of images required for training, while maintaining a good segmentation accuracy. The GAN generator produces a segmentation label map and its discriminator outputs a confidence map, which gives the probability of a pixel coming from the label or from the generator. Moreover, we propose a new formulation of the GAN loss function based on distance transform and pixel-wise cross-entropy. This new loss function provides a better segmentation of boundary pixels, by favoring the correct classification of those pixels rather than focusing on pixels that are farther away from the boundary between anatomical structures. Our proposed method achieves a mean Hausdorff distance of 2.16 mm ± 0.42 mm (2.28 mm ± 0.21 mm for U-Net) and a Dice score of 0.88 ± 0.08 (0.91 ± 0.12 for U-Net) for the endocardium segmentation, using 50% of the annotated data. For the epicardium segmentation, we achieve a mean Hausdorff distance of 2.23 mm ± 0.35 mm (2.34 mm ± 0.39 mm for U-Net) and a Dice score of 0.93 mm ± 0.04 mm (0.89 ± 0.09 for U-Net). For the myocardium segmentation, we achieve a mean Hausdorff distance of 2.98 mm ± 0.43 mm (3.04 mm ± 0.27 mm for U-Net) and a Dice score of 0.79 mm ± 0.10 mm (0.74 ± 0.04 for U-Net). This new model could be very useful for the automatic analysis of cardiac MRI and for conducting large-scale studies based on MRI readings, with a limited amount of training data.
Badminton Activity Recognition Using Accelerometer Data

A thorough analysis of sports is becoming increasingly important during the training process of badminton players at both the recreational and professional level. Nowadays, game situations are usually filmed and reviewed afterwards in order to analyze the game situation, but these video set-ups tend to be difficult to analyze, expensive, and intrusive to set up. In contrast, we classified badminton movements using off-the-shelf accelerometer and gyroscope data. To this end, we organized a data capturing campaign and designed a novel neural network using different frame sizes as input. This paper shows that with only accelerometer data, our novel convolutional neural network is able to distinguish nine activities with 86% precision when using a sampling frequency of 50 Hz. Adding the gyroscope data causes an increase of up to 99% precision, as compared to, respectively, 79% and 88% when using a traditional convolutional neural network. In addition, our paper analyses the impact of different sensor placement options and discusses the impact of different sampling frequenciess of the sensors. As such, our approach provides a low cost solution that is easy to use and can collect useful information for the analysis of a badminton game.
Securing SDN-Controlled IoT Networks Through Edge Blockchain

The Internet of Things (IoT) connected by software-defined networking (SDN) promises to bring great benefits to cyber-physical systems. However, the increased attack surface offered by the growing number of connected vulnerable devices and separation of SDN control and data planes could overturn the huge benefits of such a system. This article addresses the vulnerability of the trust relationship between the control and data planes. To meet this aim, we propose an edge computing-based Blockchain as a Service (BaaS), enabled by an external BaaS provider. The proposed solution provides verification of inserted flows through an efficient, edge-distributed, blockchain solution. We study two scenarios for the blockchain reward purpose: 1) information symmetry, in which the SDN operator has direct knowledge of the real effort spent by the BaaS provider and 2) information asymmetry, in which the BaaS provider controls the exposure of information regarding spent effort. The latter yields the so-called “moral hazard,” where the BaaS may claim higher than actual effort. We develop a novel mathematical model of the edge BaaS solution and propose an innovative algorithm of a fair reward scheme based on game theory that takes into account moral hazard. We evaluate the viability of our solution through analytical simulations. The results demonstrate the ability of the proposed algorithm to maximize the joint profits of the BaaS and SDN operator, i.e., maximizing the social welfare.
Overshooting of sovereign emerging eurobond yields in the context of COVID-19

This paper illustrates the phenomenon of overshooting yields on eurobonds issued by emerging and developing countries in the context of COVID-19. Using panel data from 48 emerging and developing countries, the results show that daily reports of confirmed cases have led to increases in yields and announcements of international creditor assistance to developing and emerging countries, which have calmed investor concerns.
A Systematic Literature Review of Intelligent Tutoring Systems With Dialogue in Natural Language

Intelligent tutoring systems (ITSs) are computer programs that provide instruction adapted to the needs of individual students. Dialog systems are computer programs that communicate with human users by using natural language. This paper presents a systematic literature review to address ITSs that incorporate dialog systems and have been implemented in the last twenty years. The review found 33 ITSs and focused on answering the following five research questions. a) What ITSs with natural language dialogue have been developed? b) What is the main purpose of the tutoring dialogue in each system? c) What are the pedagogical features of the teaching process performed by the ITSs with natural language dialogue? d) What natural language understanding approach does each system employ to understand students’ utterances? e) What evidence exists related to the evaluation of ITSs with natural language dialogue? The results of this review reveal that most ITSs are directed toward science, technology, engineering, and mathematics (STEM) domains at the university level, and the majority of the selected ITSs implement the expectations and misconceptions tailored approach. Furthermore, most ITSs use dialog to help students learn how to solve a problem by applying rules, laws, etc. (the apply level in Bloom’s taxonomy). With regard to the instructional approach, the selected ITSs help students write correct explanations or answers for deep questions; assist students in problem solving; or support a reflective dialogue motivated by either previously provided content or the result of a simulation. Additionally, we found empirical evaluations for 90.91% of the selected ITSs that measure the learning gains and/or assess the impacts of different tutoring strategies.
Compensation strategies in speaking activities for non-English department students: poor and competent speakers

&#x0D; &#x0D; &#x0D; &#x0D; This study investigated the compensation strategies which were used by competent and poor speakers to avoid communication gap in speaking activities. This study used descriptive quantitative desaign. The use of descriptive method because of this study was not searching about compararison or making prediction, but only finding out about mostly strategies used either competent or poor speakers in developing their speaking activities to avoid communication gap. Descriptive research is also chosen because it explores and catches social situation particularly in the class wholisticly, widely and deeply. Data collection used an observation sheet and a questionnaire. Population of this study was 83 students categorized as competent and poor speakers of semester 4 at English Study Program in one of the universities in Kendari. The findings of this study showed that the strategy which dominant used by either competent speakers or poor speakers as their group tendencies that was selecting the topic with the same overall average score of 4.0; competent speakers mostly used selecting the topic with the overall average of 4.0, adjusting or approximating the message with the overall average of 3.6 and using mime or gesture with the overall average of 3.5; poor speakers also mostly used selecting the topic with the overall average of 4.0 and coining word of 3.5. Competent speakers much more used compensation strategies than poor speakers. Thus, its major implication for pedagogy is that compensation strategies are extremely useful as guidance to avoid communication gap in speaking activities&#x0D; &#x0D; &#x0D; &#x0D;
DyERNIE: Dynamic Evolution of Riemannian Manifold Embeddings for Temporal Knowledge Graph Completion

There has recently been increasing interest in learning representations of temporal knowledge graphs (KGs), which record the dynamic relationships between entities over time.Temporal KGs often exhibit multiple simultaneous non-Euclidean structures, such as hierarchical and cyclic structures.However, existing embedding approaches for temporal KGs typically learn entity representations and their dynamic evolution in the Euclidean space, which might not capture such intrinsic structures very well.To this end, we propose Dy-ERNIE, a non-Euclidean embedding approach that learns evolving entity representations in a product of Riemannian manifolds, where the composed spaces are estimated from the sectional curvatures of underlying data.Product manifolds enable our approach to better reflect a wide variety of geometric structures on temporal KGs.Besides, to capture the evolutionary dynamics of temporal KGs, we let the entity representations evolve according to a velocity vector defined in the tangent space at each timestamp.We analyze in detail the contribution of geometric spaces to representation learning of temporal KGs and evaluate our model on temporal knowledge graph completion tasks.Extensive experiments on three real-world datasets demonstrate significantly improved performance, indicating that the dynamics of multi-relational graph data can be more properly modeled by the evolution of embeddings on Riemannian manifolds.
On the Language Neutrality of Pre-trained Multilingual Representations

Multilingual contextual embeddings, such as multilingual BERT and XLM-RoBERTa, have proved useful for many multi-lingual tasks. Previous work probed the cross-linguality of the representations indirectly using zero-shot transfer learning on morphological and syntactic tasks. We instead investigate the language-neutrality of multilingual contextual embeddings directly and with respect to lexical semantics. Our results show that contextual embeddings are more language-neutral and, in general, more informative than aligned static word-type embeddings, which are explicitly trained for language neutrality. Contextual embeddings are still only moderately language-neutral by default, so we propose two simple methods for achieving stronger language neutrality: first, by unsupervised centering of the representation for each language and second, by fitting an explicit projection on small parallel data. Besides, we show how to reach state-of-the-art accuracy on language identification and match the performance of statistical methods for word alignment of parallel sentences without using parallel data.
The Origins and Prevalence of Texture Bias in Convolutional Neural Networks

Recent work has indicated that, unlike humans, ImageNet-trained CNNs tend to classify images by texture rather than by shape. How pervasive is this bias, and where does it come from? We find that, when trained on datasets of images with conflicting shape and texture, CNNs learn to classify by shape at least as easily as by texture. What factors, then, produce the texture bias in CNNs trained on ImageNet? Different unsupervised training objectives and different architectures have small but significant and largely independent effects on the level of texture bias. However, all objectives and architectures still lead to models that make texture-based classification decisions a majority of the time, even if shape information is decodable from their hidden representations. The effect of data augmentation is much larger. By taking less aggressive random crops at training time and applying simple, naturalistic augmentation (color distortion, noise, and blur), we train models that classify ambiguous images by shape a majority of the time, and outperform baselines on out-of-distribution test sets. Our results indicate that apparent differences in the way humans and ImageNet-trained CNNs process images may arise not primarily from differences in their internal workings, but from differences in the data that they see.
Boundary-sensitive Pre-training for Temporal Localization in Videos

Many video analysis tasks require temporal localization for the detection of content changes. However, most existing models developed for these tasks are pre-trained on general video action classification tasks. This is due to large scale annotation of temporal boundaries in untrimmed videos being expensive. Therefore, no suitable datasets exist that enable pre-training in a manner sensitive to temporal boundaries. In this paper for the first time, we investigate model pre-training for temporal localization by introducing a novel boundary-sensitive pretext (BSP) task. Instead of relying on costly manual annotations of temporal boundaries, we propose to synthesize temporal boundaries in existing video action classification datasets. By defining different ways of synthesizing boundaries, BSP can then be simply conducted in a self-supervised manner via the classification of the boundary types. This enables the learning of video representations that are much more transferable to downstream temporal localization tasks. Extensive experiments show that the proposed BSP is superior and complementary to the existing action classification-based pre-training counterpart, and achieves new state-of-the-art performance on several temporal localization tasks. Please visit our website for more details https://frostinassiky.github.io/bsp.
Acceleration for Compressed Gradient Descent in Distributed and  Federated Optimization

Due to the high communication cost in distributed and federated learning problems, methods relying on compression of communicated messages are becoming increasingly popular. While in other contexts the best performing gradient-type methods invariably rely on some form of acceleration/momentum to reduce the number of iterations, there are no methods which combine the benefits of both gradient compression and acceleration. In this paper, we remedy this situation and propose the first accelerated compressed gradient descent (ACGD) methods. In the single machine regime, we prove that ACGD enjoys the rate $O\Big((1+\omega)\sqrt{\frac{L}{\mu}}\log \frac{1}{\epsilon}\Big)$ for $\mu$-strongly convex problems and $O\Big((1+\omega)\sqrt{\frac{L}{\epsilon}}\Big)$ for convex problems, respectively, where $\omega$ is the compression parameter. Our results improve upon the existing non-accelerated rates $O\Big((1+\omega)\frac{L}{\mu}\log \frac{1}{\epsilon}\Big)$ and $O\Big((1+\omega)\frac{L}{\epsilon}\Big)$, respectively, and recover the optimal rates of accelerated gradient descent as a special case when no compression ($\omega=0$) is applied. We further propose a distributed variant of ACGD (called ADIANA) and prove the convergence rate $\widetilde{O}\Big(\omega+\sqrt{\frac{L}{\mu}}+\sqrt{\big(\frac{\omega}{n}+\sqrt{\frac{\omega}{n}}\big)\frac{\omega L}{\mu}}\Big)$, where $n$ is the number of devices/workers and $\widetilde{O}$ hides the logarithmic factor $\log \frac{1}{\epsilon}$. This improves upon the previous best result $\widetilde{O}\Big(\omega + \frac{L}{\mu}+\frac{\omega L}{n\mu} \Big)$ achieved by the DIANA method of Mishchenko et al. (2019). Finally, we conduct several experiments on real-world datasets which corroborate our theoretical results and confirm the practical superiority of our accelerated methods.
Intraoral image generation by progressive growing of generative adversarial network and evaluation of generated image quality by dentists

Abstract Dentists need experience with clinical cases to practice specialized skills. However, the need to protect patient's private information limits their ability to utilize intraoral images obtained from clinical cases. In this study, since generating realistic images could make it possible to utilize intraoral images, progressive growing of generative adversarial networks are used to generate intraoral images. A total of 35,254 intraoral images were used as training data with resolutions of 128 × 128, 256 × 256, 512 × 512, and 1024 × 1024. The results of the training datasets with and without data augmentation were compared. The Sliced Wasserstein Distance was calculated to evaluate the generated images. Next, 50 real images and 50 generated images for each resolution were randomly selected and shuffled. 12 pediatric dentists were asked to observe these images and assess whether they were real or generated. The d prime of the 1024 × 1024 images was significantly higher than that of the other resolutions. In conclusion, generated intraoral images with resolutions of 512 × 512 or lower were so realistic that the dentists could not distinguish whether they were real or generated. This implies that the generated images can be used in dental education or data augmentation for deep learning, without privacy restrictions.
Color image encryption based on joint permutation and diffusion

Image encryption plays an essential role in the community of image security. Most existing image encryption approaches adopt a permutation–diffusion scheme to permute pixel positions and change pixel values separately. One limitation of this scheme is that it has a high risk of being cracked. To solve this problem, we propose an approach that jointly permutates and diffuses (JPD) the pixels in a color image for encryption. First, a 4D hyperchaotic system with two positive Lyapunov exponents is used to generate a sequence for almost all encryption procedures. To enhance security, the plain image’s information is introduced to the hyperchaotic system’s initial parameters. Then, the hyperchaotic sequence is used to permute and diffuse the pixels in images jointly. More specifically, two index matrices that determine which pixels will be permuted and diffused and one mask matrix that determines how the pixels will be diffused are generated by the hyperchaotic sequence. We test the proposed JPD with several popular images. Experimental results and security analysis demonstrate that the JPD is capable of resisting various types of attacks. Moreover, the JPD can accelerate the encryption process. All these indicate that the JPD is effective and efficient for color image encryption.
A European Assessment of the Solar Energy Cost: Key Factors and Optimal Technology

Solar energy has become one of the most important sources of energy all around the world. Only in the European Union, between 2010 and 2019, solar photovoltaic (PV) electricity generation capacity increased from 1.9 to over 133 GW. Throughout this work, an economic analysis of the production of photovoltaic solar energy utility scale facilities is performed, previously defining some theoretical concepts relating to electricity generation by means of photovoltaic modules, as well as commenting on studies that have inspired the project. In order to carry out this economic analysis, the locations of twenty capital cities within European Union countries are selected, in order to estimate their yearly solar PV energy produced under specific conditions. The Levelized Costs of Energy (LCOE) is calculated with the goal of comparing the profitability of each photovoltaic tracking technology: fixed, one-axis tracking systems (vertical or inclined) and two-axis tracking systems; including LCOE maps country-wise for each technology. A sensitivity analysis is also presented, in order to evaluate the significance and impact of the main variables involved in the analysis. The results show that one-axis tracking systems are the best option in all countries, reducing LCOE by more than 20% when compared to two-axis tracking system. The impact of wages is also significant. In higher latitudes, in most cases, wages also increase, hence the LCOE is higher and consequently less interesting for a potential investor.
Dual Side Deep Context-aware Modulation for Social Recommendation

Social recommendation is effective in improving the recommendation performance by leveraging social relations from online social networking platforms. Social relations among users provide friends’ information for modeling users’ interest in candidate items and help items expose to potential consumers (i.e., item attraction). However, there are two issues haven’t been well-studied: Firstly, for the user interests, existing methods typically aggregate friends’ information contextualized on the candidate item only, and this shallow context-aware aggregation makes them suffer from the limited friends’ information. Secondly, for the item attraction, if the item’s past consumers are the friends of or have a similar consumption habit to the targeted user, the item may be more attractive to the targeted user, but most existing methods neglect the relation enhanced context-aware item attraction.