Reviewerly/exHarmony huggingface dataset:
Multi-node Bert-pretraining: Cost-efficient Approach

Recently, large scale Transformer-based language models such as BERT, GPT-2, and XLNet have brought about exciting leaps in state-of-the-art results for many Natural Language Processing (NLP) tasks. One of the common trends in these recent models is a significant increase in model complexity, which introduces both more weights and computation. Moreover, with the advent of large-scale unsupervised datasets, training time is further extended due to the increased amount of data samples within a single training epoch. As a result, to train these models within a reasonable time, machine learning (ML) programmers often require advanced hardware setups such as the premium GPU-enabled NVIDIA DGX workstations or specialized accelerators such as Google's TPU Pods. Our work addresses this limitation and demonstrates that the BERT pre-trained model can be trained within 2 weeks on an academic-size cluster of widely available GPUs through careful algorithmic and software optimizations. In this paper, we present these optimizations on how to improve single device training throughput, distribute the training workload over multiple nodes and GPUs, and overcome the communication bottleneck introduced by the large data exchanges over the network. We show that we are able to perform pre-training on BERT within a reasonable time budget (12 days) in an academic setting, but with a much less expensive and less aggressive hardware resource requirement than in previously demonstrated industrial settings based on NVIDIA DGX machines or Google's TPU Pods.
UD-based Linear Filtering for Discrete-Time Systems with Multiplicative and Additive Noises

This paper addresses the problem of constructing new linear filtering algorithms for discrete-time linear stochastic systems with multiplicative and additive noises, which are robust to machine roundoff errors. Two algorithms based on UDU <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</sup> decomposition of covariance and information matrices are proposed. The first of these algorithms is the UD-based covariance filter, and the second one is the UD-based information filter. Both of these filters have an extended array form, and their computational schemes allow for updating all required filter quantities with the use of the numerically stable MWGS orthogonalization procedure. The results of numerical experiments show the robustness of the proposed algorithms against roundoff errors.
METHODS OF SCHOOL CHEATING FROM THE STUDENTS´ POINT OF VIEW

Appears in: EDULEARN20 Proceedings Publication year: 2020Pages: 1309-1315ISBN: 978-84-09-17979-4ISSN: 2340-1117doi: 10.21125/edulearn.2020.0428Conference name: 12th International Conference on Education and New Learning TechnologiesDates: 6-7 July, 2020Location: Online Conference
Ubiquitous information literacy education in college libraries

Changes in the information environment caused by information technology advancements necessitate reforms in cultivation of information literacy. From the angles of ubiquitous knowledge environment and ubiquitous services in libraries, this paper proposed the idea of ubiquitous information literacy education in colleges. By using the SWOT method, we made comprehensive analysis of the strengths, weaknesses, opportunities and threats of carrying out ubiquitous information literacy education in college libraries, and put forward strategies in terms of education ideas, teaching faculty development, curriculum development and platform resources development, in hopes of offering feasible solutions to reforms in college education of information literacy and improving the information literacy of students and teachers in colleges.
Approximate is Good Enough: Probabilistic Variants of Dimensional and Margin Complexity

We present and study approximate notions of dimensional and margin complexity, which correspond to the minimal dimension or norm of an embedding required to approximate, rather then exactly represent, a given hypothesis class. We show that such notions are not only sufficient for learning using linear predictors or a kernel, but unlike the exact variants, are also necessary. Thus they are better suited for discussing limitations of linear or kernel methods.
SimulEval: An Evaluation Toolkit for Simultaneous Translation

Simultaneous translation on both text and speech focuses on a real-time and low-latency scenario where the model starts translating before reading the complete source input. Evaluating simultaneous translation models is more complex than offline models because the latency is another factor to consider in addition to translation quality. The research community, despite its growing focus on novel modeling approaches to simultaneous translation, currently lacks a universal evaluation procedure. Therefore, we present SimulEval, an easy-to-use and general evaluation toolkit for both simultaneous text and speech translation. A server-client scheme is introduced to create a simultaneous translation scenario, where the server sends source input and receives predictions for evaluation and the client executes customized policies. Given a policy, it automatically performs simultaneous decoding and collectively reports several popular latency metrics. We also adapt latency metrics from text simultaneous translation to the speech task. Additionally, SimulEval is equipped with a visualization interface to provide better understanding of the simultaneous decoding process of a system. SimulEval has already been extensively used for the IWSLT 2020 shared task on simultaneous speech translation. Code will be released upon publication.
An evolution-based approach for efficient differentiable architecture search

We propose a hybrid neural architecture search method of evolution-based and gradient-based approaches. In this method, the architecture variables are optimized by evolutionary algorithm and exploited by gradient-based optimization. This hybridity allows an efficient architecture search and also maintains the extensibility of the evolution-based search. On the CIFAR-10 dataset, our method discovers architectures that achieve competitive performance with state-of-the-art models.
Knowledge Visualization Centers and University Infographics: International and Domestic Experience

Infographics as a young science of converting massive volumes of knowledge into concise emergent visualizations, i.e. the methodology that works with transcoded knowledge, has become capable, at the digital stage of its own evolution, to integrate the most diverse fields of knowledge. This is possible due to the advantages that the graphic language provides along with all other ways of expressing and formalizing knowledge. The potentials of infographics for the university environment provide an opportunity to visually operate with vast volumes of knowledge, to search for new interdisciplinary nexus-points and heuristic intersections in the network structure of university science. Along the way, the problems of inventive tasks and the promotion of scientific knowledge are solved through digital infographic visualizations on the Internet, within and between universities. The article develops an idea of socio-epistemological and interdisciplinary potentials of the phenomenon of scientific and philosophical infographics for the sustainable growth of the competitive advantages of the Russian education system. The authors analyze the world and domestic experience in infographics functioning in science, business, management and consider a new project for a domestic university education on creating a knowledge visualization center on the example of a particular university. The authors give the examples of successful international data visualization projects and pay particular attention to the creation of a holistic interdisciplinary catalog of infographics. University digital library can serve as a source of databases and potentials of the visualization center, as a tool for visualizing this data. The authors discuss the possibility of infographic expertise procedure formation as a new methodology for analyzing and improving various program documents, social development programs, business projects, and dissertations. The article makes the case for developing infographic competencies of innovation actors in education, business and public administration.
Secure Generalized Deduplication via Multi-Key Revealing Encryption

Cloud Storage Providers (CSPs) offer solutions to relieve users from locally storing vast amounts of data, including personal and sensitive ones. While users may desire to retain some privacy on the data they outsource, CSPs are interested in reducing the total storage space by employing compression techniques such as deduplication. We propose a new cryptographic primitive that simultaneously realizes both requirements: Multi-Key Revealing Encryption (MKRE). The goal of MKRE is to disclose the result of a pre-defined function over multiple ciphertexts, even if the ciphertexts were generated using different keys, while revealing nothing else about the data. We present a formal model and a security definition for MKRE and provide a construction of MKRE for generalized deduplication that only uses symmetric key primitives in a black-box way. Our construction allows (a) cloud providers to reduce the storage space by using generalized deduplication to compress encrypted data across users, and (b) each user to maintain a certain privacy level for the outsourced information. Our scheme can be proven secure in the random oracle model (and we argue that this is a necessary evil). We develop a proof-of-concept implementation of our solution. For a test data set, our MKRE construction achieves secure generalized deduplication with a compression ratio of 87% for 1 KB file chunks and 82.2% for 8 KB chunks. Finally, our experiments show that, compared to generalized deduplication setup with un-encrypted files, adding privacy via MKRE introduces a compression overhead of less than $$3\%$$ and reduces the storage throughput by at most $$6.9\%$$ .
Research on the Big Data Platform and Its Key Technologies for the Railway Locomotive System

In order to improve the efficiency of locomotive organization and the quality of locomotive operation, this paper analyzes and discusses the big data platform and some key technologies suitable for the big data application of the railway locomotive system. Firstly, the definition of big data of the railway locomotive system is proposed, and the current data characteristics of the railway locomotive system are summarized, then the status quo and demands of big data application of the railway locomotive system are analyzed. Secondly, the overall architecture of the big data platform for the railway locomotive system is proposed. Furthermore, seven application scenarios available for the big data platform are analyzed, including locomotive running organization, high-speed railway, repair, maintenance and other fields. Finally, some key technologies, which consist of data collection system of front-line operations, locomotive equipment portrait analysis, staff portrait analysis, transmission and analysis of locomotive video, intelligent auxiliary driving system, are provided to increase efficiency of the locomotive organization and capability of safety management. The obtained results can play a positive role in the construction and application of big data of the railway locomotive system.
Neural Named Entity Recognition for Kazakh

We present several neural networks to address the task of named entity recognition for morphologically complex languages (MCL). Kazakh is a morphologically complex language in which each root/stem can produce hundreds or thousands of variant word forms. This nature of the language could lead to a serious data sparsity problem, which may prevent the deep learning models from being well trained for under-resourced MCLs. In order to model the MCLs' words effectively, we introduce root and entity tag embedding plus tensor layer to the neural networks. The effects of those are significant for improving NER model performance of MCLs. The proposed models outperform state-of-the-art including character-based approaches, and can be potentially applied to other morphologically complex languages.
Proving Ground Test of a DDPG-based Vehicle Trajectory Planner

The paper presents real-world test cases of an optimal trajectory design solution that combines modern control techniques with machine learning. The first step of the current research is to train a reinforcement learning agent in a simulated environment, where the conditions and the applied vehicle are modeled. System dynamics is described by a nonlinear single-track vehicle with dynamic wheel model. The designed trajectory is evaluated by driving the vehicle using a control loop. The reward of the method is based on the sum of different measures considering safety and passenger comfort. The proposed method forms a special one-step reinforcement learning task handled by Deep Deterministic Policy Gradient (DDPG) learning agent. As a result, the learning process provides a real-time neural-network-based motion planner and a tracking algorithm. The evaluation of the algorithm under real conditions is made by using an experimental test vehicle. The test setup contains a high precision GPS module, an automotive inertial sensor, an industrial PC, and communication interface devices. The test cases were performed on the ZalaZone automotive proving ground.
A Search Method for Reassembling the Elements of a Broken 2D Object

Searching for and reassembling the elements that used to form one whole is a very common issue faced by archaeologists.This is because preparing an interesting museum exhibition consists in the presentation of the objects that have been put together, not a pile of messily disassembled puzzle pieces.The article presents the concept of using the linguistic methods in the process of joining the elements of a 2D jigsaw puzzle.The method developed in the first stage creates the edge description of an object by defined unit vectors of the same length but different directions, and assigns them a designation in the form of letters, which leads to the creation of abstract words in the form of a sequence of signs.In the second stage, the words with a defined length of strings belonging to two different objects are compared.The authors have created a program that performs an exhaustive search until the pool of available elements is fully exhausted.The conducted numerical experiments indicate the correctness of the method and effectiveness in determining the places of joining elements.The developed method will be useful to automate the reassembly of 2D elements from archaeological excavations.
Attacking and Defending Machine Learning Applications of Public Cloud

Adversarial attack breaks the boundaries of traditional security defense. For adversarial attack and the characteristics of cloud services, we propose Security Development Lifecycle for Machine Learning applications, e.g., SDL for ML. The SDL for ML helps developers build more secure software by reducing the number and severity of vulnerabilities in ML-as-a-service, while reducing development cost.
Contrastive Variational Model-Based Reinforcement Learning for Complex Observations.

Deep model-based reinforcement learning (MBRL) has achieved great sample-efficiency and generalization in decision making for sophisticated simulated tasks, such as Atari games. However, real-world robot decision making requires reasoning with complex natural visual observations. This paper presents Contrastive Variational Reinforcement Learning (CVRL), an MBRL framework for complex natural observations. In contrast to the commonly used generative world models, CVRL learns a contrastive variational world model by maximizing the mutual information between latent states and observations discriminatively by contrastive learning. Contrastive learning avoids modeling the complex observation space and is significantly more robust than the standard generative world models. For decision making, CVRL discovers long-horizon behavior by online search guided by an actor-critic. CVRL achieves comparable performance with the state-of-the-art (SOTA) generative MBRL approaches on a series of Mujoco tasks, and significantly outperforms SOTAs on Natural Mujoco tasks, a new, more challenging continuous control RL benchmark with complex observations introduced in this paper.
The Essence of the Universe and Humankind

The Essence of the three-dimensional Universe came from the energy from the multi-dimensional void, deprived of all matters.Dimensions above the three-dimensional Universe postulated in the M-theory are all abstract mathematical dimensions, and they could be grouped into the zero-dimensional Universe.The present known Universe is probably the highest dimension and it is one of the infinite numbers of pocket Universe floating in the zero-dimensional kong, or void.Matters, including humans, are the products from the combination of energy and materials.What sets humans apart from the rest of the living matters is the difference in the intelligent energy bestowed during the initial stage in the creation of human cells.
Estimation of Missing Data in Remote Sensing Images Using t-SVD based tensor completion

This paper focuses on estimating the lost information from hyperspectral remote sensing images affected by missing samples. During acquisition or transmission remote sensing images may suffer from missing information. When it is affected by missing information, it cannot be used for further applications such as object detection and classification. Therefore the need to predict the missing data in existing remote sensing images becomes a necessity. The performance of the existing tensor completion techniques are poor when data is missing in all bands of a hyperspectral remote sensing image because they do not adequately utilise the spectral information in all bands. This paper proposes a new technique to overcome these problems by using Tensor Singular Value Decomposition (t-SVD) based Tensor Completion (TC). The spectral as well as spatial information in all bands of a hyperspectral remote sensing image could be utilized to detect all missing data, wherefore existing tensor completion techniques are a failure. The proposed method have been experimentally proved to outperform the existing methods when data is missing in either all of the bands or some of the bands.
Building an effective information security awareness program

Many researchers and experts in the field of information security agree that the user is the weakest link in an organization’s chain of information security. Even if the system’s and the stored data’s physical and logical protection is well developed, the human factor exposes security to significant risk. The effective protection against the threats is to provide security awareness through implementing a well-developed and successful Information Security Awareness Program.&#x0D; Although organizations are able to recognize the importance of information security awareness, the implementation of the awareness programs can be difficult. The aim of this study is to help organizations to develop an effective Information Security Awareness Program tailored to the characteristics of the organization. The paper presents how we can build a program that influences and improves the user’s knowledge, attitude and behavior the most towards information security and makes positive changes in the security culture of an organization. To achieve that goal, the study identifies the key elements of the implementation, compares traditional awareness programs with modern trainings and highlights the importance of communication channels and methods. There is no single solution to improve information security, the essay summarizes and shows the most effective techniques that experts can use in order to seize the user’s attention toward information security, to establish credibility and trust, and to motivate action.
Character and Student Ability of Covid-19 Understanding in Digital Era in 2020

The purpose of this research is to describe the character of students towards the understanding of Covid-19 in the digital age, to describe the ability of students to translate understanding of terminology relating to Covid-19 in the digital age, and to describe the effectiveness of digital campus on understanding covid-19 by students in the digital age. The data analysis technique in this research is quantitative descriptive. The result shows that The ability of students to translate intralingually new words that were present during the Covid-19 period is very good and dominant in number. The dominant character of students in putting an interest in the knowledge, learning and translation of new words during the Covid-19 pandemic to deal with the spread of Covid-19 was very good, interested, uplifting and enthusiastic. The level of effectiveness of digital campuss by students when the corona pandemic is very high and dominant. Thank you to the LPPM &amp; PM Universitas Samudra from the DIPA 2020 funds and all those who assisted our research activities so that we can complete the flagship basic research.
AUTOMATIC BENGGANG RECOGNITION BASED ON LATENT SEMANTIC FUSION OF UHR DOM AND DSM FEATURES

Abstract. Benggang are characterized by deep-cut slopes with various shapes and depressions on the vast weathered crust slopes in southern China. The gully heads have been continuously collapsed and eroded to form a chair-like erosion landforms. It develops rapidly, and leads to large amounts of erosion, with the hazards of damaging land resources, destroying basic farmland, and deteriorating ecological environment. To study and manage Benggang, the primary task is to discover it. Traditional methods based on local in-situ investigations, which are not only labour-consuming but also inefficient. These methods are difficult to meet the needs of large-scale investigations of Benggang. This paper proposes a method for automatic Benggang recognition based on Ultra-High Resolution (UHR) DOM (Digital Orthophoto Map) and DSM (Digital Surface Model) obtained from UAV (Unmanned Aerial Vehicle) survey. This method adopts a Bag of Visual-Topographical Words (BoV-TW) model. The local features extracted from DOM and DSM are represented based on BoV-TW, and fused by Latent Dirichlet Allocation (LDA). Finally Support Vector Machine (SVM) is adopted as a supervised classifier to achieve high-precision automatic Benggang recognition. Experimental results prove that the total accuracy of our method can be maintained at about 95%, with recall and precision above 80% (the highest are 97.22% and 94.44%, respectively), which are significantly higher than the methods of using only DOM local features and using only BoV-TW.