Reviewerly/exHarmony huggingface dataset:
Towards Accelerating Offline RL based Recommender Systems

Reinforcement learning optimizes an objective function by learning an optimal policy for taking a sequence of actions in an environment. Offline RL is a mechanism to learn policy offline from pre-generated traces of an agent’s interactions in an environment, accelerating an agent’s initial learning phase. Real-time deployment of RL-based recommenders across geographies could mix online and offline RL algorithms to explore new users’ behavior and exploit old knowledge to learn recommendation policy. In such a scenario, RL agents will be distributed and deployed in multiple locations. In this paper, we share our experiences and learnings in accelerating our in-house offline RL-based recommender system. The recommender system employs a mix of Batch Constrained (BCQ) and distributional RL algorithms for building policy models. We present various acceleration techniques for this system, such as operators’ fusion, performance anti-patterns, heterogeneous deployments, and design space of synchronous and asynchronous distributed training over generative and policy models of the algorithm. We have shown that the presented acceleration techniques could speed up the training of the RL agents on A100 GPUs by a factor of 47 × over a naive code (written by the ML practitioner) implementation on A100.
Multi-agent reinforcement learning for prostate localization based on multi-scale image representation

The analysis of magnetic resonance (MR) images plays an important role in medicine diagnosis. The localization of the anatomical structure of lesions or organs is a very important pretreatment step in clinical treatment planning. Furthermore, the accuracy of localization directly affects the diagnosis. We propose a multi-agent deep reinforcement learning-based method for prostate localization in MR image. We construct a collaborative communication environment for multi-agent interaction by sharing parameters of convolution layers of all agents. Because each agent needs to make action strategies independently, the fully connected layers are separate for each agent. In addition, we present a coarse-to-fine multi-scale image representation method to further improve the accuracy of prostate localization. The experimental results show that our method outperforms several state- of-the-art methods on PROMISE12 test dataset.
Metric Calibration of Aerial On-Board Multiple Non-overlapping Cameras Based on Visual and Inertial Measurement Data

Recently, the on-board cameras of the unmanned aerial vehicles are widely used for remote sensing and active visual surveillance. Compared to a conventional single aerial on-board camera, the multi-camera system with limited or non-overlapping field of views (FoVs) could make full use the FoVs and would therefore capture more visual information simultaneously, benefiting various aerial vision applications. However, the lack of common FoVs makes it difficult to adopt conventional calibration approaches. In this paper, a metric calibration method for aerial on-board multiple non-overlapping cameras is proposed. Firstly, based on the visual consistency of a static scene, pixel correspondence among different frames obtained from the moving non-overlapping cameras are established and are utilized to estimate the relative poses via structure from motion. The extrinsic parameters of non-overlapping cameras is then computed up to an unknown scale. Secondly, by aligning the linear acceleration differentiated from visual estimated poses and that obtained from inertial measurements, the metric scale factor is estimated. Neither checkerboard nor calibration pattern is needed for the proposed method. Experiments of real aerial and industrial on-board non-overlapping cameras calibrations are conducted. The average rotational error is less than \(0.2^{\circ }\), the average translational error is less than 0.015 m, which shows the accuracy of the proposed approach.
A Permissioned Blockchain with Proof of Location for Digital Cinema

This paper presents the use of a Hyperledger Fabric blockchain to manage the inventory of digital cinema projectors. It is necessary for the protected distribution of digital movies to digital theaters. The document discloses a commitment-based protocol that proves a device's location.
Resource Allocation and Power Management in Cloud Servers Using Deep Reinforcement Learning

Resource allocation and power management have become challenging tasks in cloud computing. Since workloads change over time, dynamic cloud resource allocation and power management solutions are needed, that can intrinsically adapt to the demand of the resources. This paper demonstrates the use of deep reinforcement learning algorithms for the task dispatching and power management in the simulated Alibaba cluster environment with the goal of achieving better power and latency management than traditional task scheduling algorithms. In this paper, we propose a hierarchical RL model for resource allocation and power management which obtains better results in terms of latency as well as power usage, than traditional task scheduling.
Channel Estimation with an Interpolation Trained Deep Neural Network

This paper proposes a deep learning-based channel estimation method for orthogonal frequency-division multiplexing (OFDM) systems.The existing OFDM receiver has low estimation accuracy when estimating channel state information (CSI) with fewer pilots.To tackle the problem, in this paper, a deep learning model is first trained by the interpolated channel frequency responses (CFRs) and then used to denoise the CFR estimated by least square (LS) estimation.The proposed deep neural network (DNN) can also be trained in a short time because it only learns the CFR and the network structure is simple.According to the simulation results, the performance of the DNN estimator can be compared with the minimum mean-square error (MMSE) estimator.Furthermore, the DNN approach is more robust than conventional methods when fewer pilots are used.In summary, deep learning is a promising tool for channel estimation in wireless communications.
Pharmaceutical anti-counterfeiting traceability system based on block chain double chain

In order to solve the problems of centralization, tampering, incomplete storage and privacy of patient information, a medical anti-counterfeiting traceability system based on block chain “double chain” is proposed. The system solves the problem of poor expansibility and low throughput in single chain applications, stores traceability information and consumer information separately, and realizes access control in untrusted environment by using the decentralization characteristic of block chain and the on-chain code of intelligent contract, which effectively protects consumer privacy data. The system is developed on the Fabric block chain platform of super account book (Hyperledger). The system environment is equipped with four organizations: pharmaceutical manufacturer, dealer, hospital and consumer. The chain code is developed by Java language, the client program is written by using Node.js, and the query request is initiated with the drug traceability function in the chain code. Ultimately, the certificate-certified user account can achieve drug information in the web page query. The data of block chain is difficult to tamper with, time stamp and transaction traceability can be well applied to the pharmaceutical anti-counterfeiting traceability system, which makes the traceability function of the system more perfect, and consumers can get all traceability information including drug production information, logistics information and use information.
CLC-RS

With the ever-increasing size of legal cases in China, relevant legal case retrieval given a user query has attracted considerable attention. Conventional keyword-based retrieval systems look for matching cases that contain one or more words specified by the user. However, keyword search is sharply focused on finding the exact terms specified in the query, making the retrieval systems miss many relevant documents. In addition, it is difficult for new users to identify appropriate keywords for accurate legal case retrieval. In this paper, we develop a novel Chinese legal case retrieval system (called CLC-RS), which improves the quality of semantic search with natural language queries in the legal domain. CLC-RS performs legal case retrieval in a two-stage fashion. First, we employ a classic token-based ranking method to efficiently reduce the solution space, returning a subset of candidate legal cases. Then, we deploy a novel masked language ranking model to re-rank the candidate legal cases. The experimental results show that the proposed system is both efficient and effective, providing a practical information retrieval (IR) system for retrieving Chinese legal cases. The web site for the developed CLC-RS system is available at: https://www.delilegal.com/.
Sample-based Frontier-Block Detection for Autonomous Robot Exploration

Autonomous exploration requires a robot to find frontiers as quickly as possible. The rapidly-exploring random tree (RRT) in the motion planning field has been widely adopted recently. However, the classical RRT and its variant algorithms are still relatively aimless and inefficient for the reason that there are lots of repeated useless sampling operations and the process of finding the nearest point costs too much time. So a sample-based frontier-block detection method (SFBD) is proposed to avoid directly searching for the nearest point from a set with a large number of points and SFBD uses the block structure to record collision thus reducing useless sampling. Then, SFBD uses an improved block operation, which can greatly reduce the counts each block is operated. In order to demonstrate that SFBD does not depend on local tree excessively, only global RRT and two trees are respectively used in two simulation environments. The experimental results indicate that SFBD can more efficiently complete the exploration tasks comparing to other algorithms.
Shared Latent Space of Font Shapes and Their Noisy Impressions

Styles of typefaces or fonts are often associated with specific impressions, such as heavy, contemporary, or elegant. This indicates that there are certain correlations between font shapes and their impressions. To understand the correlations, this paper constructs a shared latent space where a font and its impressions are embedded nearby. The difficulty is that the impression words attached to a font are often very noisy. This is because impression words are very subjective and diverse. More importantly, some impression words have no direct relevance to the font shapes and will disturb the construction of the shared latent space. We, therefore, use DeepSets for enhancing shape-relevant words and suppressing shape irrelevant words automatically while training the shared latent space. Quantitative and qualitative experimental results with a large-scale font-impression dataset demonstrate that the shared latent space by the proposed method describes the correlation appropriately, especially for the shape-relevant impression words.
Developing an online course for marine engineering

Developing an English Academic Online Course for engineering is a complicated endeavor. The ultimate goal of any teacher or professor is to make the students learn circadian and consciously in order to remember the theory and to be able to apply it in the future job. Assessment of every progress of the student is almost impossible during a traditional course, taking into account the fact that the groups of students are quite large and the course or the seminar time is limited. Starting from these premises we have tried to develop an online course that gives the professor the opportunity to check, by means of computer, every progress in learning terminology in a foreign language. The structure of the course and the levels of learning are described in the paper.
Creativity and Media Culture in Modern Kazakhstan

The development of network technologies and new media increases the information flow, improves the quality of socio-economic and spiritual and cultural processes. The mass media are becoming a catalyst for the cultural development of modern society, which forms the civilization of the 21st century. Currently, they won first place in the information impact on the individual and society. Mass media not only translate the existing system of values but also actively form a new cognitive information space in Kazakhstan. With the new media came another media culture that changes our consciousness, our tastes, opens up new media opportunities, and expands new horizons of the media sphere. The digital age has fundamentally changed the media environment. Changes in public consciousness were especially pronounced in the 21st century under the influence of information technology, the development of new media, the Internet, and social networks. The approval of new thinking as one of the conditions for solving global problems of modern society is associated with a reassessment of values, a change in existing spiritual guidelines and established social attitudes under the influence of digital technologies. Socio-technical imaginaries is the way in which different cultures and communities imagine their future in terms of symbolically charged scientific and technical achievements. This research articulates the socio-technical imaginary that grows up around digital technology in Kazakhstan. The 21st century has created a new information space, a new media sphere, where the formation and development of a new media culture of the century is taking place.
Implementing game requirements using design patterns

Abstract Game mechanics are high‐level descriptions of common game requirements; however, they do not provide any guidance on their code implementation. Nevertheless, their implementation involves high volumes of essential complexity, which in turn leads to the introduction of accidental complexity (long methods, code repetition, etc.). A possible solution to this problem is to map game mechanics to design patterns, in order to provide template instantiations that handle structural complexity. The aim of this study is twofold: (a) introduce template instantiations of game mechanics with Gang of Four (GoF) patterns and (b) evaluate such mappings in terms of extendibility and reusability. To achieve these objectives, we developed an online repository of mappings between GoF patterns and game mechanics and conducted an experiment to explore the benefits of the mapping. The results of the study suggest that the implementation of game mechanics with GoF patterns is beneficial, because the time to extent the current implementation or reuse code chunks is not prolonged, whereas the implementations are less faulty. Finally, the suggested mapping will equip game mechanics with sample implementations (that adhere to good design principles—ensuring extendibility) that can be reused, acting as a starting point for source code development.
JanusNet: Detection of Moving Objects from UAV Platforms

In this paper, we present JanusNet, an efficient CNN model that can perform online background subtraction and robustly detect moving targets using resource-constrained computational hardware on-board unmanned aerial vehicles (UAVs). Most of the existing work on background sub-traction either assume that the camera is stationary or make limiting assumptions about the motion of the camera, the structure of the scene under observation, or the apparent motion of the background in video. JanusNet does not have these limitations and therefore, is applicable to a variety of UAV applications. JanusNet learns to extract and combine motion and appearance features to separate background and foreground to generate accurate pixel-wise masks of the moving objects. The network is trained using a simulated video dataset (generated using Unreal Engine 4) with ground-truth labels. Results on UCF Aerial and Kaggle Drone videos datasets show that the learned model transfers well to real UAV videos and can robustly detect moving targets in a wide variety of scenarios. Moreover, experiments on CDNet dataset demonstrate that even without explicitly assuming that the camera is stationary, the performance of JanusNet is comparable to traditional background subtraction methods.
Application of Deep Learning in Classification of Encrypted Images

In 2020, Citizen Lab in University of Toronto found that the popular video conferencing application, Zoom, was not only ignoring the use of AES-256, but was instead using AES-128 in the Electronic Code Book (ECB) mode. ECB has never been a secure encryption strategy as it preserves the patterns present in the original input. This paper shows a simple deep learning based attack on images encrypted using four different encryption schemes (AES-128, GIFT-64, PICO and PRESENT) under the ECB mode of operation. We build a deep learning model that learns to classify encrypted images with 100% accuracy. We also explore the reasons behind why small size images might not look intelligible to the human eye but are still easily classifiable using a neural network. We also express how this attack can be scaled and further used to exploit encrypted data.
Towards Efforts to Enhance Tax Revenue Mobilisation in Africa: Exploring Synergies between Industrialisation and ICTs

Motivated by the momentous rise in ICT diffusion, the implementation of the African Continental Free Trade Area agreement, and the expected rebound of foreign direct investment inflow to Africa from 2022, this study examines the joint effects of industrialisation and ICT diffusion on resource mobilisation in Africa. To this end, we use data on 42 African countries for the period 1996 – 2020 for the analysis. First, we provide evidence robust to several specifications from the dynamic system GMM to show that although unconditionally both industrialisation and ICT diffusion enhance (i) goods and services tax (GST), and (ii) profits, corporate and income tax (PCIT) mobilisation efforts in Africa, the effects of the former are rather remarkable in the presence of the latter. Particularly, the results show that, while ICTs amplify the effect of industrialisation on GST, only ICT usage and ICT skills matter for PCIT. Second, the study unveils ICT thresholds for complementary policies. Accordingly, industrialisation and ICTs are necessary and sufficient conditions for tax revenue mobilisation only below some ICT thresholds. Above these ICT thresholds, complementary policies are needed to maintain the overall positive incidence on tax revenue mobilisation. Policy recommendations are provided in the end.
Improved Conditional GAN for Aerial Image Segmentation

Generative Adversarial Network (GAN) is an algorithmic architecture containing two neural networks, placed against each other to generate new synthetic images and it has been used successfully in image segmentation. The paper analyzes different GAN implementations for segmentation of images acquired by aerial robots in a real context of a rural zone in Romania. To improve the segmentation performance, a new GAN network is proposed by adding a new layer. Data augmentation was done by the following techniques: mirroring, rotation, scaling, gray scaling, blurring, sharpening, etc. Five classes of region of interest are considered: floods, vegetations, buildings, roads, and dry land. GAN implementations were tested on CPU, GPU, and TPU, on individual computing devices and in the cloud. A new layer was added. The performances were analyzed in terms of learning time, operating time, and statistical indicators. The batch size was generally low: batches of 1, 4 or 16 images were used in this paper. The results confirm that the use of batch achieves the best training and generalization performance in terms of computational cost, for a wide range of experiments.
Towards Optimal Supercomputer Energy Consumption Forecasting Method

Accurate prediction methods are generally very computationally intensive, so they take a long time. Quick prediction methods, on the other hand, are not very accurate. Is it possible to design a prediction method that is both accurate and fast? In this paper, a new prediction method is proposed, based on the so-called random time-delay patterns, named the RTDP method. Using these random time-delay patterns, this method looks for the most important parts of the time series’ previous evolution, and uses them to predict its future development. When comparing the supercomputer infrastructure power consumption prediction with other commonly used prediction methods, this newly proposed RTDP method proved to be the most accurate and the second fastest.
Cybersecurity and the New Age of Ransomware Attacks

One of the most concerning health care information technology headlines is the growing number of cyber security challenges faced by clinical agencies. Cyber security threats and attacks can take many shapes and forms, but the most pernicious and aggressive are those that result in a complete shut-down of one or more vital technology systems or the rendering of health care data to be inaccessible. The intense dependency on technology in health care for everything from billing to intravenous drip rates has elevated concerns about these attacks and increased the risk for direct harm to patients.
Application of Computer Multimedia Technology in the Design of English Education Curriculum

With the vigorous development of CMT (computer multimedia technology), it is widely used in teaching. At present, mankind is entering a new period of technological development. The emergence of CMT provides a new material foundation for the reform of English teaching methods and teaching models. The wide application of MT (multimedia technology) in teaching provides students with a brand-new learning form and promotes the innovation of traditional teachin. Many years of teaching method reforms have consumed a lot of financial and manpower, but the students’ English proficiency has not been significantly improved. How to organically combine CM technology with traditional teaching methods to form teaching advantages is particularly important. The application of CM technology in the design of EE (English education) curriculum is the focus of this article. First, analyze the results of the two classes. The results show that the multimedia teaching mode can improve students’ listening, reading, vocabulary, writing, and students’ comprehensive English scores. The average score of the control group is 83.3. 12.4 points higher than the control group. Secondly, analyze the effect of CM technology on English courses. 41% of students believe that MT teaching can stimulate students’ interest in learning English. 22 and 19 people respectively think that MT can strengthen the communication between teachers and students and enrich the teaching content. The remaining 18 people pointed out that this teaching model is conducive to cultivating teachers’ quality, improving teaching structure and efficiency.
Estimating Dense Optical Flow of Objects for Autonomous Vehicles

Autonomous vehicles need to be able to perceive both the presence and motion of objects in the surrounding environment to navigate in the real world. In this work, we propose to solve the tasks of identifying objects and estimating the corresponding motion by viewing them as a single unified task known as instance flow. Instance flow provides the pixel-wise instance mask of an object and the dense optical flow within it. To achieve this, we extended the state of the art object detection model to include a dense optical flow estimator. The estimator is used to estimate the optical flow for each region of interest only, instead of the entire image. We tested the approach by carrying out experiments on publicly available datasets for autonomous driving research, VKITTI, KITTI and HD1K. Furthermore, we also introduced a new instance flow quality metric to evaluate the instance flow estimation.