Reviewerly/exHarmony huggingface dataset:
PipeFB: An Optimized Pipeline Parallelism Scheme to Reduce the Peak Memory Usage

Neural network models is developing toward deeper and wider to obtain higher accuracy and robustness. However, the limited physical memory capacity of existing hardware devices limits the scale of the neural network that can be trained, and the limited computing capacity resulting in excessively long training time. Therefore, the distributed parallelism scheme based on multi-accelerator machines becomes an effective method to training large-scale neural networks. The pipeline parallelism is one of the distributed parallelism scheme, which has large advantages in the training speed. But it also significantly increases the peak memory usage and communication overhead, because it needs to store multiply versions of activations. Our previous work has proposed a data transfer mechanism and applied it to the PipeDream design (a mature pipeline parallelism scheme), which offloads activations in the pipeline to other memory devices, such as the CPU memory. The data transfer mechanism greatly reduces the peak memory usage of the PipeDream, but it brings a large amount of communication, which makes the PipeDream lost a lot of training speed. This paper proposes an optimized pipeline parallelism scheme, the PipeFB, for applying the data transfer mechanism. The PipeFB deploys the forward propagation and backward propagation of the neural network on different computing nodes, which is different from the traditional pipeline parallelism scheme. We implements the PipeFB and applies the data transfer mechanism to it. The experimental results shows that our design has the same peak memory usage as the PipeDream with the data transfer mechanism, but the training speed of our design is 1.48 to 2.27 times faster.
Efficient-Secure k-means Clustering Guaranteeing Personalized Local Differential Privacy

One highly discussed research topic is user privacy protection and the usability of models in data mining tasks. Currently, the most k-means clustering approach using differential privacy is based on trusted third-party servers. However, malicious servers exist in many applications and cause privacy leakages of user data. The Personalized Local Differential Privacy k-means algorithm (PLDP k-means) is proposed in this paper. To satisfy the PLDP mechanism, a perturbation mechanism is used to perturb the user data at the local side. Then clustering is completed by iteration between the local and server sides. The third-party server remains inaccessible to the real user data and considers the users’ personalized privacy demands in the proposed algorithm. In addition, the iterative centroid perturbation algorithm is proposed in this paper for resisting inference attacks and improving the utility of clustering via a privacy budget allocation sequence. Theoretical analysis demonstrates the privacy of the proposed algorithm. Experimental results indicate that the proposed algorithm effectively preserves the utility of clustering while satisfying the PLDP mechanism.
Compromise Privacy in Large-Batch Federated Learning via Malicious Model Parameters

Federated Learning (FL) is a distributed learning paradigm, in which users share their gradients instead of local data to preserve privacy. Previous works have shown that the server in FL can reveal user’s local data by inverting shared gradients. However, a large batchsize can defense against these attacks effectively by obfuscating gradients calculated on each data. In this paper, we propose a novel Gradient Inversion Attack which can compromise privacy in large-batch FL. Firstly, the server constructs malicious model parameters which can mitigate the confusion of gradients. Then users’ model parameters are tampered purposely by the server. From users’ shared gradients computed on malicious model parameters, the server can recover private local trainsets perfectly in large-batch FL. Experiments on CIFAR100 show that our method can recover 92%, 77% and 54% of the data points in a batch with batchsize 128, 256 and 512, respectively. Compared with previous works, our method has a higher performance and versatility.
A tentative study on the causes of math underachievers in junior high schools and the conversion strategies

Underachievers are an objective dissimilated group in the learning process. Math underachievers are a practical problem which should be addressed by present math educators. Through field survey and upon analysis of individual characteristics of underachievers from multiple perspectives of teachers, family education and students, a targeted conversion strategy is formulated. With focuses on teaching and learning method, the strategies for conversion of math underachievers in junior high schools are explored and summarized, followed by practical discussion on individual cases.
Warm Starting Variational Quantum Algorithms with Near Clifford Circuits

As a mainstream approach in the quantum machine learning field, variational quantum algorithms (VQAs) are frequently mentioned among the most promising applications for quantum computing. However, VQAs suffer from inefficient training methods. Here, we propose a pretraining strategy named near Clifford circuits warm start (NCC-WS) to find the initialization for parameterized quantum circuits (PQCs) in VQAs. We explored the expressibility of NCCs and the correlation between the expressibility and acceleration. The achieved results suggest that NCC-WS can find the correct initialization for the training of VQAs to achieve acceleration.
High-Precision Leveled Homomorphic Encryption for Rational Numbers

In most homomorphic encryption schemes based on RLWE, native plaintexts are represented as polynomials in a ring Zt[x]/xN+1, where t is a plaintext modulus and xN+1 is a cyclotomic polynomial with a degree power of two. An encoding scheme should be used to transform some natural data types (such as integers and rational numbers) into polynomials in the ring. After homomorphic computations on the polynomial aare finished, the decoding procedure is invoked to obtain the results. We employ the Hensel code for encoding rational numbers and construct a high-precision leveled homomorphic encryption scheme with double-CRT. The advantage of our scheme is that the limitations of previous works are avoided, such as unexpected decoding results and loss of precision. Moreover, the plaintext space can be adjusted simply by changing a hyper-parameter to adapt to different computation tasks.
Targeted Clean-Label Poisoning Attacks on Federated Learning

Federated Learning (FL) has become one of the most extensively utilized distributed training approaches since it allows users to access large datasets without really sharing them. Only the updated model parameters are exchanged with the central server after the model has been trained locally on the devices holding the data. Because of the distributed nature of the FL technique, there is a possibility of adversarial attacks that aim to manipulate the behavior of the model. This paper explores targeted clean-label attack in which adversaries inject poisoned images into compromised clients’ dataset to alter the behaviour of the model on a specific target image at test time. The standard CIFAR10 dataset is used in this study to conduct various experiments and manipulate the image classifier. This study discovered that the behavior of a FL model can be altered maliciously towards a specific target image without significantly affecting the model’s overall accuracy. In addition, the attack’s impact grows in direct proportion to the number of injected poisonous images and malicious client (i.e. controlled by adversaries) participating in the FL process.
Enabling Edge Devices Using Federated Learning and Big Data for Proactive Decisions

Big data and cloud computing lay a new paradigm of data analytics and pave an insight for new types of intelligent devices. These smart devices continuously capture, store, and transfer data to the centrally controlled devices. In recent years, there has been an exponential increase in smart devices, which has increased the amount of data generated by these devices. The communication of the generated data to the central nodes results in a high communication rate. This type of data communication is vulnerable to network intrusions and multiple security threats. Realtime processing of the data is also a huge challenge, and it is an arduous task to provide support to the applications working on realtime decision making. This paper proposes a message passing approach among intelligent edge devices, which leverages the advantages of the big data analytics and federated learning to tackle the challenges as mentioned above. The experimental analysis of several benchmark datasets supports the proposed approach’s theoretical base. The proposed approach with suitable number of edge devices reduces the overall training time maximum by one third while maintaining a significant amount of accuracy.
HeightNet: Monocular Object Height Estimation

Monocular depth estimation is a traditional computer vision task that predicts the distance of each pixel relative to the camera from one 2D image. Relative height information about objects lying on a ground plane can be calculated through several processing steps from the depth image. In this paper, we propose a height estimation method for directly predicting the height of objects from a 2D image. The proposed method utilizes an encoder-decoder network for pixel-wise dense prediction based on height consistency. We used the CARLA simulator to generate 40,000 training datasets from different positions in five areas within the simulator. The experimental results show that the object’s height map can be estimated regardless of the camera’s location.
A Semisupervised Concept Drift Adaptation via Prototype-Based Manifold Regularization Approach with Knowledge Transfer

Data stream mining deals with processing large amounts of data in nonstationary environments, where the relationship between the data and the labels often changes. Such dynamic relationships make it difficult to design a computationally efficient data stream processing algorithm that is also adaptable to the nonstationarity of the environment. To make the algorithm adaptable to the nonstationarity of the environment, concept drift detectors are attached to detect the changes in the environment by monitoring the error rates and adapting to the environment’s current state. Unfortunately, current approaches to adapt to environmental changes assume that the data stream is fully labeled. Assuming a fully labeled data stream is a flawed assumption as the labeling effort would be too impractical due to the rapid arrival and volume of the data. To address this issue, this study proposes to detect concept drift by anticipating a possible change in the true label in the high confidence prediction region. This study also proposes an ensemble-based concept drift adaptation approach that transfers reliable classifiers to the new concept. The significance of our proposed approach compared to the current baselines is that our approach does not use a performance measur as the drift signal or assume a change in data distribution when concept drift occurs. As a result, our proposed approach can detect concept drift when labeled data are scarce, even when the data distribution remains static. Based on the results, this proposed approach can detect concept drifts and fully supervised data stream mining approaches and performs well on mixed-severity concept drift datasets.
ABSTRACTION IN REFLECTION BY KISHAN MEENA: TANTRISM IN VISUALS (THE SOUL OF WATER)

The thinking of the modern artists created a vision of looking at artwork and new subjects in a new way. As a result, abstract art emerged as a feature of modern art. The artists present abstract art with his different perspective and medium. Photography is a modern and fast medium that is able to present the artist's thinking in a different way. Fine arts photography presents the viewer by shaping the artist's vision. The artist uses photography as a medium to create his own ideas. Photography has presented Tantra art as well by providing a visual look with modernity and digitally as well. Indian artists like Ajit Mukherjee, S.H. Raza, Biren De and Sohan Qadri have visually presented Indian mythology through painting as tantra art. We are going to talk about an artist Kishan Meena who took his Photography on another level of modernity in the Digital era. He has introduced the art of Tantra in the form of abstract art with the method of Digital photography and has nurtured the cycle of life through it.
Makespan and Security-Aware Workflow Scheduling for Cloud Service Cost Minimization Using Firefly Optimizer

As efficient commercial information technology, cloud computing has attracted more and more users to submit their requests to the cloud platform and pay for them based on the amount and quality of services. One of the most important concerns of cloud service providers is to minimize the monetary cost for completing users' requests while meeting constraints on the quality of services such as makespan and security. To solve this concern, this paper proposes a makespan and security-aware workflow scheduling method based on an improved firefly optimizer that consists of a new solution initialization scheme, a firefly position updating scheme, a firefly-to-solution mapping operator, and two task-to-virtual machine assignment schemes. We carry out extensive simulation experiments on real-world workflows to validate our proposed workflow scheduling method. The results reveal that the monetary cost reduction achieved by our method can be up to 54.0% compared to a baseline and two state-of-the-art approaches.
Heterogeneous Graph Based Long- And Short-Term Preference Learning Model for Next POI Recommendation

The next POI recommendation aiming at recommending the venues that people are likely interested in has become a popular service provided by location-based social networks such as Foursquare and Gowalla. Many existing methods attempt to improve the recommendation accuracy by modeling the long- and short-term preferences of people. However, these methods learn users’ preferences only from their own historical check-in records, which leads to bad recommendation performance in sparse dataset. To this end, we propose a novel approach named long- and short-term preference learning model based on heterogeneous graph convolution network and attention mechanism (LSPHGA) for next POI recommendation. Specifically, we design a heterogeneous graph convolution network to learn the higher-order structural relations between User-POI-Categories and obtain the long-term preferences of users. As for the short-term preference, we encode the recent check-in records of users through self-attention mechanism and aggregate the short-term preference by spatio-temporal attention. Finally, the long- and short-term preference is linearly combined into a unified preference with personalized weights for different users. Extensive experiments on two real-world datasets consistently validate the effectiveness of the proposed method for improving recommendation.
Chest X-ray Image Super-Resolution via Deep Contrast Consistent Feature Network

This paper proposes a chest X-ray image super-resolution reconstruction method - Deep Contrast Consistent Feature Network, which articulates the contrast consistency as a task to learn spatial contrast enhancement curve followed by a depth contrast network. The proposed method trains a network with visual quality measures through learning non-reference loss - Contrast Consistency Loss, which aims to overcome contrast overstretching and contrastive variation. Specifically, this network does not need to reference images at the time of loss formulation. We evaluated the proposed method for chest X-ray 5K datasets over several benchmarks metrics and models for quantitative and qualitative analysis. The extensive experiments report that the proposed model outperforms other approaches.
Optimizing Yinyang K-Means Algorithm on ARMv8 Many-Core CPUs

K-Means algorithm is one of the most common clustering algorithms widely applied in various data analysis applications. Yinyang K-Means algorithm is a popular enhanced K-Means algorithm that avoids most unnecessary calculations using triangle inequality. However, Yinyang K-Means algorithm is time-consuming when the problem size is large. Due to the influence of performance and energy-efficiency, ARM CPUs have appeared in high performance computing. Therefore, it is very interesting to accelerate Yinyang K-Means algorithm on ARM CPUs. In this paper, we propose an efficient parallel implementation of Yinyang K-Means algorithm on ARMv8 many-core CPUs by means of vectorization, NUMA affinity memory optimization and data layout optimization. The experiment on two ARMv8 many-core CPUs has shown that our implementation can achieve up to 5.6 times faster than the open-source multi-threaded one of Yinyang K-Means algorithm. To the best of our knowledge, this is the first work that studies the optimization of Yinyang K-Means algorithms on ARMv8 CPUs.
Analysis of changes in learning activities of university students' by comparing trends in web information search volume before and after the outbreak of COVID-19

In this study, the impact of un-tact learning on changes in learning activities in the 19-24 age group, mainly composed of university students', was analyzed by comparing the trend of web information search volume for physics terms before and after the COVID-19 outbreak. As the search terms for the research target, 'moment of inertia' and 'Fourier transform', which are basic concepts in the field of physics of the university, were selected, and the search volume information was data provided by Naver Data Lab. As a result of the analysis, it was observed that there was a unique repetitive search pattern for each search word before the COVID-19, and the deviation of web information per week was strong during the semester, but after the COVID-19, some such patterns changed and the deviation of search volume was relatively weakened. Periodic search volume deviation was defined as a kind of resonance phenomenon that occurs from active learning according to interactions between members, and changes in learning activities that appeared in university students' were interpreted based on changes in resonance phenomena that occurred after COVID-19. Based on this, future tasks that the instructor and the university should pay attention to were discussed.
Polya Decision Processes: A New History-Dependent Framework for Reinforcement Learning

We propose a new framework for sequential decision making, Polya Decision Processes (PDP); it can express the agent’s history-dependent transitions by using the Polya urn model. We show that PDP can be converted into a new type of Belief-MDP, whose belief update equation requires only urn model parameters. We introduce its theory, value iteration algorithm, and reinforcement learning algorithm for PDP using the belief state representation. Their effectiveness is confirmed by numerical experiments.
Partially Observable Markov Decision Subject to Total Variation Distance Ambiguity

In this paper we consider such Markov decision problems on finite spaces, when the probability distribution of the output conditioned on the sufficient statistic, belongs to a ball, with respect to the total variation distance metric, centered around a nominal conditional distribution. We formulate the Markov decision problem, as a minimax optimization problem in which the minimization is over the admissible set of control strategies, while the maximization is over the conditional distributions which lie in the total variation distance ball. Then we present a two-step procedure to determine the optimal control strategies. In step 1, we determine the maximizing conditional distribution, as a function of the nominal conditional distribution and the radius of the ball; it is defined on a finite partition of the observation space of aggregating the output states to form new states. In step 2, the minimax problem is transformed into a minimization problem over the control strategies, and a new dynamic programming equation is derived, where the conditional expectation is taken with respect to the maximizing conditional distribution. Surprisingly, although the current paper deals with partially observable Markov decision problems, the mathematical analysis is tractable.
Optimal convergence rates of totally asynchronous optimization

Asynchronous optimization algorithms are at the core of modern machine learning and resource allocation systems. However, most convergence results consider bounded information delays and several important algorithms lack guarantees when they operate under total asynchrony. In this paper, we derive explicit convergence rates for the proximal incremental aggregated gradient (PIAG) and the asynchronous block-coordinate descent (Async-BCD) methods under a specific model of total asynchrony, and show that the derived rates are order-optimal. The convergence bounds provide an insightful understanding of how the growth rate of the delays deteriorates the convergence times of the algorithms. Our theoretical findings are demonstrated by a numerical example.
On Using Hamiltonian Monte Carlo Sampling for RL

Q-Learning and other value function based reinforcement learning (RL) algorithms learn optimal policies from datasets of actions, rewards, and state transitions. However, generating independent and identically distributed (IID) data samples poses a significant challenge when the state transition dynamics are stochastic and high-dimensional; this is due to intractability of the associated normalizing integral. We address this challenge with Hamiltonian Monte Carlo (HMC) sampling since it offers a computationally tractable way to generate data for training RL algorithms in stochastic and high-dimensional contexts. We introduce Hamiltonian Q-Learning and use it to demonstrate, theoretically and empirically, that Q values can be learned from a dataset generated by HMC samples of actions, rewards, and state transitions. Hamiltonian Q-Learning also exploits underlying low-rank structure of the Q function using a matrix completion algorithm for reconstructing the Q function from Q value updates over a much smaller subset of state-action pairs. Thus, by providing an efficient way to apply Q-Learning in stochastic, high-dimensional settings, the proposed approach broadens the scope of RL algorithms for real-world applications.
Output-Feedback Path Planning with Robustness to State-Dependent Errors

We consider the problem of sample-based feed-back motion planning from measurements affected by systematic errors. Our previous work presented output feedback controllers that use measurements from landmarks in the environment to navigate through a cell-decomposable environment using duality, Control Lyapunov and Barrier Functions (CLF, CBF), and Linear Programming. In this paper, we build on this previous work with a novel strategy that allows the use of measurements affected by systematic errors in perceived depth (similarly to what might be generated by vision-based sensors), as opposed to accurate displacements measurements. As a result, our new method has the advantage of providing more robust performance (with quantitative guarantees) when inaccurate sensors are used. We test the proposed algorithm in the simulation to evaluate the performance limits of our approach predicted by our theoretical derivations.