Reviewerly/exHarmony huggingface dataset:
Research on K-means Clustering Algorithm Based on MapReduce Distributed Programming Framework

As a classical clustering algorithm, K-means algorithm has a profound research background. In the of big data era, K-means algorithms will play a greater advantage, being able to quickly divide similar data into the same cluster. Combining K-means algorithm with MapReduce distributed computing framework and running on Hadoop big data platform can significantly improve the clustering effect. Based on MapReduce framework structure, this paper studies K-means model, including K-means principle, distance calculation, content validity index and external validity index. On this basis, the K-means clustering flow based on MapReduce big data programming framework is proposed, and the execution process of the algorithm flow is described in detail, which provides a guide for the algorithm implementation.
Online education based on distributed multi-layer data processing technology

In order to promote the intelligent development of online education and optimize the status quo of online education, an online education method based on distributed multi-layer data processing technology is proposed. The method integrates the database information and builds the BTREE index by calculating the distributed database resource information, the initial information sequence, the main information set and the trade-off coefficient of information attributes. The bidirectional index method is used to extract the core data segment of integrated data and establish the time series table. The schedule is used to constrain the data twice, and the coincidence degree of the initial data transmission path is compared with the actual path parameters after evaluation, so as to realize the automatic matching of the data transmission path. The experimental results show that compared with the traditional transmission path matching method, the designed automatic matching method improves the matching accuracy of the complete data transmission path by 24% and the broken data transmission path by 19%, which has practical advantages. It is proved that this method can promote the precise personality teaching of online education, so as to achieve the effect of optimizing online education. It can also provide reference for the intelligent development of other online education.
Big Data Architecture Design of Smart Grid Based On Big Data Platform

A novel big data-driven approach has been proposed to address the challenges of extensive data and user application difficulties in the smart grid. A smart grid big data architecture based on a big data platform has been designed. The random matrix theory is applied to establish a random matrix algorithm model, which samples data from the smart grid big database and undergoes training and learning to construct a data model for user requirements. Through calculations, the macroscopic data hidden in the smart grid big data is transformed into microscopic data for users' reference, enabling the identification of parameters that affect the normal operation of the smart grid at its core. The computed data can be displayed locally and also uploaded remotely to the SG186 marketing system for various user applications. Additionally, the data can be permanently stored in the cloud through wireless communication. By incorporating the random matrix theory algorithm into the smart grid big data architecture, the proposed solution not only enhances the intuitive visualization of smart grid big data but also provides technical references for future work.
English Machine translation System Based on Neural Network Algorithm

In order to solve the problems of machine translation efficiency and translation quality, this paper proposes a research method of machine English translation evaluation system based on BP neural network algorithm. This method provides users with a more intelligent machine translation service experience. With the help of BP neural network algorithm, taking English online translation as the research object, Google's translation quality is the best, with an error frequency of only 167, while Baidu translation and iFLYTEK translation in China have a high error rate of 266 and 301 respectively, which is much higher than Google translation. A model of machine translation evaluation system based on neural network algorithm is proposed to better solve the disadvantages of traditional English machine translation. The results show that the machine translation system based on neural network algorithm can further optimize the problems existing in machine translation, such as insufficient use of information and large scale of model parameters, and further improve the performance of neural network machine translation.
Design and Implementation of Digitization Resource Sharing System Based on Hadoop Platform

The basic idea of Hadoop is to use multiple independently working computers to form a cluster of computers that can be stored and calculated logically, and do not require other expensive facilities to run, but integrate cheap equipment together. Hadoop technology is produced in the context of big data, to achieve the mass storage of big data, the most prominent advantage is the cost advantage. Based on Hadoop architecture, this paper designs the LZW compression algorithm for improving the storage efficiency of teaching resources, including the algorithm principle and algorithm flow. The system functions of resource collection, storage, management, migration and retrieval are designed. and designed the function module of each system. The function modules of the system form a whole system through parameter transfer, which has low coupling property. The research results assist the construction of an integrated sharing system of storage, management, transfer and retrieval of teaching resources to meet the demands of teaching resource sharing in the era of big data.
Development of Network Public Opinion Analysis System in Big Data Environment Based On Hadoop Architecture

The rapid development of the Internet has given individuals the freedom to fully express their own opinions and value demands, and also accelerated the occurrence and evolution of online public opinions. In essence, online public opinion reflects social conditions and public opinions. In the era of big data, various kinds of data generated by the Internet every day are rapidly growing at the scale of PB level, which poses new challenges to the collation and analysis of online public opinion information. In order to comprehensively improve the authenticity and comprehensiveness of online public opinion governance, realize scientific guidance of public opinion, and optimize the way of online public opinion governance, this paper develops an online public opinion analysis system in the big data environment based on Hadoop architecture, and adopts data mining technology to analyze public opinion in three stages before, during and after the event. It evaluates the risks, predicts the development trend, puts forward early warning in time, and provides theoretical guidance for the governance reform of online public opinions.
Analysis of Spatio-temporal Behavior of Research Travelers Based on Spark Big Data Computing Model

There are still many defects in the process of data processing, storage and analysis, which cannot adapt to the needs of big data processing and analysis. In order to more fully reflect the value of big data processing, it is necessary to realize efficient data processing. Therefore, people invented the Spark operating system and integrated the idea of SIMBA, built a big data analysis system, and searched the information through the database query method. In Spark operating system can import search mechanism, in order to improve the search effect of information, through the establishment of segment tree storage method can improve the retrieval effect, in big data analysis can use partition strategy method for big data partition, and adopted the global filtering and local search method, make in information retrieval can achieve high throughput and low delay performance.
Network information data processing method based on artificial intelligence

Because of the complexity and development of network information data processing, many conventional technical means can not process information data, and the artificial itself is powerless, so it needs to use artificial intelligence technology for processing. In view of this, this paper will introduce the basic concept of artificial intelligence technology and its advantages in information data processing, and then combine the technology to put forward the network information data processing method. Through the research, the method in this paper can give full play to the role of artificial intelligence technology, fast and in-depth processing of network information data, and can give artificial decision support.
Electric power marketing system network security protection technical scheme

Electric power marketing system to electric power group provides the convenience, makes the power organization economic gains and also solve the many problems, but with the deepening of the application system, electric power group met with a lot of network attacks, shows that the system is under a state of risk, if attacked, the key information of power marketing may be leaked, resulting in economic and other losses. Therefore, in order to better protect the electric power marketing system, this paper has carried on the research. This paper introduces the current situation of network security protection of electric power marketing system, and then puts forward the protection technology scheme. The research shows that the technical scheme in this paper has better protection intensity and can protect the safety of electric power marketing system more effectively.
Software Entity Automated Visual Recognition Method Based on Deep Learning Algorithm

Automated visual recognition is a process that utilizes computer vision technology to analyze, process, and recognize images and videos, thereby achieving automation. It is an important branch of computer vision technology, bringing great convenience to people's production and life. This article mainly studies software entity automated visual recognition methods based on deep learning algorithms. Combining image processing technology and deep learning theory, a visual recognition method based on convolutional neural networks is proposed. On this basis, the advantages and disadvantages of this method were analyzed, and improvement suggestions were proposed to address its shortcomings. The experimental results show that the visual recognition method based on deep learning algorithm exhibits excellent performance in terms of accuracy and robustness.
Developing a Car Object Detection Module for an Autonomous-driving System

This study compares the performance of YOLOv5 and R-CNN, two state-of-the-art object detection models, in car detection tasks. Both models were trained and evaluated on the same ”Car Object Detection” dataset. The YOLOv5 model leveraged transfer learning, while the R-CNN model used the Selective Search algorithm for region proposals. The paper provides insights for choosing an appropriate object detection model for specific applications.
Classify Human Activities Based On Deepforest

One important aspect of human behavior understanding is the recognition and monitoring of daily activities. An accurate activity recognition system can improve the quality of life in many key areas. The multi-metric feature extraction and DeepForest classifier designed in this paper effectively had solved the problems of incomprehensive feature extraction and insufficient classifier accuracy. First, this paper extracts a total of 450 dimensional feature vectors using mean, variance, maximum, skewness, minimum, kurtosis, regression of independent variables and sample entropy as feature indicators, so that the features of the original vectors can be presented comprehensively. The dimensionality of the feature set is too large, so PCA is used to reduce the dimensionality of the high-dimensional feature vector. After studying the relationship between the dimensionality reduction and the cumulative contribution rate, it is concluded that the cumulative contribution rate reaches 90% when the dimensionality is 15, which can retain the original features better. Then DeepForest was used as the classifier, and the reduced-dimensional feature set was used as the sample set to divide the training set and test set by 3:1. The model was tested on the test set after training, and the classification accuracy of the test set was 98.202%. Six classifiers, GaussianNB, SVM, K-NN, XGBoost, RandomForest and DecisionTree, were selected as the control experimental group, of which only the RandomForest model reached 97%, while the rest of the control models did not achieve 95% effect, indicating that DeepForest was more accurate in classifying human activities. Next, the generalisation ability of the DeepForest classifier was assessed using Monte Carlo Cross Validation (MCCV), K-fold and its confusion matrix. The MCCV validation used 50% of the data as the training set and 30% of the data as the test set, and set the number of splits to 10, resulting in a mean accuracy of 98.227%. The size of K in the K-fold validation was determined to be 8 based on the number of people conducting the human activity experiment, and the final mean accuracy value obtained was 97.790%. The combined confusion matrix from the K-fold validation (which aggregates the confusion matrix for each classification result) was calculated, and the results showed that the highest accuracy reached 100% for A9 and A14 classifications, where A5 and A10, A15 and A10, A12 and A5 were more likely to be confused, with the highest error rate for A5 classification, which was 4.167%, and the rest of the activity classifications were better.
Evaluation on Fine-grained Cross Modal Image and Text Retrieval Method Based on Deep Adversarial Hashing Technology

Fine-grained cross modal image and text retrieval is one of the hot topics in recent years, with the goal of achieving efficient and accurate image and text retrieval in different modal datasets. In this paper, a new type of cross-modal graphic retrieval method based on deep adversarial hashing is used to learn the feature representation of images and text through deep network, and realize the feature hash representation through adversarial training. The adversary network is introduced into the network structure. By maximizing the mutual information of hash coding and minimizing the anti classification loss of hash coding, the feature representation of image and text is highly consistent and separable in the hash coding space. Experiments have proved that compared with the existing algorithm, this algorithm can be retrieved better.
Design and implementation of English Teaching Analysis System based on mobile terminal

In order to complete the design of English teaching analysis system on mobile terminal and improve the quality of English teaching, this paper will carry out relevant research. This paper first introduces the design significance of the English teaching analysis system, and then puts forward the system design scheme and implementation method. Through this study, the design of English teaching analysis system on mobile terminal can effectively improve the quality of teaching, indicating that the system has certain application value.
The Construction and Application of an Automatic Scoring System for English Writing

This article utilizes big data technology to design an automatic scoring system for English writing. The research of this thesis specifically answers the following questions: (1) whether the automatic scoring system can effectively improve university student' English writing ability. Which is better to use only traditional teacher feedback or a combination of automatic scoring system and teachers in English teaching feedback? (2) How does the automatic scoring system affect university student? (3) Do students prefer teacher feedback or a combination of automatic scoring system and teacher feedback? Through pre-test and post test, pre-test and post test questionnaires, and interview research methods, the experiment analyzes the impact and changes of writing feedback teaching combined with automatic scoring system and teachers on students' writing learning. In the process of the experiment, two classes of university were selected as the objects of the experiment. The experimental results show that 93% of the students in class A and 92% of the students in class B scored above 90 points at ordinary times. The two data values are similar. The students in both classes have a certain English foundation and the English foundation is similar. More than half of the students can well understand the writing feedback provided by the website. Most of the students held a positive attitude towards the feedback of the automatic scoring system to promote writing revision. The system builds a teaching model with complementary resource advantages to maximize the benefit of feedback teaching in English writing.
Computer Graphics Algorithm Demonstration System

With the rapid development of computer technology. Many scholars pay close attention to computer graphics algorithms, which makes computer graphics widely used in various fields and enters a new revolutionary era. The purpose of this paper is to study the evaluation of computer graphics algorithm demonstration systems. Combining theory with practice, on the basis of careful analysis of the principle of graphics algorithm, a new idea of synchronization of graphics production process and a specific algorithm for realizing graphics are proposed, and most graphics in computer graphics are visualized. It breaks the limitation that only two-dimensional graphic simulation effects can be realized, and increases the interactive user control system. The algorithm complexity is significantly reduced, the graphics creation speed is increased, and the simulation effect is more realistic. After the introduction of this visual teaching demonstration system, these abstract principles can be explained intuitively, which can significantly improve the teaching quality.
Distributed Computer Function Assisted Evaluation Based on Virtual Reality

Distributed computers have been widely used in various fields in today's society, including military and aviation. It has many advantages, but its shortcomings cannot be ignored. This article used virtual reality technology to upgrade and improve distributed computers, while conducting auxiliary evaluations from three aspects: compatibility, structural integrity, and security. Through experimental calculations, it was found that after using virtual reality technology to upgrade distributed computers, the average compatibility reached 96%, which was 8.6% higher than the previous 87.4%. The experimental results indicated that virtual reality technology had a very significant improvement effect on distributed computers.
Case Study of Ant Group's "Double Link" Financing Based On Blockchain Technology

Through the investigation of the representative enterprises of supply chain finance, the operation and service mode, development status and development bottleneck of supply chain finance are analyzed. The "blockchain + supply chain financing" mode of Ant Group is discussed at the theoretical level by taking the "double link" financing mode as an example, and the underlying framework is built with the help of blockchain technology. Enable supply chain finance service providers to establish a comprehensive service platform for all participants in the supply chain, and solve the difficulties of supply chain finance in supporting the financing of small and medium-sized enterprises through the combined application of multiple functional modules, sum up experience, and put forward optimization ideas and strategies. In order to enrich the research content of supply chain financing optimization and innovation, and improve the theoretical system of "blockchain + supply chain financing".
Risk-sensitive Markov Decision Process and Learning under General Utility Functions

Reinforcement Learning (RL) has gained substantial attention across diverse application domains and theoretical investigations. Existing literature on RL theory largely focuses on risk-neutral settings where the decision-maker learns to maximize the expected cumulative reward. However, in practical scenarios such as portfolio management and e-commerce recommendations, decision-makers often persist in heterogeneous risk preferences subject to outcome uncertainties, which can not be well-captured by the risk-neural framework. Incorporating these preferences can be approached through utility theory, yet the development of risk-sensitive RL under general utility functions remains an open question for theoretical exploration.In this paper, we consider a scenario where the decision-maker seeks to optimize a general utility function of the cumulative reward in the framework of a Markov decision process (MDP). To facilitate the Dynamic Programming Principle and Bellman equation, we enlarge the state space with an additional dimension that accounts for the cumulative reward. We propose a discretized approximation scheme to the MDP under enlarged state space, which is tractable and key for algorithmic design. We then propose a modified value iteration algorithm that employs an epsilon-covering over the space of cumulative reward. In the case with access to a simulator, we demonstrate that our algorithm can efficiently identify a near-optimal policy witha sample complexity of O(H7SAκ^2/ε^2), where S represents the number of states, A the numberof actions, H the length of an episode, and κ the Lipschitz coefficient of the utility function. In the case without an accessible simulator, our algorithm utilizes an upper-confidence-boundtype of exploration approach, achieving a regret order of O(\sqrt{H^4S^2ATκ(λ+η)}), where Trepresents the total number of time steps, and λ and η are the regularity constants of the reward distribution. For both of the algorithms, we match the theoretical lower bounds for the risk-neutral setting in terms of ε and T, respectively.
The Copyright, Text &amp;amp; Data Mining and the Innovation dimension of Generative AI

The rise of the Generative AI has raised many a questions from the perspective of copyright. From the lens of copyright and database rights, questions revolve not only around the authorship of these AI-generated works, but also the very process that leads to the generation of these works, namely the process of text and data mining (TDM). Does TDM infringe the economic rights of the right holders? How does the TDM-debate transform and transmute in the age of Generative AI? Generative AI-tools create works that substitute the content creators whose very work that they learn from, and successively improvise themselves with every sequential iteration. Generative AI, thus, also presents a much larger question as they substitute the romaticised human author that sits at the center of copyright. In addition, as Generative AI tools, such as ChatGPT, can now also crawl the web, questions thus, transcends the frontiers of copyright, and touch upon the issue of innovation and competition in the market for web browsers. This research article contemplates on some of these issues, and makes some recommendations to creative a balancing framework, whereby incentives to innovate are preserved, and the interests of the human author are suitably safeguarded in the age of text and data mining and Generative AI.