Reviewerly/exHarmony huggingface dataset:
Research on Multimodal Human Behavior Recognition Based on Double Flow Network

How to use computer vision technology to automatically identify and analyze human behavior in video has become a research hotspot. In traditional behavior recognition methods, features need to be extracted manually, and the recognition effect of features largely depends on the experience of feature designers. This paper takes the dual-stream convolutional neural network as the basic theory, and uses the TSN (Temporal Segment Networks) model as the basic framework to analyze the shortcomings and shortcomings of the single-stream network and the original dual-stream network. A multi-modal human behavior recognition model based on dual-stream network is proposed. In order to extract video-level features effectively, this model adopts two attention mechanisms, which are used to learn image frame features and video-level feature transfer. Then, CNN is used to extract global motion features, and finally, it is fused with spatio-temporal features. The fusion feature is evaluated on the public data set, and the results show that the two features are complementary, and their fusion makes the features more expressive, and the recognition result on the public data set is greatly improved compared with the single spatio-temporal feature.
Event identification by deep transfer learning with dual transformations

We evaluated the modeling of dual transformations (DT) in neural networks on ranking microblogs for events. We showed the effectiveness of DT in comparing with single-direction transformations and dense modeling with more neural-networks layers. Further, we showed DT-based model was effective on implicit reference cases of real-world microblog data.
The Role of Key Words and the Use of the Multilingual Eurovoc Thesaurus when Searching for Legal Regulations of the Republic of Croatia - Research Results

As legislation today is one of the largest collections of documents in electronic form, access to such documents should be publicly available by making legal regulations accessible to both legal experts and all interested citizens, in a transparent way. The most comprehensive, free, and permanent access to legislation is provided by the Central State Office for the Development of the Digital Society through the CADIAL search engine. The collection of legal regulations in electronic form contains legal regulations of the Republic of Croatia published in the electronic edition of the Official Gazette. All regulations can be searched by full text, and the collection is organized as a system of interrelated texts of regulations and information about them. Unique content analysis of texts and carefully selected key words that describe the document allow the user to quickly search for documents. The Eurovoc Thesaurus enables the content of documents to be unambiguously displayed in a documentation system, and their metadata to be created in high quality. It also ensures the unambiguity of user queries addressed to the documentation and information system. The paper describes the results of a three-month user queries analysed through five case studies, with detailed processing of semantic queries by examining their correspondence with the words in the title, the words in the text of the regulation and the correspondence with Eurovoc descriptors. As part of the research, a survey of CADIAL search engine users' satisfaction was conducted.
Deep Reinforcement Learning based Compute-Intensive Workload Allocation in Data Centers with High Energy Efficiency

Recently the huge amount of energy consumption has become a barrier to the widespread deployment of data centers serving various Internet of Things applications. The reasonable allocation of compute-intensive workloads to physical servers is an efficient way to improve the data center's energy efficiency. Though existing works has proposed some algorithms to manage workloads or virtual machines for energy saving, most of them did not comprehensively consider the high dynamics of server states, and lacked in high scalability in their implementation. In this paper, the Actor Critic based Compute-Intensive Workload Allocation Scheme (AC-CIWAS) is proposed, which can both guarantee the Quality of Service (QoS) of workloads and reduce the computational energy consumption of physical servers. To achieve rational workload allocation, AC-CIWAS captures the dynamic feature of server states continuously, and takes the impact of different workloads on energy consumption into consideration. AC-CIWAS employs the Deep Reinforcement Learning (DRL) based Actor Critic (AC) algorithm to evaluate the expected cumulative return over time, while the cumulative return guides to allocate workloads with high energy efficiency. Simulation results have demonstrated that compared to existing baseline allocation methods, the proposed AC-CIWAS can achieve an approximately 20 percent decrease in server power consumption with QoS guarantee.
A Recommender System that safeguards the user privacy through Federated Learning

Nowadays, there are more and more legal and ethical limitations when it comes to managing personal data. This is a serious problem for recommender systems (RS) or similar aplications since personal data (e.g. socioeconomic, demographics, behavioural, etc.) are the most useful for generating tailored correlations and predictions. To cope with this issue, this work proposes a recommender system that safeguards the user privacy by using a Federated Learning approach (FL). To this end, this article takes as the baseline an already existing centralized RS that uses all the data from users in a clear manner. This baseline RS is based on Factor Machines and it aims to employ persuasion strategies adapted to the user to increase energy awareness and change their consumption habits in the work environment. In order to test the performance of the FL-based distributed RS, the real dataset used (N=678) have been separated into four subsets mimicking a segmentation by country of origin (Austria, UK, Spain and Greece). Each country can create an artificial intelligence model suitable for its users that will be sent to a central server where the aggregation of models will take place and the improved global model will be returned back to each country. The simulation of this FL strategy is performed with four Raspberry Pi's reflecting each country and an NVIDIA Jetson Nano is used as the aggregation server. The generated model not only increases the privacy of the users as no raw data travels to the central server but also improves the reliability of the recommendations.
Layered Model Aggregation based Federated Learning in Mobile Edge Networks

With the continuous improving of performance of the IoT and mobile devices, a new type of machine learning architecture, federated learning came into being. And there is also an increasing need to implement artificial intelligence frameworks on edge nodes. In this paper, we propose a federated learning system deployed in edge computing network, which realizes the server part in distributed form and uses layered model aggregation and dynamic topology to reduce bandwidth usage and time consuming. The experiment shows the effectiveness of federated learning algorithm in our system. And simulation results show that the time cost of our system increases logarithmically with the number of nodes rather than linearly in the traditional system.
Adaptive Video Transmission Strategy Based on Ising Machine

With the dramatically increasing video streaming in the total network traffic, it is critical to develop effective algorithms to ensure the quality of content delivery service. Adaptive bitrate (ABR) control is the most essential technique which determines the proper bitrate to be chosen based on network conditions, thus realize high-quality video streaming. In this paper, a novel ABR strategy is proposed based on Ising machine by using the quadratic unconstrained binary optimization (QUBO) method and Digital Annealer (DA) for the first time. The proposed method is evaluated by simulation with the real-world measured throughput and compared with other state-of-the-art methods. Experiment results show that the proposed QUBO-based method can outperform the existing methods, which demonstrating the superior of the proposed QUBO-based method.
Consistent Semantic Attacks on Optical Flow

We present a novel approach for semantically targeted adversarial attacks on Optical Flow. In such attacks the goal is to corrupt the flow predictions of a specific object category or instance. Usually, an attacker seeks to hide the adversarial perturbations in the input. However, a quick scan of the output reveals the attack. In contrast, our method helps to hide the attacker's intent in the output flow as well. We achieve this thanks to a regularization term that encourages off-target consistency. We perform extensive tests on leading optical flow models to demonstrate the benefits of our approach in both white-box and black-box settings. Also, we demonstrate the effectiveness of our attack on subsequent tasks that depend on the optical flow.
A Hierarchical Machine Learning Frame Work to Classify Breast Tissue for Identification of Cancer

In this work a study is conducted on the Breast tissues using machine learning algorithms to identify the Breast cancer. The paper proposes a hierarchical two stage classification frame work utilizing the features provided by the Breast tissue dataset of UCI machine learning database with SVM_RBF, kNN and Decision tree algorithms. The experimental results and their analysis at stage-1 binary classification indicated better performance of the method. Further the analysis under multi class classification specifies the robustness of the methodology by providing reduction in false alarm rates.
Monitor Learning for Virtual Classroom

Face recognition technology is widely used to identify or verify the subject through images or videos. There are various face recognition techniques that are developed in the past and used in various applications. The phenomenal extension and subsequent increase of use of technology and innovations has led to an increase in their use in higher education over the past few decades. The adaptation to the virtual classroom has increased in most educational institutions, especially after the spread of the Novel Coronavirus though there is failure to engage students in their learning. In this paper, a solution is proposed to monitor student(s) in virtual classrooms. “Monitor Learning for Virtual Classroom” detects the person(s) face from the camera or through the video and identifies them firstly which helps to ensure the presence of the person for attendance. A trained model detects facial expression of a person and also whether the subject is using any objects like mobile or not. This technique differentiates people in various conditions, light conditions as well in different posture. Using the microphone of the devices this system also detects the surrounding noise which helps to prepare a report. This solution deals with the current problem as well as in the future that help to ensure the quality of education for virtual classrooms, distant learning or some other form of remote learning.
Bayesian Covariance Representation with Global Informative Prior for 3D Action Recognition

For the merits of high-order statistics and Riemannian geometry, covariance matrix has become a generic feature representation for action recognition. An independent action can be represented by an empirical statistics over all of its pose samples. Two major problems of covariance include the following: (1) it is prone to be singular so that actions fail to be represented properly, and (2) it is short of global action/pose-aware information so that expressive and discriminative power is limited. In this article, we propose a novel Bayesian covariance representation by a prior regularization method to solve the preceding problems. Specifically, covariance is viewed as a parametric maximum likelihood estimate of Gaussian distribution over local poses from an independent action. Then, a Global Informative Prior (GIP) is generated over global poses with sufficient statistics to regularize covariance. In this way, (1) singularity is greatly relieved due to sufficient statistics, (2) global pose information of GIP makes Bayesian covariance theoretically equivalent to a saliency weighting covariance over global action poses so that discriminative characteristics of actions can be represented more clearly. Experimental results show that our Bayesian covariance with GIP efficiently improves the performance of action recognition. In some databases, it outperforms the state-of-the-art variant methods that are based on kernels, temporal-order structures, and saliency weighting attentions, among others.
Experiment on Handwriting Generation with Recurrent Neural Networks using Small Datasets

This paper examines the performance of a specific Long-Short Term Memory Recurrent Neural Network for cursive handwriting generation when training with small datasets. The RNN can generate complex structure sequences by predicting one data point at a time. Then, by predicting the overall writing structure, the handwriting can be synthesized. The resulting network can generate different handwriting style parameters.
A Novel Method for Color Forged Image Detection

Digital images permeate almost all areas of our lives, mainly in news media, scientific discoveries, medical imaging, and judicial evidence. However, in recent years, due to the wide application of deep learning in image processing technology, these technologies or software have not only brought convenience to people, but also made it easier for people to forge or tamper with digital images without leaving any traces. The authenticity of digital images has been affected. A serious threat to modify and threaten. These forged or tampered images will bring serious threats to judicial justice, social stability, and the medical industry, and cause huge negative effects. Therefore, this article proposes an authenticity detection algorithm for generating color forged images based on deep learning. The corresponding color channel features of the real and forged image datasets are extracted and FIsher encoded, respectively, and the encoded color channel features are used to train the SVM model. Experiments prove that our proposed method achieves better results in detecting image color tampering.
The time-dependent polaron frame

The single-photon emitters in solid-state photonic devices have become crucial for quantum technology implementations. The exciton–phonon interaction degrades the quality of the photons even when the coherent scattering is dominant. Some models describing an emitter’s trajectory in a phonon bath are the weak-coupling master equation and the standard and variational polaron master equations. The polaron theory based on a steady polaron transformation cannot capture the response of the phonon reservoir upon the exciton creation in its transformed Hamiltonian. Here, a time-dependent phonon displacement leads to the time evolution of the phonon-related energies and correlations. Variables as the phonon-induced energy shift and the phonon first-order correlation function converge to the standard polaron theory results. However, the expectation value of the phonon displacement operator differs. According to the time-dependent polaron master equation, the mitigation of the coupling between the single-photon emitter and the coherent driving field by the phonon reservoir evolves to be worse than predicted in the standard polaron theory. • A time-dependent polaron unitary transformation is derived • Time evolution of the exciton–phonon interaction • The path from the weak-coupling Hamiltonian to the polaron Hamiltonian
Plaques Inpainting Algorithm Based on Edge-connect Network

Plaques of the old films are common damage. Due to the randomness of the shape and distribution of the plaques, the mask image obtained by making the difference according to the reference frame contains the plaques area and the local motion area. It is difficult to inpainting texture information in the local motion area, and traditional plaques inpainting algorithms cannot meet the inpainting requirements. The edge-connect network is used for plaques inpainting, the image completion network is adjusted to the single-channel model, and the old film data set training network is made. The comparative experiment shows that the edge-connect network can inpainting the plaques. At the same time, the texture information restoration of the motion area is better than the traditional algorithm, which can meet the restoration needs of old films.
Client-Based Secure IoT Data Sharing using Untrusted Clouds

IoT systems commonly rely on cloud services. However, utilizing cloud providers can be problematic in terms of data security. Data stored in the cloud need to be secured from unauthorized malicious nodes and from the cloud providers themselves. Using a simple symmetric cipher can encrypt the data before uploading and decrypt it while retrieving. However, such a solution can be only applied between two parties with no support for multiple nodes. Whereas in IoT scenarios, many smart devices communicate and share data with each other. This paper proposes a solution that tackles the issue of sharing data securely between IoT devices by implementing a system that allows secure sharing of encrypted data in untrusted clouds. The implementation of the system performs the computation on connectionless clients with no involvement of the cloud server nor any third party. The cloud server is only used as a passive storage server. Analysis of the implemented prototype demonstrates that the system can be used in real-life applications with relatively small overhead. Based on the used hardware, key generation takes about 60 nanoseconds and the storage overhead is only a few kilobytes for large number of files and/or users.
“How to best say it?” : Translating Directives in Machine Language into Natural Language in the Blocks World

We propose a method to generate optimal natural language for block placement directives generated by a machine’s planner during human-agent interactions in the blocks world. A non user-friendly machine directive, e.g., move(ObjId, toPos), is transformed into visually and contextually grounded referring expressions that are much easier for the user to comprehend. We describe an algorithm that progressively and generatively transforms the machine’s directive in ECI (Elementary Composable Ideas)-space, generating many alternative versions of the directive. We then define a cost function to evaluate the ease of comprehension of these alternatives and select the best option. The parameters for this cost function were derived empirically from a user study that measured utterance-to-action timings.
Industry 4.0 and Supply Chain Performance in the Oil and Gas Industry

The purpose of this study is to investigate industry 4.0 and supply chain performance. Extant literature was reviewed. Secondary data such as E-books, journal articles projects were used. Exploratory research design was used for the study. Based on the review literature, it was revealed that, integrated supply chain performance measurement system (SCPMS) is paramount for an efficient supply chain management (SCM) at inter-organizational and beyond-the-boundary processes. The recent technological innovations (internet of things (IoTs), big data, and web-based communication systems) revamp the SCPMS through effective data collection, information sharing, and framework integration among the different SC partners across nations. The study recommend for further research on the obstacles facing industry 4.0 and supply chain performance in the oil and gas industries across nations.
Multi branch deepfake detection based on double attention mechanism

With the continuous development of artificial face technology, it is more and more difficult to distinguish true and false faces with naked eyes. This paper proposes a multi branch artifact detection algorithm based on double attention mechanism, which can detect subtle artifacts. Firstly, dlib is used for face detection, and the local images of eyes, nose and mouth are segmented. A multi branch detection network model based on double attention mechanism is proposed. The model fully learns the context semantic information of local artifacts and global artifacts. The performance of the proposed method is evaluated on the deepfake data set, and the experimental results show that the test accuracy of the proposed method reaches 96.45%. At the same time, compared with other methods, this method also has a good performance in anti compression test.
Understanding Sentiments on Corona Vaccine using Social Media Analysis

Covid-19 crossed all international, national, regional boundaries so declared as Pandemic across the globe. Since one and half year every country is struggling to deal with this disease either by using preventive measures like lockdown, sanitization or with cure measure like treatment, isolation etc. Most important preventive measure adopted by every country now is Vaccination. But due to confusion, uncertainty and lack of awareness people have different opinion and sentiment about the vaccination drive. There is a drastic requirement to understand the People's sentiments on the vaccine, which may vary for different regions. Especially when these sentiments became vocal on social media, it may create adverse impact on the Vaccination Drive. This paper discusses predictive analytics and visualization methods for analyzing and predicting the summarized sentiments using social media channel: twitter. This approach is useful to provide support the vaccination drive by spreading awareness about the vaccine in the region with extreme negative sentiments. Corona Vaccine related tweets are analyzed and classified using various classification techniques as Bernoulli Naïve Bayes, Linear SVC and Random Forest. Visualization techniques are discussed to showcase summarized sentiments region-wise. This approach is effective and useful for support for vaccination drive for Covid-19 spread prevention
Quadratic Convex Reformulation for Solving Task Assignment Problem with Continuous Hopfield Network

This research is an optimal allocation of tasks to processors in order to minimize the total costs of execution and communication. This problem is called the Task Assignment Problem (TAP) with nonuniform communication costs. To solve the latter, the first step concerns the formulation of the problem by an equivalent zero-one quadratic program with a convex objective function using a convexification technique, based on the smallest eigenvalue. The second step concerns the application of the Continuous Hopfield Network (CHN) to solve the obtained problem. The calculation results are presented for the instances from the literature, compared to solutions obtained both the CPLEX solver and by the heuristic genetic algorithm, and show an improvement in the results obtained by applying only the CHN algorithm. We can see that the proposed approach evaluates the efficiency of the theoretical results and achieves the optimal solutions in a short calculation time.