Reviewerly/exHarmony huggingface dataset:
Video captioning based on vision transformer and reinforcement learning

Global encoding of visual features in video captioning is important for improving the description accuracy. In this paper, we propose a video captioning method that combines Vision Transformer (ViT) and reinforcement learning. Firstly, Resnet-152 and ResNeXt-101 are used to extract features from videos. Secondly, the encoding block of the ViT network is applied to encode video features. Thirdly, the encoded features are fed into a Long Short-Term Memory (LSTM) network to generate a video content description. Finally, the accuracy of video content description is further improved by fine-tuning reinforcement learning. We conducted experiments on the benchmark dataset MSR-VTT used for video captioning. The results show that compared with the current mainstream methods, the model in this paper has improved by 2.9%, 1.4%, 0.9% and 4.8% under the four evaluation indicators of LEU-4, METEOR, ROUGE-L and CIDEr-D, respectively.
VISIONE at Video Browser Showdown 2022

VISIONE is a content-based retrieval system that supports various search functionalities (text search, object/color-based search, semantic and visual similarity search, temporal search). It uses a full-text search engine as a search backend. In the latest version of our system, we modified the user interface, and we made some changes to the techniques used to analyze and search for videos.
Biometric Authentication for Intelligent and Privacy-Preserving Healthcare Systems

Secure identification is a critical system requirement for patients seeking health-related services. In the event of critical, aged, or disabled patients who require frequent health treatments, quick and easy identification is vital. Researchers describe the notion of the unprotected environment in this study, in which patients can receive health services from the hospital’s smart and intelligent surroundings without the use of explicit equipment. Patients would interact directly with the environment and be identified through it. We suggest a biometric-based authentication technique for the unprotected hospital environment that also safeguards the patient’s identity privacy. Furthermore, we demonstrate that this authentication technique is resistant to many well-known assaults, including insider attacks, replay attacks, and identity privacy. Doctors and other staff members showed enthusiastic responses after installing 2-factor authentications, as it makes their workflow efficient and makes things easier for patients. It also lets them focus on other factors rather than worrying about data security; hence, we need biometric authentication in intelligent and privacy-preserving healthcare systems. The paper deals with two-factor biometric authentication, and despite the added security, two-factor authentication adoption is said to be poor. It is due to a lack of awareness and difficulty to use and configure two-factor authentication (2FA) into a particular application by some individuals who struggle with the concept of authentication and its technology. Also, many 2FA methods in widespread use today have not been subjected to adequate usability testing. Research focuses on the point that there is still a large section of people unaware of the use of biometric systems to protect their online data. Researchers collected quantitative and qualitative data from 96 individuals during a two-week between-subjects usability survey of some common and rarely used 2FA approaches. The survey allowed the researcher to investigate which authentication methods are given higher priority and why, along with the relationship between different usage patterns and perceived usability, and identify user misconceptions and insecure habits to determine ease of use. It was observed that the biometric-based method was given the utmost preferability.
CMY Color Spaced-Based Visual Cryptography Scheme for Secret Sharing of Data

Visual cryptography technique allows visual information to be encrypted in such a way that the decrypted information appears as a visual image. Visual cryptography allows digital images to be separate into few shares called transparent shares. For security reasons, it ensures that hackers cannot find any clues about the secret image from a single cover image. The proposed technique uses CMY (cyan, magenta, yellow) color space to enhance visual cryptography, and mean visual cryptography can be used on color images. The other techniques used were color decomposition and error diffusion. Error diffusion is a type of half toning in which the quantization residual is distributed to neighboring pixels that have not yet been processed. The image will be encrypted by four shares which is cyan, magenta, yellow, and mask. Mask will be using random function to random generate half black and half white pixel in <math xmlns="http://www.w3.org/1998/Math/MathML" id="M1"> <mn>2</mn> <mo>×</mo> <mn>2</mn> </math> block. Stack all the shares for reviewing the secret image. After that, do the image preprocessing of the image and use OCR (optical character recognition) to recognize the message.
DeSmoke-LAP: improved unpaired image-to-image translation for desmoking in laparoscopic surgery

Robotic-assisted laparoscopic surgery has become the trend in medicine thanks to its convenience and lower risk of infection against traditional open surgery. However, the visibility during these procedures may severely deteriorate due to electrocauterisation which generates smoke in the operating cavity. This decreased visibility hinders the procedural time and surgical performance. Recent deep learning-based techniques have shown the potential for smoke and glare removal, but few targets laparoscopic videos.We propose DeSmoke-LAP, a new method for removing smoke from real robotic laparoscopic hysterectomy videos. The proposed method is based on the unpaired image-to-image cycle-consistent generative adversarial network in which two novel loss functions, namely, inter-channel discrepancies and dark channel prior, are integrated to facilitate smoke removal while maintaining the true semantics and illumination of the scene.DeSmoke-LAP is compared with several state-of-the-art desmoking methods qualitatively and quantitatively using referenceless image quality metrics on 10 laparoscopic hysterectomy videos through 5-fold cross-validation.DeSmoke-LAP outperformed existing methods and generated smoke-free images without applying ground truths (paired images) and atmospheric scattering model. This shows distinctive achievement in dehazing in surgery, even in scenarios with partial inhomogenenous smoke. Our code and hysterectomy dataset will be made publicly available at https://www.ucl.ac.uk/interventional-surgical-sciences/weiss-open-research/weiss-open-data-server/desmoke-lap .
Rapid and Label-Free Classification of Blood Leukocytes for Immune State Monitoring

A fully automated and label-free sample-to-answer white blood cell (WBC) cytometry platform for rapid immune state monitoring is demonstrated. The platform integrates (1) a WBC separation process using the multidimensional double spiral (MDDS) device and (2) an imaging process where images of the separated WBCs are captured and analyzed. Using the deep-learning-based image processing technique, we analyzed the captured bright-field images to classify the WBCs into their subtypes. Furthermore, in addition to cell classification, we can detect activation-induced morphological changes in WBCs for functional immune assessment, which could allow the early detection of various diseases. The integrated platform operates in a rapid (<30 min), fully automated, and label-free manner. The platform could provide a promising solution to future point-of-care WBC diagnostics applications.
Influence of an External Classical Field on a ♢ Four-Level Atom Inside a Quantized Field

In this paper, we study the effect of detuning parameters and driven external classical field parameters on a quantum system consisting of a ♢-configuration four-level atom inside a quantized cavity field. Under some canonical conditional of dressed states, the exact solution of the Schrödinger equation is obtained. The occupation of atomic levels and statistical population inversion is studied. Our results show that the classical field parameter dissolved the collapse periods and increased the maximum bounds of the upper state, while decreasing the lower bounds of the lower state. The detuning parameters reduce the minimum bounds of atomic levels and their inversion. On the other hand, the linear entropy and l1 norm of coherence are employed to measure the temporal evolution of the mixedness and coherence. It is found that the driven classical field improves the temporal evolution of the mixedness and lower bounds of coherence. However, the detuning parameters have a destructive effect on the mixedness and lower bounds of coherence. The intensity of the external classical field is regarded as a control parameter with different values of detuning parameters.
An embedded toolset for human activity monitoring in critical environments

In many working and recreational activities, there are scenarios where both individual and collective safety have to be constantly checked and properly signaled, as occurring in dangerous workplaces or during pandemic events like the recent COVID-19 disease. From wearing personal protective equipment to filling physical spaces with an adequate number of people, it is clear that a possibly automatic solution would help to check compliance with the established rules. Based on an off-the-shelf compact and low-cost hardware, we present a deployed real use-case embedded system capable of perceiving people’s behavior and aggregations and supervising the appliance of a set of rules relying on a configurable plug-in framework. Working on indoor and outdoor environments, we show that our implementation of counting people aggregations, measuring their reciprocal physical distances, and checking the proper usage of protective equipment is an effective yet open framework for monitoring human activities in critical conditions.
Silicon microring resonator based on all-optical reversible modified Fredkin gate: design and analysis

The conventional logic circuits dissipate energy into the environment due to the loss of information; hence these are termed irreversible logic circuits. The reversible logic (RL) circuits are capable of minimizing the power dissemination, quantum cost, and garbage outputs. The RL circuits based on all-optical technology have been adopted by many researchers in recent. The Fredkin gate is a significant RL gate. A modified Fredkin gate (MFG) made of a silicon all-optical microring resonator is realized in MATLAB. The MFG is capable of designing 16-Boolean functions with very small complexity. The figure of merit of the proposed MFG is achieved through numerical simulation. The parameters are selected from the determined specifications, and this could provide a practicable implementation of the proposed design.
Research on Face Recognition Technology Based on PCA and SVM

Face recognition is a hot topic in the field of artificial intelligence and pattern recognition, which is mainly conducive to machine learning algorithm to judge whether face samples belong to a certain category. Based on the discussion and analysis of PCA and SVM technology, this paper expounds the basic principle and process of face recognition, and puts forward face recognition method based on PCA and SVM. Finally, this method is used to recognize LFW (labeled faces in the wild) face data set, and the parameter adjuster named gridsearchcv is used to adjust the parameters to further improve the precision and recall of face recognition. The experimental results show that the face recognition technology based on PCA and SVM not only reduces the dimension of face data, but also increases the recognition accuracy, which is a feasible face recognition method.
CVSS-BERT: Explainable Natural Language Processing to Determine the Severity of a Computer Security Vulnerability from its Description

When a new computer security vulnerability is publicly disclosed, only a textual description of it is available. Cybersecurity experts later provide an analysis of the severity of the vulnerability using the Common Vulnerability Scoring System (CVSS). Specifically, the different characteristics of the vulnerability are summarized into a vector (consisting of a set of metrics), from which a severity score is computed. However, because of the high number of vulnerabilities disclosed everyday this process requires lot of manpower, and several days may pass before a vulnerability is analyzed. We propose to leverage recent advances in the field of Natural Language Processing (NLP) to determine the CVSS vector and the associated severity score of a vulnerability from its textual description in an explainable manner. To this purpose, we trained multiple BERT classifiers, one for each metric composing the CVSS vector. Experimental results show that our trained classifiers are able to determine the value of the metrics of the CVSS vector with high accuracy. The severity score computed from the predicted CVSS vector is also very close to the real severity score attributed by a human expert. For explainability purpose, gradient-based input saliency method was used to determine the most relevant input words for a given prediction made by our classifiers. Often, the top relevant words include terms in agreement with the rationales of a human cybersecurity expert, making the explanation comprehensible for end-users.
Graph Convolutional Networks with Dual Message Passing for Subgraph Isomorphism Counting and Matching

Graph neural networks (GNNs) and message passing neural networks (MPNNs) have been proven to be expressive for subgraph structures in many applications. Some applications in heterogeneous graphs require explicit edge modeling, such as subgraph isomorphism counting and matching. However, existing message passing mechanisms are not designed well in theory. In this paper, we start from a particular edge-to-vertex transform and exploit the isomorphism property in the edge-to-vertex dual graphs. We prove that searching isomorphisms on the original graph is equivalent to searching on its dual graph. Based on this observation, we propose dual message passing neural networks (DMPNNs) to enhance the substructure representation learning in an asynchronous way for subgraph isomorphism counting and matching as well as unsupervised node classification. Extensive experiments demonstrate the robust performance of DMPNNs by combining both node and edge representation learning in synthetic and real heterogeneous graphs.
Word2Vec: Optimal hyperparameters and their impact on natural language processing downstream tasks

Abstract Word2Vec is a prominent model for natural language processing tasks. Similar inspiration is found in distributed embeddings (word-vectors) in recent state-of-the-art deep neural networks. However, wrong combination of hyperparameters can produce embeddings with poor quality. The objective of this work is to empirically show that Word2Vec optimal combination of hyper-parameters exists and evaluate various combinations. We compare them with the publicly released, original Word2Vec embedding. Both intrinsic and extrinsic (downstream) evaluations are carried out, including named entity recognition and sentiment analysis. Our main contributions include showing that the best model is usually task-specific, high analogy scores do not necessarily correlate positively with F 1 scores, and performance is not dependent on data size alone. If ethical considerations to save time, energy, and the environment are made, then relatively smaller corpora may do just as well or even better in some cases. Increasing the dimension size of embeddings after a point leads to poor quality or performance. In addition, using a relatively small corpus, we obtain better WordSim scores, corresponding Spearman correlation, and better downstream performances (with significance tests) compared to the original model, which is trained on a 100 billion-word corpus.
Prediction and Influencing Factors of College Students' Career Planning Based on Big Data Mining

College students' career planning refers to the process of establishing their own career development goals based on the analysis of their own subjective and objective factors, searching for a career, and taking necessary actions to achieve their career goals. Career planning is an important aspect of human resource management, and it is a new trend in recent research to use the theory and method of psychology to study it. This project understands the current situation of contemporary college students' career planning through investigation and finds the common problems in career planning among college graduates. Systematic analysis of these problems can be used as the research premise to study college students' career planning, help college students to make correct self-assessment and career orientation, improve their self-awareness and evaluation ability, and form a good career self. In the research, the author found that, in the guidance of college students' career development direction, it is necessary to predict the future career development direction of college students. Effective prediction methods can provide objective and simple data support and theoretical basis for educators to help them implement education guidance smoothly. Therefore, this paper has carried out a series of research work. The author firstly analyzes the problems existing in the process of college career education, then analyzes the relevant theories and common methods of educational data mining and the dimension analysis of the influencing factors of college students' career development direction, so as to determine the research dimension of this paper, and then carries out the relevant research such as the modeling of prediction model, the implementation of algorithm, and the case test of prediction method. According to the above research, the author draws the following conclusions: the grey prediction algorithm has better performance in small- and medium-sized data prediction. Based on the comprehensive quality evaluation data, the prediction conclusion is more objective and accurate. However, there are still some shortcomings in the algorithm design in this study. In the future research, we can try to adjust the relevant parameters through more data and more experimental verification, so as to further optimize the algorithm.
SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning

Answering complex questions about images is an ambitious goal for machine intelligence, which requires a joint understanding of images, text, and commonsense knowledge, as well as a strong reasoning ability. Recently, multimodal Transformers have made a great progress in the task of Visual Commonsense Reasoning (VCR), by jointly understanding visual objects and text tokens through layers of cross-modality attention. However, these approaches do not utilize the rich structure of the scene and the interactions between objects which are essential in answering complex commonsense questions. We propose a Scene Graph Enhanced Image-Text Learning (SGEITL) framework to incorporate visual scene graph in commonsense reasoning. In order to exploit the scene graph structure, at the model structure level, we propose a multihop graph transformer for regularizing attention interaction among hops. As for pre-training, a scene-graph-aware pre-training method is proposed to leverage structure knowledge extracted in visual scene graph. Moreover, we introduce a method to train and generate domain relevant visual scene graph using textual annotations in a weakly-supervised manner. Extensive experiments on VCR and other tasks show significant performance boost compared with the state-of-the-art methods, and prove the efficacy of each proposed component.
A survey on effects of class imbalance in data pre-processing stage of classification problem

Classifier learning with datasets suffering from imbalance class distribution is a challenging task and it hinders the performance of machine learning algorithms. This imbalance occurs when a particular class is highly outnumbered than that of another class. Such kind of data distribution in the real world applications caught the attention of many researchers. This paper presents the review of various state of the art sampling techniques and ensemble techniques to resolve class imbalance. This paper also investigates the other factors such as threshold of distribution, inter or within class imbalance, etc., that make class imbalance a more complex issue. Comparisons of various approaches viz. data sampling, cost sensitive methods, bagging, boosting which alleviate the class imbalance problem are investigated in detail for their effects on class imbalance problem. Different parameters have been reviewed for measuring and evaluating the performance of the model. Accuracy is majorly used as evaluation parameter in machine learning problems, but from reviews it is found that there are different parameters such as precision, recall and AU-ROC which provide statistical measures for evaluating the model. The paper gives research directions in the domain of class imbalance problems.
Hybrid e-rehabilitation services: SMART-system for remote support of rehabilitation activities and services

One of the most effective solutions in medical rehabilitation assistance is remote patient / person-centered rehabilitation. Rehabilitation also needs effective methods for the "Physical therapist - Patient - Multidisciplinary team" system, including the statistical processing of large volumes of data. Therefore, along with the traditional means of rehabilitation, as part of the "Transdisciplinary intelligent information and analytical system for the rehabilitation processes support in a pandemic (TISP)" in this paper, we introduce and define: the basic concepts of the new hybrid e-rehabilitation notion and its fundamental foundations; the formalization concept of the new Smart-system for remote support of rehabilitation activities and services; and the methodological foundations for the use of services (UkrVectōrēs and vHealth) of the remote Patient / Person-centered Smart-system. The software implementation of the services of the Smart-system has been developed.
M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database

The emotional state of a speaker can be influenced by many different factors in dialogues, such as dialogue scene, dialogue topic, and interlocutor stimulus.The currently available data resources to support such multimodal affective analysis in dialogues are however limited in scale and diversity.In this work, we propose a Multi-modal Multi-scene Multi-label Emotional Dialogue dataset, M 3 ED, which contains 990 dyadic emotional dialogues from 56 different TV series, a total of 9,082 turns and 24,449 utterances.M 3 ED is annotated with 7 emotion categories (happy, surprise, sad, disgust, anger, fear, and neutral) at utterance level, and encompasses acoustic, visual, and textual modalities.To the best of our knowledge, M 3 ED is the first multimodal emotional dialogue dataset in Chinese.It is valuable for cross-culture emotion analysis and recognition.We apply several state-of-the-art methods on the M 3 ED dataset to verify the validity and quality of the dataset.We also propose a general Multimodal Dialogue-aware Interaction framework, MDI, to model the dialogue context for emotion recognition, which achieves comparable performance to the state-of-the-art methods on the M 3 ED.The full dataset and codes are available 1 .