Reviewerly/exHarmony huggingface dataset:
Large Scale Holistic Video Understanding

Video recognition has been advanced in recent years by benchmarks with rich annotations. However, research is still mainly limited to human action or sports recognition - focusing on a highly specific video understanding task and thus leaving a significant gap towards describing the overall content of a video. We fill this gap by presenting a large-scale "Holistic Video Understanding Dataset"~(HVU). HVU is organized hierarchically in a semantic taxonomy that focuses on multi-label and multi-task video understanding as a comprehensive problem that encompasses the recognition of multiple semantic aspects in the dynamic scene. HVU contains approx.~572k videos in total with 9 million annotations for training, validation, and test set spanning over 3142 labels. HVU encompasses semantic aspects defined on categories of scenes, objects, actions, events, attributes, and concepts which naturally captures the real-world scenarios. We demonstrate the generalization capability of HVU on three challenging tasks: 1.) Video classification, 2.) Video captioning and 3.) Video clustering tasks. In particular for video classification, we introduce a new spatio-temporal deep neural network architecture called "Holistic Appearance and Temporal Network"~(HATNet) that builds on fusing 2D and 3D architectures into one by combining intermediate representations of appearance and temporal cues. HATNet focuses on the multi-label and multi-task learning problem and is trained in an end-to-end manner. Via our experiments, we validate the idea that holistic representation learning is complementary, and can play a key role in enabling many real-world applications.
Video Object Segmentation using Space-Time Memory Networks

We propose a novel solution for semi-supervised video object segmentation. By the nature of the problem, available cues (e.g. video frame(s) with object masks) become richer with the intermediate predictions. However, the existing methods are unable to fully exploit this rich source of information. We resolve the issue by leveraging memory networks and learn to read relevant information from all available sources. In our framework, the past frames with object masks form an external memory, and the current frame as the query is segmented using the mask information in the memory. Specifically, the query and the memory are densely matched in the feature space, covering all the space-time pixel locations in a feed-forward fashion. Contrast to the previous approaches, the abundant use of the guidance information allows us to better handle the challenges such as appearance changes and occlussions. We validate our method on the latest benchmark sets and achieved the state-of-the-art performance (overall score of 79.4 on Youtube-VOS val set, J of 88.7 and 79.2 on DAVIS 2016/2017 val set respectively) while having a fast runtime (0.16 second/frame on DAVIS 2016 val set).
An Image Encryption Algorithm Based on Improved Lifting-Like Structure and Cross-Plane Zigzag Transform

With the increasing development of information technology, image information security has received more and more attention.Furthermore, image encryption is one of the most powerful methods to ensure image information security and plays a vital role in modern society.This paper proposes an image encryption algorithm based on an improved lifting-like structure and cross-plane zigzag transform.First, to overcome the problem of simple original lifting-like structure and single update function.We modularize the original image and add new update functions and prediction functions to handle the information between different modules.Our proposed structure has higher encryption quality and can better spread the information between each image pixel.Secondly, we propose a cross-plane zigzag transform, which can effectively eliminate the correlation between adjacent elements of the original image by applying the zigzag transform to the replacement between different planes, overcoming the disadvantage that the traditional zigzag transform is limited in scanning mode and challenging to resist brute-force attack.The experimental results show that compared with other state-of-the-art schemes, this scheme has higher information entropy, can well resist selective plaintext attacks and differential attacks, and has better security.
On Measuring Social Biases in Sentence Encoders

The Word Embedding Association Test shows that GloVe and word2vec word embeddings exhibit human-like implicit biases based on gender, race, and other social constructs (Caliskan et al., 2017). Meanwhile, research on learning reusable text representations has begun to explore sentence-level texts, with some sentence encoders seeing enthusiastic adoption. Accordingly, we extend the Word Embedding Association Test to measure bias in sentence encoders. We then test several sentence encoders, including state-of-the-art methods such as ELMo and BERT, for the social biases studied in prior work and two important biases that are difficult or impossible to test at the word level. We observe mixed results including suspicious patterns of sensitivity that suggest the test's assumptions may not hold in general. We conclude by proposing directions for future work on measuring bias in sentence encoders.
Explaining individual predictions when features are dependent: More accurate approximations to Shapley values

Explaining complex or seemingly simple machine learning models is an important practical problem. We want to explain individual predictions from a complex machine learning model by learning simple, interpretable explanations. Shapley values is a game theoretic concept that can be used for this purpose. The Shapley value framework has a series of desirable theoretical properties, and can in principle handle any predictive model. Kernel SHAP is a computationally efficient approximation to Shapley values in higher dimensions. Like several other existing methods, this approach assumes that the features are independent, which may give very wrong explanations. This is the case even if a simple linear model is used for predictions. In this paper, we extend the Kernel SHAP method to handle dependent features. We provide several examples of linear and non-linear models with various degrees of feature dependence, where our method gives more accurate approximations to the true Shapley values. We also propose a method for aggregating individual Shapley values, such that the prediction can be explained by groups of dependent variables.
Learning with Sets in Multiple Instance Regression Applied to Remote Sensing

In this paper, we propose a novel approach to tackle the multiple instance regression (MIR) problem. This problem arises when the data is a collection of bags, where each bag is made of multiple instances corresponding to the same unique real-valued label. Our goal is to train a regression model which maps the instances of an unseen bag to its unique label. This MIR setting is common to remote sensing applications where there is high variability in the measurements and low geographical variability in the quantity being estimated. Our approach, in contrast to most competing methods, does not make the assumption that there exists a prime instance responsible for the label in each bag. Instead, we treat each bag as a set (i.e, an unordered sequence) of instances and learn to map each bag to its unique label by using all the instances in each bag. This is done by implementing an order-invariant operation characterized by a particular type of attention mechanism. This method is very flexible as it does not require domain knowledge nor does it make any assumptions about the distribution of the instances within each bag. We test our algorithm on five real world datasets and outperform previous state-of-the-art on three of the datasets. In addition, we augment our feature space by adding the moments of each feature for each bag, as extra features, and show that while the first moments lead to higher accuracy, there is a diminishing return.
RocketRML - A NodeJS implementation of a use-case specific RML mapper

The creation of Linked Data from raw data sources is, in theory, no rocket science (pun intended). Depending on the nature of the input and the mapping technology in use, it can become a quite tedious task. For our work on mapping real-life touristic data to the schema.org vocabulary we used RML but soon encountered, that the existing Java mapper implementations reached their limits and were not sufficient for our use cases. In this paper we describe a new implementation of an RML mapper. Written with the JavaScript based NodeJS framework it performs quite well for our uses cases where we work with large XML and JSON files. The performance testing and the execution of the RML test cases have shown, that the implementation has great potential to perform heavy mapping tasks in reasonable time, but comes with some limitations regarding JOINs, Named Graphs and inputs other than XML and JSON - which is fine at the moment, due to the nature of the given use cases.
Sentiment analysis of Chinese comments on OTA website using BERT and LSTM

Under the background of "Internet + tourism", many OTA sites have large numbers of users and have accumulated huge amounts of user review information. In this paper, web crawler is used to grab Chinese comment data on OTA website for pre-processing. BERT pre-training model is used to represent text feature vectors at sentence level. The results are then input into LSTM model to obtain high-level semantic features. Thus, the emotional tendency of the comments can be classified. The comparison of experimental data shows that BERT-LSTM model has higher classification accuracy, recall rate and F1 value. It shows that the effectiveness of text sentiment analysis has been improved. An emotional analysis of these emotional commentary texts is conducive to the monitoring of management departments, providing reference for the government and enterprises to make decisions. And this research has very high application value for promoting the development of tourism.
The Impact Of Video Integrated With Bloom’s Taxonomy On The Improvement Of English-Speaking Performance

Speaking has an essential role in students’ performance in the English language subject. This study investigates 30 English language students who were given video in the experimental class. The results show that there is an improvement of participants’ English-speaking performance which is indicated by the increase of scores between pre-test and post-test through four speaking components, such as fluency, precision, lexical, and syntactical. The most significant increase was found in the precision aspect with pre-test value of 1.00 and post-test value of 3.07. The results also revealed that both written and verbal communication can be improved by using this method of video integrated with Bloom’s taxonomy.
SIG-Former: monocular surgical instruction generation with transformers

Automatic surgical instruction generation is a crucial part for intra-operative surgical assistance. However, understanding and translating surgical activities into human-like sentences are particularly challenging due to the complexity of surgical environment and the modal gap between images and natural languages. To this end, we introduce SIG-Former, a transformer-backboned generation network to predict surgical instructions from monocular RGB images.Taking a surgical image as input, we first extract its visual attentive feature map with a fine-tuned ResNet-101 model, followed by transformer attention blocks to correspondingly model its visual representation, text embedding and visual-textual relational feature. To tackle the loss-metric inconsistency between training and inference in sequence generation, we additionally apply a self-critical reinforcement learning approach to directly optimize the CIDEr score after regular training.We validate our proposed method on DAISI dataset, which contains 290 clinical procedures from diverse medical subjects. Extensive experiments demonstrate that our method outperforms the baselines and achieves promising performance on both quantitative and qualitative evaluations.Our experiments demonstrate that SIG-Former is capable of mapping dependencies between visual feature and textual information. Besides, surgical instruction generation is still at its preliminary stage. Future works include collecting large clinical dataset, annotating more reference instructions and preparing pre-trained models on medical images.
Face Detection Analysis of Digital Photos Using Mean Filtering Method

Face detection in digital photos aims to get the face area in the digital photo. Usually, a lot of noise occurred when detecting faces in digital photos. This study applies the mean filtering method to improve digital photos by reducing noise. The accuracy of the mean filtering method is calculated using a confusion matrix, while the ability of this method is measured using the parameters of Mean Square Error (MSE) and Peak Noise to Signal Ratio (PNSR). Viola-Jones method was used to detect faces in this research. This method was chosen because it is one of the face detection procedures with high accuracy and good computational ability. Testing the mean filtering method obtained the lowest MSE of 9.33, while the highest PNSR of 14.37. The accuracy obtained by the mean filtering method using confusion is 90%. Based on these results, it can be concluded that the mean filtering method is feasible to be used in the case of face detection in digital photos.
An Alternative Proof of Minimum Trace Reconciliation

Minimum trace reconciliation, developed by Wickramasuriya et. al. (2019), is an innovation in the literature of forecast reconciliation.The proof, however, is indirect and not easy to extend to more general situations.This paper provides an alternative proof based on the first-order condition in the space of nonsquare matrix and argues that it is not only simpler but also can be extended to incorporate more general results on minimum weighted trace reconciliation in Panagiotelis et.al. (2021).Thus, our alternative proof not only has pedagogical value but also connects the results in the literature from a unified perspective.
Improvement Potential and Sustainability Index of Photovoltaic Thermal Solar Air Collector

Photovoltaic thermal (PVT) system is the future technology that will help in decreasing carbon emissions because this PVT technology generates electricity and thermal simultaneously. The advantage of PVT technology is that it is clean, saves space and cost, et cetera. This research applies an experimental approach in the laboratory and outdoors. The objective of this comparison is to generate real and controlled data and parameters. The analysis conducted is the improvement potential and sustainability index analysis. Both analyses are rarely conducted in PVT technology studies. Mass flow rate used was 0.01 kg/s-0.05 kg/s and sun intensity used was 800 W/m2. The result indicated the impact of mass flow rate on improvement potential and sustainability index. The higher the mass flow rate, the lower the improvement potential decreased; the higher the mass flow rate used, the higher the sustainability index increased.
Label Selection Algorithm Based on Iteration Column Subset Selection for Multi-label Classification

In multi-label classification, each sample can be associated with a set of class labels. When the number of labels grows to the hundreds or even thousands, existing multi-label classification methods often become computationally inefficient. To this end, dimensionality reduction strategy is applied to label space via exploiting label correlation information, resulting in label embedding and label selection techniques. Compared with a lot of label embedding work, less attention has been paid to label selection techniques due to its difficulty. Therefore, it is a challenging task to design more effective label selection techniques for multi-label classification. Column subset selection is the problem of selecting a small portion of columns from a large data matrix as one form of interpretable data summarization. So, the column subset selection problem translates naturally to this purpose, as it provides simple linear models for low-rank data reconstruction. Iterative column subset selection is one of the methods to solve the problem of column subset selection, and this method can achieve a good result in the problem. In this paper, we first execute iterative column subset selection to select a small portion of columns from a large label matrix, in the prediction stage, we do some processing on the recovery matrix. So, a new method of multi-label classifier based on iterative column subset selection is proposed. The new method is tested on six publicly available datasets with varying numbers of labels. The experimental evaluation shows that the new method works particularly well on datasets with a large number of labels.
PBRE: A Rule Extraction Method from Trained Neural Networks Designed for Smart Home Services

Abstract Designing smart home services is a complex task when multiple services with a large number of sensors and actuators are deployed simultaneously. It may rely on knowledge-based or data-driven approaches. The former can use rule-based methods to design services statically, and the latter can use learning methods to discover inhabitants’ preferences dynamically. However, neither of these approaches is entirely satisfactory because rules cannot cover all possible situations that may change, and learning methods may make decisions that are sometimes incomprehensible to the inhabitant. In this paper, PBRE (Pedagogic Based Rule Extractor) is proposed to extract rules from learning methods to realize dynamic rule generation for smart home systems. The expected advantage is that both the explainability of rule-based methods and the dynamicity of learning methods are adopted. We compare PBRE with an existing rule extraction method, and the results show better performance of PBRE. We also apply PBRE to extract rules from a smart home service represented by an NRL (Neural Network-based Reinforcement Learning). The results show that PBRE can help the NRL-simulated service to make understandable suggestions to the inhabitant.
The Importance of Augmented Reality Technology in Science Education: A Scoping Review

The use of digital technology such as augmented reality AR technology has been an important topic of research in several fields, uncovering diverse benefits regarding its impacts. Although there have been numerous empirical studies on the design and evaluation techniques of the AR approach to enhance physical, cognitive, personal, social abilities in science education, their outcomes varied in different science disciplines, and there is a lack of reviews on how the AR has been applied in the field of science education. The aim of this study is to conduct a scoping review on the positivity of AR applications in science education. This study presents a scoping literature review of 26 studies published between 2015-2020 on AR in science education. The results indicated an overall positive impact of AR in science education. The results from this systematic review are expected to provide valuable information regarding the AR usage in science fields.
Comparison of Sequence Variants and the Application in Electronic Medical Records

A sequence variant is a data structure that represents elements with partial order, and it can be regarded as a sequence with branches. The comparison of sequence variants is an important problem in practical applications. To conduct comparisons, it is necessary to detect the common and uncommon parts of sequence variants, but a suitable method is not available. In this study, we developed a method for comparing sequence variants by providing appropriate definitions and algorithms. The longest common subsequence variant is defined based on the longest common subsequence and a merged sequence variant is proposed for comparison. We also propose algorithms for calculating these sequence variants. As an example, we applied the methods to real data from electronic medical records (EMRs) to determine the diversity of the treatment patterns in different hospitals, before presenting the results to medical workers to help them recognize the differences and improve their medical actions. Frequent treatment patterns were extracted from a real treatment order database of EMRs by using sequential pattern mining, and the differences in the treatment patterns were calculated and visualized as longest common subsequence variants and merged sequence variants.
Data Act: Legal Implications for the Digital Agriculture Sector

On 23 February 2022, the proposal of the Data Act containing ‘harmonised rules on fair access and use of data’ was released. It is a horizontal framework for data access to unlock competition and innovation. The emerging Digital Agriculture sector is one of the non-personal data-driven industries that can be affected by this horizontal intervention. Therefore, it is highly relevant to investigate the possible implications of the Data Act from the sectoral perspective. This paper aims to demonstrate what the Data Act brings, to what extent it is helpful to overcome the sectoral problems connected to ag-data access, and how it can be fine-tuned to increase its effectiveness for the sector. Thus, the paper also aims to identify the remaining issues that may need to be addressed by a follow-up sectoral regulation.
Baseline Study of COVID-19 and Biometric Technologies

COVID-19 is a pathogenic viral infection caused by severe acute respiratory syndrome corona virus 2 (SARS-CoV-2), which emerged in Wuhan, China in December 2019 and has spread to several countries of the world resulting in economic hardship and travel restrictions. This paper presents findings on the baseline study of COVID-19 and biometric technologies. The study included succinct discussions on biometric technologies prior to and since outbreak of COVID-19 and an online survey involving 2438 randomly selected individuals via questionnaire that centered on the world's economy with daily application of biometric technologies. The questionnaire featured indices on biometric technologies and global security, the rating of each biometric mode in the global security performance scale among others. Analysis of data from the survey established the paradigm shift in biometric applications from contact-based to contact-free since the outbreak of the disease, low risk level between COVID-19 and biometric technologies and diminishing cash flow in biometric market.