Reviewerly/exHarmony huggingface dataset:
Office Activity Recognition Using HTN Planning

The problem of office activity recognition is characterised by common occupant activities that may have composition relationships and involve expected objects with sensors providing both logical and numeric information. While this problem has been tackled using learning and ontological approaches, the associated disadvantages are obvious: a lot of training data is needed, the learned activities can be specific to the data, and ontologies suffer from flexibility and maintainability issues. We propose to address office activity recognition using Hierarchical Task Network (HTN) planning, a widely used Artificial Intelligence planning technique developed to model agents’ behaviour. We argue that HTN planning has the power to deal with all the characteristics of the office activity recognition problem while ensuring generality, flexibility, and maintainability. In addition, we suggest our approach be designed and implemented as a service not only to support the portability of off-the-shelf HTN planners but also to enable and encourage the integration of the approach in building automation systems. We present a use case to demonstrate the proposal’s feasibility.
Latent-Space Analysis for Improved Overhead Imagery Explainability

With the ever-growing quantity of information generated by commercial and intelligence platforms, it is imperative that the community moves toward solutions that provide not just automated outputs, but a more accurate explanation into why automated outputs are predicted. Automated Target Recognition (ATR) solutions that rely on deep learning are being leveraged to derive intelligence automatically, but these networks can produce incorrect answers in some circumstances, reducing user trust. Most analysts cautiously trust these results, but they typically still require inefficient manual verification. To help address this challenge, we introduce an ATR anomaly metric created specifically for GEOINT systems that determines if a newly detected object fits into the distribution of the network’s original training dataset. This approach has been tested on different sets of targets, such as ships, airplanes, and vehicles, spanning multiple publicly available EO and SAR commercial GEOINT datasets. Low anomaly scores generated by this approach help to validate model performance and increase trust in a model’s outputs, while high anomaly scores help identify unique images that were not captured within a model’s training distribution. For instance, some examples of high anomaly scores were found to correspond to mislabeled data used in training, identifying aircraft in pre/post flight configurations and in various stages of disassembly, detecting boats out of water, and identifying aircraft that were viewed through clouds. More importantly, for detections with a low anomaly score, our analysis shows that the results are highly accurate, allowing the analyst to trust the results and focus on harder problems. Additionally, we show a preliminary proof of concept analysis in which we invert this anomaly metric to perform semi-supervised active learning on brand new datasets.
Multimodal Knowledge Reasoning for Enhanced Visual Question Answering

In the domains of Natural Language Processing (NLP) and Computer Vision (CV) Visual Question Answering (VQA) is a multidisciplinary task, in which an image and a question are given to a VQA system, which is responsible for giving the answer. The VQA system is used for a variety of real-world applications, such as providing situational information based on visual material, making judgments using a vast quantity of surveillance data, interacting with robots, and helping individuals who are blind or visually impaired. Although it is required yet challenging to complete comprehensive VQA, Fact-based VQA (FVQA) approaches in which external knowledge is required to process with image and question. Existing FVQA methods combine all types of data without fine-grained selection, thereby generating unexpected noise while reasoning about the final result. The problem solution should be able to collect complementary-information evidence based on question-attention. We represent an image with different layers of information by a multimodal knowledge graph relating to the features of visual, factual, and semantic. We propose a multimodal knowledge graph-convolutional-network (GCN) to collect relevant-information evidence from different information layers based on the given question. In particular, intra-modal knowledge graph attention takes evidence from each modality, while inter-modal knowledge graph attention gets evidence across the different information layers. To get an optimal answer, we stack this process multiple times to perform a reasoning mechanism. Over the FVQA dataset, we achieved state-of-the-art results by improving 10.86% test accuracy, which demonstrates the usefulness and interpretability of our approach.
Optimization Scheme for Online Viewpoint Planning of Active Optical Navigation System in Orthopedic Surgeries

Optical tracking system (OTS) is widely used in robot orthopedic surgeries. During the surgery, the locomotion of the surgical instrument along different osteotomy planes and the adjustment of the patient’s body will lead to obvious variations on the spatial relationship among the positioning tools. Usually, the OTS is manually placed at a fixed viewpoint. As the size of the measurement volume (MV) of the OTS is limited, some positioning tools might move towards the boundaries of the MV, increasing the risk of navigation failure. This paper proposes a two-phase optimization scheme for the active online adjusting of the OTS viewpoint to guarantee the consistency of navigation. A novel optimization structure is proposed with the consideration of continuous state inequality constraints. The dynamic accuracy of the OTS during the active adjusting is also demonstrated to be reliable. Ultimately, simulations and experiments are performed on an active navigation system to verify the effectiveness of the proposed two-phase optimization scheme. Experimental results show that the OTS can be moved to the optimal viewpoint smoothly when confronted with significant variations on the spatial relationship among the positioning tools, guaranteeing the consistency of navigation. The navigation robot successfully keeps the positioning tools around the central region of the MV at different surgical phases.
Flexible On-Orbit Calibration for Monocular Camera and Laser Rangefinder Integrated Pose Measurement System

Owing to its low cost and ease of manufacture, integrated monocular space camera and laser rangefinders are commonly used in many space applications (i.e., pose measurement of non-cooperative targets). This and other composite systems estimate poses with high precision using a combination of high-resolution lateral visual and high-precision distance information. Notably, any system that is tested on ground must again receive precise on-orbit calibration because the accompanying spaceborne turntables and arms must be frequently manipulated to meet field-of-view and distance measurement requirements and require complicated realignments. To accommodate flexible multiple target calibration, this study proposed an on-orbit model that obtains accurate pose relationships jointly using the monocular camera and laser rangefinder. First, we invented a robust calibration model and derive its pose transformation matrix based on the given component positions. We then provided an improved (simplified) orthogonal iteration algorithm that optimizes the given matrix for calibration. The influences of different factors on calibration accuracy are quantitatively analyzed via simulations, and the new method’s performance is verified with real-world experiments. This system enables several novel and significant space and military applications.
Computational Super-Resolution Imaging With a Sparse Rotational Camera Array

A sparse camera array utilizes several identical cameras for multi-frame super-resolution. However, the influence of camera layout on super-resolution performance remains unclear. In conventional camera arrays, all cameras share a similar observation model, resulting in redundancy during information collection. This study presents a novel super-resolution framework that actively controls cameras in a sparse camera array to rotate to certain degrees. Through a detailed analysis of the forward imaging model, it is shown that the rotation of cameras uses the rotational asymmetry property of pixel layouts and shape. The proposed camera layout method provides more diverse and informative spatial sampling than a conventional camera array with only translations between cameras. A physics-enabled deep neural network was introduced for multi-frame super-resolution. With the explicitly defined gradients and affine models as guidance, it can effectively decode additional information collected by the rotational camera array and overcome the large-scale spatial feature alignment problem. The experimental results on synthetic and real prototype datasets validated the proposed theory and solution. We believe our study can inspire innovations for future task-specific camera array system design.
Adaptor: Improving the Robustness and Imperceptibility of Watermarking by the Adaptive Strength Factor

In watermarking, the watermark embedding strength is crucial, and the introduction of the strength factor can adjust the trade-off between the quality of the encoded image and the accuracy of the recovered message, thus enabling good imperceptibility and robustness of the encoded image. In traditional watermarking methods, the strength factor is selected in relation to the cover image, and based on different images, different strength factors are manually selected or algorithmically derived to adjust the visual effect of the watermarked image. However, due to the subjectivity and inflexibility of traditional algorithms, they can not achieve the effect of adaptive adjustment of watermarked images. Recently, watermarking methods combined with deep learning have gradually occupied the mainstream of this field. In the testing stage, to balance the overall robustness and imperceptibility, the strength factor is no longer selected based on the cover image as in traditional methods. Instead, it is set to a universal value. Therefore, the watermarking method based on deep learning is still in the primary stage of the trial-and-error method. To solve the subjectivity of the hand-designed embedding strength algorithm of the traditional watermarking methods and the low elasticity of the strength factor of the learning method so as to realize the adaptive embedding of watermarks, we propose an adaptive watermarking method with separate training. The proposed method adds a new component, the Adaptor, compared to other frameworks. The Adaptor can adaptively select strength factors to control the embedding strength of the watermark relying on the cover image and secret message. A two-stage training method is used to maintain the stability of the training and to achieve the best results for each component. With the results obtained from our experiments, our proposed method can find the appropriate strength factor and optimize it, resulting in improved robustness and imperceptibility of the watermark. The proposed method shows better results compared to the current state-of-the-art algorithms.
A Multiview Text Imagination Network Based on Latent Alignment for Image-Text Matching

In image-text matching fields, one of the keys to improving performance is to extract features with more semantic information. Existing works demonstrate that semantic enrichment through knowledge expansion can improve performance. Most of them expand image features, however, the shortage of semantic information in text modality and the unilateral character of the view are often bottlenecks that limit the performance of image-text matching models. To solve the two problems, we aggregate knowledge from multiple views and propose a word imagination graph (WIG). A WIG can be used to expand textual semantic information by imagination based on input images. Then, utilizing WIG, we construct a novel multiview text imagination network (MTIN). A MTIN enables latent alignment of images and texts on tags, which can assist matching on a semantic level. Results from the Flickr30K and MS-COCO datasets demonstrate the effectiveness of our method. The source code has been released on GitHub <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/smileslabsh/Multiview-Text-Imagination-Network</uri> .
Trusted Execution Environments for Telecoms: Strengths, Weaknesses, Opportunities, and Threats

Building on our experience and other experiments, we perform a strengths, weaknesses, opportunities, and threats analysis of x86 platform trusted execution environments (TEEs) in the telecom industry, including recent developments from CPU and cloud vendors and the vibrant TEE research domain.
Outsourced and Privacy-Preserving Collaborative k-Prototype Clustering for Mixed Data via Additive Secret Sharing

Outsourced cloud computing can be considered as an effective way to overcome the data island among users and relieve the pressure of limited resources. However, due to the concerns about trust in cloud servers, outsourcing the users’ data and model training task has considerable privacy disclosure risks. This article presents a PriKPM scheme by using additive secret sharing (ASS), so as to implement the privacy-preserving <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${k}$ </tex-math></inline-formula> -prototype clustering for mixed data (i.e., including numerical and categorical attributes). In PriKPM, data samples are randomly split into two shares and delivered offline to two collaborative servers. We design a secure initialization method for determining the location and number of cluster centers. Then, both servers securely calculate the mixed distance between samples and cluster centers, and execute the samples partion and cluster updating operations. An efficient and secure comparison protocol is developed to offer flexibly the “less than or equal” and “equal” functions during the entire clustering process. Furthermore, theoretical analysis proves the effectiveness and security of PriKPM. Sufficient experiments demonstrate that PriKPM is computationally more efficient than existing secure clustering works. PriKPM can achieve the approximate accuracy of the plaintext <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${k}$ </tex-math></inline-formula> -prototype clustering scheme.
Six-Granularity Based Chinese Short Text Classification

Short text classification is an important task in Natural Language Processing (NLP). The classification result for Chinese short text is always not ideal due to the sparsity problem of them. Most of the previous classification models for Chinese short text are based on word or character, considering that Chinese radical can also represent the meaning individually, so word, character and radical are all used to build a Chinese short text classification model in this paper, which solves the data sparsity problem of short text. In addition, in the process of segmenting sentences into words, considering that jieba will cause the loss of key information and ngram will generate noise words, both jieba and ngram are used to construct a six-granularity (i.e. word-jieba, word-jieba-radical, word-ngram, word-ngram-radical, character and character-radical) based Chinese short text classification (SGCSTC) model. Additionally, different weights are assigned to the six granularities and are automatically updated in the process of back-propagation using cross-entropy loss due to the different influence of them on the classification results. The classification Accuracy, Precision, Recall and F1 of SGCSTC in THUCNews-S dataset are 93.36%, 94.47%, 94.15% and 94.31% respectively, and that in CNT dataset are 92.67%, 92.38%, 93.15% and 92.76% respectively, and multiple comparative experiment results on THUCNews-S and CNT datasets show that SGCSTC outperforms the state-of-the-art text classification models.
Robot-Person Tracking in Uniform Appearance Scenarios: A New Dataset and Challenges

Person-tracking robots have many applications including security, surveillance, and autonomous driving. Despite the abundance of uniform appearance in many contexts and the challenges they exhibit, there is a lack of video datasets dedicated to benchmarking tracking algorithms in such contexts. In this article, we propose a new high-quality RGB-D benchmark called PTUA for robot–person tracking in uniform appearance scenarios. PTUA is recorded using an RGB-D sensor on top of a moving robot and consists of 45 sequences containing more than 85 K frames. Each frame is manually annotated with a bounding box and attributes, making PTUA the largest and the most challenging person tracking RGB-D dataset. To the best of our knowledge, such a densely annotated and properly synchronized RGB-D tracking benchmark does not exist in the literature. Each sequence comprises various challenges deriving from real-life scenarios where the target person appears highly similar to the background or distractors. By releasing PTUA, we expect to provide the community with a large-scale challenging RGB-D benchmark with high quality for the robust evaluation of trackers on uniform appearance scenarios for autonomous robots. We also present a rigorous experimental evaluation of the state-of-the-art trackers on the PTUA dataset with a comprehensive analysis. The findings evidence the challenges of person tracking in a uniform appearance scenario for both target tracking and robot–person tracking, and the need to bridge the performance gap. In addition, we propose a new RGB-D tracker that extracts features from RGB-D frames and it achieves the best performance on each challenging scenario of PTUA.
Multi-Object Navigation Using Potential Target Position Policy Function

Visual object navigation is an essential task of embodied AI, which is letting the agent navigate to the goal object under the user's demand. Previous methods often focus on single-object navigation. However, in real life, human demands are generally continuous and multiple, requiring the agent to implement multiple tasks in sequence. These demands can be addressed by repeatedly performing previous single task methods. However, by dividing multiple tasks into several independent tasks to perform, without the global optimization between different tasks, the agents' trajectories may overlap, reducing the efficiency of navigation. In this paper, we propose an efficient reinforcement learning framework with a hybrid policy for multi-object navigation, aiming to maximally eliminate noneffective actions. First, the visual observations are embedded to detect the semantic entities (such as objects). And the detected objects are memorized and projected into semantic maps, which can also be regarded as a long-term memory of the observed environment. Then a hybrid policy consisting of exploration and long-term planning strategies is proposed to predict the potential target position. In particular, when the target is directly oriented, the policy function makes long-term planning for the target based on the semantic map, which is implemented by a sequence of motion actions. In the alternative, when the target is not oriented, the policy function estimates an object's potential position toward exploring the most possible objects (positions) that have close relations to the target. The relation between different objects is obtained with prior knowledge, which is used to predict the potential target position by integrating with the memorized semantic map. And then a path to the potential target is planned by the policy function. We evaluate our proposed method on two large-scale 3D realistic environment datasets, Gibson and Matterport3D, and the experimental results demonstrate the effectiveness and generalization of the proposed method.
Tunable Lau effect: Optical frequency sweeping

For controlling the visibility of the Lau patterns, we disclose the use of a zoom system that implements a zero-throw, tunable magnification of the source grating. The zoom system implements a nonmechanical technique, for controlling the in-register condition of the Lau patterns. We show that one can increase the sharpness the Lau fringes, by exploiting the noncyclic autocorrelation of the Fourier coefficients, associated to the grating acting as spatial filter. The described technique can be thought of as a frequency sweeping technique, which tunes the visibility and the shape of the Lau fringes.
Personal factors of professional success of employees of internal affairs bodies

Professional success is determined by a combination of factors of objective and subjective content that ensure the achievement of a significant result in activities, job satisfaction and the development of the personal potential of the organization's employees. Objective performance results and positive subjective experiences of professional success are a powerful motivator of professional activity and help to increase its effectiveness. The purpose of the study is determined by the need to establish the features of the factor structure of personality traits that act as resources for professional success. Subject of study: factorial structure of personality traits. Research methods: personality traits were considered in accordance with the classes of factors proposed in the psychodiagnostic methodology "California Personality Questionnaire - CPI". The sample of the study was represented by male employees of the internal affairs bodies (n=97), divided into two groups of "successful" (n=65) and "unsuccessful" (n=32) based on an expert assessment. The results of the study made it possible to establish important characteristics of personal factors of professional success, which constitute a significant psychological resource for the professional development of subjects of professional activity.It has been established that the main factors are subjective, reflecting the effectiveness of effective interpersonal interaction, the normativity and maturity of the individual, the focus on achieving and developing intellectual potential, a constructively optimistic lifestyle, which determines psychological well-being and job satisfaction. The scope of the research results is determined by the specifics of the organization of psychological support for the activities of employees of internal affairs bodies, the tasks of moral and psychological training, the goals of personality development and the formation of skills of official behavior. Prospects for further research lie in the plane of analyzing the parameters of the identified factors, identifying patterns in the formation of professional success and its subjective components.
Algorithm for Predicting Pedestrian Behavior on Public Roads

Human Pose Estimation (HPE) is a way to identify and to classify the nodes of the human body. Essentially, it is a way of determining coordinates of each node (arm, head, torso, etc.) called a key point and defining the position of the human body. The relationship between these points is called a pair. The article analyzes existing approaches to predicting human behavior and compares them. The article contains a comparison and description of tools and technologies for the implementation of human behavior detection systems. It describes the implementation of the system, the development of a module to detect such a sign of deviant behavior as falling, the development of an application that acts as a client application to display notifications of the detection of human falls, manual testing of the implemented program.