Reviewerly/exHarmony huggingface dataset:
Egocentric Activity Recognition and Localization on a 3D Map

Given a video captured from a first person perspective and the environment context of where the video is recorded, can we recognize what the person is doing and identify where the action occurs in the 3D space? We address this challenging problem of jointly recognizing and localizing actions of a mobile user on a known 3D map from egocentric videos. To this end, we propose a novel deep probabilistic model. Our model takes the inputs of a Hierarchical Volumetric Representation (HVR) of the 3D environment and an egocentric video, infers the 3D action location as a latent variable, and recognizes the action based on the video and contextual cues surrounding its potential locations. To evaluate our model, we conduct extensive experiments on the subset of Ego4D dataset, in which both human naturalistic actions and photo-realistic 3D environment reconstructions are captured. Our method demonstrates strong results on both action recognition and 3D action localization across seen and unseen environments. We believe our work points to an exciting research direction in the intersection of egocentric vision, and 3D scene understanding.
Malicious traffic detection based on GWO-SVM model

With the rapid development of Internet technology, people gradually rely on intelligent devices for information exchange and communication through the network. With the popularity of smart connected devices, security issues are growing, including malicious traffic attacks. Therefore, the design of an efficient malicious traffic detection system is of great significance. This paper makes two improvements based on the traditional malicious traffic classification model for malicious traffic detection. One is to adopt a brand new combined dataset named Composed Encrypted Malicious Traffic Dataset and conduct a correlation coefficient matrix analysis on the data set. The second is to design an improved SVM model based on the gray wolf optimization algorithm. We abandon the idea of relying on expert experience to set parameters and optimize parameters through intelligent algorithms. Through six groups of comparative tests with GWO-SVM, Xgboost, RF, SVM,GA, GA-BP, the classification effect of the improved model is the best, and the classification accuracy is 0.99625, the standard deviation for cross-validation is 0.01.
Concept-Enhanced Relation Network for Video Visual Relation Inference

Video visual relation inference aims at extracting the relation triplets in the form of < subject-predicate-object <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$&gt;$ </tex-math></inline-formula> in videos. With the development of deep learning, existing approaches are designed based on data-driven neural networks. But the datasets are always biased in terms of objects and relation triplets, which make relation inference challenging. Existing approaches often describe the relationships from visual, spatial, and semantic characteristics. The semantic description plays a key role to indicate the potential linguistic connections between objects, that are crucial to transfer knowledge across relationships, especially for the determination of novel relations. However, in these works, the semantic features are not emphasized, but simply obtained by mapping object labels, which can not reflect sufficient linguistic meanings. To alleviate the above issues, we propose a novel network, termed Concept-Enhanced Relation Network (CERN), to facilitate video visual relation inference. Thanks to the attributes and linguistic contexts implied in concepts, the semantic representations aggregated with related concept knowledge of objects are of benefit to relation inference. To this end, we incorporate retrieved concepts with local semantics of objects via the gating mechanism to generate the concept-enhanced semantic representations. Extensive experimental results show that our approach has achieved state-of-the-art performance on two public datasets: ImageNet-VidVRD and VidOR.
Factorization of Dynamic Games over Spatio-Temporal Resources

Dynamic games feature a state-space complexity that scales superlinearly with the number of players. This makes this class of games often intractable even for a handful of players. We introduce the factorization process of dynamic games as a transformation leveraging the independence of players at equilibrium to build a leaner game graph. When applicable, it yields fewer nodes, fewer players per game node, hence much faster solutions. While for the general case checking for independence of players requires to solve the game itself, we observe that for dynamic games in the robotic domain there exist exact heuristics based on the spatio-temporal occupancy of the individual players. We validate our findings in realistic autonomous driving scenarios showing that already for a 4-player intersection we have a reduction of game nodes and solving time close to 99%.
MGRM: A Multi-Segment Greedy Rewriting Method to Alleviate Data Fragmentation in Deduplication-Based Cloud Backup Systems

Data deduplication has been broadly used in Cloud due to its storage space saving ability. An issue of deduplication is the contiguous data chunks in a segment may be scattered in different containers. This phenomenon is called data fragmentation. Because of data fragmentation, a restore process must reference various containers across a wide variety of segments, thereby hurting the restore performance. Capping methods that rewrite the data chunks of low Container Reference Ratio ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CRR</i> ) containers are developed to alleviate data fragmentation. We analyze and observe from real traces that a number of segments only point to low <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CRR</i> containers, while some others only contain high <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CRR</i> containers. This interesting observation is ignored by the existing capping methods which sort containers from a single segment, falling short in searching multiple segments collectively. Thus, the reference count of selected containers in the existing capping methods is still high. To address this problem, we propose a multi-segment greedy rewriting method named MGRM. MGRM sorts containers of segments in a sequential way. More specifically, given the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i-th</i> segment currently being processed, MGRM will sort all the containers in the top <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i-th</i> segments. This salient searching feature enables MGRM to select and rewrite the true low-reference container set. Moreover, to achieve a good balance between deduplication ratio and restore performance, MGRM has two working modes: an optimal rewriting mode and a radical rewriting mode. When working in the optimal rewriting mode, MGRM aims to improve the deduplication ratio; when the radical rewriting mode, MGRM strives to improve the restore performance. MGRM adaptively switches the working mode according to workload. Furthermore, unlike the existing capping methods that improve restore performance at the cost of the deduplication ratio, MGRM pays attention to both aspects. Our extensive experimental results show that MGRM achieves high restore performance, coupled with a high deduplication ratio. In particular, compared with the two state-of-art schemes FC and FLC, MGRM improves the deduplication ratio and restore performance by up to 114.83% and 99.34%, respectively.
Face Identification and Real Time Emotion Tracker

Traditional method of detecting emotions from facial expressions is easier for humans but performing the same with machine intelligence is a challenging task. Now using the unique and distinct Face Emotion Recognition mechanism based on CNN (FERC) model detects emotions in real-time from photographs or videos. The FERC model which was proposed in 2014 is used to build a project which is based on a convolutional neural network (CNN). This model includes two main functions, one is removing the background from the image and the next function focuses on the facial feature extraction. This model applied to the public data contains of 10,000 photographs that had been saved (154 persons).In this paper, uses facial feature vectors of length 24 values for identifying and classifying the current emotion of the individuals with five different types of regular facial expressions and the model outperformed with 98% of accuracy.
Research Methodology

In this chapter, the research purpose and questions are specified first. The research design and data sources are summarized and then the participants are introduced, followed by demonstrations on the data collection instruments, research procedures, and methods used for data analyses.
A Statistical Approach to Detecting Low-Throughput Exfiltration through the Domain Name System Protocol

The Domain Name System (DNS) is a critical network protocol that resolves human-readable domain names to IP addresses. Because it is an essential component necessary for the Internet to function, DNS traffic is typically allowed to bypass firewalls and other security services. Additionally, this protocol was not designed for the purpose of data transfer, so is not as heavily monitored as other protocols. These reasons make the protocol an ideal tool for covert data exfiltration by a malicious actor. A typical company or organization has network traffic containing tens to hundreds of thousands of DNS queries a day. It is impossible for an analyst to sift through such a vast dataset and investigate every domain to ensure its legitimacy. An attacker can use this as an advantage to hide traces of malicious activity within a small percentage of total traffic. Recent research in this field has focused on applying supervised machine learning (ML) or one-class classifier techniques to build a predictive model to determine if a DNS domain query is used for exfiltration purposes; however, these models require labelled datasets. In the supervised approach, models require both legitimate and malicious data samples, but it is difficult to train these models since realistic network datasets containing known DNS exploits are rarely made public. Instead, prior studies used synthetic curated datasets, but this has the potential to introduce bias. In addition, some studies have suggested that ML algorithms do not perform as well in situations where the ratio between the two classes of data is significant, as is the case for DNS exfiltration datasets. In the one-class classifier approach, these models require a dataset known to be void of exfiltration data. Our model aims to circumvent these issues by identifying cases of DNS exfiltration within a network, without requiring a labelled or curated dataset. Our approach eliminates the need for a network analyst to sift through a high volume of DNS queries, by automatically detecting traffic indicative of exfiltration.
Robust Kalman Filter Enhanced by Projection Statistic Detector for Multisensor Navigation in Urban Canyon Environment

Multisensor navigation via redundancy and complementary has been widely applied to safety-critical services, such as self-driving vehicles. To improve the robustness and reliability of the multisensor navigation system in urban canyon environments, a robust Kalman filter (RKF) method enhanced by the projection statistic (PS) detector is proposed in this work. Based on a statistical consistency check, the availability of measurements from observation sensors is preevaluated by the PS criterion. The new iterative Huber’s M-estimation with the exclusion function is implemented on the linear regression model for robust state estimation. The effectiveness of the proposed algorithm was verified by a dynamic test in the representative urban canyon environment. Our approach shows significant superiority and robustness among the comparative experiments, where the 3-D root-mean-square (rms) error is limited to 3.33 m. The biased measurements can be effectively identified and removed by the PS detector with a given significance level. The new iterative Huber’s M-estimation assisted by the prior availability knowledge enables the integration solution more robust and reliable. The presented robust method is pretty suitable for multisensor navigation in urban canyon environments.
An Evaluation of Caret Navigation Methods for Text Editing in Augmented Reality

A common task in text editing is navigating the text caret as part of the text editing process. We explore three key dimensions in the design space of caret navigation techniques: cursor placement method, visual expansion in the form of a magnifying lens, and display distance. In addition to two commonly used cursor placement methods, direct touch and raycast, we also present a novel multimodal text cursor placement method combining eye gaze and touch gestures for optical see-through augmented reality (AR). This method allows the user to refine the caret position with an indirect mid-air virtual touch block after the caret has snapped to a location provided by an initial eye fixation. We derive eight combinations from three design dimensions and study their performance in a user study with 24 participants. Our results reveal that: 1) raycast delivered the fastest completion times among all combinations evaluated; 2) in near-field conditions the multimodal method can achieve similar performance as direct touch input with less physical effort; and 3) magnifying lenses offer no significant performance advantages for caret navigation in AR.
TransVLAD: Focusing on Locally Aggregated Descriptors for Few-Shot Learning

This paper presents a transformer framework for few-shot learning, termed TransVLAD, with one focus showing the power of locally aggregated descriptors for few-shot learning. Our TransVLAD model is simple: a standard transformer encoder following a NeXtVLAD aggregation module to output the locally aggregated descriptors. In contrast to the prevailing use of CNN as part of the feature extractor, we are the first to prove self-supervised learning like masked autoencoders (MAE) can deal with the overfitting of transformers in few-shot image classification. Besides, few-shot learning can benefit from this general-purpose pre-training. Then, we propose two methods to mitigate few-shot biases, supervision bias and simple-characteristic bias. The first method is introducing masking operation into fine-tuning, by which we accelerate fine-tuning (by more than 3x) and improve accuracy. The second one is adapting focal loss into soft focal loss to focus on hard characteristics learning. Our TransVLAD finally tops 10 benchmarks on five popular few-shot datasets by an average of more than 2%.
Application of magnetic susceptibility measurement for mapping and assessment of ecological quality in urban topsoils

The article presents the results of geostatistical mapping of the magnetic susceptibility of the urban top soil. Soil magnetometry is well suited for rapid monitoring of pollution in urban areas due to its high sensitivity, ease of measurement, rapidity, high reproducibility of analysis, and low cost. This method allows obtaining large datasets with high resolution. The purpose of the study: spatial modeling of magnetic susceptibility and ecological-geochemical assessment of the top soil of Kudymkar city. The research area covers a section of the city with a total area of 32 km². Spatial modeling was carried out by the geostatistical method based on 51 soil samples. The background magnetic susceptibility of the soils of the city is 3–4 times higher than the magnetic susceptibility of the soils of the non-contaminated regional background. Soils with high and very high magnetic susceptibility occupy more than 30 % of the city area. Anomalous zones of soil magnetic pollution or magnetic “hot spots” formed near industrial facilities, heating boilers, on roadside soils with heavy traffic. A scale for the volumetric magnetic susceptibility of soils was developed on the centile analysis of the data. Strongly magnetic soils contain elevated concentrations of Zn, Cu, and Ni. The concentrations and names of pollutant metals in urban soils depend on the techno-geochemical specialization of cities. It can be recommended that the Environmental Services of cities use measurements of the magnetic susceptibility to monitor the ecological and geochemical state of soils and identify areas of city soils contaminated with heavy metals.
AI Food Recommendation Systems

In an information-rich generation, recommendation systems is very important. Hence, in this paper we use the Google Map API and the feedback record of user to build data set on database for the AI (Artificial Intelligence) food recommendation systems. In order to make a recommendation system that knows more than the user, we combine the indispensable food and drink according to the preferences of users to recommend the favorite of users. That is, the AI food recommendation system uses GAE (Graph Autoencoder) and regional filtering to predict the favorite of users. Besides, the AI food recommendation system is proposed to use GAE and regional filtering to predict the user's favorite degree by means of deep learning, and then push it to the user through the Line Bot that can recommend the store to the favorite food of users on nearby, and then feedback to our system after the user uses it, which makes the proposed system more complete and accurate.
VTC: Improving Video-Text Retrieval with User Comments

Multi-modal retrieval is an important problem for many applications, such as recommendation and search. Current benchmarks and even datasets are often manually constructed and consist of mostly clean samples where all modalities are well-correlated with the content. Thus, current video-text retrieval literature largely focuses on video titles or audio transcripts, while ignoring user comments, since users often tend to discuss topics only vaguely related to the video. Despite the ubiquity of user comments online, there is currently no multi-modal representation learning datasets that includes comments. In this paper, we a) introduce a new dataset of videos, titles and comments; b) present an attention-based mechanism that allows the model to learn from sometimes irrelevant data such as comments; c) show that by using comments, our method is able to learn better, more contextualised, representations for image, video and audio representations. Project page: https://unitaryai.github.io/vtc-paper .
Enhanced Plastic Recycling using RGB+Depth Fusion with MassFaster and MassMask R-CNN

The rapid increase in waste generation from electrical and electronic equipment (WEEE) has created the need for more advanced sensor-based systems to sort this complex type of waste. Therefore, this study proposes a method for object detection, instance segmentation, and mass estimation of plastics and contaminants using the fusion of RGB and depth (D) images. The methodology is based on the Faster and Mask R-CNN with an extra head for the mass estimation. In addition, a pre-processing method to enhance the depth image (ED) is proposed. To evaluate the data fusion and pre-processing method, two data sets of plastics and impurities were created containing images with and without overlapping samples. The first data set contains 174 RGB images and depth (D) maps of 3146 samples, excluding their mass value, while the second data set contains 42 RGB and D images of 766 pieces together with their mass. The first and second data sets were used to evaluate the performance of Mask and Faster R-CNN. Further, the second data set was used to evaluate the network’s performance with the additional head for mass estimation.The proposed method achieved 0.75 R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , 1.39 RMSE, and 0.81 MAE with an IoU greater than 50% using the network Resnet50_FPN_RGBED. Hence, it can be concluded that the presented method can distinguish plastics from other materials with reasonable accuracy. Furthermore, the mass of each detected particle can be estimated individually, which is of great relevance for the recycling sector. Knowing the mass distribution and the percentage of contaminants in a waste stream of mixed plastics can be valuable for adjusting the parameters of upstream and downstream sorting processes.
The Effect of the JPEG Implementation on the Cover-Source Mismatch Error in Image Steganalysis

One of the challenges in steganalysis based on machine learning is cover-source mismatch (CSM). It occurs when a detector is trained on cover and stego pairs generated from one data source and then applied to data of unknown type from a different source. The mismatch leads to an increase of the classification error, which corresponds to a loss of detection performance. This paper studies how much the implementation of the JPEG compression and decompression contributes to the CSM of color image steganalysis. Specifically, it studies the differences between libjpeg versions 6b and 7. The impact on the CSM is measured for J-UNIWARD embedding using JPEG and spatial rich models with an ensemble classifier. It is observed that the use of different libjpeg versions may cause CSM. The error is large when different libjpeg versions are used for feature extraction, and barely measurable when different versions are used for the generation of cover and stego pairs.
Out-of-Distribution Detection with Semantic Mismatch Under Masking

This paper proposes a novel out-of-distribution (OOD) detection framework named MoodCat for image classifiers. MoodCat masks a random portion of the input image and uses a generative model to synthesize the masked image to a new image conditioned on the classification result. It then calculates the semantic difference between the original image and the synthesized one for OOD detection. Compared to existing solutions, MoodCat naturally learns the semantic information of the in-distribution data with the proposed mask and conditional synthesis strategy, which is critical to identify OODs. Experimental results demonstrate that MoodCat outperforms state-of-the-art OOD detection solutions by a large margin. Our code is available at https://github.com/cure-lab/MOODCat .