Reviewerly/exHarmony huggingface dataset:
DLMetaChain: An IoT Data Lake Architecture Based on the Blockchain

Nowadays, the IoT ecosystem is evolving rapidly, with multiple heterogeneous sources producing high volumes of data and processes transforming this data into meaningful or “smart” information. These volumes of data, including IoT data, need to be stored in repositories that can host raw, unprocessed, relational and non-relational types of data, such as Data Lakes. Due to the weakness of metadata management, security & access control is one of the main challenges of Big Data storage architectures as Data Lakes can be replaced without oversight of the contents. Recently, the Blockchain technology has been introduced as an effective solution to build trust between different entities, where trust is either nonexistent or unproven, and to address security and privacy concerns. In this paper we introduce DLMetaChain, an extended Data Lake metadata mechanism that consists of data from heterogeneous data sources which interact with IoT data. The extended mechanism mainly focuses on developing an architecture to ensure that the data in the Data Lake is not modified or altered by taking into advantage the capabilities of the Blockchain.
Neural-Sim: Learning to Generate Training Data with NeRF

Training computer vision models usually requires collecting and labeling vast amounts of imagery under a diverse set of scene configurations and properties. This process is incredibly time-consuming, and it is challenging to ensure that the captured data distribution maps well to the target domain of an application scenario. Recently, synthetic data has emerged as a way to address both of these issues. However, existing approaches either require human experts to manually tune each scene property or use automatic methods that provide little to no control; this requires rendering large amounts of random data variations, which is slow and is often suboptimal for the target domain. We present the first fully differentiable synthetic data pipeline that uses Neural Radiance Fields (NeRFs) in a closed-loop with a target application's loss function. Our approach generates data on-demand, with no human labor, to maximize accuracy for a target task. We illustrate the effectiveness of our method on synthetic and real-world object detection tasks. We also introduce a new "YCB-in-the-Wild" dataset and benchmark that provides a test scenario for object detection with varied poses in real-world environments. Code and data could be found at .
CAViT: Contextual Alignment Vision Transformer for Video Object Re-identification

Video object re-identification (reID) aims at re-identifying the same object under non-overlapping cameras by matching the video tracklets with cropped video frames. The key point is how to make full use of spatio-temporal interactions to extract more accurate representation. However, there are dilemmas within existing approaches: (1) 3D solutions model the spatio-temporal interaction but are often troubled with the misalignment of adjacent frames, and (2) 2D solutions adopt a divide-and-conquer strategy against the misalignment but cannot take advantage of the spatio-temporal interactions. To address the above problems, we propose a Contextual Alignment Vision Transformer (CAViT) to the spatio-temporal interaction with a 2D solution. It contains a Multi-shape Patch Embedding (MPE) module and a Temporal Shift Attention (TSA) module. MPE is designed to retain spatial semantic information against the misalignment caused by pose, occlusion, or misdetection. TSA is designed to achieve contextual spatial semantic feature alignment and jointly model spatio-temporal clues. We further propose a Residual Position Embedding (RPE) to guide TSA in focusing on the temporal saliency clues. Experimental results on five video person reID datasets demonstrate the superiority of the proposed CAViT. Additionally, the experiment conducted on VVeRI-901-trial also shows the effectiveness of CAViT for the video vehicle reID. Our code is available on https://github.com/KimWu1994/CAViT .
Quantifying the Effectiveness of Project-Based Editing Operations in Virtual Reality

With the rapid evolution of virtual reality solutions as general-purpose computing platforms, the ability of users to integrate their complete digital life into 3D spaces is becoming increasingly important. In this paper, we focus on ways to manipulate arrangements – or layouts – of 2D content in 3D spaces. We compare three different approaches to layout creation based on variations and combinations of previously proposed solutions. Through controlled experiments, we show that users can create content layouts with high accuracy and significantly less time if they can re-use already existing content layouts. These results support the idea that the effectiveness of virtual reality platforms as universal computing platforms can be improved by incorporating a 3D file system in which spatial dashboard layouts emerge as a central organizing concept.
MaCLR: Motion-Aware Contrastive Learning of Representations for Videos

We present MaCLR, a novel method to explicitly perform cross-modal self-supervised video representations learning from visual and motion modalities. Compared to previous video representation learning methods that mostly focus on learning motion cues implicitly from RGB inputs, MaCLR enriches standard contrastive learning objectives for RGB video clips with a cross-modal learning objective between a Motion pathway and a Visual pathway. We show that the representation learned with our MaCLR method focuses more on foreground motion regions and thus generalizes better to downstream tasks. To demonstrate this, we evaluate MaCLR on five datasets for both action recognition and action detection, and demonstrate state-of-the-art self-supervised performance on all datasets. Furthermore, we show that MaCLR representation can be as effective as representations learned with full supervision on UCF101 and HMDB51 action recognition, and even outperform the supervised representation for action recognition on VidSitu and SSv2, and action detection on AVA.
Automatic Life Event Tree Generation for Older Adults

Studies have shown that learning personal stories could help provide individualized eldercare services. However, personal stories are often disordered because of the scattered collection, including informal interviews or daily interactions, which brings difficulties in acquiring valuable information quickly. One solution to this problem is to extract events from personal stories and automatically organize them in chronological order. Events extracted by current methods from social media or news corpus are mainly organized in a linear structure. These works usually focus on the event time and ignore the consistency of event contents when organizing events. This paper aims to organize events into a tree structure based on an event network, with stem nodes representing key event topics and branch nodes representing detailed events. Social workers or caregivers can clarify the life experience of the older adults quickly through the event tree and have a preliminary understanding of them. The experiments show that the event tree generated by our method has a better performance in consistency than current event organization methods. A survey study shows that our method achieves the highest logical coherence for the event tree branches compared with other algorithms.
Characterizing Multipreference Closure with System W

System W is an approach to reasoning from conditional beliefs that exhibits many properties desirable for nonmonotonic reasoning like extending rational closure, avoiding the drowning problem, and complying with syntax splitting. Multipreference closure, initially considered for reasoning in description logics with exceptions, has recently been reconstructed as a method for reasoning with conditionals based on propositional logic by Giordano and Gliozzi; this reconstruction is rather involved and complex. In this paper, we show how system W can be used to obtain a significantly less involved semantical characterization of MP-closure. To do this, we first present a representation theorem for system W using a canonical preferential model $$\mathcal {M}^{\textsf {w}}(\varDelta )$$ for a belief base $$\varDelta $$ that is obtained straightforwardly from system W. We then prove our main result, stating that $$\mathcal {M}^{\textsf {w}}(\varDelta )$$ also induces the MP-closure of $$\varDelta $$ ; furthermore, this implies that MP-closure coincides with system W inference.
Contrastive Learning for Cross-Domain Open World Recognition

The ability to evolve is fundamental for any valuable autonomous agent whose knowledge cannot remain limited to that injected by the manufacturer. Consider for example a home assistant robot: it should be able to incrementally learn new object categories when requested, but also to recognize the same objects in different environments (rooms) and poses (hand-held/on the floor/above furniture), while rejecting unknown ones. Despite its importance, this scenario has started to raise interest in the robotic community only recently and the related research is still in its infancy, with existing experimental testbeds but no tailored methods. With this work, we propose the first learning approach that deals with all the previously mentioned challenges at once by exploiting a single contrastive objective. We show how it learns a feature space perfectly suitable to incrementally include new classes and is able to capture knowledge which generalizes across a variety of visual domains. Our method is endowed with a tailored effective stopping criterion for each learning episode and exploits a self-paced thresholding strategy that provides the classifier with a reliable rejection option. Both these novel contributions are based on the observation of the data statistics and do not need manual tuning. An extensive experimental analysis confirms the effectiveness of the proposed approach in establishing the new state-of-the-art. The code is available at https://github.com/FrancescoCappio/Contrastive_Open_World.
A Single Point of Contact for Privacy Management in Cyber-Physical Systems

Cyber-Physical Systems (CPS) are a pervasive generation of socio-technical systems that become manifest in complex domains, such as healthcare. This progression challenges system architecting due to the heterogeneity, connectivity, and dynamic adaptation of CPS. In particular, cross-cutting issues, such as privacy management, require dedicated development effort, as they affect both individual components and the behavior of the overall system. The presented research introduces a Single Point of Contact architecture for handling privacy issues at design and run time.
SeNMFk-SPLIT

As the amount of text data continues to grow, topic modeling is serving an important role in understanding the content hidden by the overwhelming quantity of documents. One popular topic modeling approach is non-negative matrix factorization (NMF), an unsupervised machine learning (ML) method. Recently, Semantic NMF with automatic model selection (SeNMFk) has been proposed as a modification to NMF. In addition to heuristically estimating the number of topics, SeNMFk also incorporates the semantic structure of the text. This is performed by jointly factorizing the term frequency-inverse document frequency (TF-IDF) matrix with the co-occurrence/word-context matrix, the values of which represent the number of times two words co-occur in a predetermined window of the text. In this paper, we introduce a novel distributed method, SeNMFk-SPLIT, for semantic topic extraction suitable for large corpora. Contrary to SeNMFk, our method enables the joint factorization of large documents by decomposing the word-context and term-document matrices separately. We demonstrate the capability of SeNMFk-SPLIT by applying it to the entire artificial intelligence (AI) and ML scientific literature uploaded on arXiv.
AR Cloud: Towards Collaborative Augmented Reality at a Large-Scale

Augmented Reality (AR) appears as one of the most promising tech-nologies for the next decades with a lot of applications in Industry 4.0. Unfortunately, most of existing systems are limited to small environments and running on a single device. To cope with these limitations, we present in this paper an AR Cloud solution for real-time crowd mapping. We propose an AR cloud service architecture that is able to 1) continuously relocalize AR devices in relation to the reference coordinate system of a common global 3D map in order to estimate the initial pose of the AR device and to correct tracking drift as soon as possible; 2) locally map in 3D the real environment surrounding each AR device; 3) fuse these 3D local maps with the global 3D map in order to keep it up-to-date, mandatory feature when real environment changes over time. Finally, we show experiments of our AR cloud platform at a large scale.
Declassification Predicates for Controlled Information Release

Declassification refers to the controlled release of sensitive information by a program. It is well recognised that security analyses, formal or otherwise, need to take declassification into account to be practically usable. This paper introduces the concept of declassification predicates which enable a programmer to define precisely what can be declassified in a given program, and where in the program it can be released. We show how declassification predicates can be added to an existing information flow logic, and how the extended logic can be implemented within the Dafny program verifier.
Attention-based Broad Self-guided Network for Low-light Image Enhancement

Low-light image enhancement is widely used in many fields, such as target detection, face recognition, and image segmentation. In recent years, Deep Learning methods have achieved impressive breakthroughs in low-light image enhancement. However, most of them mine high-dimensional features of images by stacking network structures and deepening the depth of the network, which causes more runtime costs for single image enhancement. To reduce inference time while fully extracting local features and global features of low-light images, we propose an Attention-based Broad Self-guided Network for real-world low-light image Enhancement. Compared to U-net structure [1], such a self-guided structure requires a smaller number of parameters and enables us to achieve better effectiveness. To extract local information more efficiently, we designed a Multilevel Guided Densely Connected Attention block, which can be considered as a novel extension of the densely connected blocks in feature space. In addition, we also offer a more efficient module to extract global information from images, which is called the Global Spatial Attention module. The proposed network is validated by lots of mainstream benchmarks. Many experimental results show that the proposed network outperforms most state-of-the-art low-light image Enhancement methods. Our code is available at https://github.com/paullenwyue/ABSGNet.
DenseNet-based Attention Network to recognize Handwritten Mathematical Expressions

Machine learning-based translation models have recently hit the trend line and impacted the pace of research in Natural Language Processing. The emerging trend of the era witnesses' transformation networks overpowering all existing influences of deep learning-based systems: the intelligent transformer system's attention-based encoder-decoder's robustness and exceptional performance display scope for modest research. In terms of handwritten mathematical text recognition, these systems need more attention. This article presents an attention-based encoder-decoder-based transformation model that has been produced and trained for an extensive database of mathematical expressions that have been collected from localities of Punjab and Madhya Pradesh schools, colleges and universities. Mathematical expressions are an essential component of education and scientific learning opportunities. Thus, the article's novel approach is to predict them based on their handwritten source using an encoder-decoder-based neural network. Massively, 101400 images from the corpus have been preprocessed, segmented, and recognized using the built network. The proposed attention-based dense encoder with GRU-based Decoder with attention mechanics gives us an ExpRate of 57.4% on the created corpus. It exhibits a competent ExpRate of 58% on CROHME 2016, outperforming many state-of-the-art models.
Towards Refining ID Cards Presentation Attack Detection Systems using Face Quality Index

Remote identity verification systems which rely on official documents, such as government-issued ID cards, require the use of Presentation Attack Detection (PAD) systems in order to combat impostor attacks. These attacks might use composition or splicing techniques, a printed ID card, or an image of the document displayed on a screen, altered prior to its presentation to the verification system. Based on recent efforts, Face Image Quality Assessment may help PAD systems in identifying ID card image tampering by utilising the photo ID section of the document, where a low face image quality could be related to printed, screen, or composite spoofed ID cards. This work presents an early approach towards complementing ID cards PAD systems by employing MagFace, a quality-aware face recognition method, in conjunction with a Chilean ID card PAD system, analysing the influence of the quality index of photo IDs on the system and its effectiveness when a prior rejection of low quality ID card images is made.
CodeBERT for Code Clone Detection: A Replication Study

Large pre-trained models have dramatically improved the state-of-the-art on a variety of natural language processing (NLP) tasks. CodeBERT is one such pre-trained model for natural language (NL) and programming language (PL) which captures the semantics in natural language and programming language, and produces general-purpose representations. While it has been shown to support natural language code search and code documentation generation tasks, its effectiveness for code clone detection is not explored in depth. In this paper, we aim to replicate and evaluate the performance of CodeBERT for code clone detection on multiple datasets with varying functionalities to understand (1) whether CodeBERT can generalize to unseen code, (2) how fine-tuning can effect CodeBERT’s performance on unseen code, and (3) how CodeBERT performs for detecting various code clone types. To this end, we consider three different datasets of Java methods. We derive the first dataset from Big-CloneBench. We use Java clone pairs from SemanticCloneBench to derive our second dataset, and our third dataset consists of Java methods from Android applications. Our experiments indicate that CodeBERT performs the best for detecting Type-1 and Type-4 clones with a 100% and 96% recall on average respectively. We also find that there is limited generalizability on unseen functionalities where recall drops by 15% and 40% on the SemanticCloneBench and Android datasets respectively. Furthermore, we observe that fine-tuning can improve the recall by 22% and 30% on the SemanticCloneBench and Android datasets respectively.
OERL: Enhanced Representation Learning via Open Knowledge Graphs

The sparseness and incompleteness of knowledge graphs (KGs) trigger considerable interest in enhancing the representation learning with external corpora. However, the difficulty of aligning entities and relations with external corpora leads to inferior performance improvement. Open knowledge graphs (OKGs) consist of entity-mentions and relation-mentions that are represented by noncanonicalized freeform phrases, which generally do not rely on the specification of ontology schema. The roughness of the nonontological construction method leads to a specific characteristic of OKGs: diversity, where multiple entity-mentions (or relation-mentions) have the same meaning but different expressions. The diversity of OKGs can provide potential textual and structural features for the representation learning of KGs. We speculate that leveraging OKGs to enhance the representation learning of KGs can be more effective than using pure text or pure structure corpora. In this paper, we propose a new <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OERL</i> , <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</b> pen knowledge graph <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</b> nhanced <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</b> epresentation <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</b> earning of KGs. OERL automatically extracts textual and structural connections between KGs and OKGs, models and transfers refined profitable features to enhance the representation learning of KGs. The strong performance improvement and exhaustive experimental analysis prove the superiority of OERL over state-of-the-art baselines.
Connecting the Complementary-view Videos: Joint Camera Identification and Subject Association

We attempt to connect the data from complementary views, i.e., top view from drone-mounted cameras in the air, and side view from wearable cameras on the ground. Collaborative analysis of such complementary-view data can facilitate to build the air-ground cooperative visual system for various kinds of applications. This is a very challenging problem due to the large view difference between top and side views. In this paper, we develop a new approach that can simultaneously handle three tasks: i) localizing the side-view camera in the top view; ii) estimating the view direction of the side-view camera; iii) detecting and associating the same subjects on the ground across the complementary views. Our main idea is to explore the spatial position layout of the subjects in two views. In particular, we propose a spatial-aware position representation method to embed the spatial-position distribution of the subjects in different views. We further design a cross-view video collaboration framework composed of a camera identification module and a subject association module to simultaneously perform the above three tasks. We collect a new synthetic dataset consisting of top-view and side-view video sequence pairs for performance evaluation and the experimental results show the effectiveness of the proposed method.