Reviewerly/exHarmony huggingface dataset:
Multiple layered Security using combination of Cryptography with Rotational, Flipping Steganography and Message Authentication

Data or information are being transferred at an enormous pace and hence protecting and securing this transmission of data are very important and have been very challenging. Cryptography and Steganography are the most broadly used techniques for safeguarding data by encryption of data and hiding the existence of data. A multi-layered secure transmission can be achieved by combining Cryptography with Steganography and by adding message authentication ensuring the confidentiality of the message. Different approach towards Steganography implementation is proposed using rotations and flips to prevent detection of encoded messages. Compression of multimedia files is set up for increasing the speed of encoding and consuming less storage space. The HMAC (Hash-based Authentication Code) algorithm is chosen for message authentication and integrity. The performance of the proposed Steganography methods is concluded using Histogram comparative analysis. Simulations have been performed to back the reliability of the proposed method.
Episodic Memory Question Answering

Egocentric augmented reality devices such as wearable glasses passively capture visual data as a human wearer tours a home environment. We envision a scenario wherein the human communicates with an AI agent powering such a device by asking questions (e.g., “where did you last see my keys?”). In order to succeed at this task, the egocentric AI assistant must (1) construct semantically rich and efficient scene memories that encode spatio-temporal infor-mation about objects seen during the tour and (2) possess the ability to understand the question and ground its answer into the semantic memory representation. Towards that end, we introduce (1) a new task - Episodic Memory Question Answering (EMQA) wherein an egocentric AI assistant is provided with a video sequence (the tour) and a question as an input and is asked to localize its answer to the question within the tour, (2) a dataset of grounded questions designed to probe the agent's spatio-temporal understanding of the tour, and (3) a model for the task that encodes the scene as an allocentric, top-down semantic feature map and grounds the question into the map to localize the answer. We show that our choice of episodic scene memory outperforms naive, off-the-shelf solutions for the task as well as a host of very competitive baselines and is robust to noise in depth, pose as well as camera jitter.
Models and mechanisms for spatial data fairness

Fairness in data-driven decision-making studies scenarios where individuals from certain population segments may be unfairly treated when being considered for loan or job applications, access to public resources, or other types of services. In location-based applications, decisions are based on individual whereabouts, which often correlate with sensitive attributes such as race, income, and education. While fairness has received significant attention recently, e.g., in machine learning, there is little focus on achieving fairness when dealing with location data. Due to their characteristics and specific type of processing algorithms, location data pose important fairness challenges. We introduce the concept of spatial data fairness to address the specific challenges of location data and spatial queries. We devise a novel building block to achieve fairness in the form of fair polynomials. Next, we propose two mechanisms based on fair polynomials that achieve individual spatial fairness, corresponding to two common location-based decision-making types: distance-based and zone-based. Extensive experimental results on real data show that the proposed mechanisms achieve spatial fairness without sacrificing utility.
Fake Speech Detection Using OpenSMILE Features

With the advancement of technology in deep learning, we have developed methods that generate fake speech, which is impossible to differentiate from a natural speech by an ordinary person perceptually. Fake speech can be used maliciously to harm society or a person (impersonation, fake news spreading, etc.), so we need to develop methods to detect fake speech. Several features have been proposed in the literature that can identify fake speech. Each of those features has different contributions to the detection task. In this work, we propose to use the most common speech features together for fake speech detection. openSMILE toolkit is an open-source library that extracts the most common speech features and stores them in the vector of 88 dimensions. We use these features over machine learning models to detect fake speech. To check the robustness of the proposed method, we test it over various datasets that contain session, gender, domain, and synthesizer variability. The experimental results on the different variabilities showed that the openSMILE features were able to detect the fake speech in session, gender and synthesizer variability with high performance, whereas the performance is low with the domain variability conditions.
Surface Vessels Control: Case Study of a Complex Project with Related Courses

This paper addresses the development of the robotic vessel setup for academic purposes. The setup provides experimental testing of algorithms for computer vision to measure coordinates of the vessel establishing the feedback and motion control based on this feedback steering the vessel towards the given reference. The paper describes the structure of the setup and discusses problems and approaches relevant to dynamic positioning. The developed setup can be helpful for students in gaining hands-on experience in implementing the computer vision and motion control algorithms to the scaled but still real-life application.
Workshop YOUR study design! Participatory critique and refinement of participants' studies

HRI is an interdisciplinary field that requires researchers to be knowledgeable in broad areas ranging from social sciences to engineering.Study design is a multifaceted aspect of HRI that is hard to develop and perfect.Thus, the second edition of the "Workshop Your Study Design" workshop aims to improve the quality of future HRI studies by training researchers and boosting the accessibility of HRI as a field.Participants will have the opportunity to receive guidance and feedback on their study from an expert mentor.Researchers from all avenues of HRI will be invited to submit a 2-4 page paper on an HRI study they are currently designing, including a brief introduction and a complete methods section.Accepted submissions will be discussed in small groups led by mentors with relevant expertise.Prior to the workshop, papers will be shared within each group.Participants will be encouraged to read other submissions.During the workshop, attendees will work within their mentee-mentor groups to discuss each paper and provide feedback.There will also be a session where mentors lead mini discussions on topics important to study design, such as balancing qualitative and quantitative design, power analysis, and research ethics.The workshop will end with a session where all participants can share important lessons that they learned with fellow attendees.
INTERACTION: A Generative XAI Framework for Natural Language Inference Explanations

XAI with natural language processing aims to produce human-readable explanations as evidence for AI decision-making, which addresses explainability and transparency. However, from an HCI perspective, the current approaches only focus on delivering a single explanation, which fails to account for the diversity of human thoughts and experiences in language. This paper thus addresses this gap, by proposing a generative XAI framework, INTERACTION (explain aNd predicT thEn queRy with contextuAl CondiTional varIational autO-eNcoder). Our novel framework presents explanation in two steps: (step one) Explanation and Label Prediction; and (step two) Diverse Evidence Generation. We conduct intensive experiments with the Transformer architecture on a benchmark dataset, e-SNLI [1]. Our method achieves competitive or better performance against state-of-the-art baseline models on explanation generation (up to 4.7% gain in BLEU) and prediction (up to 4.4% gain in accuracy) in step one; it can also generate multiple diverse explanations in step two.
Fisheye object detection based on standard image datasets with 24-points regression strategy

Fisheye object detection is a difficult task in robotics and autonomous driving. One of the reasons is that the fisheye datasets are inferior to standard image datasets in scale and quantity, which inspires the idea of using standard image datasets for fisheye object detection. However, the models trained on standard image datasets do not perform well with fisheye data. In this work, we explore the effect of fisheye images on different stages of the YOLOX with published weights generated by standard image datasets. We also propose a new regression strategy for 24-points object representation method, which is insensitive to image distortion. The experiments show that the feature extraction part is robust to fisheye image features, while the regression part of location and category performs poorly. The strategy can achieve the position of discrete points without calculating the IOU of irregular-shaped boxes. Theoretically, the strategy can be widely adopted to regress the irregular bounding boxes composed of discrete points. Source code is at https://github.com/IN2-ViAUn/Exploration-of-Potential.
Deep Hyperspectral-Depth Reconstruction Using Single Color-Dot Projection

Depth reconstruction and hyperspectral reflectance reconstruction are two active research topics in computer vision and image processing. Conventionally, these two topics have been studied separately using independent imaging setups and there is no existing method which can acquire depth and spectral reflectance simultaneously in one shot without using special hardware. In this paper, we propose a novel single-shot hyperspectral-depth reconstruction method using an off-the-shelf RGB camera and projector. Our method is based on a single color-dot projection, which simultaneously acts as structured light for depth reconstruction and spatially-varying color illuminations for hyperspectral reflectance reconstruction. To jointly reconstruct the depth and the hyperspectral reflectance from a single color-dot image, we propose a novel end-to-end network architecture that effectively incorporates a geometric color-dot pattern loss and a photometric hyperspectral reflectance loss. Through the experiments, we demonstrate that our hyperspectral-depth reconstruction method outperforms the combination of an existing state-of-the-art single-shot hyperspectral reflectance reconstruction method and depth reconstruction method.
Collaborative Rank Aggregation in Recommendation Systems

Over the years, various techniques of generating recommendations have been developed. However, it turns out that when we compare the recommendations generated by different algorithms in the context of a particular user, the quality of such recommendations for different techniques may differ. The use of the aggregation techniques, the aim of which is to combine several rankings into one, can be a solution to this problem. In theory it should improve the quality of the recommendations. Additionally, in order to personalize the recommendations better, a metaheuristic algorithm, which, by assigning different weights to each feature, tries to represent the preference of the active user, was used. This paper also presents a suggestion to include additional rankings generated for other users in the system in the aggregation process. The idea will be supported by research results that clearly show that taking into account rankings of other users can improve the quality of the generated recommendations.
Attention Mechanism in Convolutional Recurrent Neural Network for Improving Recognition Accuracy in Printed Devanagari Text

Modeling accurate text recognizers for scanned Devanagari scripts is a challenging task, primarily due to the ingrained complexities of characters, vowels, conjuncts, and modifiers in the word formation of the script. This paper presents an attention-based convolutional recurrent neural network (CRNN) for improving the text recognition accuracy of the Devanagari script images. The proposed model is an encoder–decoder network with an attention mechanism. The encoder is a convolutional neural network (CNN), which extracts the significant and high-level features from the input images. The encoder also generates the extracted features in an ordered sequence. The decoder uses a bidirectional long short-term (BLSTM) network with an attention mechanism, followed by a connectionist temporal classification (CTC) layer to predict the text in images. The attention mechanism enables the decoder to selectively exploit the high-level features of the encoder congruently. We have trained, validated, and tested the performance of the proposed model on three datasets, viz. publicly available IIIT-H scene image dataset, synthetic dataset, and real dataset curated during this work. We have also compared the proposed model results with work already attempted in the area for the three datasets. We found that the performance of our proposed model outdoes the outcomes obtained from other models on the three datasets.
DeepPCT: Single Object Tracking in Dynamic Point Cloud Sequences

Three-dimensional single object tracking (SOT) in dynamic point cloud sequences is a fundamental problem for numerous intelligent systems. In this article, we propose a deep neural architecture named DeepPCT to achieve accurate and robust tracking in sequential point clouds. Instead of relying on the bounding box classification and regression to localize the object, the proposed framework directly formulates the intersection-over-union (IoU) prediction as its objectives for accurate 3-D localization. A Center IoU (CIoU) is further proposed to mitigate the ambiguity of standard IoU. In addition, we also investigate the impact of an additional re-detection module to the overall tracking performance, since tracking failures frequently occur due to the large variations and error accumulation. Experimental results on the KITTI benchmark demonstrate that the proposed tracker achieves competitive performance, with a 3-D precision rate of 63.7% and a 3-D success rate of 45.0%. Moreover, our tracker achieves compelling performance in the wild with severe illumination variations and occlusions, and even outperforms several 2-D counterpart trackers.
OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses

Dataset bias and spurious correlations can significantly impair generalization in deep neural networks. Many prior efforts have addressed this problem using either alternative loss functions or sampling strategies that focus on rare patterns. We propose a new direction: modifying the network architecture to impose inductive biases that make the network robust to dataset bias. Specifically, we propose OccamNets, which are biased to favor simpler solutions by design. OccamNets have two inductive biases. First, they are biased to use as little network depth as needed for an individual example. Second, they are biased toward using fewer image locations for prediction. While OccamNets are biased toward simpler hypotheses, they can learn more complex hypotheses if necessary. In experiments, OccamNets outperform or rival state-of-the-art methods run on architectures that do not incorporate these inductive biases. Furthermore, we demonstrate that when the state-of-the-art debiasing methods are combined with OccamNets ( https://github.com/erobic/occam-nets-v1 ) results further improve.
Benchmarking Augmentation Methods for Learning Robust Navigation Agents: the Winning Entry of the 2021 iGibson Challenge

Recent advances in deep reinforcement learning and scalable photorealistic simulation have led to increasingly mature embodied AI for various visual tasks, including navigation. However, while impressive progress has been made for teaching embodied agents to navigate static environments, much less progress has been made on more dynamic environments that may include moving pedestrians or movable obstacles. In this study, we aim to benchmark different augmentation techniques for improving the agent's performance in these challenging environments. We show that adding several dynamic obstacles into the scene during training confers significant improvements in test-time generalization, achieving much higher success rates than baseline agents. We find that this approach can also be combined with image augmentation methods to achieve even higher success rates. Additionally, we show that this approach is also more robust to sim-to-sim transfer than image augmentation methods. Finally, we demonstrate the effectiveness of this dynamic obstacle augmentation approach by using it to train an agent for the 2021 iGibson Challenge at CVPR, where it achieved 1 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">st</sup> place for Interactive Navigation.
Immersive WYSIWYG virtual meteorological sandbox

It has emerged that a novel way for the visual representation of meteorological data with the maturing of Immersive Analytics(IA). We propose an immersive meteorological virtual sandbox to resolve the problems of restricted representation and data perception in 2D analysis. It is a new visual method that allows users to interact directly with data by utilizing non-contact aerial gestures (NCAG). The fact that IA is currently not prevalent in weather visualization. In our system, we propose a new flow of visual exploration in meteorology based on the “What You See Is What You Get” (WYSIWYG) concept from scientific visualization, which can address the problems of weather data analysis in an immersive environment. Finally, we conduct a questionnaire for users who are invited to test our system and assess our work. The evaluation demonstrates that our system can reduce the cognitive burden, reduce the mental workload, and strengthen user's spatial memory.
MCL: A Contrastive Learning Method for Multimodal Data Fusion in Violence Detection

Multimodal learning among video and audio has shown significant performance improvement in violence detection. However, video and audio do not contribute consistently, and the video modality tends to dominate when determining whether a certain scene contains violent events. In fact, a few recent multimodal learning methods for violence detection do not fully consider data differences between various modalities, which lead to optimization imbalance problem during training, and ultimately result in insufficient performance. To address this issue, we propose a Multimodal Contrastive Learning (MCL) method to make full use of video and audio information for violence detection. In specific, to avoid the video modality dominating the model training, we design a multi-encoder framework to perform task-driven feature encoding on video and audio respectively. To reduce information loss during multimodal fusion, we introduce a contrastive learning task to capture semantically consistent representations. We conduct extensive experiments on XD-Violence dataset, showing that our proposed MCL achieves an average precision improvement of 2.34% against the state-of-the-art baseline.
OSANet: Object Semantic Attention Network for Visual Sentiment Analysis

Visual sentiment analysis aims to predict human emotional responses to visual stimuli. It has attracted considerable attention owing to the increasing popularity of online image sharing. Most researchers have focused on improving emotion recognition using holistic and local information derived from given images. Relatively less attention has been paid to the semantic information of objects in images, which influences human emotional responses to the images. Therefore, we propose a novel object semantic attention network (OSANet) that attempts to unravel the semantic information of objects in images that contribute to emotion detection. The OSANet combines both global representation and semantic information of objects to predict the emotion elicited by a given image. First, the holistic features that represent the entire image are extracted using convolutional blocks. Subsequently, the object-level semantic information is obtained from pre-trained word embedding and then weighted according to the relative importance of the object using the attention mechanism. Notably, a new loss function to address the subjectivity of sentiment analysis is introduced, which improves the performance of the emotion detection task. Extensive experiments on three image emotion datasets demonstrated the superiority and interpretability of the OSANet. The results show that the OSANet outperforms extant image emotion detection frameworks.
PANDORA: Polarization-Aided Neural Decomposition of Radiance

Reconstructing an object’s geometry and appearance from multiple images, also known as inverse rendering, is a fundamental problem in computer graphics and vision. Inverse rendering is inherently ill-posed because the captured image is an intricate function of unknown lighting, material properties and scene geometry. Recent progress in representing scene through coordinate-based neural networks has facilitated inverse rendering resulting in impressive geometry reconstruction and novel-view synthesis. Our key insight is that polarization is a useful cue for neural inverse rendering as polarization strongly depends on surface normals and is distinct for diffuse and specular reflectance. With the advent of commodity on-chip polarization sensors, capturing polarization has become practical. We propose PANDORA, a polarimetric inverse rendering approach based on implicit neural representations. From multi-view polarization images of an object, PANDORA jointly extracts the object’s 3D geometry, separates the outgoing radiance into diffuse and specular and estimates the incident illumination. We show that PANDORA outperforms state-of-the-art radiance decomposition techniques. PANDORA outputs clean surface reconstructions free from texture artefacts, models strong specularities accurately and estimates illumination under practical unstructured scenarios.
INVERSOR OF DIGITS OF TWO-BASE G–REPRESENTATION OF REAL NUMBERS AND ITS STRUCTURAL FRACTALITY

In the paper, we introduce a new two-symbol system of representation for numbers from segment $[0;0,5]$ with alphabet (set of digits) $A=\{0;1\}$ and two bases 2 and $-2$: \[x=\dfrac{\alpha_1}{2}+\dfrac{1}{2}\sum\limits^\infty_{k=1}\dfrac{\alpha_{k+1}}{2^{k-(\alpha_1+\ldots+\alpha_k)}(-2)^{\alpha_1+\ldots+\alpha_k}}\equiv \Delta^{G}_{\alpha_1\alpha_2\ldots\alpha_k\ldots}, \;\;\; \alpha_k\in \{0;1\}.\] We compare this new system with classic binary system. The function $I(x=\Delta^G_{\alpha_1\ldots \alpha_n\ldots})=\Delta^G_{1-\alpha_1,\ldots, 1-\alpha_n\ldots}$, such that digits of its $G$--representation are inverse (opposite) to digits of $G$--representation of argument is considered in detail. This function is well-defined at points having two $G$--representations provided we use only one of them. We prove that inversor is a function of unbounded variation, continuous function at points having a unique $G$--representation, and right- or left-continuous at points with two representations. The values of all jumps of the function are calculated. We prove also that the function does not have monotonicity intervals and its graph has a self-similar structure.