Reviewerly/exHarmony huggingface dataset:
Guide dog versus robot dog: assembling visually impaired people with non-human agents and achieving assisted mobility through distributed co-constructed perception

Guide dogs are sense-able agents that can assist Visually Impaired Persons (VIP) to achieve mobility. But could a guide dog be replaced by a robot dog? Based on video recordings and ethnomethodological 'conversation analysis' of VIPs who are mobile in a street environment with a remotely operated robodog or a guide dog, respectively, this paper shows the multisensory and semiotic capacities of non-human agents as assistants in navigational activities. It also highlights the differences between their type of agency and sense-ability, and thus their different roles in situations of assisted mobility and disability mobility. This paper contributes to research in assisted and disability mobility between humans and non-humans by showing how they work not as individual agents, but as 'VIP + guide dog' and 'VIP + robodog + operator' assemblages, and by demonstrating that these assemblages distribute and co-construct the practical perception of the material world which is necessary for accomplishing mobility.
A New Robust and Secure 3-Level Digital Image Watermarking Method Based on G-BAT Hybrid Optimization

This contribution applies tools from the information theory and soft computing (SC) paradigms to the embedding and extraction of watermarks in aerial remote sensing (RS) images to protect copyright. By the time 5G came along, Internet usage had already grown exponentially. Regarding copyright protection, the most important responsibility of the digital image watermarking (DIW) approach is to provide authentication and security for digital content. In this paper, our main goal is to provide authentication and security to aerial RS images transmitted over the Internet by the proposal of a hybrid approach using both the redundant discrete wavelet transform (RDWT) and the singular value decomposition (SVD) schemes for DIW. Specifically, SC is adopted in this work for the numerical optimization of critical parameters. Moreover, 1-level RDWT and SVD are applied on digital cover image and singular matrices of LH and HL sub-bands are selected for watermark embedding. Further selected singular matrices SLH and SHL are split into 3×3 non-overlapping blocks, and diagonal positions are used for watermark embedding. Three-level symmetric encryption with low computational cost is used to ensure higher watermark security. A hybrid grasshopper–BAT (G-BAT) SC-based optimization algorithm is also proposed in order to achieve high quality DIW outcomes, and a broad comparison against other methods in the state-of-the-art is provided. The experimental results have demonstrated that our proposal provides high levels of imperceptibility, robustness, embedding capacity and security when dealing with DIW of aerial RS images, even higher than the state-of-the-art methods.
A Consistent Dual-MRC Framework for Emotion-cause Pair Extraction

Emotion-cause pair extraction (ECPE) is a recently proposed task that aims to extract the potential clause pairs of emotions and its corresponding causes in a document. In this article, we propose a new paradigm for the ECPE task. We cast the task as a two-turn machine reading comprehension (MRC) task, i.e., the extraction of emotions and causes is transformed to the task of identifying answer clauses from the input document specific to a query. This two-turn MRC formalization brings several key advantages: First, the QA manner provides an explicit pairing way to identify causes specific to the target emotion; second, it provides a natural way of jointly modeling the emotion extraction, the cause extraction, and the pairing of emotion and cause; and third, it allows us to exploit the well-developed MRC models. Based on the two-turn MRC formalization, we propose a dual-MRC framework to extract emotion-cause pairs in a dual-direction way, which enables a more comprehensive coverage of all pairing cases. Furthermore, we propose a consistent training strategy for the second-turn query, so the model is able to filter the errors produced by the first turn at inference. Experiments on two benchmark datasets demonstrate that our method outperforms previous methods and achieves state-of-the-art performance. All the code and data of this work can be obtained at https://github.com/zifengcheng/CD-MRC .
Exploring Multi-lingual, Multi-task, and Adversarial Learning for Low-resource Sentiment Analysis

Deep learning has become most prominent in solving various Natural Language Processing (NLP) tasks including sentiment analysis. However, these techniques require a considerably large amount of annotated corpus, which is not easy to obtain for most of the languages, especially under the scenario of low-resource settings. In this article, we propose a deep multi-task multi-lingual adversarial framework to solve the resource-scarcity problem of sentiment analysis by leveraging the useful and relevant knowledge from a high-resource language. To transfer the knowledge between the different languages, both the languages are mapped to the shared semantic space using cross-lingual word embeddings. We evaluate our proposed architecture on a low-resource language, Hindi, using English as the high-resource language. Experiments show that our proposed model achieves an accuracy of 60.09% for the movie review dataset and 72.14% for the product review dataset. The effectiveness of our proposed approach is demonstrated with significant performance gains over the state-of-the-art systems and translation-based baselines.
Integration of Multi-Head Self-Attention and Convolution for Person Re-Identification

Person re-identification is essential to intelligent video analytics, whose results affect downstream tasks such as behavior and event analysis. However, most existing models only consider the accuracy, rather than the computational complexity, which is also an aspect to consider in practical deployment. We note that self-attention is a powerful technique for representation learning. It can work with convolution to learn more discriminative feature representations for re-identification. We propose an improved multi-scale feature learning structure, DM-OSNet, with better performance than the original OSNet. Our DM-OSNet replaces the 9×9 convolutional stream in OSNet with multi-head self-attention. To maintain model efficiency, we use double-layer multi-head self-attention to reduce the computational complexity of the original multi-head self-attention. The computational complexity is reduced from the original O((H×W)2) to O(H×W×G2). To further improve the model performance, we use SpCL to perform unsupervised pre-training on the large-scale unlabeled pedestrian dataset LUPerson. Finally, our DM-OSNet achieves an mAP of 87.36%, 78.26%, 72.96%, and 57.13% on the Market1501, DukeMTMC-reID, CUHK03, and MSMT17 datasets.
Efficient approach of sliding window-based high average-utility pattern mining with list structures

Data mining has been actively studied, and it has become more important due to the development of information technology and the demands of diverse applications, such as the retail market, medical field, and manufacturing plants. High average-utility pattern mining is a type of data mining that finds valuable patterns by dividing the utility of a pattern by the length of the pattern. It considers the length of the pattern, so it can handle the demerits of utility pattern mining and extract more important patterns. Meanwhile, stream data is produced in real time, and the importance of the recent data gradually increases as opposed to the old data. However, most of the previous high average utility pattern mining approaches cannot find the latest valuable patterns over the data streams. Although research using a sliding window model was conducted to handle only the recent data, it has a problem due to the data structure. Since the previous work is a tree-based approach, numerous candidate patterns are generated, and an additional database scan is required to calculate the actual utilities of candidates. Creating the candidate patterns reduces the efficiency of runtime and memory and makes real-time data processing difficult. We propose an efficient algorithm that uses a list structure for sliding window-based high average-utility pattern mining in this paper. The proposed algorithm scans the stream data once, mines the complete set of the recent high average-utility patterns without generating candidates, and reduces the search space by using a novel pruning technique. The experiment results show that the proposed algorithm outperforms the state-of-the-art algorithms in terms of the runtime, memory, and scalability in real and synthetic datasets.
Space-time dual quantum Zeno effect: Interferometric engineering of open quantum system dynamics

Superposition of trajectories, which modify quantum evolutions by superposing paths through interferometry, has been utilized to enhance various quantum communication tasks. However, little is known about its impact from the viewpoint of open quantum systems. Thus, we examine this subject from the perspective of system-environment interactions. We show that the superposition of multiple trajectories can result in quantum state freezing, suggesting a space-time dual to the quantum Zeno effect. Moreover, non-trivial Dicke-like super(sub)radiance can be triggered without utilizing multi-atom correlations.
Reservoir computing based on an external-cavity semiconductor laser with optical feedback modulation

We numerically and experimentally investigate reservoir computing based on a single semiconductor laser with optical feedback modulation. In this scheme, an input signal is injected into a semiconductor laser via intensity or phase modulation of the optical feedback signal. We perform a chaotic time-series prediction task using the reservoir and compare the performances of intensity and phase modulation schemes. Our results indicate that the feedback signal of the phase modulation scheme outperforms that of the intensity modulation scheme. Further, we investigate the performance dependence of reservoir computing on parameter values and observe that the prediction error improves for large injection currents, unlike the results in a semiconductor laser with an optical injection input. The physical origin of the superior performance of the phase modulation scheme is analyzed using external cavity modes obtained from steady-state analysis in the phase space. The analysis indicates that high-dimensional mapping can be achieved from the input signal to the trajectory of the response laser output by using phase modulation of the feedback signal.
An analysis of emotions and the prominence of positivity in #BlackLivesMatter tweets

Emotions are a central driving force of activism; they motivate participation in movements and encourage sustained involvement. We use natural language processing techniques to analyze emotions expressed or solicited in tweets about 2020 Black Lives Matter protests. Traditional off-the-shelf emotion analysis tools often fail to generalize to new datasets and are unable to adapt to how social movements can raise new ideas and perspectives in short time spans. Instead, we use a few-shot domain adaptation approach for measuring emotions perceived in this specific domain: tweets about protests in May 2020 following the death of George Floyd. While our analysis identifies high levels of expressed anger and disgust across overall posts, it additionally reveals the prominence of positive emotions (encompassing, e.g., pride, hope, and optimism), which are more prevalent in tweets with explicit pro-BlackLivesMatter hashtags and correlated with on the ground protests. The prevalence of positivity contradicts stereotypical portrayals of protesters as primarily perpetuating anger and outrage. Our work offers data, analyses, and methods to support investigations of online activism and the role of emotions in social movements.
3DRRDB: Super Resolution of Multiple Remote Sensing Images using 3D Residual in Residual Dense Blocks

The rapid advancement of Deep Convolutional Neural Networks helped in solving many remote sensing problems, especially the problems of super-resolution. However, most state-of-the-art methods focus more on Single Image Super-Resolution neglecting Multi-Image Super-Resolution. In this work, a new proposed 3D Residual in Residual Dense Blocks model (3DRRDB) focuses on remote sensing Multi-Image Super-Resolution for two different single spectral bands. The proposed 3DRRDB model explores the idea of 3D convolution layers in deeply connected Dense Blocks and the effect of local and global residual connections with residual scaling in Multi-Image Super-Resolution. The model tested on the Proba-V challenge dataset shows a significant improvement above the current state-of-the-art models scoring a Corrected Peak Signal to Noise Ratio (cPSNR) of 48.79 dB and 50.83 dB for Near Infrared (NIR) and RED Bands respectively. Moreover, the proposed 3DRRDB model scores a Corrected Structural Similarity Index Measure (cSSIM) of 0.9865 and 0.9909 for NIR and RED bands respectively
Attention Consistency on Visual Corruptions for Single-Source Domain Generalization

Generalizing visual recognition models trained on a single distribution to unseen input distributions (i.e. domains) requires making them robust to superfluous correlations in the training set. In this work, we achieve this goal by altering the training images to simulate new domains and imposing consistent visual attention across the different views of the same sample. We discover that the first objective can be simply and effectively met through visual corruptions. Specifically, we alter the content of the training images using the nineteen corruptions of the ImageNet-C benchmark and three additional transformations based on Fourier transform. Since these corruptions preserve object locations, we propose an attention consistency loss to ensure that class activation maps across original and corrupted versions of the same training sample are aligned. We name our model Attention Consistency on Visual Corruptions (ACVC). We show that ACVC consistently achieves the state of the art on three single-source domain generalization benchmarks, PACS, COCO, and the large-scale DomainNet <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Self-Attention with Convolution and Deconvolution for Efficient Eye Gaze Estimation from a Full Face Image

This paper proposes a whole new face image-based eye gaze estimation network to solve low generalization performance. Due to the high variance of facial appearance and environmental conditions, conventional methods in gaze estimation have low generalization performance and are easily overfitted to training subjects. To solve this problem, we adopt a self-attention mechanism that has better generalization performance. Nevertheless, applying self-attention directly to an image incurs a high computational cost. Thus, we introduce a new projection that uses convolution in the entire face image to accurate model the local context and reduce the computational cost of self-attention. The pro-posed model also includes deconvolution that transforms the down-sampled global context to the same size as the input so that spatial information is not lost. We confirmed through observations that the new method achieved state of the art on the EYEDIAP, MPIIFaceGaze, Gaze360 and RT-GENE datasets and achieved a performance increase of 0.02° to 0.30° compared to the other state of the art model. In addition, we show the generalization performance of the proposed model through a cross-dataset evaluation.
Continually Learning Self-Supervised Representations with Projected Functional Regularization

Recent self-supervised learning methods are able to learn high-quality image representations and are closing the gap with supervised approaches. However, these methods are unable to acquire new knowledge incrementally – they are, in fact, mostly used only as a pre-training phase over IID data. In this work we investigate self-supervised methods in continual learning regimes without any replay mechanism. We show that naive functional regularization, also known as feature distillation, leads to lower plasticity and limits continual learning performance. Instead, we propose Projected Functional Regularization in which a separate temporal projection network ensures that the newly learned feature space preserves information of the previous one, while at the same time allowing for the learning of new features. This prevents forgetting while maintaining the plasticity of the learner. Comparison with other incremental learning approaches applied to self-supervision demonstrates that our method obtains competitive performance in different scenarios and on multiple datasets.
Efficient Two-stage Model Retraining for Machine Unlearning

With the rise of the General Data Protection Regulation (GDPR), user data holders should guarantee the "individual’s right to be forgotten". It means user data holders must completely remove user data when they receive the request. However, enabling a deep learning model to exclude specific data used during training is challenging. We cannot easily define the meaning of "forgetting" in deep learning and how to achieve it. To address this issue, we propose an efficient machine unlearning architecture to be used for computer vision classification models. Our approach consists of two-stage models, where in the first stage we enables a deep learning model that loses information with contrastive labels in the requested dataset. Second, we retrain the first stage output model with knowledge distillation (KD). Using this two-stage approach, we can substantiate the removal or forgetness of the requested dataset in the deep learning model. With various datasets used for multimedia applications, we demonstrate that our approach achieves performance on par or even higher accuracy than the original model, while effectively removing the requested data.
Box-Grained Reranking Matching for Multi-Camera Multi-Target Tracking

Multi-Camera Multi-Target tracking (MCMT) is an essential task in intelligent transportation systems. It is highly challenging due to several problems such as heavy occlusion and appearance variance caused by various camera perspectives and congested vehicles. In this paper, we propose a practical framework for dealing with the city-scale MCMT task, consisting of four modules. The vehicles detection and ReID feature extraction are the first two modules, which locate all vehicles and extract the appearance features for all cameras. The third module is Single-Camera Multi-Target tracking (SCMT), which tracks multiple vehicles to generate candidate trajectories within each camera on the basis of the detected boxes and appearance features. The last module is Inter-Camera Association (ICA), which associates all candidate trajectories between two successive cameras using the K-reciprocal nearest neighbors algorithm, and combines all successively matched trajectories for final results. The ICA module takes the constraints of traveling time, road topology structures, and traffic rules into consideration to reduce the searching space and accelerate the matching speed. Experiments results on the public test set of 2022 AI CITY CHALLENGE Track1 demonstrate the effectiveness of our method, which achieves IDF1 of 84.86%, ranking 1st on the leaderboard.
TorMentor: Deterministic dynamic-path, data augmentations with fractals

We propose the use of fractals as a means of efficient data augmentation. Specifically, we employ plasma fractals for adapting global image augmentation transformations into continuous local transforms. We formulate the diamond square algorithm as a cascade of simple convolution operations allowing efficient computation of plasma fractals on the GPU. We present the TorMentor image augmentation framework that is totally modular and deterministic across images and point-clouds. All image augmentation operations can be combined through pipelining and random branching to form flow networks of arbitrary width and depth. We demonstrate the efficiency of the proposed approach with experiments on document image segmentation (binarization) with the DIBCO datasets. The proposed approach demonstrates superior performance to traditional image augmentation techniques. Finally, we use extended synthetic binary text images in a self-supervision regiment and outperform the same model when trained with limited data and simple extensions.
Visual Domain Bridge: A source-free domain adaptation for cross-domain few-shot learning

Due to the covariate shift, deep neural networks performance always degrades when applied to novel domains. In order to mitigate this problem, domain adaptation techniques require samples from target data during the feature extraction training, which is not always applicable in real-world scenarios. Batch Normalization is a known component of computer vision models, aiming at reducing the training-time covariate shift. However, facing distribution shift results in an internal state mismatch inside the Batch-Norm layers during the inference time. In favor of alleviating the induced mismatch, this paper proposes a sourcefree, lightweight and straightforward approach by introducing the “Visual Domain Bridge” concept reducing the BatchNorm’s internal mismatch in the cross-domain settings. Compared to the other BatchNorm-based source-free domain adaptation techniques such as AdaBN and Prediction-BN, our method formed a new state-of-the-art cross-domain few-shot fine-tuning method neglecting extra augmentations; while improving the performance in near-domain settings too. The proposed method can integrate with other domain adaptation methods and enhance their performance requiring just a few lines of modification in the BatchNorm’s implementation. Implementations are available in https://github.com/MosyMosy/VDB
Adapting Blockchain for Energy Constrained IoT in Healthcare Environment

The previous proposal aims to minimize energy consumption in these nodes and expand the lifespan of them. The habiliment devices affixed to the patients gather enormous amount of data. The above layer consists of the fog devices that administer the data. A case study is considered in the proposal. A grouping framework is suggested. They group the devices and choose a principal node. This node is responsible to assemble the data of their respective cluster and transmit the processed one to the fog devices. To solve delay issue, these nodes are deployed at the boundary. This data can be retrieved by the authenticated user (doctors, personnel staff). The mist figuring climate is comprised of exceptional organization gadgets called haze hubs, which perform different processing assignments at the edge of the organization. Each mist hub can offer types of assistance, including preparing, memory, stockpiling, and organization transfer speed. In the lower layer, wearable sensors are conveyed on the patient’s body to accumulate information. The wearable sensors screen and gather physiological information from patients as an EKG, blood oxygen, and other wellbeing-related data. The sensors conveyed assist patients with diminishing the burden of regular visits to specialists. Wearable sensors utilized in the lower layer may have restricted force, memory, preparation, and correspondence, so we will probably carry out a grouping strategy to amplify the organizational life of remote sensor organizations. The center layer comprises mist hubs. They are into preparing, stockpiling, memory, and organization transmission capacity abilities. As the medical care information gathered by the basic handheld gadget can expand, information mining and investigation of this enormous information are required. The mist hub in the center layer can deal with the crude information gathered in the lower layer for investigation. The suggestion aims in minimizing the energy consumption by using hashing. The energy is conserved by 55.33% compared to the previous contribution.