Reviewerly/exHarmony huggingface dataset:
Co-Designing Interactions between Pedestrians in Wheelchairs and Autonomous Vehicles

In the near future, mixed traffic consisting of manual and autonomous vehicles (AVs) will be common. Questions surrounding how vulnerable road users such as pedestrians in wheelchairs (PWs) will make crossing decisions in these new situations are underexplored. We conducted a remote co-design study with one of the researchers of this work who has the lived experience as a powered wheelchair user and applied inclusive design practices. This allowed us to identify and reflect on interface design ideas that can help PWs make safe crossing decisions at intersections. Through an iterative five-week study, we implemented interfaces that can be placed on the vehicle, on the wheelchair, and on the street infrastructure and evaluated them during the co-design sessions using a VR simulator testbed. Informed by our findings, we discuss design insights for implementing inclusive interfaces to improve interactions between autonomous vehicles and vulnerable road users.
VASP: An autoencoder-based approach for multivariate anomaly detection and robust time series prediction with application in motorsport

The aim is to provide a framework for robust time series prediction in the presence of anomalies. The framework is developed based on a data set from motorsport but is not limited to this specific area. In motorsport, the usage of sensors during races is generally restricted. Estimating the outputs of these missing sensors therefore provides an advantage over the competition. Deep learning approaches such as long short-term memory (LSTM) neural networks have proven to be useful for that task, however, their accuracy decreases significantly if anomalies occur in the input signals. To overcome this problem, we propose the variational autoencoder based selective prediction (VASP) framework which combines the tasks of anomaly detection and time series prediction. VASP consists of a variational autoencoder (VAE), an anomaly detector and LSTM predictors. Depending on the anomaly detector, a subset of the inputs may be replaced by the VAE, allowing a more robust prediction. To the best of our knowledge the approach of using a VAE to only selectively replace anomalous input data before prediction has not yet been published. Our contributions are clear implementation guidelines and a comparison to other VAE-based methods and a LSTM approach as baseline. We simulate anomalies with three approaches and show that VASP outperforms other methods by having no trade-off between accuracy and robustness. VASP is as accurate as the baseline for regular data, but for anomalous inputs the error is reduced by 13% to 33% on average and up to 70% in special cases.
Developing a Model to Highlight the Relation of Digital Trust With Privacy and Security for the Blockchain Technology

Digitalization uses digital technology to change a business model and provide new revenue models and value-producing opportunities. Blockchain is a type of database that stores various kinds of information in blocks that form a chain of information. It is one of the secured ways of transferring and storage of data. Blockchain is helping in creating trust for digitalization among its users. This research aims to study the impact of trust in blockchain by analyzing the privacy and security concerns that can impact the user attitude and its intention to the adoption process. For this structure, literature review is performed. Five variables are used, and they are attitude, privacy, trust, security, and intention. A questionnaire is developed for survey-based research in the software firms, banking sector, and digital marketing companies. For analysis, exploratory factor analysis and structural equation modeling are used. A model is developed that shows a good fit, and the parameters are satisfied.
Language and innovation

Abstract Strong‐future‐time reference (FTR) languages require speakers to grammatically mark future events, while weak‐FTR languages do not. Using data from 34 countries, we find that firms in countries where strong‐FTR languages are spoken have fewer patent counts and citations than those in countries where weak‐FTR languages are spoken. Further evidence shows that strong‐FTR languages affect inventors’ perceptions and beliefs about future rewards from innovation. Moreover, due to interactions between people speaking different languages, globalization attenuates the negative impact of language FTR on innovation. To further support these findings, we provide evidence from a single country with a multilingual environment to control for omitted country‐level characteristics. Our study emphasizes the impact of language on corporate innovation and sheds light on the importance of informal institutions in economic outcomes.
Cryptanalysis of Reversible Data Hiding in Encrypted Images by Block Permutation and Co-Modulation

Reversible data hiding in encrypted images (RDH-EI) technology is commonly used in cloud storage images for privacy protection. Most existing RDH-EI techniques reported in the literature applied block permutation and co-modulation (BPCM) encryption to generate encrypted images. This work analyses the security of the RDH-EI algorithm based on BPCM encryption under known plaintext attacks (KPAs). Different from the existing KPAs, this paper considers that attackers can perform KPAs based on marked encrypted images and shows that BPCM encryption has the risk of information leakage. To find the constant features of a block before and after co-modulation, the first-pixel difference block (FDB) of a block is first defined. Then, a pseudo cypher difference image of the cyphertext image is constructed to eliminate the changed FDBs so that the differences in the cyphertext FDBs are the same as the FDBs in the corresponding plaintext difference image. Finally, we design an FDB-based block permutation key estimation method according to the plaintext difference image and pseudocyphertext difference image. The influence of block size on key estimation accuracy and the time complexity of the proposed KPA algorithm are analysed and discussed. Experimental results show that the correct rate of key estimation is positively correlated with the block size and the number of plain-cyphertext pairs. The average correct rate of key estimation reaches 63% when the block size is greater than 3×3.
PROTECTOR: An optimized deep learning-based framework for image spam detection and prevention

Image spam is a spamming technique that integrates spam text content into graphical images in order to bypass conventional text-based spam filters. In order to detect image spam efficiently, it is important to analyze the image data. The existing image spam detection techniques in literature focus on textual or graphic features of the image. None of the existing techniques considered the link information of the image which results in low accuracy and performance degradation. So, to fill these gaps, in this paper, we analyzed the link properties of image, for image spam detection and prevention. We propose an optimized framework called as PROTECTOR. In PROTECTOR, the rank score is generated by using the linkage information of the image, textual information of the image, and metadata information of the image. The computed rank score indicates the relevance of an image. This rank score is then used to train a deep neural network design of deep learning, which yields the accuracy of 96% with respect to various performance evaluation metrics. Also, the optimization algorithm, i.e., genetic algorithm is fitted in the results according to the defined fitting function. The proposed framework is validated with standard dataset of Image spam Hunter.
Learning a maximized shared latent factor for cross-modal hashing

Due to the enormous advantages of high efficiency and low storage, hashing has already gained wide attention in the field of cross-modal search. Currently, most hashing methods learn unified hash codes for retrieval tasks by leveraging the inner characteristics of original data or the similarity graphs constructed by supervised information. However, both the two schemes neglect the discrepancy between distinct modalities in real-world scenarios that leads to poor search performance as well as limited search scalability. Toward this end, in this paper, we propose a novel supervised hashing method, termed Maximized Shared Latent Factor (MSLF) wherein the common property and unique properties of different modalities are taken into account simultaneously. Specifically, the common property (i.e., shared latent factor) generated by available label supervision without the similarity graph construction, can reduce the computation cost of hashing methods and thus make it suitable for cross-modal search applications. Meanwhile, the unique property (i.e., individual latent factor) obtained by each modality and the cross-correlation matching from data instances are designed to alleviate the structure difference between modalities. Furthermore, we combine these properties to learn a maximized shared latent factor for obtaining the hash codes, which enhances the search performance of MSLF. Extensive quantitative experiments on three popular datasets prove that the proposed MSLF outperforms the state-of-the-art hashing methods in both search performance and scalability.
Media Forensics Considerations on DeepFake Detection with Hand-Crafted Features

DeepFake detection is a novel task for media forensics and is currently receiving a lot of research attention due to the threat these targeted video manipulations propose to the trust placed in video footage. The current trend in DeepFake detection is the application of neural networks to learn feature spaces that allow them to be distinguished from unmanipulated videos. In this paper, we discuss, with features hand-crafted by domain experts, an alternative to this trend. The main advantage that hand-crafted features have over learned features is their interpretability and the consequences this might have for plausibility validation for decisions made. Here, we discuss three sets of hand-crafted features and three different fusion strategies to implement DeepFake detection. Our tests on three pre-existing reference databases show detection performances that are under comparable test conditions (peak AUC &gt; 0.95) to those of state-of-the-art methods using learned features. Furthermore, our approach shows a similar, if not better, generalization behavior than neural network-based methods in tests performed with different training and test sets. In addition to these pattern recognition considerations, first steps of a projection onto a data-centric examination approach for forensics process modeling are taken to increase the maturity of the present investigation.
Local Moment Driven PVO Based Reversible Data Hiding

Pixel-value-ordering (PVO) is one of the most widely used reversible data hiding (RDH) framework which efficiently utilizes smooth pixels of the cover image to provide high-fidelity stego-image but with limited embedding capacity. This letter proposes an RDH scheme based on local moment driven pixel value ordering (LM-PVO) which further enhances the effectiveness of smooth pixel's utilization by dividing the fixed-size blocks into two groups. Pixels of each group are sub-divided into two sub-groups based on the local moment of the block so that correlation among the pixels of each sub-group is enhanced. Thus doing, the pixels of each sub-group are grouped based on their intensity values instead of their position as in the conventional PVO-based schemes; this enables information hider to embed a higher amount of secret data while also enhancing the stego-image quality. Experimental results also validate the superiority of the proposed scheme over the existing PVO-based RDH schemes.
Communication-efficient Decentralized Machine Learning over Heterogeneous Networks

In the last few years, distributed machine learning has been usually executed over heterogeneous networks such as a local area network within a multi-tenant cluster or a wide area network connecting data centers and edge clusters. In these heterogeneous networks, the link speeds among worker nodes vary significantly, making it challenging for state-of-the-art machine learning approaches to perform efficient training. Both centralized and decentralized training approaches suffer from low-speed links. In this paper, we propose a decentralized approach, namely NetMax, that enables worker nodes to communicate via high-speed links and, thus, significantly speed up the training process. NetMax possesses the following novel features. First, it consists of a novel consensus algorithm that allows worker nodes to train model copies on their local dataset asynchronously and exchange information via peer-to-peer communication to synchronize their local copies, instead of a central master node (i.e., parameter server). Second, each worker node selects one peer randomly with a fine-tuned probability to exchange information per iteration. In particular, peers with high-speed links are selected with high probability. Third, the probabilities of selecting peers are designed to minimize the total convergence time. Moreover, we mathematically prove the convergence of NetMax. We evaluate NetMax on heterogeneous cluster networks and show that it achieves speedups of 3.7×, 3.4×, and 1.9× in comparison with the state-of-the-art decentralized training approaches Prague, Allreduce-SGD, and AD-PSGD, respectively.
Saliency-Guided Image Translation

In this paper, we propose a novel task for saliency-guided image translation, with the goal of image-to-image translation conditioned on the user specified saliency map. To address this problem, we develop a novel Generative Adversarial Network (GAN)-based model, called SalG-GAN. Given the original image and target saliency map, SalG-GAN can generate a translated image that satisfies the target saliency map. In SalG-GAN, a disentangled representation framework is proposed to encourage the model to learn diverse translations for the same target saliency condition. A saliency-based attention module is introduced as a special attention mechanism for facilitating the developed structures of saliency-guided generator, saliency cue encoder and saliency-guided global and local discriminators. Furthermore, we build a synthetic dataset and a real-world dataset with labeled visual attention for training and evaluating our SalG-GAN. The experimental results over both datasets verify the effectiveness of our model for saliency-guided image translation.
Intentonomy: a Dataset and Study towards Human Intent Understanding

An image is worth a thousand words, conveying information that goes beyond the mere visual content therein. In this paper, we study the intent behind social media images with an aim to analyze how visual information can facilitate recognition of human intent. Towards this goal, we introduce an intent dataset, Intentonomy, comprising 14K images covering a wide range of everyday scenes. These images are manually annotated with 28 intent categories derived from a social psychology taxonomy. We then systematically study whether, and to what extent, commonly used visual information, i.e., object and context, contribute to human motive understanding. Based on our findings, we conduct further study to quantify the effect of attending to object and context classes as well as textual information in the form of hashtags when training an intent classifier. Our results quantitatively and qualitatively shed light on how visual and textual information can produce observable effects when predicting intent. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Distill on the Go: Online knowledge distillation in self-supervised learning

Self-supervised learning solves pretext prediction tasks that do not require annotations to learn feature representations. For vision tasks, pretext tasks such as predicting rotation, solving jigsaw are solely created from the input data. Yet, predicting this known information helps in learning representations useful for downstream tasks. However, recent works have shown that wider and deeper models benefit more from self-supervised learning than smaller models. To address the issue of self-supervised pre-training of smaller models, we propose Distill-on-the-Go (DoGo), a self-supervised learning paradigm using single-stage online knowledge distillation to improve the representation quality of the smaller models. We employ deep mutual learning strategy in which two models collaboratively learn from each other to improve one another. Specifically, each model is trained using self-supervised learning along with distillation that aligns each model’s softmax probabilities of similarity scores with that of the peer model. We conduct extensive experiments on multiple benchmark datasets, learning objectives, and architectures to demonstrate the potential of our proposed method. Our results show significant performance gain in the presence of noisy and limited labels, and in generalization to out-of-distribution data.
Learning Asynchronous and Sparse Human-Object Interaction in Videos

Human activities can be learned from video. With effective modeling it is possible to discover not only the action labels but also the temporal structure of the activities, such as the progression of the sub-activities. Automatically recognizing such structure from raw video signal is a new capability that promises authentic modeling and successful recognition of human-object interactions. Toward this goal, we introduce Asynchronous-Sparse Interaction Graph Networks (ASSIGN), a recurrent graph network that is able to automatically detect the structure of interaction events associated with entities in a video scene. ASSIGN pioneers learning of autonomous behavior of video entities including their dynamic structure and their interaction with the coexisting neighbors. Entities’ lives in our model are asynchronous to those of others therefore more flexible in adapting to complex scenarios. Their interactions are sparse in time hence more faithful to the true underlying nature and more robust in inference and learning. ASSIGN is tested on human-object interaction recognition and shows superior performance in segmenting and labeling of human sub-activities and object affordances from raw videos. The native ability of ASSIGN in discovering temporal structure also eliminates the dependence on external segmentation that was previously mandatory for this task.
Scene-aware Generative Network for Human Motion Synthesis

We revisit human motion synthesis, a task useful in various real-world applications, in this paper. Whereas a number of methods have been developed previously for this task, they are often limited in two aspects: 1) focus on the poses while leaving the location movement behind, and 2) ignore the impact of the environment on the human motion. In this paper, we propose a new framework, with the interaction between the scene and the human motion taken into account. Considering the uncertainty of human motion, we formulate this task as a generative task, whose objective is to generate plausible human motion conditioned on both the scene and the human’s initial position. This framework factorizes the distribution of human motions into a distribution of movement trajectories conditioned on scenes and that of body pose dynamics conditioned on both scenes and trajectories. We further derive a GAN-based learning approach, with discriminators to enforce the compatibility between the human motion and the contextual scene as well as the 3D-to-2D projection constraints. We assess the effectiveness of the proposed method on two challenging datasets, which cover both synthetic and real-world environments.
Distractor-Aware Fast Tracking via Dynamic Convolutions and MOT Philosophy

A practical long-term tracker typically contains three key properties, i.e. an efficient model design, an effective global re-detection strategy and a robust distractor awareness mechanism. However, most state-of-the-art long-term trackers (e.g., Pseudo and re-detecting based ones) do not take all three key properties into account and therefore may either be time-consuming or drift to distractors. To address the issues, we propose a two-task tracking framework (named DMTrack), which utilizes two core components (i.e., one-shot detection and re-identification (re-id) association) to achieve distractor-aware fast tracking via Dynamic convolutions (d-convs) and Multiple object tracking (MOT) philosophy. To achieve precise and fast global detection, we construct a lightweight one-shot detector using a novel dynamic convolutions generation method, which provides a unified and more flexible way for fusing target information into the search field. To distinguish the target from distractors, we resort to the philosophy of MOT to reason distractors explicitly by maintaining all potential similarities’ tracklets. Benefited from the strength of high recall detection and explicit object association, our tracker achieves state-of-the-art performance on the LaSOT, Ox-UvA, TLP, VOT2018LT and VOT2019LT benchmarks and runs in real-time (3x faster than comparisons) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Ball Motion Control in the Table Tennis Robot System Using Time-Series Deep Reinforcement Learning

One of the biggest challenges hindering a table tennis robot to play as well as a professional player is the ball's accurate motion control, which depends on various factors such as the incoming ball's position, linear, spin velocity and so forth.Unfortunately, some factors are almost impossible to be directly measured in real practice, such as the ball's spin velocity, which is difficult to be estimated from vision due to the little texture on the ball's surface.To perform accurate motion control in table tennis, this study proposes to learn a ball stroke strategy to guarantee desirable ''target landing location'' and the ''over-net height'' which are two key indicators to evaluate the quality of a stroke.To overcome the spin velocity challenge, a deep reinforcement learning (DRL) based stroke approach is developed with the spin velocity estimation capability, through which the system can predict the relative spin velocity of the ball and stroke it back accurately by iteratively learning from the robot-environment interactions.To pre-train the DRL-based strategy effectively, this paper develops a virtual table tennis playing environment, through which various simulated data can be collected.For the real table tennis robot implementation, experimental results demonstrate the superior performance of the proposed control strategy compared to that of the traditional aerodynamics-based method with an average landing error around 80 mm and the landing-within-table probability higher than 70%.