Miracle-dz/UniFAR-FTUs huggingface dataset:
Online Camera Pose Optimization for the Surround-view System

Surround-view system is an important information medium for drivers to monitor the driving environment. A typical surround-view system consists of four to six fish-eye cameras arranged around the vehicle. From these camera inputs, a top-down image of the ground around the vehicle, namely the surround-view image can be generated with well calibrated camera poses. Although existing surround-view system solutions can estimate camera poses accurately in off-line environment, how to correct the camera poses' change in online environment is still an open issue. In this paper, we propose a camera pose optimization method for surround-view system in online environment. Our method consists of two models: Ground Model and Ground-Camera Model, both of which correct the camera poses by minimizing photometric errors between ground projections of adjacent cameras. Experiments show that our method can effectively correct the geometric misalignment of the surround-view image caused by camera poses' change. Since our method is highly automated with low requirement of calibration site and manual operation, it has a wide range of applications and is convenient for the end-users. To make the results reproducible, the source code is publicly available at https://cslinzhang.github. io/CamPoseOpt/.
Fruit Image Classification Based on MobileNetV2 with Transfer Learning Technique

Fruit image classification is the key technology for robotic picking which can tremendously save costs and effectively improve fruit producer's competitiveness in the international fruit market. In the image classification field, deep learning technologies especially DCNNs are state-of-the-art technologies and have achieved remarkable success. But the requirements of high computation and storage resources prohibit the usages of DCNNs on resource-limited environments such as automatic harvesting robots. Therefore, we need to choose a lightweight neural network to achieve the balance of resource limitations and recognition accuracy. In this paper, a fruit image classification method based on a lightweight neural network MobileNetV2 with transfer learning technique was used to recognize fruit images. We used a MobileNetV2 network pre-trained by ImageNet dataset as a base network and then replace the top layer of the base network with a conventional convolution layer and a Softmax classifier. We applied dropout to the new-added conv2d at the same time to reduce overfitting. The pre-trained MobileNetV2 was used to extract features and the Softmax classifier was used to classify features. We trained this new model in two stages using Adam optimizer of different learning rate. This method finally achieved a classification accuracy of 85.12% in our fruit image dataset including 3670 images of 5 fruits. Compared with other network such as MobileNetV1, InceptionV3 and DenseNet121, this hybrid network implemented by Google open source deep learning framework Tensorflow can make a good compromise between accuracy and speed. Since MobileNetV2 is a lightweight neural network, the method in this paper can be deployed in low-power and limited-computing devices such as mobile phone.
ALIC: A Superpixel Segmentation Algorithm Based on Autonomous Attachment

In this paper, we reconsider the problem of low computational efficiency in traditional superpixel segmentation methods based on clustering. We propose a superpixel segmentation method based on autonomous attachment named ALIC, which use a simpler and more efficient distance measurement method to accelerate the algorithm. Simultaneously, the allocation of pixel autonomous attached fully considers the natural continuity between pixels, and each pixel can share label with its neighbors, which allows us to obtain better boundary performance. In the experiment, our method only achieves convergence in five iterations. On the basis of obtaining more sensitive boundaries, our algorithm improves the operation speed. (On a simple CPU core, it only takes about 0.1s to segment a 481×321 image into 400 homogenic superpixels).
Deep Multimodal Network: a multi-signal input neural network for Ad Click Predictions

Online advertisement is an important source of revenue for internet companies, so increasing click-through rates (CTR) on ads is crucial. the traditional CTR prediction model only extracts the classification features and numerical features of the advertisement, but ignores the title, description and other text features of the advertisement. However, we know that such information is very important for an advertisement. Therefore, we propose a deep multimodal network (DMN) to solve this problem. on the basis of the traditional deep model, DMN add the text features of cyclic neural network learning, so as to improve the performance of the model. we also did a lot of visual data analysis on the dataset, Finally, we conducted a comparative experiment on the real-world dataset.
What Could Go Wrong?: Raising Mobile Privacy and Security Awareness Through a Decision-Making Game

Uninformed actions in privacy and security settings of mobile devices might cause data leaks. In many occasions, users may not be aware of such issues due to lack of knowledge on the topic. To raise interest and awareness towards this issue, extra effort might be needed. Game-based learning has proven to be an effective approach in terms of motivation and promotion of understanding. Humour is also considered to have the ability to create a positive attitude to the subject matter and reduce anxiety. To examine the interplay of games and humor, we developed What Could Go Wrong, a humorous decision-making game that helps users to better understand the consequences of applying security changes on a mobile device. We conducted a preliminary study comparing the game to two more models (a serious animated video and a humorous animated video), to analyze its impact on user's motivation as well as raising awareness of the topic. Our user study (n=21) shows that the game-based approach is more successful in engaging and raising awareness.
Relation Understanding in Videos: A Grand Challenge Overview

ACM Multimedia 2019 Video Relation Understanding Challenge is the first grand challenge aiming at pushing video content analysis at the relational and structural level. This year, the challenge asks the participants to explore and develop innovative algorithms to detect object entities and their relations based on a large-scale user-generated video dataset. The tasks will advance the foundation of future visual systems that are able to perform complex inferences. This paper presents an overview of the grand challenge, including background, detailed descriptions of the three proposed tasks, the corresponding datasets for training, validation and testing, and the evaluation process.
Band and Quality Selection for Efficient Transmission of Hyperspectral Images

Due to recent technological advances in capturing and processing devices, hyperspectral imaging is becoming available for many commercial and military applications such as remote sensing, surveillance, and forest fire detection. Hyperspectral cameras provide rich information, as they capture each pixel along many frequency bands in the spectrum. The large volume of hyperspectral images as well as their high dimensionality make transmitting them over limited-bandwidth channels a challenge. To address this challenge, we present a method to prioritize the transmission of various components of hyperspectral data based on the application needs, the level of details required, and available bandwidth. This is unlike current works that mostly assume offline processing and the availability of all data beforehand. Our method jointly and optimally selects the spectral bands and their qualities to maximize the utility of the transmitted data. It also enables progressive transmission of hyperspectral data, in which approximate results are obtained with small amount of data and can be refined with additional data. This is a desirable feature for large-scale hyperspectral imaging applications. We have implemented the proposed method and compared it against the state-of-the-art in the literature using hyperspectral imaging datasets. Our experimental results show that the proposed method achieves high accuracy, transmits a small fraction of the hyperspectral data, and significantly outperforms the state-of-the-art; up to 35% improvements in accuracy was achieved.
Gamification Strategies as Socialization Tools in the Context of Pediatric Hospitalization

Pediatric hospitalization causes a series of psychological problems and can have negative consequences in children's social life derived from their internment and the lack of contact with peers and family. In order to improve this aspect of pediatric admittance, and to alleviate the difficulties they might suffer, this work aims to provide the children with a set of tools and games that allow them to interact with other people from inside the hospital in an effective way, by testing different gaming mechanics and strategies to determine which are the ones that foster interaction between the participants the most. In the end, a social network of games and communication will be developed and evaluated against a control group to see what the impact is and how it changes the hospitalization experience to those children exposed to it.
The Role of Service Quality on Location and Price Perception on the Purchasing Decision Process

This study is aimed at investigating the role of service quality on location and price perception to purchase decision. It is argued that moviegoers have unique behaviours on the process of purchasing decision in a cinema. In this study, 150 moviegoers are employed as the participants. Reinforcement motivation is used to analyse the purchasing decision process of this research. It is believed that service quality provided by cinemas' management has a significant effect on the participants' decision making process. The result shows that the service quality does not mediate the influence of location and price perception on purchase decision. Some limitation and future research are discussed.
Attacking Gait Recognition Systems via Silhouette Guided GANs

This paper investigates a new attack method to gait recognition systems. Different from typical spoofing attacks that require impostors to mimic certain clothing or walking styles, it proposes to intercept the video stream captured by the on-site camera and replace it with synthesized samples. To this end, we present a novel Generative Adversarial Network (GAN) based approach, which is able to render a faked video from the source walking sequence of a specified subject and the target scene image with both good visual effects and sufficient discriminative details. A new generator architecture is built, where the features of the source foreground sequence and the target background image are combined at multiple scales, making the synthesized video vivid. To fool recognition systems, the silhouette-conditioned losses are specially designed to constrain the static and dynamic consistency between the subjects in the source and generated videos. The person re-identification similarity based triplet loss is exploited to guide the generator, which keeps the personalized appearance properties stable. The edge and flow-related losses further regulate the generation of the attacking video. Two state-of-the-art gait recognition systems are used for evaluation, namely GaitSet and CNN-Gait, and we analyze their performance under attacking. Both the visual fidelity and attacking ability of the generated videos validate the effectiveness of the proposed method.
Early Detection of Fake News "Before It Flies High"

Currently, social media for news consumption is preferred over the conventional media and attracted many people due to its low cost, easy access, simplistic way of commenting & sharing, more timely nature, and rapid information sharing capabilities. On the other hand, it aggravates the prompt and wide spreading of fake news. Fake news may be fabricated for the purpose of, commercial gain, political propaganda, seeking attention, and intent of defamation. Interest of individuals, and various groups to influence events and policies around the globe is the other reason for fake news generation and dissemination. The extensive spread of fake news is progressively becoming a threat to individuals and society as a whole. It disrupts the authenticity balance of the news ecosystem; induces biased or false beliefs into consumers; creates real-life fears in the society and threatens freedom of speech, freedom of the press and democracy. The craving to mitigate the undesirable effects of fake news, recently makes fake news detection on social media an emerging research area attracting tremendous attention. Following this warm concern, various researches have been conducted and showed promising results. In this work, we propose a model for early detection of fake news using deep learning, and news content. Deep learning and heterogeneous dataset has been used to create a more generic model that could perform better in the real world. We conducted experiments on two real world datasets and a third dataset which is obtained by combining the two datasets and randomly shuffled them. Our experiment results have shown that early detection of fake news using news content and deep learning models, without waiting for news propagation, is achievable and should be given better attention to combat fake news effectively before it proliferates and misleads many people. The experimental results obtained are interesting.
DaNet: Decompose-and-aggregate Network for 3D Human Shape and Pose Estimation

Reconstructing 3D human shape and pose from a monocular image is challenging despite the promising results achieved by most recent learning based methods. The commonly occurred misalignment comes from the facts that the mapping from image to model space is highly non-linear and the rotation-based pose representation of the body model is prone to result in drift of joint positions. In this work, we present the Decompose-and-aggregate Network (DaNet) to address these issues. DaNet includes three new designs, namely UVI guided learning, decomposition for fine-grained perception, and aggregation for robust prediction. First, we adopt the UVI maps, which densely build a bridge between 2D pixels and 3D vertexes, as an intermediate representation to facilitate the learning of image-to-model mapping. Second, we decompose the prediction task into one global stream and multiple local streams so that the network not only provides global perception for the camera and shape prediction, but also has detailed perception for part pose prediction. Lastly, we aggregate the message from local streams to enhance the robustness of part pose prediction, where a position-aided rotation feature refinement strategy is proposed to exploit the spatial relationship between body parts. Such a refinement strategy is more efficient since the correlations between position features are stronger than that in the original rotation feature space. The effectiveness of our method is validated on the Human3.6M and UP-3D datasets. Experimental results show that the proposed method significantly improves the reconstruction performance in comparison with previous state-of-the-art methods. Our code is publicly available at https://github.com/HongwenZhang/DaNet-3DHumanReconstrution .
AVEC'19: Audio/Visual Emotion Challenge and Workshop

The ninth Audio-Visual Emotion Challenge and workshop AVEC 2019 was held in conjunction with ACM Multimedia'19. This year, the AVEC series addressed major novelties with three distinct tasks: State-of-Mind Sub-challenge (SoMS), Detecting Depression with Artificial Intelligence Sub-challenge (DDS), and Cross-cultural Emotion Sub-challenge (CES). The SoMS was based on a novel dataset (USoM corpus) that includes self-reported mood (10-point Likert scale) after the narrative of personal stories (two positive and two negative). The DDS was based on a large extension of the DAIC-WOZ corpus (c.f. AVEC 2016) that includes new recordings of patients suffering from depression with the virtual agent conducting the interview being, this time, wholly driven by AI, i.e., without any human intervention. The CES was based on the SEWA dataset (c.f. AVEC 2018) that has been extended with the inclusion of new participants in order to investigate how emotion knowledge of Western European cultures (German, Hungarian) can be transferred to the Chinese culture. In this summary, we mainly describe participation and conditions of the AVEC Challenge.
Measuring Fun with Adolescents: Introducing the Spanish and Dutch Adaptation of the FunQ

It is increasingly understood how fun is an essential aspect in interaction for children in or beyond game play, as well as supporting learning and other activities involving children, e.g., for participatory design. While it is often proclaimed that such activities are fun for children participants, empirical evidence is hard to come by as measuring fun is not straightforward. Currently available assessment tools suitable for adolescents are scarce. FunQ is a recently developed measurement tool specifically designed for adolescents for the assessment of the experienced fun. Here we discuss ongoing work for adapting FunQ - to Spanish and Dutch. We report on the process followed to adapt the questionnaire; the reliability measures based on the initial results (ωoverall-SP= 0.876 and ωpartial-SP= 0.859; ωoverall-NL= 0.819 and ωpartial-NL= 0.804); and discuss our findings in reflection of the current state of the adaptation procedure.
Krylov Methods for Low-Rank Regularization

This paper introduces new solvers for the computation of low-rank approximate solutions to large-scale linear problems, with a particular focus on the regularization of linear inverse problems. Although Krylov methods incorporating explicit projections onto low-rank subspaces are already used for well-posed systems that arise from discretizing stochastic or time-dependent PDEs, we are mainly concerned with algorithms that solve the so-called nuclear norm regularized problem, where a suitable nuclear norm penalization on the solution is imposed alongside a fit-to-data term expressed in the 2-norm: this has the effect of implicitly enforcing low-rank solutions. By adopting an iteratively reweighted norm approach, the nuclear norm regularized problem is reformulated as a sequence of quadratic problems, which can then be efficiently solved using Krylov methods, giving rise to an inner-outer iteration scheme. Our approach differs from the other solvers available in the literature in that: (a) Kronecker product properties are exploited to define the reweighted 2-norm penalization terms; (b) efficient preconditioned Krylov methods replace gradient (projection) methods; (c) the regularization parameter can be efficiently and adaptively set along the iterations. Furthermore, we reformulate within the framework of flexible Krylov methods both the new inner-outer methods for nuclear norm regularization and some of the existing Krylov methods incorporating low-rank projections. This results in an even more computationally efficient (but heuristic) strategy, that does not rely on an inner-outer iteration scheme. Numerical experiments show that our new solvers are competitive with other state-of-the-art solvers for low-rank problems, and deliver reconstructions of increased quality with respect to other classical Krylov methods.
INTEL-TAU: A Color Constancy Dataset

In this paper, we describe a new large dataset for illumination estimation. This dataset, called INTEL-TAU, contains 7022 images in total, which makes it the largest available high-resolution dataset for illumination estimation research. The variety of scenes captured using three different camera models, namely Canon 5DSR, Nikon D810, and Sony IMX135, makes the dataset appropriate for evaluating the camera and scene invariance of the different illumination estimation techniques. Privacy masking is done for sensitive information, e.g., faces. Thus, the dataset is coherent with the new General Data Protection Regulation (GDPR). Furthermore, the effect of color shading for mobile images can be evaluated with INTEL-TAU dataset, as both corrected and uncorrected versions of the raw data are provided. Furthermore, this paper benchmarks several color constancy approaches on the proposed dataset.
On decentralized oracles for data availability

Nakamoto consensus, the protocol underlying Bitcoin, has the potential to secure a new class of systems which agree on non-mathematical truths. As an example of this capability, we propose a design for a trustless, data availability oracle. This exposition reduces the problem of determining whether or not a registered datum is publicly available to the problem of constructing a network in which either almost all nodes can download a given datum, or almost none of them can.
Restless Hidden Markov Bandits with Linear Rewards

This paper presents an algorithm and regret analysis for the restless hidden Markov bandit problem with linear rewards. In this problem the reward received by the decision maker is a random linear function which depends on the arm selected and a hidden state. In contrast to previous works on Markovian bandits, we do not assume that the decision maker receives information regarding the state of the system, but can only infer/estimate it based on its actions and the received reward. Additionally, it is assumed that the decision maker knows in advance that the reward is a random linear function which depends on the selected arm, the action, and hidden states. However, the decision maker does not know in advance the probability distributions of these hidden states; thus we call this side information structural side information. Surprisingly, we can still maintain logarithmic regret in the case of polyhedral action set. Furthermore, we show that the structural side information leads to expected regret that does not depend on the number of extreme points in the action space.
Scaling laws in axisymmetric magnetohydrodynamic duct flows

We report on a numerical study of axisymmetric flow of liquid metal in a circular duct with rectangular cross-section. The flow is forced through the combination of an axial magnetic field and a radial current. Sweeping a wide range of forcing parameters, we identify the different regimes which characterize the flows and explicit the associate scaling laws. Experimental results are interpreted in the light of our numerical simulations.
Propositional Dynamic Logic for Hyperproperties

Information security properties of reactive systems like non-interference often require relating different executions of the system to each other and following them simultaneously. Such hyperproperties can also be useful in other contexts, e.g., when analysing properties of distributed systems like linearizability. Since common logics like LTL, CTL, or the modal mu-calculus cannot express hyperproperties, the hyperlogics HyperLTL and HyperCTL* were developed to cure this defect. However, these logics are not able to express arbitrary omega-regular properties. In this paper, we introduce HyperPDL-Delta, an adaptation of the Propositional Dynamic Logic of Fischer and Ladner for hyperproperties, in order to remove this limitation. Using an elegant automata-theoretic framework, we show that HyperPDL-Delta model checking is asymptotically not more expensive than HyperCTL* model checking, despite its vastly increased expressive power. We further investigate fragments of HyperPDL-Delta with regard to satisfiability checking.