Miracle-dz/UniFAR-FTUs huggingface dataset:
The Development of "Orbit": The Collaborative BCI Game for Children with AD(H)D

Children with Attention Deficit (Hyperactivity) Disorder or AD(H)D can require treatment for which they need to experience long-lasting neurofeedback sessions. Children might not adhere to at-home treatment activities because of the nature of these sessions; thus, not getting the benefits of the program. To increase adherence and effectiveness of the treatment itself, we propose neurofeedback gaming and social encouragement. Our hypothesis is that by playing a collaborative neurofeedback game, children will be more adherent to their treatment and therefore derive a stronger benefit. For this purpose we designed the game "Orbit", a first multiplayer prototype that was evaluated in a pilot study with five neuropsychologists. It was found that collaborative multiplayer games are suitable from a therapeutic standpoint and long-term use because of its higher social motivation and collaboration between children with AD(H)D; albeit there are some drawbacks including unreliability of electroencephalography (EEG) input and the risk for the collaborative environment to be distracting for the player.
A Toolbox for Information System Evaluation

An information system (IS) measures toolbox is proposed for evaluating implemented IS. The toolbox supports the evaluator in his action from the definition stage to the results one. The software is based on the content, context, process (CCP) framework with adaptations and the IS systemic view examination. An IS evaluation models and measures library is integrated to the toolbox in order to give material to evaluators about previous studies and their results when addressing the main evaluation questions what is being evaluated, with what measures, why evaluation is being done, who is interested by the evaluation and how to conduct evaluation. The toolbox makes possible the cumulative tradition to be carried forward within the discipline.
GP-BPR: Personalized Compatibility Modeling for Clothing Matching

Owing to the recent advances in the multimedia processing domain and the publicly available large-scale real-world data provided by online fashion communities, like the IQON and Chictopia, researchers are enabled to investigate the automatic clothing matching solutions. In a sense, existing methods mainly focus on modeling the general item-item compatibility from the aesthetic perspective, but fail to incorporate the user factor. In fact, aesthetics can be highly subjective, as different people may hold different clothing preferences. In light of this, in this work, we attempt to tackle the problem of personalized compatibility modeling from not only the general aesthetics but also the personal preference perspectives. In particular, we present a personalized compatibility modeling scheme GP-BPR, comprising of two essential components: general compatibility modeling and personal preference modeling, which characterize the item-item and user-item interactions, respectively. In particular, due to the concern that both the modalities (e.g., the image and context description) of fashion items can deliver important cues regarding user personal preference, we present a comprehensive personal preference modeling method. Moreover, for evaluation, we create a large-scale dataset, IQON3000, from the online fashion community IQON. Extensive experiment results on IQON3000 verify the effectiveness of the proposed scheme. As a byproduct, we have released the dataset, codes, and involved parameters to benefit other researchers.
Vision-based Price Suggestion for Online Second-hand Items

Different from shopping in physical stores, where people have the opportunity to closely check a product (e.g., touching the surface of a T-shirt or smelling the scent of perfume) before making a purchase decision, online shoppers rely greatly on the uploaded product images to make any purchase decision. The decision-making is challenging when selling or purchasing second-hand items online since estimating the items' prices is not trivial. In this work, we present a vision-based price suggestion system for the online second-hand item shopping platform. The goal of vision-based price suggestion is to help sellers set effective prices for their second-hand listings with the images uploaded to the online platforms. To provide effective price suggestions for second-hand items with their images, first we propose to better extract representative visual features from the images with the aid of some other image-based item information (e.g., category, brand). Then, we design a vision-based price suggestion module which takes the extracted visual features along with some statistical item features from the shopping platform as the inputs to determine whether an uploaded item image is qualified for price suggestion by a binary classification model, and provide price suggestions for items with qualified images by a regression model. According to the two demands from the platform operator, two different objective functions are proposed to jointly optimize the classification model and the regression model. For better training these two models, we also propose a warm-up training strategy for the joint optimization. Extensive experiments on a large real-world dataset demonstrate the effectiveness of our vision-based price prediction system.
Android Malware Detection Based on Convolutional Neural Networks

Due to the open source and fragmentation of the Android system, its security is increasingly challenged. Currently, Android malware detection has certain deficiencies in large-scale and automation detection. In this paper, we proposed an Android malware detection framework based on Convolutional Neural Network (CNN). We used static analysis tools and python scripts to automatically extract 1003 static features, and transformed the features of each sample into a two-dimensional matrix as input to the CNN model. We selected 5000 malicious samples and 5000 benign samples for verification. The experimental results show that the detection accuracy of CNN reaches 99.68%, which is much higher than other algorithms.
DanSync: Playful Track Selection through Synchronized Movements

Experiencing synchrony with others is a powerful and engaging experience that fosters empathy, happiness, and connectedness between people. However, synchrony is underexplored in social co-located play. We developed DanSync, a system that enables multiple players to explore personalized musical characteristics through movement synchrony. We discuss the potential of synchrony between players for interactive play experiences.
Geographic Diversification of Recommended POIs in Frequently Visited Areas

In the personalized Point-Of-Interest (POI) (or venue) recommendation, the diversity of recommended POIs is an important aspect. Diversity is especially important when POIs are recommended in the target users’ frequently visited areas, because users are likely to revisit such areas. In addition to the (POI) category diversity that is a popular diversification objective in recommendation domains, diversification of recommended POI locations is an interesting subject itself. Despite its importance, existing POI recommender studies generally focus on and evaluate prediction accuracy. In this article, geographical diversification (geo-diversification), a novel diversification concept that aims to increase recommendation coverage for a target users’ geographic areas of interest, is introduced, from which a method that improves geo-diversity as an addition to existing state-of-the-art POI recommenders is proposed. In experiments with the datasets from two real Location Based Social Networks (LSBNs), we first analyze the performance of four state-of-the-art POI recommenders from various evaluation perspectives including category diversity and geo-diversity that have not been examined previously. The proposed method consistently improves geo-diversity (CPR(geo)@20) by 5 to 12% when combined with four state-of-the-art POI recommenders with negligible prediction accuracy (Recall@20) loss and provides 6 to 18% geo-diversity improvement with tolerable prediction accuracy loss (up to 2.4%).
Tell Me Where It is Still Blurry: Adversarial Blurred Region Mining and Refining

Mobile devices such as smart phones are ubiquitously being used to take photos and videos, thus increasing the importance of image deblurring. This study introduces a novel deep learning approach that can automatically and progressively achieve the task via adversarial blurred region mining and refining (adversarial BRMR). Starting with a collaborative mechanism of two coupled conditional generative adversarial networks (CGANs), our method first learns the image-scale CGAN, denoted as iGAN, to globally generate a deblurred image and locally uncover its still blurred regions through an adversarial mining process. Then, we construct the patch-scale CGAN, denoted as pGAN, to further improve sharpness of the most blurred region in each iteration. Owing to such complementary designs, the adversarial BRMR indeed functions as a bridge between iGAN and pGAN, and yields the performance synergy in better solving blind image deblurring. The overall formulation is self-explanatory and effective to globally and locally restore an underlying sharp image. Experimental results on benchmark datasets demonstrate that the proposed method outperforms the current state-of-the-art technique for blind image deblurring both quantitatively and qualitatively.
Supporting Dynamic Workflows with Automatic Extraction of Goals from BPMN

Organizations willing to employ workflow technology have to be prepared to undertake a significant investment of time and effort due to the exceptionally dynamic nature of the business environment. Today, it is unlikely that processes are modeled once to be repeatedly executed without any changes. Goal-oriented dynamic workflows are a promising approach to provide flexibility to the execution of business processes. Many goal-oriented frameworks exist in the literature to be used for the purpose. However, modeling goals is a burden for the business analyst. This work proposes an automatic approach for extracting goals from a business process for supporting adaptive workflows. The approach consists of a static analysis of the global workflow state. Goals derive from individual BPMN elements and their interactions. For validating the theory, we developed the BPMN2Goals tool, which has been used for supporting a middleware for self-adaptation.
CVAE-Attention: CVAE based Semi-Supervised Sentiment Classification using Attention

Text sentiment classification is an important domain in NLP, and the related technical research has been mature. The sentiment classification of text with the "but" contrastive marker is a challenging problem. In this paper, a semi-supervised framework based on conditional variational autoencoder using attention, called CVAE-Attention, is proposed for sentiment classification. In the CVAE-Attention framework, the attention mechanism is introduced to cope with the contrastive structure. The latent semantic information of the clause after "but" (but-clause) is extracted through the attention model, and is incorporated into the generative model to enlarge the effect of the but-clause. Experiments show that the proposed method is effective compared with other state-of-the-art semi-supervised methods.
Pseudo-Cyclic Network for Unsupervised Colorization with Handcrafted Translation and Output Spatial Pyramids

We present a novel pseudo-cyclic adversarial learning approach for unsupervised colorization of grayscale images. We investigate the use of a non-trainable, lightweight and well-defined Handcrafted Translation to enforce the generation of realistic images and replace one of the two deep convolutional generative adversarial neural networks classically used in cyclic models. Additionally, we propose to use Output Spatial Pyramids to jointly constrain the deep latent spaces of an encoder-decoder generator to preserve spatial structures and improve the quality of the generated images. We demonstrate the interest of our approach compared with the state of the art on standard datasets (paintings, landscapes, aerial, thumbnails) that we modified for the purpose of colorization. We evaluate colorization quality of the generated images along the training with deterministic and reproducible criteria. In complement, we demonstrate the ability of our method to generate representations that are prone to make a classification network generalize well to slightly different color spaces. We believe our approach has potential applications in arts and cultural heritage to produce alternative representations without requiring paired data.
PAN: Persistent Appearance Network with an Efficient Motion Cue for Fast Action Recognition

Despite the remarkable performance in video-based action recognition over the past several years, current state-of-the-art approaches heavily rely on the optical flow as motion representation. However, computing the optical flow in advance is computationally expensive, which restricts action recognition to be real-time. In this paper, we shed light on fast action recognition by lifting the reliance on optical flow. Inspired by Persistence of Vision in human visual system, we design a novel motion cue called Persistence of Appearance (PA), which enables the network to distill motion information directly from adjacent RGB frames. Our PA derives from optical flow and focuses on the small displacements of motion boundaries. Compared with other motion representations, our PA enables the network to achieve competitive accuracy on UCF101. Meanwhile, the inference speed reaches 1855 fps, which is over 120x faster than that of the traditional optical flow based methods. Besides, we devise a decision strategy called Various-timescale inference Pooling (VIP) to empower the network with the ability of long-range temporal modeling across various timescales. We further incorporate the proposed PA and VIP to form a unified framework called Persistent Appearance Network (PAN). Compared with methods using only RGB frames, our delicately designed PAN achieves state-of-the-art results on three benchmark datasets: UCF101, HMDB51 and Kinetics, where it reaches 96.2%, 74.8% and 82.5% accuracy respectively with the run-time speed as high as 595 fps. The code for this project is available at: https://github.com/zhang-can/PAN-PyTorch .
Learning Using Privileged Information for Food Recognition

Food recognition for user-uploaded images is crucial in visual diet tracking, an emerging application linking multimedia and healthcare domains. However, it is challenging due to the various visual appearances of food images. This is caused by different conditions when taking the photos, such as angles, distances, light conditions, food containers, and background scenes. To alleviate such a semantic gap, this paper presents a cross-modal alignment and transfer network (ATNet), which is motivated by the paradigm of learning using privileged information (LUPI). It additionally utilizes the ingredients in food images as an "intelligent teacher" in the training stage to facilitate cross-modal information passing. Specifically, ATNet first uses a pair of synchronized autoencoders to build the base image and ingredient channels for information flow. Subsequently, the information passing is enabled through a two-stage cross-modal interaction. The first stage of interaction adopts a two-step method, called partial heterogeneous transfer, to 1) alleviate the intrinsic heterogeneity between images and ingredients and 2) align them in a shared space to make their carried information about food classes interact. In the second stage, ATNet learns to map the visual embeddings of images to the ingredient channel for food recognition from the view of "teacher''. This leads a refined recognition by a multi-view fusion. Experiments on two real-world datasets show that ATNet can be incorporated with any state-of-the-art CNN models to consistently improve their performance.
HD3: Distributed Dueling DQN with Discrete-Continuous Hybrid Action Spaces for Live Video Streaming

Live streaming applications are becoming increasingly popular recently, and it exposes new technical challenges compared to regular video streaming. High video quality and low latency are two main requirements in live streaming scenarios. A live streaming application needs to make bitrate and target buffer level decisions as well as sets a continuous latency limit value to skip video frames. We formulate the live streaming task as a reinforcement learning problem with discrete-continuous hybrid action spaces, then propose a novel deep reinforcement learning (DRL) algorithm HD3 which can take hybrid actions to solve it. We compare HD3 with several state-of-the-art DRL algorithms on various network environments, and the simulation results show that HD3 can outperform all the other comparison schemes. We emphasize that HD3 generates a single agent which can perform well on different network conditions and video scenes.
3D Singing Head for Music VR: Learning External and Internal Articulatory Synchronicity from Lyric, Audio and Notes

We propose a real-time 3D singing head system to enhance the talking head on model integrity, keyframe generation and song synchronicity. The individual head appearance meshes are first obtained by matching multi-view visible images with face prior for accuracy, and then used to reconstruct entire head model by integrating with generic internal articulatory meshes for efficiency. After embedding physiology, the keyframes of each phoneme-music note correspondence are substantially synthesized from real articulation data. The song synchronicity of articulators is learned using a deep neural network to train visual co-articulation model (VCM) on parallel audio-visual data. Finally, the keyframes of adjacent phoneme-music note correspondences are blended by VCM to produce song synchronized animation. Compared to state-of-the-art baselines, our system can not only clearly distinguish phonemes and notes, but also significantly reduce the dependence on training data.
Malicious Android Application Detection Based on Composite Features

With the use of mobile phones, malicious applications are constantly developing, affecting the normal use of mobile phones by users. For the malicious application of Android platform, a detection model based on combined features is proposed. The model extracts the dynamic and static features and select the importance of them. Selecting Combination Features from important features. Taking the combined features as new features, and combing the single features to detect Android malicious applications. Experiments are carried out using different classification algorithm. which verifies the proposed Android malicious application detection model is feasible and superior, and the detection accuracy is up to 97.12%.
A Survey on Motion Capture Data Compression Algorithm

With the rapid development of data-driven animation technologies, huge motion capture data has been accumulated. Motion capture data is a kind of spatio-temporal high dimensional data, which needs a lot of storage space. Efficient compression and transmission of motion capture data has become a hot topic in computer animation. In this paper, we induce the characteristics of motion capture data and the general processing pipeline of motion capture data compression algorithm. Then we review the research achievements in the field of motion capture data compression for the last twenty years. According to reduced dimensions, running platform, lossy or lossless, motion data format, environment contact processing and progressivity, we put these motion capture data compression algorithms into different categories. Finally, we look forward to the future research trend.
Seeing the Invisible

The latest techniques in video motion magnification and relevant small motion analysis are surveyed. The main motion magnification techniques are discussed in chronological fashion, highlighting the inherent limitations of predecessor techniques in comparison with subsequent variants. The focus is then shifted to the specific stages within the motion magnification framework to discuss advancements that have been proposed in the literature, namely for spatial decomposition and for emphasizing, representing, and distinguishing different motion signals. The survey concludes with a treatment of different problems in varying application contexts that have benefited from motion magnification and small motion analysis.
Fast Intra Coding Algorithm for 360-Degree Videos Based on Texture Characteristics

In this paper, we proposed a fast intra mode and block partition decision algorithm for Versatile Video Coding (VVC), to reduce the complexity of the intra coding. By analyzing the non-zero coefficients of image block after DCT and its texture direction, the candidate modes of RMD can be quickly reduced. In addition, the decision of coding block partition is made through the combination of texture complexity and non-zero coefficients of image block after DCT. Experimental results show that out proposed algorithm can save 49.76% intra coding time with 0.95% BD-rate loss on average in All Intra configuration when compared with standard JEM 7.0.
Ingredient-Guided Cascaded Multi-Attention Network for Food Recognition

Recently, food recognition is gaining more attention in the multimedia community due to its various applications, e.g., multimodal foodlog and personalized healthcare. Most of existing methods directly extract visual features of the whole image using popular deep networks for food recognition without considering its own characteristics. Compared with other types of object images, food images generally do not exhibit distinctive spatial arrangement and common semantic patterns, and thus are very hard to capture discriminative information. In this work, we achieve food recognition by developing an Ingredient-Guided Cascaded Multi-Attention Network (IG-CMAN), which is capable of sequentially localizing multiple informative image regions with multi-scale from category-level to ingredient-level guidance in a coarse-to-fine manner. At the first level, IG-CMAN generates the initial attentional region from the category-supervised network with Spatial Transformer (ST). Taking this localized attentional region as the reference, IG-CMAN combined ST with LSTM to sequentially discover diverse attentional regions with fine-grained scales from ingredient-guided sub-network in the following levels. Furthermore, we introduce a new dataset ISIA Food-200 with 200 food categories from the list in the Wikipedia, about 200,000 food images and 319 ingredients. We conducted extensive experiment on two popular food datasets and newly proposed ISIA Food-200, and verified the effectiveness of our method. Qualitative results along with visualization further show that IG-CMAN can introduce the explainability for localized regions, and is able to learn relevant regions for ingredients.