Reviewerly/exHarmony huggingface dataset:
All-in-depth via Cross-baseline Light Field Camera

Light-field (LF) camera holds great promise for passive/general depth estimation benefited from high angular resolution, yet suffering small baseline for distanced region. While stereo solution with large baseline is superior to handle distant scenarios, the problem of limited angular resolution becomes bothering for near objects. Aiming for all-in-depth solution, we propose a cross-baseline LF camera using a commercial LF camera and a monocular camera, which naturally form a 'stereo camera' enabling compensated baseline for LF camera. The idea is simple yet non-trivial, due to the significant angular resolution gap and baseline gap between LF and stereo cameras.
ATF

Face alignment is an important task in the field of multi-media. Together with the impressive progress of algorithms, various benchmark datasets have been released in recent years. Intuitively, it is meaningful to integrate multiple labeled datasets with different annotations to achieve higher performance on a target landmark detector. Although numerous efforts have been made in joint usage, there yet remain three shortages in recent works, e.g., additional computation, limitation of the markups scheme, and limited support for the regression method. To address the above problems, we proposed a novel Alternating Training Framework (ATF), which leverages similarity and diversity across multi-media sources for a more robust detector. Our framework mainly contains two sub-modules: Alternating Training with Decreasing Proportions (ATDP) and Mixed Branch Loss (mathcal LMB). In particular, ATDP trains multiple datasets simultaneously to take advantage of the diversity between them, while mathcal LMB utilizes similar landmark pairs to constrain different branches of corresponding datasets. Extensive experiments on various benchmarks show the effectiveness of our framework, and ATF is feasible for both heatmap-based network and direct coordinate regression. Specifically, the mean error even reaches 3.17 on the experiment on 300W leveraging WFLW, which significantly outperforms state-of-the-art methods. Both in an ordinary convolutional network (OCN) and HRNET, ATF achieves up to 9.96% relative improvement. Our source codes are made publicly available at https://github.com/starhiking/ATF.
Improving fuzzy clustering algorithm for probability density functions and applying in image recognition

This study introduces a measure called coefficient of within-cluster proximity (CWP) to evaluate the similarity of probability density functions (DFs) within clusters. After surveying the under and upper, and the computational problems of CWP, a fuzzy clustering algorithm for DFs is proposed. This algorithm can determine the suitable number of clusters and find the probability for each DF to belong to specific cluster. The convergence of the algorithm is considered in theory and illustrated by the numerical examples. The algorithm is applied to image recognition. The results show strong advantages of it in comparison to other algorithms. They also indicate the potential of the proposed approach in application to the data of different types.
A Quantitative Comparison of Different Machine Learning Approaches for Human Spermatozoa Quality Prediction Using Multimodal Datasets

Despite remarkable advances in medical data analysis fields, they are severely restrained from the limited property of the employed single modality, usually medical imaging data. However, other modalities (such as patient-related information) should also be taken into account in the process of clinical decision. How to fully employ the multi-modal dataset is still under-explored. In this paper, we make a quantitative comparison of different machine learning approaches for the human spermatozoa quality prediction task, leveraging multiple modalities dataset. To empirically investigate the advantages and disadvantages of different machine learning approaches, we perform extensive experiments. Leveraging different features, we achieve state-of-the-art performance on most of the tasks. The obtained results show that simple models can provide better performance, which emphasizes the importance of avoiding overfitting. For the sake of reproducibility, we have released our code to facilitate the research community.
A Fair and Secure Reverse Auction for Government Procurement

With the development of e-commerce, the electronic auction is attracting the attention of many people. Many Internet companies, such as eBay and Yahoo!, have launched online auction systems. Many researchers have studied the security problems of electronic auction systems, but few of them are multi-attribute-based. In 2014, Shi proposed a provable secure, sealed-bid, and multi-attribute auction protocol based on the semi-honest model. We evaluated this protocol and found that it has some design weaknesses and is vulnerable to the illegal operations of buyers, which results in unfairness. In this paper, we improved this protocol by replacing the Paillier’s cryptosystem with the elliptic curve discrete (ECC), and we designed a novel, online, and multi-attribute reverse-auction system using the semi-honest model. In our system, sellers’ identities are not revealed to the buyers, and the buyers cannot conduct illegal operations that may compromise the fairness of the auction.
Multi-Feature Fusion Method Based on Salient Object Detection for Beauty Product Retrieval

Beauty and Personal care product retrieval has attracted more and more attention due to its wide application value. However, due to the diversity of data and the complexity of image background, this task is very challenging. In this paper, we propose a multi-feature fusion method based on salient object detection to improve retrieval performance. The key of our method is to extract the foreground objects of the query set by using the salient object detection network, so as to eliminate the background interference. Then the foreground target images and dataset are put into the multi-classification networks to extract multiple fusion features for retrieval. We use the perfect-500k dataset for experiments, and the results show that our method is effective. Our method ranked 2st in the Grand Challenge of AI Meets Beauty in ACM Multimedia 2020 with a MAP score of 0.43729. We released our code on GitHub:github.com/R-M-Yan/ACMMM2020AIMeetBeauty.
Avoid Crowding in the Battlefield

Crisis situations often require authorities to convey important messages to a large population of varying demographics. An example of such a message is maintain a distance of 6 ft from others in times of the present COVID-19 crisis. In this paper, we propose a method to programmatically place such messages in existing entertainment media as overlays at semantically relevant locations. For this purpose, we use generic semantic annotations on the media and subsequent spatio-temporal querying on these annotations to find candidate locations for message placement. We then propose choosing the final locations optimally using parameters such as spacing of messages, length of the messages and confidence of query results. We present preliminary results for optimal placement of messages in popular entertainment media.
The use of proactive communication through knowledge management to create awareness and educate clients on e-banking fraud prevention

Purpose: This study sets out to address the lack of studies to educate clients and create awareness on fraud prevention in this sector, specifically on e-banking fraud that falls into two main categories: phishing/vishing/Smishing and malware practices. Design/methodology/approach: An interpretivist approach was used to analyse a real online website that is available to all users. The subject under study was the website of the South African Banking Risk Information Centre (SABRIC). The three concurrent cyclical flow of activity of the data analysis interactive model was utilised in the research. An abductive approach was used to report on the findings based on descriptions and interpretive comments relating it to and drawing on the theoretical thrusts identified. The research was conducted during the two time frames of 2017 and 2018. Findings/results: It was found that SABRIC did use its website to communicate proactively with clients to create awareness and educate them on e-banking fraud prevention measures. A significant increase in proactive communication through various activities was evident when the 2017 and 2018 statistics were compared. Practical implications: The main themes that emerged from the findings include the need for collaborative efforts, member training, intervention by government and law enforcement agencies and the importance of awareness of protective measures. Originality/value: This study contributes to the limited body of knowledge and literature investigating the need for proactive communication through knowledge management typologies in emerging economic contexts to create awareness and educate clients on e-banking fraud prevention to address the intensification of cases in fraud.
Phonology based Fuzzy Phoneme Recognition

Speech enhanced applications are fundamentally based on an effective separation and recognition of phonetic units. The variable nature of speech signal makes it challenging due to higher rate of variability in the phonetic units. This nature of variability is addressed with the help of Fuzzy Logic in this work for identification of phonemes. The feature parameters representing the speech signal variability in the work are Zero Crossing Rate (ZCR) and Short Term Energy (STE). The approach used in the presented work demonstrate to be more effective than other methods such as Artificial Neural Networks (ANN). A Fuzzy Logic model is tuned using features with various phonemes. Phonology based fuzzy decision logic is used to infer about phonetic units separation point. Phonetic analysis has been done for different spoken words of English language identify the vowel recognition.
Knowledge-Building Approach for Tsunami Impact Analysis Aided by Citizen Science

Large tsunami events are devastating even though the impacts are limited to coastal areas. Knowledge is the key to minimize the losses from natural disasters by more proactive and better preparation. Targeting on natural hazards mitigation in Asia, a collaboration among 10 Asian and 2 European countries based on deeper understanding approach has been conducted since 2015. Deeper understanding is to discover the physical mechanisms and drivers behind a hazard event. Innovative model and simulation facilities are developed correspondingly to achieve more accurate numerical simulations of the whole lifespan of the target event. Application framework composed by the knowledge, data, simulation facility, software tools and case studies is designed to provide advanced estimation of hazard risk and would be evolved progressively with more case studies and observation data. For tsunami, based on the COMCOT (COrnell Multi-grid Coupled Tsunami Model), the simulation portal (iCOMCOT) implementing parallelized tsunami wave propagation calculation over distributed cloud had been established. iCOMCOT system finished the simulation of whole lifecycle of the 2011 Great East Japan Earthquake Tsunami in one minute. In this regional collaboration, case studies on historical events and tsunami impact analysis were conducted. The goal is to capture the physical characteristics of tsunami as much as possible such as tsunami wave propagation, tsunami refraction, tsunami run-up on land, as well as their drivers and root causes. The whole processes of tsunami from its initiation to its impacts in selected locations then could be simulated accurately by iCOMCOT based on the scientific explorations and the revised models quantitatively. Sulawesi Tsunami (2018) case is presented to demonstrate the processes of deeper understanding approach and how to achieve the capacity building. At the same time, the ways to take advantages of citizen science are also explored. Citizen science model is valuable in supporting data collection such as data of run up height, inundation range, flow depth, disruption information, impact area, from publication, news report, and interviews of local people. According to experiences on case studies, suggestions to simplify and optimize the integration of citizen science model with deeper understanding approach to lower operation cost are provided.
Learning to Remember Beauty Products

This paper develops a deep learning model for the beauty product image retrieval problem. The proposed model has two main components- an encoder and a memory. The encoder extracts and aggregates features from a deep convolutional neural network at multiple scales to get feature embeddings. With the use of an attention mechanism and a data augmentation method, it learns to focus on foreground objects and neglect background on images, so can it extract more relevant features. The memory consists of representative states of all database images as its stacks, and it can be updated during training process. Based on the memory, we introduce a distance loss to regularize embedding vectors from the encoder to be more discriminative. Our model is fully end-to-end, requires no manual feature aggregation and post-processing. Experimental results on the Perfect-500K dataset demonstrate the effectiveness of the proposed model with a significant retrieval accuracy.
Applying User Centered design to Mobile Application for Children

Smartphone technology is growing rapidly. Children are one of the active smartphone users. This encourages the development of language applications for children. Child Centered Design (CCD) is a methodology used in this development. This methodology involves the child in every stage of development from the data collection stage to testing. The steps contained in the CCD methodology are determining the end user, collecting data or needs. Data obtained through observation and interviews with children and parents. The data were analyzed and the results of the analysis were designed according to the children needs. Designs are evaluated directly by the children. This activity is done in several iterations for usability to be achieved. Usability testing is used to measure application usability. The results of this research is the application of Batak, Indonesian and English language introduction for children. The application has four main content: animals, plants, family, and organs. In addition, there are picture and sound quiz features. Applications have met the children needs, both content and interface design.
Predicting change in newly created files in a software product line project

Abstract At the beginning of the testing phase and before the deployment phase of a project's development cycle, we need to predict files with a high chance of change. Software products are always prone to change due to several reasons, including fixing errors or improvements. In this work, we used the Eclipse (releases from 2.0 to 3.5) to investigate how prediction models can perform when learning from a release and predicting in the subsequent one, which contains new files that models have not seen. We compared the performance of these models with models that are trained and tested on the same release. We found no differences between predicting the same release or subsequent release on two pre Europa releases. Predicting change in newly created files helps improve maintenance planning for software project managers and reduce cost. It will also help to enhance the quality of software by improving the practices of developers. This study used the Adaptive Boost classifier with the decision tree J48 algorithm and combined it with the re‐sampling method. We find this to be better than using a meta classifier alone or combine the re‐sampling with the standard classification. We compared our results with related works and found that our results are outperforming.
Towards capsule routing as reconstruction with sparsity constraints

The most recently-proposed concept of capsule network consists of capsules, a structural group of neurons with activation, as building blocks, and dynamic routing between them as connections. Semantic information for final tasks can be extracted by stacking such capsule layers as construction of deep models. In this paper, we formulate the dynamic routing problem from the perspective of feature compression and reconstruction, proposing a novel routing algorithm which explicitly minimizes the reconstruction error with sparsity constraints. By alternately updating poses and routing weights with closed-form solutions, the method ensures a theoretical convergence of the dynamic routing. Benefitting from the strict convergence, we further prompt the efficiency of routing with the use of an adaptive number of iterations. Experimental results on different datasets prove that our proposed routing method outperforms existing routing methods on both performance as well as efficiency with obvious margin. Furthermore, we empirically show that the reconstruction based routing method can be embedded into backbone networks as a representative compression module. The integration of convolutional networks and capsule routing delivers both reduction of model parameters and improvement of performance to original backbones.
Large-Scale Network Representation Learning Based on Improved Louvain Algorithm and Deep Autoencoder

In recent years, feature learning of nodes in network has become a research hot spot. However, with the growth of the network scale, network structure has become more and more complicated, which makes it extremely difficult for network representation learning in large and complex networks. This paper proposes a fast large-scale network representation learning method based on improved Louvain algorithm and deep autoencoder. First, it quickly folds large and complex network into corresponding small network kernel through effective improved Louvain strategy. Then based on network kernel, a deep autoencoder method is conducted to represent nodes in kernel. Finally, the representations of the original network nodes are obtained by a coarse-to-refining procedure. Extensive experiments show that the proposed method perform well on large and complex real networks and its performance is better than most network representation learning methods.
Grouping of Medicinal Drugs Used for Similar Symptoms by Mining Clusters from Drug Benefits Reviews

Clustering is an unsupervised process for partitioning a set of data into a set of meaningful sub-classes or sub-datasets, called clusters. In this paper, the k-means clustering algorithm is used on a textual dataset of unlabeled reviews of medicinal drugs in order to group the drugs with similar usage and benefits. Two alternative clustering algorithms, agglomerative clustering and BIRCH (balanced iterative reducing and clustering using hierarchies), are used on the same dataset for comparison. The methodology for text preprocessing and feature extraction is explained. The results are analysed using quantitative performance metrics (Silhouette Score and Calinski-Harabasz Score), cluster sizes, as well as visualization by the means of scatter plot of clusters, distances from final centroids, word clouds and dendrogram.
Robust data hiding with multiple backups and optimized reference matrix

Most data hiding methods have limitations in resisting cropping and noise attacks. Aiming at this problem, a robust data hiding with multiple backups and optimized reference matrix is proposed in this paper. Specifically, secret data is divided into a set of groups and multiple backups of each group data are generated according to the number of backups. The cover image is divided into several blocks. A reference matrix is constructed by four constraints to assist data hiding and data extraction. The proposed method aims to extract exactly at least one backup of each group data so that the correct backups can construct the secret data well if the stego-image is corrupted. Experimental results show that the proposed algorithm is robust to cropping and noise attacks.
Multi-Scale Generalized Attention-Based Regional Maximum Activation of Convolutions for Beauty Product Retrieval

The application of beauty and personal-care product retrieval seems to be evident in our daily life, and it has attracted increasing research interests during the last decade. However, the retrieval task is suffered from different image variations and complicated backgrounds. Recent works have demonstrated that Generalized-attention Regional Maximal Activation of Convolutions (GRMAC) descriptor can provide state-of-the-art performance for the retrieval task. However, GRMAC descriptor is restrained from the essentially limited property of the employed feature from a single layer. Features from a single layer are not robust enough for scale variations, shape deformation, and heavy occlusion. In this paper, we propose a novel descriptors, named Multi-Scale Generalized Attention-Based Regional Maximum Activation of Convolutions (MS-GRMAC). This method introduces multi-scale generalized attention mechanism to reduce the influence of scale variations, thus, can boost the performance of the retrieval task. To empirically investigate the effectiveness of the proposed approach, we conduct extensive experiments on the dataset containing more than half-million personal-care products (Perfect-500K) and obtain satisfactory results without ensemble.
EmotionTracker: A Mobile Real-time Facial Expression Tracking System with the Assistant of Public AI-as-a-Service

Public AI-as-a-Service (AIaaS) is a promising next-generation computing paradigm that attracts resource-limited mobile users to outsource their machine learning tasks. However, the time delay between cloud/edge servers and end users makes it hard for real-time mobile artificial intelligence applications. In this demonstration, we present EmotionTracker, a real-time mobile facial expression tracking system combining AIaaS and mobile local auxiliary computing, including facial expression tracking and the corresponding task offloading. Mobile facial expression tracking iteratively estimates the facial expression with the help of sparse optical flow and neural network. Task offloading dynamically estimate the moment of task offloading with machine learning method. According to the results in a real-world environment, EmotionTracker successfully fulfills the mobile real-time facial expression tracking requirements.
Expressional Region Retrieval

Image retrieval is a long-standing topic in the multimedia community due to its various applications, e.g., product search and artworks retrieval in museum. The regions in images contain a wealth of information. Users may be interested in the objects presented in the image regions or the relationships between them. But previous retrieval methods are either limited to the single object of images, or tend to the entire visual scene. In this paper, we introduce a new task called expressional region retrieval, in which the query is formulated as a region of image with the associated description. The goal is to find images containing the similar content with the query and localize the regions within them. As far as we know, this task has not been explored yet. We propose a framework to address this issue. The region proposals are first generated based on region detectors and language features are extracted. Then the Gated Residual Network (GRN) takes language information as a gate to control the transformation of visual features. In this way, the combined visual and language representation is more specific and discriminative for expressional region retrieval. We evaluate our method on a new established benchmark which is constructed based on the Visual Genome dataset. Experimental results demonstrate that our model effectively utilizes both visual and language information, outperforming the baseline methods.
Events and Machine Learning

Hebblewhite, Hohwy, and Drummond view the special issue from a machine learning perspective drawing close relations to model‐based, hierarchical reinforcement learning.