Reviewerly/exHarmony huggingface dataset:
Time and Temporality in HCI Research

Abstract ‘Time’ and ‘temporality’ are increasingly addressed in human-computer interaction (HCI) research. From issues related to mapping and visualizing data along timelines via explorations of temporality as a design material to studies of lag and the rhythms of work, time is a recurring perspective in HCI research. In fact, time has been a recurring aspect in HCI research for the past 30 years. Based on this continuous and growing interest in HCI research, we find it to be a good time to explore if ‘time’ can be used as a way to structure and organize HCI research. In this paper, we make one such attempt based on a literature study in which we have focused on how time and temporality has been addressed in HCI research during the past 30 years. In our overview of the field, we explore how time and temporality has played out in HCI along the two dimensions of what and how it has been studied. Based on these two dimensions, we created a 4 × 4 matrix that allowed us to filter the material and categorize HCI research in relation to time and temporality. As a result of our explorations, we have identified a turn to temporality as a design material in HCI, an interest in methods for temporality studies in HCI and in temporality as a theoretical lens. We end with a discussion of some implications of our findings around the notions of (i) waves of time and temporality studies in HCI research, (ii) the potential use of the 4 × 4 matrix and (iii) the consequences of a more fundamental shift from things to events.
Precise error estimation for sketch-based flow measurement

As a class of approximate measurement approaches, sketching algorithms have significantly improved the estimation of network flow information using limited resources. While these algorithms enjoy sound error-bound analysis under worst-case scenarios, their actual errors can vary significantly with the incoming flow distribution, making their traditional error bounds too "loose" to be useful in practice. In this paper, we propose a simple yet rigorous error estimation method to more precisely analyze the errors for posterior sketch queries by leveraging the knowledge from the sketch counters. This approach will enable network operators to understand how accurate the current measurements are and make appropriate decisions accordingly (e.g., identify potential heavy users or answer "what-if" questions to better provision resources). Theoretical analysis and trace-driven experiments show that our estimated bounds on sketch errors are much tighter than previous ones and match the actual error bounds in most cases.
Beyond linear subspace clustering: A comparative study of nonlinear manifold clustering algorithms

Subspace clustering is an important unsupervised clustering approach. It is based on the assumption that the high-dimensional data points are approximately distributed around several low-dimensional linear subspaces. The majority of the prominent subspace clustering algorithms rely on the representation of the data points as linear combinations of other data points, which is known as a self-expressive representation. To overcome the restrictive linearity assumption, numerous nonlinear approaches were proposed to extend successful subspace clustering approaches to data on a union of nonlinear manifolds. In this comparative study, we provide a comprehensive overview of nonlinear subspace clustering approaches proposed in the last decade. We introduce a new taxonomy to classify the state-of-the-art approaches into three categories, namely locality preserving, kernel based, and neural network based. The major representative algorithms within each category are extensively compared on carefully designed synthetic and real-world data sets. The detailed analysis of these approaches unfolds potential research directions and unsolved challenges in this field.
Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels

The confusion matrix, a ubiquitous visualization for helping people evaluate machine learning models, is a tabular layout that compares predicted class labels against actual class labels over all data instances. We conduct formative research with machine learning practitioners at Apple and find that conventional confusion matrices do not support more complex data-structures found in modern-day applications, such as hierarchical and multi-output labels. To express such variations of confusion matrices, we design an algebra that models confusion matrices as probability distributions. Based on this algebra, we develop Neo, a visual analytics system that enables practitioners to flexibly author and interact with hierarchical and multi-output confusion matrices, visualize derived metrics, renormalize confusions, and share matrix specifications. Finally, we demonstrate Neo's utility with three model evaluation scenarios that help people better understand model performance and reveal hidden confusions.
Hand Alphabet Recognition for Dactylology Conversion to English Print Using Streaming Video Segmentation

research-article Share on Hand Alphabet Recognition for Dactylology Conversion to English Print Using Streaming Video Segmentation Authors: Manuel Bautista Garcia FEU Institute of Technology, Philippines FEU Institute of Technology, PhilippinesView Profile , Teodoro Feria Revano Jr. FEU Institute of Technology, Philippines FEU Institute of Technology, PhilippinesView Profile , Armi Cunanan Yabut FEU Institute of Technology, Philippines FEU Institute of Technology, PhilippinesView Profile Authors Info & Claims ICCCM '21: The 2021 9th International Conference on Computer and Communications ManagementJuly 2021 Pages 46–51https://doi.org/10.1145/3479162.3479169Online:28 October 2021Publication History 0citation26DownloadsMetricsTotal Citations0Total Downloads26Last 12 Months26Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Learning Personalized Image Aesthetics From Subjective and Objective Attributes

Due to the widespread popularity of social media, researchers have developed a strong interest in learning the personalized image aesthetics of online users. Personalized image aesthetics assessment (PIAA) aims to study the aesthetic preferences of individual users for images, which should be affected by the properties of both users and images. Existing PIAA approaches usually use the generic aesthetics learned from images as a prior model and adapt it to PIAA models through a small number of data annotated by individual users. However, the prior model merely learns the objective attributes of images, which is agnostic to the subjective attributes of users, complicating efficient learning of the personalized image aesthetics of individual users. Therefore, we propose a personalized image aesthetics assessment method that integrates the subjective attributes of users and objective attributes of images simultaneously. To characterize these two attributes jointly, an attribute extraction module is introduced to learn users’ personality traits and image aesthetic attributes. Then, an aesthetic prior model is built from numerous individual users’ annotated data, which leverages the personality traits of users and the aesthetic attributes of rated images as prior knowledge to model both the image aesthetic distribution and users’ residual scores relative to generic aesthetics simultaneously. Finally, a PIAA model is obtained by fine-tuning the aesthetic prior model with an individual user’s annotated data. Experiments demonstrate that the proposed method is superior to existing PIAA methods in learning individual users’ personalized image aesthetics.
Multi-caption Text-to-Face Synthesis: Dataset and Algorithm

Text-to-Face synthesis with multiple captions is still an important yet less addressed problem because of the lack of effective algorithms and large-scale datasets. We accordingly propose a Semantic Embedding and Attention (SEA-T2F) network that allows multiple captions as input to generate highly semantically related face images. With a novel Sentence Features Injection Module, SEA-T2F can integrate any number of captions into the network. In addition, an attention mechanism named Attention for Multiple Captions is proposed to fuse multiple word features and synthesize fine-grained details. Considering text-to-face generation is an ill-posed problem, we also introduce an attribute loss to guide the network to generate sentence-related attributes. Existing datasets for text-to-face are either too small or roughly generated according to attribute labels, which is not enough to train deep learning based methods to synthesize natural face images. Therefore, we build a large-scale dataset named CelebAText-HQ, in which each image is manually annotated with 10 captions. Extensive experiments demonstrate the effectiveness of our algorithm.
Studying the Effects of Self-Attention for Medical Image Analysis

When the trained physician interprets medical images, they understand the clinical importance of visual features. By applying cognitive attention, they apply greater focus onto clinically relevant regions while disregarding unnecessary features. The use of computer vision to automate the classification of medical images is widely studied. However, the standard convolutional neural network (CNN) does not necessarily employ subconscious feature relevancy evaluation techniques similar to the trained medical specialist and evaluates features more generally. Self-attention mechanisms enable CNNs to focus more on semantically important regions or aggregated relevant context with long-range dependencies. By using attention, medical image analysis systems can potentially become more robust by focusing on more important clinical feature regions. In this paper, we provide a comprehensive comparison of various state-of-the-art self-attention mechanisms across multiple medical image analysis tasks. Through both quantitative and qualitative evaluations along with a clinical user-centric survey study, we aim to provide a deeper understanding of the effects of self-attention in medical computer vision tasks.
Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

High dynamic range (HDR) imaging is a technique that allows an extensive dynamic range of exposures, which is important in image processing, computer graphics, and computer vision. In recent years, there has been a significant advancement in HDR imaging using deep learning (DL). This study conducts a comprehensive and insightful survey and analysis of recent developments in deep HDR imaging methodologies. We hierarchically and structurally group existing deep HDR imaging methods into five categories based on (1) number/domain of input exposures, (2) number of learning tasks, (3) novel sensor data, (4) novel learning strategies, and (5) applications. Importantly, we provide a constructive discussion on each category regarding its potential and challenges. Moreover, we review some crucial aspects of deep HDR imaging, such as datasets and evaluation metrics. Finally, we highlight some open problems and point out future research directions.
Weather and Light Level Classification for Autonomous Driving: Dataset, Baseline and Active Learning

Autonomous driving is rapidly advancing, and Level 2 functions are becoming a standard feature. One of the foremost outstanding hurdles is to obtain robust visual perception in harsh weather and low light conditions where accuracy degradation is severe. It is critical to have a weather classification model to decrease visual perception confidence during these scenarios. Thus, we have built a new dataset for weather (fog, rain, and snow) classification and light level (bright, moderate, and low) classification. Furthermore, we provide street type (asphalt, grass, and cobblestone) classification, leading to 9 labels. Each image has three labels corresponding to weather, light level, and street type. We recorded the data utilizing an industrial front camera of RCCC (red/clear) format with a resolution of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1024 \times 1084$</tex> , We collected 15k video sequences and sampled 60k images. We implement an active learning framework to reduce the dataset's redundancy and find the optimal set of frames for training a model. We distilled the 60k images further to 1.1k images, which will be shared publicly after privacy anonymization. There is no public dataset for weather and light level classification focused on autonomous driving to the best of our knowledge. The baseline ResNet18 network used for weather classification achieves state-of-the-art results in two non-automotive weather classification public datasets but significantly lower accuracy on our proposed dataset, demonstrating it is not saturated and needs further research.
Multi-Factors Aware Dual-Attentional Knowledge Tracing

With the increasing demands of personalized learning, knowledge tracing has become important which traces students' knowledge states based on their historical practices. Factor analysis methods mainly use two kinds of factors which are separately related to students and questions to model students' knowledge states. These methods use the total number of attempts of students to model students' learning progress and hardly highlight the impact of the most recent relevant practices. Besides, current factor analysis methods ignore rich information contained in questions. In this paper, we propose Multi-Factors Aware Dual-Attentional model (MF-DAKT) which enriches question representations and utilizes multiple factors to model students' learning progress based on a dual-attentional mechanism. More specifically, we propose a novel student-related factor which records the most recent attempts on relevant concepts of students to highlight the impact of recent exercises. To enrich questions representations, we use a pre-training method to incorporate two kinds of question information including questions' relation and difficulty level. We also add a regularization term about questions' difficulty level to restrict pre-trained question representations to fine-tuning during the process of predicting students' performance. Moreover, we apply a dual-attentional mechanism to differentiate contributions of factors and factor interactions to final prediction in different practice records. At last, we conduct experiments on several real-world datasets and results show that MF-DAKT can outperform existing knowledge tracing methods. We also conduct several studies to validate the effects of each component of MF-DAKT.
Using Explainable Machine Learning to Improve Intensive Care Unit Alarm Systems

Due to the continuous monitoring process of critical patients, Intensive Care Units (ICU) generate large amounts of data, which are difficult for healthcare personnel to analyze manually, especially in overloaded situations such as those present during the COVID-19 pandemic. Therefore, the automatic analysis of these data has many practical applications in patient monitoring, including the optimization of alarm systems for alerting healthcare personnel. In this paper, explainable machine learning techniques are used for this purpose, with a methodology based on age-stratification, boosting classifiers, and Shapley Additive Explanations (SHAP) proposed. The methodology is evaluated using MIMIC-III, an ICU patient research database. The results show that the proposed model can predict mortality within the ICU with AUROC values of 0.961, 0.936, 0.898, and 0.883 for age groups 18-45, 45-65, 65-85 and 85+, respectively. By using SHAP, the features with the highest impact in predicting mortality for different age groups and the threshold from which the value of a clinical feature has a negative impact on the patient's health can be identified. This allows ICU alarms to be improved by identifying the most important variables to be sensed and the threshold values at which the health personnel must be warned.
Investigating Students’ Digital Literacy Levels during Online Education Due to COVID-19 Pandemic

COVID-19 has brought serious challenges to education and many other sectors. Within the educational context, the main difficulties experienced have been reported as challenges related to not only mechanical impediments—such as technology or infrastructure—but also to methodological and personal features, such as lack of motivation or online learning/teaching experiences. The abrupt transition from face-to-face to online education has created the need for some specific abilities, such as digital literacy on the side of the learners at all educational levels. In this context, this mixed-method study aims to determine the digital literacy levels of learners belonging to different school levels and whether age, gender and school degree were significant variables. This study also investigates the technology-related challenges students experienced during COVID-19 pandemic. The sample consisted of 510 participants representing different school levels, age groups, and genders; in addition, a smaller representative group (n = 30) revealed their own evaluations related to their levels of digital literacy and the technology-related challenges they had about online learning. The findings suggest that there is a statistically significant relationship between students’ digital literacy and their gender and school degree, while age was not found to be a statistically significant variable. The qualitative self-reported data suggested that learners have sufficient levels of digital literacy, and that the major technology-related challenges were reported to be lack of the necessary technologies and difficulties in adapting to a new approach to learning.
An Integrated Design Based on Dual Thresholding and Features Optimization for White Blood Cells Detection

White blood cells (WBC) are an important component of the immune mechanism, as they protect the human body from parasites, viruses, fungi, and bacteria. The number of blood cells provides significant information related to infections such as AIDS, leukemia, deficiencies of immune and autoimmune infections. To heal an infection on time, it is critical to recognize it early on. Therefore, a method is proposed to accurately segment and classify WBC at an early stage. The RGB image is converted into HSV after which dual thresholding is applied to the saturation component to segment WBC. The 1000 features are extracted from Alexnet to FC8 layer, the Logits layer is selected for feature extraction from mobilenetv2, the node_202 layer is utilized to extract the features from the shuffle net, and the FC1000 layer is chosen from the Resnet-18 model. Four feature vectors are obtained from transfer learning models; these feature vectors are combined serially and create the final optimized vector by a non-dominated sorting genetic algorithm (NSGA). The classification results are investigated on the fusion of Alexnet, shuffle net, Resnet-18, mobilenetv2, and the fusion of mobilenetv2, shuffle net, and Resnet-18 whereas mobilenetv2 features are fused independently. The final optimized feature vector is passed to classifiers including Naïve Bayes (NB), Decision tree (DT), Ensemble, Linear Discriminant Analysis (LDA), Support vector machine (SVM), and K nearest neighbor (KNN) to classify WBC. The method is tested on three publicly available datasets as LISC, ALL_IDB1, and ALL_IDB2. The method achieved the maximum of 1.00 accuracy to classify the blast/non-blast cells, 0.9992 accuracy on Basophil cells, and 1.00 accuracy on Lymphocyte, Neutrophil, Monocyte, Eosinophil, and mixture of these cells. When compared to existing modern approaches, the proposed method produces better outcomes.
Cross‐media search method based on complementary attention and generative adversarial network for social networks

The rapid development of the social network has brought great convenience to people's lives. A large amount of cross-media big data, such as text, image, and video data, has been accumulated. A cross-media search can facilitate a quick query of information so that users can obtain helpful content for social networks. However, cross-media data suffer from semantic gaps and sparsity in social networks, which bring challenges to cross-media searches. To alleviate the semantic gaps and sparsity, we propose a cross-media search method based on complementary attention and generative adversarial networks (CAGS). To obtain high-quality feature representations, we build a complementary attention mechanism containing the focused and unfocused features of images to realize the consistent association of cross-media data in social networks. By designing the cross-media adversarial learning process, we can obtain a common semantic representation of cross-media data and further alleviate the semantic gap and sparsity issues for social networks. Finally, we perform a similarity calculation to realize an accurate cross-media search. We construct four search tasks utilizing two standard cross-media data sets to verify the search performance of the proposed CAGS.
XAI-AV: Explainable Artificial Intelligence for Trust Management in Autonomous Vehicles

Artificial intelligence (AI) is the most looked up technology with a diverse range of applications across all the fields, whether it is intelligent transportation systems (ITS), medicine, healthcare, military operations, or others. One such application is autonomous vehicles (AVs), which comes under the category of AI in ITS. Vehicular Adhoc Networks (VANET) makes communication possible between AVs in the system. The performance of each vehicle depends upon the information exchanged between AVs. False or malicious information can perturb the whole system leading to severe consequences. Hence, the detection of malicious vehicles is of utmost importance. We use machine learning (ML) algorithms to predict the flaw in the data transmitted. Recent papers that used the stacking ML approach gave an accuracy of 98.44%. Decision tree-based random forest is used to solve the problem in this paper. We achieved accuracy and F1 score of 98.43% and 98.5% respectively on the VeRiMi dataset in this paper. Explainable AI (XAI) is the method and technique to make the complex black-box ML and deep learning (DL) models more interpretable and understandable. We use a particular model interface of the evaluation metrics to explain and measure the model’s performance. Applying XAI to these complex AI models can ensure a cautious use of AI for AVs.
A faster secure content-based image retrieval using clustering for cloud

Secure indexing, ranking, searching, and retrieval in encrypted domain without compromising privacy is a challenging task. This is even more difficult in the case of images. In this paper, we propose a faster, more efficient, and highly scalable secure content-based image retrieval scheme using clustering for cloud. In the proposed scheme, the image owner extracts MPEG-7 visual descriptors from all the images and clusters them for indexing. These image features and cluster centers encrypted using Asymmetric scalar-product preserving encryption are offloaded to the cloud along with encrypted images to ensure secure retrieval of images. We also propose a copy-deterrence mechanism to identify untrustworthy query users. The proposed scheme is highly scalable, up to seven times faster in searching and up to 23 times faster in indexing with better retrieval precision compared with the fastest state-of-the-art scheme. The proposed copy deterrence mechanism ensures better speed and perceptual quality of watermarked images with better or comparable extraction accuracy when compromised.