Reviewerly/exHarmony huggingface dataset:
Intrinsic Emotion Recognition Considering the Emotional Association in Dialogues

Computer communication via text messaging or Social Networking Services (SNS) has become increasingly popular. At this time, many studies are being conducted to analyze user information or opinions and recognize emotions by using a large amount of data. Currently, the methods for the emotion recognition of dialogues requires an analysis of emotion keywords or vocabulary, and dialogue data are mostly classified as a single emotion. Recently, datasets classified as multiple emotions have emerged, but most of them are composed of English datasets. For accurate emotion recognition, a method for recognizing various emotions in one sentence is required. In addition, multi-emotion recognition research in Korean dialogue datasets is also needed. Since dialogues are exchanges between speakers. One’s feelings may be changed by the words of others, and feelings, once generated, may last for a long period of time. Emotions are expressed not only through vocabulary, but also indirectly through dialogues. In order to improve the performance of emotion recognition, it is necessary to analyze Emotional Association in Dialogues (EAD) to effectively reflect various factors that induce emotions. Therefore, in this paper, we propose a more accurate emotion recognition method to overcome the limitations of single emotion recognition. We implement Intrinsic Emotion Recognition (IER) to understand the meaning of dialogue and recognize complex emotions. In addition, conversations are classified according to their characteristics, and the correlation between IER is analyzed to derive Emotional Association in Dialogues (EAD) and apply them. To verify the usefulness of the proposed technique, IER applied with EAD is tested and evaluated. This evaluation determined that Micro-F1 of the proposed method exhibited the best performance, with 74.8% accuracy. Using IER to assess the EAD proposed in this paper can improve the accuracy and performance of emotion recognition in dialogues.
Full-Memory Transformer for Image Captioning

The Transformer-based approach represents the state-of-the-art in image captioning. However, existing studies have shown Transformer has a problem that irrelevant tokens with overlapping neighbors incorrectly attend to each other with relatively large attention scores. We believe that this limitation is due to the incompleteness of the Self-Attention Network (SAN) and Feed-Forward Network (FFN). To solve this problem, we present the Full-Memory Transformer method for image captioning. The method improves the performance of both image encoding and language decoding. In the image encoding step, we propose the Full-LN symmetric structure, which enables stable training and better model generalization performance by symmetrically embedding Layer Normalization on both sides of the SAN and FFN. In the language decoding step, we propose the Memory Attention Network (MAN), which extends the traditional attention mechanism to determine the correlation between attention results and input sequences, guiding the model to focus on the words that need to be attended to. Our method is evaluated on the MS COCO dataset and achieves good performance, improving the result in terms of BLEU-4 from 38.4 to 39.3.
Journey through a virtual tunnel: Simulated motion and its effects on the experience of time

This paper examines the relationship between time and motion perception in virtual environments. Previous work has shown that the perception of motion can affect the perception of time. We developed a virtual environment that simulates motion in a tunnel and measured its effects on the estimation of the duration of time, the speed at which perceived time passes, and the illusion of self-motion, also known as vection. When large areas of the visual field move in the same direction, vection can occur; observers often perceive this as self-motion rather than motion of the environment. To generate different levels of vection and investigate its effects on time perception, we developed an abstract procedural tunnel generator. The generator can simulate different speeds and densities of tunnel sections (visibly distinguishable sections that form the virtual tunnel), as well as the degree of embodiment of the user avatar (with or without virtual hands). We exposed participants to various tunnel simulations with different durations, speeds, and densities in a remote desktop and a virtual reality (VR) laboratory study. Time passed subjectively faster under high-speed and high-density conditions in both studies. The experience of self-motion was also stronger under high-speed and high-density conditions. Both studies revealed a significant correlation between the perceived passage of time and perceived self-motion. Subjects in the virtual reality study reported a stronger self-motion experience, a faster perceived passage of time, and shorter time estimates than subjects in the desktop study. Our results suggest that a virtual tunnel simulation can manipulate time perception in virtual reality. We will explore these results for the development of virtual reality applications for therapeutic approaches in our future work. This could be particularly useful in treating disorders like depression, autism, and schizophrenia, which are known to be associated with distortions in time perception. For example, the tunnel could be therapeutically applied by resetting patients’ time perceptions by exposing them to the tunnel under different conditions, such as increasing or decreasing perceived time.
Blockchain-based differentiated authentication mechanism for 6G heterogeneous networks

Abstract It is well known that the Sixth Generation (6G) communication system integrating multiple access networks promotes the internet of everything world-widely. However, due to the differentiated underlying network protocols, it is difficult to find a general authentication solution to support various authentication methods in different access networks. Blockchain is a new technology that supports network heterogeneity, which provides a potential solution for differentiated authentication. In this paper, we propose a blockchain-based differentiated authentication mechanism for 6G Heterogeneous Networks (HetNets), which can efficiently authenticate user identities through scheduling different authentication methods. Particularly, we analyze the authentication architecture of 6G HetNets and put forward a blockchain-based differentiated authentication framework. Besides, to improve the scalability of user authentication, it is the first time to use various blockchain authentication contracts to represent different authentication methods. Meanwhile, a differentiated authentication management contract is proposed to uniformly manage different authentication contracts to realize differentiated identity authentication. Based on the evaluation of the prototype system, the proposed mechanism can dynamically provide differentiated authentication services (e.g. EAP-MD5, 5G-AKA) with low additional time (milliseconds levels) cost.
Explainable artificial intelligence (XAI): How to make image analysis deep learning models transparent

Recently, Deep learning (DL) model has made remarkable achievements in image processing. To increase the accuracy of the DL model, more parameters are used. Therefore, the current DL models are black-box models that cannot understand the internal structure. This is the reason why the DL model cannot be applied to fields where stability and reliability are important despite its high performance. In this paper, We investigated various Explainable artificial intelligence (XAI) techniques to solve this problem. We also investigated what approaches exist to make multi-modal deep learning models transparent.
Adapted ACO Algorithm for Energy-Efficient Path Finding of Waste Collection Robot

Waste collection is a major concern of many companies with large areas of facility, e.g., buildings or factories, where there are many trash bins at various dumping points. Therefore, they require human labor to handle, which is a major cost of consideration. Currently, there are research works using robots for waste collection instead of humans. There is a challenge for waste collection robots in terms of energy consumption to pick up the waste at various dumping points efficiently. The factors related to the energy consumption of waste collection robots are directly related to the distance and waste weight that the robots have to collect and carry from the trash bins at various dump points along the paths. This paper presents the adapted ant colony optimization (ACO) algorithm to find the energy-efficient paths of the waste collection robots. The adapted ACO algorithm uses the waste weight in the trash bin as path heuristic information between two dumping points to determine the state transition probability for finding the most energy-efficient path. The experiment was conducted by the simulation to compare the result with the conventional ACO algorithm that uses distance as the path heuristic information. The simulation results expressed that the adapted ACO algorithm provided the most energy-efficient path under the number of nodes and waste weights specified better than the conventional ACO algorithm.
Performance Comparison of Human Activity Recognition for Unmanned Retails

Lately, the broad usage of technology in almost all aspects of life has led to the increase in research supporting technology advancement. One of these research topics is Human Activity Recognition (HAR) with diverse applicability which include and not limited to video surveillance, healthcare and education. In this paper, we present a study based on human activity recognition while employing the Kinect RGB and Depth sensor camera to recognize seven different human activities (7 classes). The joint angles extracted from the Kinect depth sensor each has 3 axes (X, Y, Z) for the 8 limbs employed in our experiment as the feature vectors. For the purpose of classifying the human activities, we train and test with 3 different state of the art recurrent neural network models (GRU, LSTM, Bi-LSTM). The comparison of the 3 recurrent neural network models shows that LSTM has a higher human activity classification accuracy at 96% and using the confusion matrix as the performance metric for all the models, we show classification per activity.
Preliminary study of AR navigation-guided core-needle breast biopsy

Accurate tumor localization has remained to be one of the biggest challenges in core-needle biopsy due to the malleability and texture-heterogeneity of tissues surrounding a tumor. To overcome these limitations, we explored the effectiveness of augmented reality (AR)-guidance to navigate the core-needle insertion in breast biopsy thanks to the enhanced technology of AR. Our proposed framework consists of two independent functions: tumor localization and needle pose tracking in real-time. First, tumor localization was achieved by employing the two-front environmental sensors of HoloLens 2 (HL2) for tracking the passive-reflective markers. Given the geometrical information of the reference markers, the stereo-correspondences and the marker pose can be calculated efficiently for overlaying the corresponding virtual models in HL2. Second, an external RGB camera was used to track the pose of a needle. A You-Only-Look-Once (YOLO) model with a customized training set of 200 images was employed to detect the collinear pinktape points attached to the needle. The interpoint-distance between these points will be utilized to estimate the pose of a needle that will be sent to HL2 using TCP/IP communication. Finally, real-time navigation of needle insertion can be rendered in HL2 using a more accurate transformation to assist the breast biopsy. Overall, the preliminary results demonstrate that the whole process can be executed in a reasonable running time of less than two seconds.
Blockchain and API Development to Improve Relational Database Integrity and System Interoperability

Technological developments have resulted in a trend of cryptocurrencies that use a technology called blockchain to create and record all transactions made into a digital ledger. Along with the emergence of the trend of cryptocurrencies, this development has also resulted in crimes that hit the digital world, such as data leakage and cyber espionage. This threat can be prevented by applying blockchain technology to the database that has been used. Therefore, we need a system that can facilitate the use of blockchain technology and can process data from an existing relational database into a blockchain-based database. The system developed in this study was built with the FastAPI framework that uses the Python programming language and the React framework that uses the Javascript programming language. This system was tested using Katalon and Wireshark software to perform throughput testing and man-in-the-middle attacks. Evaluation of this system is assessed based on the average throughput time and also the results of Wireshark packet capture. The system designed in this research is expected to help overcome interoperability problems when using blockchain and improve relational database integrity. The results of the test show that the system is safe from man-in-the-middle attacks while sending data through API and has a faster throughput time than BigchainDB system by 4.151 seconds.
Digital forensics on facebook messenger application in an android smartphone based on NIST SP 800-101 R1 to reveal digital crime cases

Android is the world's most popular system of operation. In the Android system, communication applications can be installed, for example, Facebook Messenger, which is one of the most popular communication applications with 988 million active users. As a medium for communication and data exchange, the Facebook Messenger application may be misused by unauthorized parties for cybercrime. This action is usually done to obtain a one-sided profit for unauthorized parties. After doing cybercrime, cybercriminals will delete all of the data to remove digital evidence. Digital forensics serves the challenge of uncovering evidence in investigations. Digital forensics on the Facebook Messenger application is needed to help the authorities organize digital evidence when the disclosure process is performed. The data revealed through the digital forensics process can reinforce existing evidence in a court. In this research, digital forensics on the Facebook Messenger application was conducted with data deletion scenarios and application uninstall scenarios. The results are digital evidence in the form of Facebook Messenger application information, photo, and video sent by the cybercriminal. The conversation and audio evidence could not be located because there was no rooting process, or the smartphone was in an untied state. When the uninstall process was carried out, the smartphone automatically deleted all the data including thumbnails. However, the digital forensics process in an unrooted state on the Facebook Messenger application can still produce digital evidence that can be used in the court.
Opinion Summarization via Submodular Information Measures

This paper focuses on opinion summarization for constructing subjective and concise summaries representing essential opinions of online text reviews. As previous works rarely focus on the relationship between opinions, topics, and sentences, we propose a set of new requirements for Opinion-Topic-Sentence, which are essential for performing opinion summarization. We prove that Opinion-Topic-Sentence can be theoretically analyzed by submodular information measures. Thus, our proposed method can reduce redundant information, strengthen the relevance to given topics, and informatively represent the underlying emotional variations. While conventional methods require human-labeled topics for extractive summarization, we use unsupervised topic modeling methods to generate topic features. We propose four submodular functions and two optimization algorithms with proven performance bounds that can maximize opinion summarization's utility. An automatic evaluation metric, Topic-based Opinion Variance, is also derived to compensate for ROUGE-based metrics of opinion summarization evaluation. Four large, diversified, and representative corpora, OPOSUM, Opinosis, Yelp, and Amazon reviews, are used in our study. The results on these online review texts corroborate the efficacy of our proposed metric and framework.
Clustering constrained on linear networks

An unsupervised classification method for point events occurring on a geometric network is proposed. The idea relies on the distributional flexibility and practicality of random partition models to discover the clustering structure featuring observations from a particular phenomenon taking place on a given set of edges. By incorporating the spatial effect in the random partition distribution, induced by a Dirichlet process, one is able to control the distance between edges and events, thus leading to an appealing clustering method. A Gibbs sampler algorithm is proposed and evaluated with a sensitivity analysis. The proposal is motivated and illustrated by the analysis of crime and violence patterns in Mexico City.
Constrained neuro fuzzy inference methodology for explainable personalised modelling with applications on gene expression data

Interpretable machine learning models for gene expression datasets are important for understanding the decision-making process of a classifier and gaining insights on the underlying molecular processes of genetic conditions. Interpretable models can potentially support early diagnosis before full disease manifestation. This is particularly important yet, challenging for mental health. We hypothesise this is due to extreme heterogeneity issues which may be overcome and explained by personalised modelling techniques. Thus far, most machine learning methods applied to gene expression datasets, including deep neural networks, lack personalised interpretability. This paper proposes a new methodology named personalised constrained neuro fuzzy inference (PCNFI) for learning personalised rules from high dimensional datasets which are structurally and semantically interpretable. Case studies on two mental health related datasets (schizophrenia and bipolar disorders) have shown that the relatively short and simple personalised fuzzy rules provided enhanced interpretability as well as better classification performance compared to other commonly used machine learning methods. Performance test on a cancer dataset also showed that PCNFI matches previous benchmarks. Insights from our approach also indicated the importance of two genes (ATRX and TSPAN2) as possible biomarkers for early differentiation of ultra-high risk, bipolar and healthy individuals. These genes are linked to cognitive ability and impulsive behaviour. Our findings suggest a significant starting point for further research into the biological role of cognitive and impulsivity-related differences. With potential applications across bio-medical research, the proposed PCNFI method is promising for diagnosis, prognosis, and the design of personalised treatment plans for better outcomes in the future.
Diagnostic validation of a portable whole slide imaging scanner for lymphoma diagnosis in resource-constrained setting: A cross-sectional study

Telepathology utilizing high-throughput static whole slide image scanners is proposed to address the challenge of limited pathology services in resource-restricted settings. However, the prohibitive equipment costs and sophisticated technologies coupled with large amounts of space to set up the devices make it impractical for use in resource-limited settings. Herein, we aimed to address this challenge by validating a portable whole slide imaging (WSI) device against glass slide microscopy (GSM) using lymph node biopsies from suspected lymphoma cases from Sub-Saharan Africa.This was part of a multicenter prospective case-control head-to-head comparison study of liquid biopsy against conventional pathology. For the portable WSI scanner validation, the study pathologists evaluated 105 surgical lymph node specimens initially confirmed by gold-standard pathology between February and December 2021. The tissues were processed according to standard protocols for Hematoxylin and Eosin (H&E) and Immunohistochemistry (IHC) staining by well-trained histotechnicians, then digitalized the H& E and IHC slides at each center. The digital images were anonymized and uploaded to a HIPAA-compliant server by the histotechnicians. Three study pathologists independently accessed and reviewed the images after a 6-week washout. The agreement between diagnoses established on GSM and WSI across the pathologists was described and measured using Cohens' kappa coefficient (κ).On GSM, 65.5% (n=84) of specimens were lymphoma; 25% were classified as benign, while 9.5% were metastatic. Morphological quality assessment on GSM and WSI established that 79.8% and 53.6% of cases were of high quality, respectively. When diagnoses by GSM were compared to WSI, the overall concordance for various diagnostic categories was 93%, 100%, and 86% for lymphoma, metastases, and benign conditions respectively. The sensitivity and specificity of WSI for the detection of lymphoma were 95.2% and 85.7%, respectively, with an overall inter-observer agreement (κ) of 0.86; 95% CI (0.70-0.95).We demonstrate that mobile whole slide imaging (WSI) is non-inferior to conventional glass slide microscopy (GSM) for the primary diagnosis of malignant infiltration of lymph node specimens. Our results further provide proof of concept that mobile WSI can be adapted to resource-restricted settings for primary surgical pathology and would significantly improve patient outcomes.
Creating and Evaluating Uncertainty Estimates with Neural Networks for Environmental-Science Applications

Abstract Neural networks (NN) have become an important tool for prediction tasks—both regression and classification—in environmental science. Since many environmental-science problems involve life-or-death decisions and policy making, it is crucial to provide not only predictions but also an estimate of the uncertainty in the predictions. Until recently, very few tools were available to provide uncertainty quantification (UQ) for NN predictions. However, in recent years the computer-science field has developed numerous UQ approaches, and several research groups are exploring how to apply these approaches in environmental science. We provide an accessible introduction to six of these UQ approaches, then focus on tools for the next step, namely, to answer the question: Once we obtain an uncertainty estimate (using any approach), how do we know whether it is good or bad? To answer this question, we highlight four evaluation graphics and eight evaluation scores that are well suited for evaluating and comparing uncertainty estimates (NN based or otherwise) for environmental-science applications. We demonstrate the UQ approaches and UQ-evaluation methods for two real-world problems: 1) estimating vertical profiles of atmospheric dewpoint (a regression task) and 2) predicting convection over Taiwan based on Himawari-8 satellite imagery (a classification task). We also provide Jupyter notebooks with Python code for implementing the UQ approaches and UQ-evaluation methods discussed herein. This article provides the environmental-science community with the knowledge and tools to start incorporating the large number of emerging UQ methods into their research. Significance Statement Neural networks are used for many environmental-science applications, some involving life-or-death decision-making. In recent years new methods have been developed to provide much-needed uncertainty estimates for NN predictions. We seek to accelerate the adoption of these methods in the environmental-science community with an accessible introduction to 1) methods for computing uncertainty estimates in NN predictions and 2) methods for evaluating such estimates.