Reviewerly/exHarmony huggingface dataset:
Neighborhood evaluator for efficient super-resolution reconstruction of 2D medical images

Authors:
raw_author_name: Zijia Liu
raw_affiliation_string: School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, 800 Dongchuan RD, Shanghai, 200240, China
raw_author_name: Jing Han
raw_affiliation_string: Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine, 639 Zhizaoju RD, Shanghai, 200011, China
raw_author_name: Jiannan Liu
raw_affiliation_string: Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine, 639 Zhizaoju RD, Shanghai, 200011, China
raw_author_name: Zhi-Cheng Li
raw_affiliation_string: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan RD, Shenzhen, 518055, China
raw_author_name: Guangtao Zhai
raw_affiliation_string: School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, 800 Dongchuan RD, Shanghai, 200240, China

Abstract: Deep learning-based super-resolution (SR) algorithms aim to reconstruct low-resolution (LR) images into high-fidelity high-resolution (HR) images by learning the low- and high-frequency information. Experts’ diagnostic requirements are fulfilled in medical application scenarios through the high-quality reconstruction of LR digital medical images. Medical image SR algorithms should satisfy the requirements of arbitrary resolution and high efficiency in applications. However, there is currently no relevant study available. Several SR research on natural images have accomplished the reconstruction of resolutions without limitations. However, these methodologies provide challenges in meeting medical applications due to the large scale of the model, which significantly limits efficiency. Hence, we suggest a highly effective method for reconstructing medical images at any desired resolution. Statistical features of medical images exhibit greater continuity in the region of neighboring pixels than natural images. Hence, the process of reconstructing medical images is comparatively less challenging. Utilizing this property, we develop a neighborhood evaluator to represent the continuity of the neighborhood while controlling the network’s depth. The suggested method has superior performance across seven scales of reconstruction, as evidenced by experiments conducted on panoramic radiographs and two external public datasets. Furthermore, the proposed network significantly decreases the parameter count by over 20× and the computational workload by over 10× compared to prior researches. On large-scale reconstruction, the inference speed can be enhanced by over 5×. The novel proposed SR strategy for medical images performs efficient reconstruction at arbitrary resolution, marking a significant breakthrough in the field. The given scheme facilitates the implementation of SR in mobile medical platforms.
Global Feature-Based Multimodal Semantic Segmentation

Authors:
raw_author_name: Suining Gao
raw_affiliation_string: Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; Key laboratory of Space-based Dynamic Rapid Optical Imaging Technology, Chinese Academy of Sciences, Changchun 130033, China; University of Chinese Academy of Sciences, Beijing 100049, China
raw_author_name: Xiubin Yang
raw_affiliation_string: Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; Key laboratory of Space-based Dynamic Rapid Optical Imaging Technology, Chinese Academy of Sciences, Changchun 130033, China; University of Chinese Academy of Sciences, Beijing 100049, China
raw_author_name: Li Jiang
raw_affiliation_string: School of Physics, Changchun University of Science and Technology, Changchun 130022, China
raw_author_name: Zongqiang Fu
raw_affiliation_string: Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; Key laboratory of Space-based Dynamic Rapid Optical Imaging Technology, Chinese Academy of Sciences, Changchun 130033, China; University of Chinese Academy of Sciences, Beijing 100049, China

Abstract: Incorporating complementary modality into RGB branch can significantly improve the effectiveness of semantic segmentation. However, fusion between the two modalities faces huge challenge due to the difference of their optical dimensions. Existed fusion methods can't keep a balance between performance and efficiency in aggregating detailed features. To address this problem, we propose a global feature-based network (GFBN) for semantic segmentation that establishes mapping function and extraction relationship among the multi-modalities. The GFBN contains three important modules, which are used for feature correction, fusion and edge enhancement. Firstly, the cross-attention rectification module (CARM) adaptively extracts mapping relationships and rectifies the RGB and complementary features. Secondly, the cross-field fusion module (CFM) integrates long-range rectified features of two branches to obtain an optimal fusion feature. Finally, the boundary guidance module (BGM) sharpens the boundary information of the fused features to effectively improve the segmentation accuracy of object boundaries. We make the experiments of GFBN on the challenging MCubeS and ZJU-RGB-Ps datasets. The results show that GFBN outperforms state-of-the-art methods by at least 0.64% and 0.7% on mean intersection over union (mIoU), respectively. It demonstrates the performance and efficiency of our proposed method.
Background no more: Action recognition across domains by causal interventions

Authors:
raw_author_name: Hazel Doughty
raw_affiliation_string: University of Amsterdam, Science Park 900, Amsterdam 1098 XH, The Netherlands
raw_author_name: Cees G.M. Snoek
raw_affiliation_string: University of Amsterdam, Science Park 900, Amsterdam 1098 XH, The Netherlands

Abstract: We aim to recognize actions under an appearance distribution shift between a source training domain and a target test domain. To enable such video domain generalization, our key idea is to intervene on the action to remove the confounding effect of the domain background on the class label using causal inference. Towards this, we propose to learn a causally debiased model on a source domain that intervenes on the action through three possible Do-operators, which separate the action and background. To better align the source and target distributions, we also introduce a test-time action intervention. Experiments on two challenging video domain generalization benchmarks reveal that causal inference is a promising tool for action recognition as it already achieves state-of-the-art results on Kinetics2Mimetics, the benchmark with the largest domain shift.
TIE- text information extraction from natural scene images using SVM

Authors:
raw_author_name: B. Sujatha
raw_affiliation_string: GIET(A), Rajahmundry, Andhra Pradesh, India
raw_author_name: L. Sumalatha
raw_affiliation_string: Dept. of CSE, JNTUK, Kakinada, Andhra Pradesh, India

Abstract: Detection of text and its localisation from natural scenic imagery plays a vital role in in Content Based Imagery Analysis. The major and unavoidable challenges are illumination effects, different orientations in the lines, typical backgrounds, different font styles and sizes. This paper presents a novel methodology Text Information Extraction (TIE) using Support Vector Machine (SVM) which robustly detects and localises the text from natural scene images. Major focus of this paper lies in detection of text object from the whole scene. Initially the image will be pre-processed for noise removal and contrast enhancement. Later, all the objects of scene will be marked and extracted in order to form an object pool. SVM technique is used to locate text object among the object pool. The SVM will be trained with supervised parameter learning. At last the well trained model will perform binary classification to differentiate between text and non-text object from the object pool. Experiments have been performed on ICDAR dataset and achieved results determine that proposed approach leads to utmost Precision and Recall performance when compared to existing state-of-the-art techniques.
Audio-visual saliency prediction for movie viewing in immersive environments: Dataset and benchmarks

Authors:
raw_author_name: Zhao Chen
raw_affiliation_string: School of Remote Sensing and Informantion Engineering, Wuhan University, Wuhan, 430079, China
raw_author_name: Kao Zhang
raw_affiliation_string: School of Artificial Intelligence/School of Future Technolog, Nanjing University of Information Science and Technology, Nanjing, 210044, China; School of Remote Sensing and Informantion Engineering, Wuhan University, Wuhan, 430079, China
raw_author_name: Hao Cai
raw_affiliation_string: School of Remote Sensing and Informantion Engineering, Wuhan University, Wuhan, 430079, China
raw_author_name: Xiaoying Ding
raw_affiliation_string: School of Information and Safety Engineering, Zhongnan University of Economics and Law, Wuhan, 430073, China
raw_author_name: Chenxi Jiang
raw_affiliation_string: School of Remote Sensing and Informantion Engineering, Wuhan University, Wuhan, 430079, China
raw_author_name: Zhenzhong Chen
raw_affiliation_string: School of Remote Sensing and Informantion Engineering, Wuhan University, Wuhan, 430079, China

Abstract: In this paper, an eye-tracking dataset of movie viewing in the immersive environment is developed, which contains 256 movie clips with 2K QHD resolution and corresponding movie genre labels from IMDb (Internet Movie Database). The dataset provides the audio-visual clues for studying the human visual attention when watching movie using a VR headset, by recording the eye movements using integrated eye tracker. To provide benchmarks for a saliency prediction for movie viewing in the immersive environment, fifteen computational models are evaluated on the dataset, including a newly developed multi-stream audio-visual saliency prediction model based on deep neural networks, named as MSAV. Detailed quantitative and qualitative comparisons and analyses are also provided. The developed dataset and benchmarks could help to facilitate the studies of visual saliency prediction for movie viewing in the immersive environments.
Stochastic reconstruction of digital cores using two-discriminator VAE-GAN

Authors:
raw_author_name: Ting Zhang
raw_affiliation_string: College of Computer Science and Technology, Shanghai University of Electric Power, Shanghai, 200090, China
raw_author_name: Tong Shen
raw_affiliation_string: College of Computer Science and Technology, Shanghai University of Electric Power, Shanghai, 200090, China
raw_author_name: Fangfang Lu
raw_affiliation_string: College of Computer Science and Technology, Shanghai University of Electric Power, Shanghai, 200090, China
raw_author_name: Xin Du
raw_affiliation_string: Department of Modern Mechanics, University of Science and Technology of China, Hefei, 230026, China

Abstract: The internal structure of rocks has a great influence on the refined reservoir exploitation. The digital core has been extensively studied to reflect the underlying characteristics of rock structures. Physical experimental methods and numerical reconstruction methods are two mainstreams in the reconstruction of digital cores. The former requires expensive equipment and is probably very time-consuming because of the complex preparation process. In contrast, numerical reconstruction methods are less expensive but have problems such as slow processes and potentially poor results. Nowadays, deep learning based on extracting features from training images has opened a new door for reconstructing digital cores. Variational autoencoder (VAE) and generative adversarial network (GAN) are two well-known deep learning models. The training process of VAE is stable but the generated image is easily blurred while GAN can generate clear images but have an unstable process. Although VAE-GAN combines VAE and GAN to hopefully completely address the above issue, it cannot solve the error problem from variational inference originated from VAE. Therefore, we add a code discriminator in VAE-GAN to solve this problem, replacing variational inference by adversarial learning. By comparing with a traditional method and classical deep generative models, the effectiveness and practicality of the new model in reconstructing digital cores is verified.
Trustworthy Artificial Intelligence -based federated architecture for symptomatic disease detection

Authors:
raw_author_name: Ricardo S. Alonso
raw_affiliation_string: AIR Institute - Deep tech lab IoT Digital Innovation Hub, Paseo de Belén 9A, Valladolid, 47011, Castille and Leon, Spain; UNIR, International University of La Rioja, Av. de la Paz, 137, Logroño, 26006, La Rioja, Spain
raw_author_name: Sara Rodríguez-González
raw_affiliation_string: BISITE Research Group, University of Salamanca, Edificio I+D+i, Calle del Espejo 2, Salamanca, 37006, Castille and Leon, Spain
raw_author_name: Javier Prieto
raw_affiliation_string: BISITE Research Group, University of Salamanca, Edificio I+D+i, Calle del Espejo 2, Salamanca, 37006, Castille and Leon, Spain
raw_author_name: Juan M. Corchado
raw_affiliation_string: AIR Institute - Deep tech lab IoT Digital Innovation Hub, Paseo de Belén 9A, Valladolid, 47011, Castille and Leon, Spain; BISITE Research Group, University of Salamanca, Edificio I+D+i, Calle del Espejo 2, Salamanca, 37006, Castille and Leon, Spain

Abstract: The recent viral outbreaks have had a significant impact on interpersonal relationships, particularly in enclosed spaces. Detecting and preventing the transmission of diseases such as COVID-19 has become a top priority. These diseases are typically identifiable through the symptoms they cause in humans. However, the collection of personal and health data for use in Artificial Intelligence models can give rise to ethical, security, and privacy issues. Therefore, it is necessary to have architectures that maintain the principles of Trustworthy Artificial Intelligence by design. This work proposes a decentralized architecture based on Federated Learning for symptomatic disease detection using the edge computing paradigm, storing the information in the device that collected it, and the foundations of Trustworthy Artificial Intelligence. The architecture is designed to be robust, secure, transparent, and responsible while maintaining data privacy. The proposed approach can be used with medical information capture systems with different user profiles.
Multimodal Risk Prediction with Physiological Signals, Medical Images and Clinical Notes

Authors:
raw_author_name: Yuanlong Wang
raw_affiliation_string: Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210, USA
raw_author_name: Changchang Yin
raw_affiliation_string: Department of Biomedical Informatics, The Ohio State University, Columbus, OH 43210, USA; Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210, USA
raw_author_name: Ping Zhang
raw_affiliation_string: Department of Biomedical Informatics, The Ohio State University, Columbus, OH 43210, USA; Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210, USA

Abstract: The broad adoption of electronic health record (EHR) systems brings us a tremendous amount of clinical data and thus provides opportunities to conduct data-based healthcare research to solve various clinical problems in the medical domain. Machine learning and deep learning methods are widely used in the medical informatics and healthcare domain due to their power to mine insights from raw data. When adapting deep learning models for EHR data, it is essential to consider its heterogeneous nature: EHR contains patient records from various sources including medical tests (e.g. blood test, microbiology test), medical imaging, diagnosis, medications, procedures, clinical notes, etc. Those modalities together provide a holistic view of patient health status and complement each other. Therefore, combining data from multiple modalities that are intrinsically different is challenging but intuitively promising in deep learning for EHR. To assess the expectations of multimodal data, we introduce a comprehensive fusion framework designed to integrate temporal variables, medical images, and clinical notes in EHR for enhanced performance in clinical risk prediction. Early, joint, and late fusion strategies are employed to combine data from various modalities effectively. We test the model with three predictive tasks: in-hospital mortality, long length of stay, and 30-day readmission. Experimental results show that multimodal models outperform uni-modal models in the tasks involved. Additionally, by training models with different input modality combinations, we calculate the Shapley value for each modality to quantify their contribution to multimodal performance. It is shown that temporal variables tend to be more helpful than CXR images and clinical notes in the three explored predictive tasks.
Dataset of Sentiment Tagged Language Resources for Bosnian Language

Authors:
raw_author_name: Sead Jahić
raw_affiliation_string: University of Primorska, FAMNIT, Glagoljaska 8, 6000 Koper, Slovenia
raw_author_name: Jernej Vičič
raw_affiliation_string: Research Centre of the Slovenian Academy of Sciences and Arts, The Fran Ramovž Institute, Novi trg 2, 1000, Ljubljana, Slovenia; University of Primorska, FAMNIT, Glagoljaska 8, 6000 Koper, Slovenia

Abstract: The Bosnian language holds significant importance as a member of the West-South Slavic subgroup within the Slavic branch of the Indo-European linguistic family. With approximately 2.5 million speakers in Europe, including 1.87 million individuals in Bosnia and Herzegovina alone, the Bosnian language constitutes the mother tongue for a considerable portion of the population. In Natural Language Processing (NLP) tasks related to the Bosnian language, besides removing stop words, it is important to consider the influence of other linguistic elements. Bosnian text contains words derived from diminishers, relative intensifiers, minimizers, maximizers, boosters, and approximators. These words contribute to the overall meaning and sentiment analysis of the text. By including these elements in NLP models and algorithms, researchers can achieve more accurate and nuanced analysis of Bosnian language data, enhancing the effectiveness of NLP applications. The two lists of sentiment annotated words that present the core of the Bosnian sentiment-annotated lexicon, a list of the stopwords, and a list of Affirmative and non-Affrimative words (AnAwords) composed mostly of intensifiers and diminishers, were used to construct a dataset that presents the base for sentiment analysis in the Bosnian language.
LPMSAEF: Lightweight Process Mining-Based Software Architecture Evaluation Framework for Security and Performance Analysis

Authors:
raw_author_name: Mahdi Sahlabadi
raw_affiliation_string: Department of Information Security Engineering, Soonchunhyang University, Chungnam, Asan-si, 31538, South Korea
raw_author_name: Ravie Chandren Muniyandi
raw_affiliation_string: Center For Cyber Security, Universiti Kebangsaan Malaysia, Bangi, Selangor, 43600, Malaysia
raw_author_name: Zarina Shukur
raw_affiliation_string: Center For Cyber Security, Universiti Kebangsaan Malaysia, Bangi, Selangor, 43600, Malaysia
raw_author_name: Morteza SaberiKamarposhti
raw_affiliation_string: Center For Cyber Security, Universiti Kebangsaan Malaysia, Bangi, Selangor, 43600, Malaysia
raw_author_name: Kangbin Yim
raw_affiliation_string: Department of Information Security Engineering, Soonchunhyang University, Chungnam, Asan-si, 31538, South Korea

Abstract: The article discusses the need for a lightweight software architecture evaluation framework that can address practitioners' concerns. Specifically, the proposed framework uses process mining and Petri nets to analyze security and performance in software development's early and late stages. Moreover, the framework has been implemented in six case studies, and the results show that it is a feasible and effective solution that can detect security and performance issues in complex and heterogeneous architecture with less time and effort. Furthermore, the article provides a detailed explanation of the framework's features, factors, and evaluation criteria. Additionally, this article discusses the challenges associated with traditional software architecture documentation methods using Unified Modeling Language diagrams and the limitations of code alone for creating comprehensive Software Architecture models. Various methods have been developed to extract implicit Software Architecture from code artifacts, but they tend to produce code-oriented diagrams instead of Software Architecture diagrams. Therefore, to bridge the model-code gap, the article proposes a framework that considers existing Software Architecture in the source code as architectural components and focuses on Software Architecture behaviors for analyzing performance and security. The proposed framework also suggests comparing Software Architecture extracted by different Process Mining algorithms to achieve consensus on architecture descriptions, using visualizations to understand differences and similarities. Finally, the article suggests that analyzing the previous version of a system's Software Architecture can lead to improvements and deviations from planned Software Architecture can be detected using traceability approaches to aid software architects in detecting inconsistencies.