Reviewerly/exHarmony huggingface dataset:
Object detection in hospital facilities: A comprehensive dataset and performance evaluation

Detecting objects in hospital indoor environments is critical for scene understanding and can have various applications in healthcare. Deep learning algorithms have proven to be effective in object recognition from images or videos, but the availability of annotated datasets plays a crucial role in their successful application. However, there is a shortage of datasets for object detection in hospital settings, hindering the advancement of hospital indoor object detection algorithms. In this paper, we present the Hospital Indoor Object Detection (HIOD) dataset, consisting of 4,417 images covering 56 object categories. The HIOD dataset represents the frequently encountered objects in hospitals and comprises 51,869 annotated objects. The dataset is characterized by dense annotation, with an average of 11.7 objects and 6.8 object categories per image. An object detection benchmark was established using the HIOD dataset and eight state-of-the-art object detectors. The benchmark provides a comprehensive evaluation of the performance of the selected object detectors on a large and diverse set of images of objects commonly seen in hospital environments. The results of the benchmark can be used to compare and analyze the performance of different object detectors and identify their strengths and weaknesses for use in hospital environments. In the benchmark, one-stage detectors have shown superior performance compared to two-stage detectors of similar parameter sizes. In particular, YOLOv6-L was able to attain a mean Average Precision (mAP) of 51.7% while operating at a detection speed of 255 FPS. The benchmark and dataset can serve as a valuable resource for researchers and practitioners in the field of computer vision and robotics, helping to advance the development of more effective and efficient object detection algorithms for developing automated operations in hospitals such as robotic disinfection and patient assistance.
Understanding CNN fragility when learning with imbalanced data

Abstract Convolutional neural networks (CNNs) have achieved impressive results on imbalanced image data, but they still have difficulty generalizing to minority classes and their decisions are difficult to interpret. These problems are related because the method by which CNNs generalize to minority classes, which requires improvement, is wrapped in a black-box. To demystify CNN decisions on imbalanced data, we focus on their latent features. Although CNNs embed the pattern knowledge learned from a training set in model parameters, the effect of this knowledge is contained in feature and classification embeddings ( FE and CE ). These embeddings can be extracted from a trained model and their global, class properties (e.g., frequency, magnitude and identity) can be analyzed. We find that important information regarding the ability of a neural network to generalize to minority classes resides in the class top-K CE and FE . We show that a CNN learns a limited number of class top-K CE per category, and that their magnitudes vary based on whether the same class is balanced or imbalanced. We hypothesize that latent class diversity is as important as the number of class examples, which has important implications for re-sampling and cost-sensitive methods. These methods generally focus on rebalancing model weights, class numbers and margins; instead of diversifying class latent features. We also demonstrate that a CNN has difficulty generalizing to test data if the magnitude of its top-K latent features do not match the training set. We use three popular image datasets and two cost-sensitive algorithms commonly employed in imbalanced learning for our experiments.
CISA: Context Substitution for Image Semantics Augmentation

Large datasets catalyze the rapid expansion of deep learning and computer vision. At the same time, in many domains, there is a lack of training data, which may become an obstacle for the practical application of deep computer vision models. To overcome this problem, it is popular to apply image augmentation. When a dataset contains instance segmentation masks, it is possible to apply instance-level augmentation. It operates by cutting an instance from the original image and pasting to new backgrounds. This article challenges a dataset with the same objects present in various domains. We introduce the Context Substitution for Image Semantics Augmentation framework (CISA), which is focused on choosing good background images. We compare several ways to find backgrounds that match the context of the test set, including Contrastive Language–Image Pre-Training (CLIP) image retrieval and diffusion image generation. We prove that our augmentation method is effective for classification, segmentation, and object detection with different dataset complexity and different model types. The average percentage increase in accuracy across all the tasks on a fruits and vegetables recognition dataset is 4.95%. Moreover, we show that the Fréchet Inception Distance (FID) metrics has a strong correlation with model accuracy, and it can help to choose better backgrounds without model training. The average negative correlation between model accuracy and the FID between the augmented and test datasets is 0.55 in our experiments.
Explainable Causal Analysis of Mental Health on Social Media Data

With recent developments in Social Computing, Natural Language Processing and Clinical Psychology, the social NLP research community addresses the challenge of automation in mental illness on social media. A recent extension to the problem of multi-class classification of mental health issues is to identify the cause behind the user's intention. However, multi-class causal categorization for mental health issues on social media has a major challenge of wrong prediction due to the overlapping problem of causal explanations. There are two possible mitigation techniques to solve this problem: (i) Inconsistency among causal explanations/ inappropriate human-annotated inferences in the dataset, (ii) in-depth analysis of arguments and stances in self-reported text using discourse analysis. In this research work, we hypothesise that if there exists the inconsistency among F1 scores of different classes, there must be inconsistency among corresponding causal explanations as well. In this task, we fine tune the classifiers and find explanations for multi-class causal categorization of mental illness on social media with LIME and Integrated Gradient (IG) methods. We test our methods with CAMS dataset and validate with annotated interpretations. A key contribution of this research work is to find the reason behind inconsistency in accuracy of multi-class causal categorization. The effectiveness of our methods is evident with the results obtained having category-wise average scores of $$81.29 \%$$ and 0.906 using cosine similarity and word mover's distance, respectively.
An Experimental Analysis of Deep Neural Network Based Classifiers for Sentiment Analysis Task

The application of natural language processing (NLP) in sentiment analysis task by using textual data has wide scale application across various domains in plethora of industries.We have methodically studied pre-existing models and proposed new models for examining sentiment analysis task.The models proposed were analysed with three widely popular word embeddings separately and in combined approach using all embeddings as unique channels.We combined deep neural network models such as Bidirectional Long Short-Term Memory (BiLSTM) and Convolutional Neural Network (CNN) so that integrated models complement each other with their unique architectures.The word embeddings used had profound impact in accuracy of models owing to performative changes.The best word embedding was Word2Vec giving highest accuracy in almost all implemented models, followed by GloVe.FastText embedding performed consistently worse, giving much lower accuracy than other embeddings.We also observed that adding transformer encoder layers with CNN improves accuracy by 2% when compared to CNN without any transformer layers.An accuracy improvement of 2-3% over CNN-BiLSTM model was also observed by utilizing transformer encoder layer in conjunction with both BiLSTM and CNN.The proposed model achieved an accuracy of 89.04% on SST-2 dataset.We also compared larger pretrained language model used in sentiment analysis task with our proposed approach.The accuracy values obtained through combination of embeddings and models can be useful for other researchers when selecting word embeddings for their models.
Deep Anomaly Detection on Tennessee Eastman Process Data

Abstract This paper provides the first comprehensive evaluation and analysis of modern (deep‐learning‐based) unsupervised anomaly detection methods for chemical process data. We focus on the Tennessee Eastman process dataset, a standard litmus test to benchmark anomaly detection methods for nearly three decades. Our extensive study will facilitate choosing appropriate anomaly detection methods in industrial applications. From the benchmark, we conclude that reconstruction‐based methods are the methods of choice, followed by generative and forecasting‐based methods.
CVit-Net: A conformer driven RGB-D salient object detector with operation-wise attention learning

Salient object detection (SOD) on Red Green Blue Depth (RGB-D) data is often confronted with ambiguous cross-modality fusion, due to three major challenges: (i) How to select complementarity of RGB and depth modalities, (ii) How to alleviate the negative affect on model performance due to low quality depth maps and (iii) How to effectively fuse multi-level cross-modalities features. To address these issues, we propose a two-way feature interaction approach to exploit the synergies of RGB and depth modalities. Considering the discrepancy of cross-modalities, local and global contextual information is extracted via Conformer encoder. A Local Detail Enhancement Module (LDE) is proposed, which effectively captures the intra-modality features at lower layers of backbone network using a novel operation-wise shuffle channel attention module and supplements the edge guidance network. The quality of depth map is explicitly investigated through bottom-up edge guidance network and affluent saliency cues are exploited via top-down global contextual aware reverse attention mechanism in Global Detail Enhancement Module (GDE). A light-weight decoder is designed to generate final prediction based on saliency maps obtained at four stages. Extensive experiments performed on 6 RGB-D salient object detection datasets demonstrate that our proposed model outperforms 22 state-of-the-art (SOTA) RGB-D SOD models.
A Hunger Games Search algorithm with opposition-based learning for solving multimodal medical image registration

Multimodal medical image registration involves multiple pieces of medical equipment collecting complementary information on the same content, resulting in information fusion. Beyond its theoretical significance, it has practical applications in areas such as disease diagnosis, patient follow-up, surgical navigation, and radiology. Determining the optimal geometric-space transformation parameters is a major challenge in this field. To address this, we propose the opposition-based Hunger Games Search (OHGS) algorithm, an improved version of the Hunger Games Search (HGS) algorithm. The OHGS algorithm aims to find the optimal translation, scaling, and shearing parameters that result in the highest similarity value between the images being registered. By incorporating opposition-based learning (OBL) and dynamically adjusting mutation criteria, the OHGS algorithm has improved exploration capabilities and the ability to escape local optima. In this study, OHGS was compared with 9 other algorithms developed in recent years on 23 well-known benchmark functions and 10 single-objective optimization functions from IEEE CEC2020. Furthermore, to achieve better registration accuracy and optimal alignment of the geometric space in multimodal medical images, OHGS and 8 other algorithms were tested using RIRE and BraTS datasets, optimizing the two similarity measures of Normalized Mutual Information (NMI) and Structural Similarity Index Measure (SSIM). The results showed that OHGS significantly improved performance on benchmark functions and multimodal medical image registration problems, thereby expanding the application range of the HGS algorithm.
Effects of <scp>AR</scp>‐ and <scp>VR</scp>‐based wearables in teaching <scp>English</scp>: The application of an <scp>ARCS</scp> model‐based learning design to improve elementary school students' learning motivation and performance

Abstract Background Previous research has examined the use of augmented reality (AR) and virtual reality (VR) in English language teaching. However, to the best of our knowledge, none of the earlier studies that investigated integrating AR and VR “wearables” into teaching and learning English based on the attention, relevance, confidence, and satisfaction (ARCS) motivation model compared the differences in learning motivation and performance between traditional teaching methods and those using AR‐ and VR‐based using wearables. Objectives To create a thematic learning environment based on learners' lived experiences and provide them with an immersive English course that extends textbook knowledge using a technology‐assisted real‐life learning context. Method This study used a quasi‐experimental research design. A total of 119 participants were spilt into three learning groups based on teaching tools: traditional methods and AR‐ and VR‐based wearable devices. The performance of each group was evaluated through three stages using appropriate educational tools for traditional, AR‐, and VR‐based classrooms, followed by semistructured interviews. Multiple regression analysis was conducted to investigate the inference learning performance (remembering and analysing) of the four ARCS dimensions with the three teaching methods. Results The results showed that confidence and relevance were positively correlated with learning performance in the VR‐based wearable group. In the context of the ARCS model, learners' confidence predicted learning performance and the remembering subdimension when VR‐based learning materials were used. Implications These results suggest that, compared to traditional teaching methods, VR‐based wearables make it easier to create a highly immersive learning environment and are more likely to help learners find the correct words, improve their accuracy, and increase their confidence and sense of accomplishment in the entire learning process.
A Real-Time Memory Updating Strategy for Unsupervised Person Re-Identification

Recently, clustering-based methods have been the dominant solution for unsupervised person re-identification (ReID). Memory-based contrastive learning is widely used for its effectiveness in unsupervised representation learning. However, we find that the inaccurate cluster proxies and the momentum updating strategy do harm to the contrastive learning system. In this paper, we propose a real-time memory updating strategy (RTMem) to update the cluster centroid with a randomly sampled instance feature in the current mini-batch without momentum. Compared to the method that calculates the mean feature vectors as the cluster centroid and updating it with momentum, RTMem enables the features to be up-to-date for each cluster. Based on RTMem, we propose two contrastive losses, i.e., sample-to-instance and sample-to-cluster, to align the relationships between samples to each cluster and to all outliers not belonging to any other clusters. On the one hand, sample-to-instance loss explores the sample relationships of the whole dataset to enhance the capability of density-based clustering algorithm, which relies on similarity measurement for the instance-level images. On the other hand, with pseudo-labels generated by the density-based clustering algorithm, sample-to-cluster loss enforces the sample to be close to its cluster proxy while being far from other proxies. With the simple RTMem contrastive learning strategy, the performance of the corresponding baseline is improved by 9.3% on Market-1501 dataset. Our method consistently outperforms state-of-the-art unsupervised learning person ReID methods on three benchmark datasets. Code is made available at:https://github.com/PRIS-CV/RTMem.
A novel cyber-Resilient solar power forecasting model based on secure federated deep learning and data visualization

Improving the accuracy of photovoltaic (PV) power forecasting is crucial to ensure more effective use of energy resources. Improvements are especially important for regions for which historical solar radiation data do not exist. This paper proposes a cyber-secure forecasting model called federated deep learning (FDL) to forecast PV power generation in various regions across Iran. The training process in each client is done by a convolutional neural network (CNN). Then, a generalizable global supermodel is generated based on the features extracted in each client, which has the ability to generalize and forecast for regions where there is no training data. Preserve data privacy and ideal performance against cyber-attacks are prominent features of the proposed method. The use of the proposed method is illustrated with a case study for Iran. The proposed FDL network is designed with 9 clients and three different scenarios were developed to test the robustness of the suggested method. In the first scenario, the PV power generation forecasting is done via the proposed technique and other conventional methods. The performance accuracy (R2) of the generated global supermodel in this scenario for PV power generation forecasting in the regions of Khomein, Meybod, Varzaneh, Taleghan, and Shiraz are obtained as 0.981, 0.989, 0.986, 0.983, and 0.987 respectively. However, it was observed that other conventional deep learning-based models such as CNN and long short-term memory were not able to provide any forecasting for these regions. The second scenario models the scaling attack as a specific pattern of false data injection attack, to evaluate the performance of forecasting models against the data integrity attack. In the third scenario, cyber-attack detection is performed based on data visualization and image processing procedures. The results presented in different scenarios emphasize the high accuracy and generalizability of the global cyber-secure supermodel in PV power generation forecasting in different regions of Iran.
PbDinEHR: A Novel Privacy by Design Developed Framework Using Distributed Data Storage and Sharing for Secure and Scalable Electronic Health Records Management

Privacy in Electronic Health Records (EHR) has become a significant concern in today’s rapidly changing world, particularly for personal and sensitive user data. The sheer volume and sensitive nature of patient records require healthcare providers to exercise an intense quantity of caution during EHR implementation. In recent years, various healthcare providers have been hit by ransomware and distributed denial of service attacks, halting many emergency services during COVID-19. Personal data breaches are becoming more common day by day, and privacy concerns are often raised when sharing data across a network, mainly due to transparency and security issues. To tackle this problem, various researchers have proposed privacy-preserving solutions for EHR. However, most solutions do not extensively use Privacy by Design (PbD) mechanisms, distributed data storage and sharing when designing their frameworks, which is the emphasis of this study. To design a framework for Privacy by Design in Electronic Health Records (PbDinEHR) that can preserve the privacy of patients during data collection, storage, access and sharing, we have analysed the fundamental principles of privacy by design and privacy design strategies, and the compatibility of our proposed healthcare principles with Privacy Impact Assessment (PIA), Australian Privacy Principles (APPs) and General Data Protection Regulation (GDPR). To demonstrate the proposed framework, ‘PbDinEHR’, we have implemented a Patient Record Management System (PRMS) to create interfaces for patients and healthcare providers. In addition, to provide transparency and security for sharing patients’ medical files with various healthcare providers, we have implemented a distributed file system and two permission blockchain networks using the InterPlanetary File System (IPFS) and Ethereum blockchain. This allows us to expand the proposed privacy by design mechanisms in the future to enable healthcare providers, patients, imaging labs and others to share patient-centric data in a transparent manner. The developed framework has been tested and evaluated to ensure user performance, effectiveness, and security. The complete solution is expected to provide progressive resistance in the face of continuous data breaches in the patient information domain.
Boosting Night-Time Scene Parsing With Learnable Frequency

Night-Time Scene Parsing (NTSP) is essential to many vision applications, especially for autonomous driving. Most of the existing methods are proposed for day-time scene parsing. They rely on modeling pixel intensity-based spatial contextual cues under even illumination. Hence, these methods do not perform well in night-time scenes as such spatial contextual cues are buried in the over-/under-exposed regions in night-time scenes. In this paper, we first conduct an image frequency-based statistical experiment to interpret the day-time and night-time scene discrepancies. We find that image frequency distributions differ significantly between day-time and night-time scenes, and understanding such frequency distributions is critical to NTSP problem. Based on this, we propose to exploit the image frequency distributions for night-time scene parsing. First, we propose a Learnable Frequency Encoder (LFE) to model the relationship between different frequency coefficients to measure all frequency components dynamically. Second, we propose a Spatial Frequency Fusion module (SFF) that fuses both spatial and frequency information to guide the extraction of spatial context features. Extensive experiments show that our method performs favorably against the state-of-the-art methods on the NightCity, NightCity+ and BDD100K-night datasets. In addition, we demonstrate that our method can be applied to existing day-time scene parsing methods and boost their performance on night-time scenes. The code is available at https://github.com/wangsen99/FDLNet.
Foundation Models and Fair Use

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI
Anomaly Detection in Surveillance Videos Using Transformer Based Attention Model

Surveillance footage can catch a wide range of realistic anomalies. This research suggests using a weakly supervised strategy to avoid annotating anomalous segments in training videos, which is time consuming. In this approach only video level labels are used to obtain frame level anomaly scores. Weakly supervised video anomaly detection (WSVAD) suffers from the wrong identification of abnormal and normal instances during the training process. Therefore it is important to extract better quality features from the available videos. With this motivation, the present paper uses better quality transformer-based features named Videoswin Features followed by the attention layer based on dilated convolution and self attention to capture long and short range dependencies in temporal domain. This gives us a better understanding of available videos. The proposed framework is validated on real-world dataset i.e. ShanghaiTech Campus dataset which results in competitive performance than current state-of-the-art methods. The model and the code are available at https://github.com/kapildeshpande/Anomaly-Detection-in-Surveillance-Videos .