Reviewerly/exHarmony huggingface dataset:
Reversible data hiding in encrypted images with block-based adaptive MSB encoding

Contemporary cloud storage and social media networks have matured and; as a result, the privacy protection of image content in these applications have attracted more attention. Reversible data hiding in encrypted images (RDHEI) is an efficient technique to embed additional data such as timestamps, watermarks, and copyright information into encrypted images to protect the image content. This paper proposes a reversible data hiding scheme in encrypted images based on a block-based adaptive MSB encoding technique (BAME-RDHEI). The content owner first uses a block-based image encryption method, including block permutation, pixel permutation in each block, and the block-based bitwise exclusive-or operation, to encrypt an image. This specific image encryption method preserves the relevance of the MSB bit planes in each block, and the block-based adaptive MSB encoding technique can be applied to classify and encode eight block types according to the number of MSB bit planes where all the values are all ‘1’ or ‘0’. After embedding the indicators that are generated by Huffman coding into the first MSB bit plane of each block, the remaining values of these MSB bit planes can be vacated for data embedding. The receiver can extract additional data, decrypted the image or recover the original image according to the different keys. The experimental results demonstrate that our proposed scheme provides enhanced security, significantly improves the embedding rate, and obtains higher visual quality for the decrypted image compared to other state-of-the-art schemes.
Deep learning-based tumor microenvironment analysis in colon adenocarcinoma histopathological whole-slide images

: Colon cancer is a fatal disease, and a comprehensive understanding of the tumor microenvironment (TME) could lead to better risk stratification, prognosis prediction, and therapy management. In this paper, we focused on the automatic evaluation of TME in giga-pixel digital histopathology whole-slide images. : A convolutional neural network is used to recognize nine different content presented in colon cancer whole-slide images. Several implementation details, including the foreground filtering and stain normalization are discussed. Based on the whole-slide segmentation, several TME descriptors are quantified and correlated with the clinical outcome by Kaplan-Meier analysis and Cox regression. Specifically, the stroma, tumor, necrosis, and lymphocyte components are discussed. : We validated the method on colon adenocarcinoma cases from The Cancer Genome Atlas project. The result shows that the stroma is an independent predictor of progression-free interval (PFI) after corrected by age and pathological stage, with a hazard ratio of 1.665 (95%CI: 1.110~2.495, p = 0.014). High-level necrosis component and lymphocytes component tend to be correlated with poor PFI, with a hazard ratio of 1.552 (95%CI: 0.943~2.554, p = 0.084) and 1.512 (95%CI: 0.979~2.336, p = 0.062), respectively. : The result reveals the complex role of the tumor microenvironment in colon adenocarcinoma, and the quantified descriptors are potential predictors of disease progression. The method could be considered for risk stratification and targeted therapy and extend to other types of cancer, leading to a better understanding of the tumor microenvironment.
(Generalized Linear) Mixed‐Effects Modeling: A Learner Corpus Example

Abstract This methods showcase article provides a detailed overview of a mixed‐effects modeling analysis of corpus data on the use of that in object and subject complementation by native speakers of English compared to its use by German and Spanish learners of English. “We emphasize that we do not claim that our illustrations are the only way to carry out these analyses, but the strategy outlined above has yielded satisfactory results.” (Bates et al., 2018, p. 5)
A stacking methodology of machine learning for 3D geological modeling with geological-geophysical datasets, Laochang Sn camp, Gejiu (China)

A stacking method was followed for the classification and prediction of rock types and for 3D geological modeling in the Laochang Sn camp, Geijiu (China). The sequence of the methodology is as follows. Initially, multi-source datasets were compiled GOCAD software, including a borehole dataset with rock types and 1:50,000 scale gravity (residual density) and magnetic susceptibility inversion interpretation using 3D-grid model. Then, the datasets were subjected to first-level classifiers using support vector machine, gradient boosting decision tree, neural network, random forest, and XGBoost. This was followed by subjecting the datasets to second-level classifier using XGBoost. Next, features (XYZ coordinates, residual density and magnetic susceptibility) were input for training and testing (70% of data for training, 30% for testing). Then, upon completion of training the model, the machine learning metrics were used to evaluate comprehensively the performance of the model, and the 3D geological model was visualized. Finally, the 3D geological modeling was validated by comparison with a robust geological dataset with mining channel dataset. The research results show the stacking method of machine learning can be used to construct effectively and quickly a 3D multi-type rocks model using geological and geophysical datasets. The obtained areas under the ROC curves were 0.93 for the granites, 0.98 for the basalts, and 0.94 for "other" rocks. The methodology can be extended to multi-source datasets of geoscience big data and even to different metallogenic models or geological settings.
B-Learning and Technology: Enablers for University Education Resilience. An Experience Case under COVID-19 in Spain

Innovative teaching methodologies begat blended learning, which seems to facilitate engineering students’ achievement of competencies required for the 21st century and has proven to be essential to keep quality standards as university education has suffered the COVID-19 pandemic. In this paper, we approach the use of b-learning and digital technologies before the pandemic started, and how it worked like a vaccine, enabling university education resilience and facilitating the sustainability of the students learning process. A questionnaire-based study is presented, in which the data came from Aerospace Engineering students following a Mathematics subject in a Technological University in Spain. ANOVA and ANCOVA analysis provided a significant difference in the appreciation of the adaptation based on the prior application of b-learning methodologies or more traditional methodologies. Results obtained indicated that the use of digital resources and educational platforms caused a noticeable change in the students’ way of learning, improving habits and digital skills.
Bridging Text and Video: A Universal Multimodal Transformer for Audio-Visual Scene-Aware Dialog

Audio-Visual Scene-Aware Dialog (AVSD) is a task to generate responses when chatting about a given video, which is organized as a track of the 8th Dialog System Technology Challenge (DSTC8). There are two challenges in this task: 1) making effective interaction among different modalities; 2) better understanding dialogues and generating informative responses. To tackle the challenges, we propose a universal multimodal transformer and introduce the multi-task learning method to learn joint representations among different modalities as well as generate informative and fluent responses by leveraging the pre-trained language model. Our method extends the natural language generation pre-trained model to multimodal dialogue generation task, which allows fine-tuning language models to capture information across both visual and textual modalities. Our system achieves the best performance in the objective evaluation in both DSTC7-AVSD and DSTC8-AVSD dataset and achieves an impressive 98.4% of the human performance based on human ratings in the DSTC8-AVSD challenge.
An Improved Gray Wolf Optimization Algorithm to Solve Engineering Problems

With the rapid development of the economy, the disparity between supply and demand of resources is becoming increasingly prominent in engineering design. In this paper, an improved gray wolf optimization algorithm is proposed (IGWO) to optimize engineering design problems. First, a tent map is used to generate the initial location of the gray wolf population, which evenly distributes the gray wolf population and lays the foundation for a diversified global search process. Second, Gaussian mutation perturbation is used to perform various operations on the current optimal solution to avoid the algorithm falling into local optima. Finally, a cosine control factor is introduced to balance the global and local exploration capabilities of the algorithm and to improve the convergence speed. The IGWO algorithm is applied to four engineering optimization problems with different typical complexity, including a pressure vessel design, a tension spring design, a welding beam design and a three-truss design. The experimental results show that the IGWO algorithm is superior to other comparison algorithms in terms of optimal performance, solution stability, applicability and effectiveness; and can better solve the problem of resource waste in engineering design. The IGWO also optimizes 23 different types of function problems and uses Wilcoxon rank-sum test and Friedman test to verify the 23 test problems. The results show that the IGWO algorithm has higher convergence speed, convergence precision and robustness compared with other algorithms.
Artificial Intelligence in Pathology

BACKGROUND Increasing digitalization enables the use of intelligence (AI) and machine learning in pathology. However, these technologies have only just begun to be implemented, and no randomized prospective trials have yet shown a benefit of AI-based diagnosis. In this review, we present current concepts, illustrate them with examples from representative publications, and discuss the possibilities and limitations of their use. METHODS This article is based on the results of a search in PubMed for articles published between January 1950 and January 2020 containing the searching terms artificial intelligence, deep learning, and digital pathology, as well as the authors' own research findings. RESULTS Current research on AI in pathology focuses on supporting routine diagnosis and on prognostication, particularly for patients with cancer. Initial data indicate that pathologists can arrive at a diagnosis faster and more accurately with the aid of a computer. In a pilot study on the diagnosis of breast cancer, involving 70 patients, sensitivity for the detection of micrometastases rose from 83.3% (by a pathologist alone) to 91.2% (by a pathologist combined with a computer algorithm). The evidence likewise suggests that AI applied to histomorphological properties of cells during microscopy may enable the inference of certain genetic properties, such as mutations in key genes and deoxyribonucleic acid (DNA) methylation profiles. CONCLUSION Initial proof-of-concept studies for AI in pathology are now available. Randomized, prospective studies are now needed so that these early findings can be confirmed or falsified.
Prediction of Solar Irradiance and Photovoltaic Solar Energy Product Based on Cloud Coverage Estimation Using Machine Learning Methods

Cloud cover estimation from images taken by sky-facing cameras can be an important input for analyzing current weather conditions and estimating photovoltaic power generation. The constant change in position, shape, and density of clouds, however, makes the development of a robust computational method for cloud cover estimation challenging. Accurately determining the edge of clouds and hence the separation between clouds and clear sky is difficult and often impossible. Toward determining cloud cover for estimating photovoltaic output, we propose using machine learning methods for cloud segmentation. We compare several methods including a classical regression model, deep learning methods, and boosting methods that combine results from the other machine learning models. To train each of the machine learning models with various sky conditions, we supplemented the existing Singapore whole sky imaging segmentation database with hazy and overcast images collected by a camera-equipped Waggle sensor node. We found that the U-Net architecture, one of the deep neural networks we utilized, segmented cloud pixels most accurately. However, the accuracy of segmenting cloud pixels did not guarantee high accuracy of estimating solar irradiance. We confirmed that the cloud cover ratio is directly related to solar irradiance. Additionally, we confirmed that solar irradiance and solar power output are closely related; hence, by predicting solar irradiance, we can estimate solar power output. This study demonstrates that sky-facing cameras with machine learning methods can be used to estimate solar power output. This ground-based approach provides an inexpensive way to understand solar irradiance and estimate production from photovoltaic solar facilities.
Hierarchical Feature Fusion With Mixed Convolution Attention for Single Image Dehazing

Single image dehazing, which aims at restoring a haze-free image from its correspondingly unconstrained hazy scene, is a fundamental yet challenging task and has gained immense popularity recently. However, the images recovered by some existing haze-removal methods often contain haze, artifacts, and color distortions, which severely degrade the visual quality and have negative impacts on subsequent computer vision tasks. To this end, we propose a network combining multi-scale hierarchical feature fusion and mixed convolution attention to progressively and adaptively enhance the dehazing performance. The haze levels and image structure information are accurately estimated by fusing multi-scale hierarchical features, thus the model restores images with less remaining haze. The proposed mixed convolution attention mechanism is capable of reducing feature redundancy, learning compact and effective internal representations and highlighting task-relevant features, thus, it can further help the model estimate images with sharper textural details and more vivid colors. Furthermore, a deep semantic loss is also proposed to highlight essential semantic information in deep features. The experimental results show that the proposed method outperforms state-of-the-art haze removal algorithms.
Prototyping and Validating a Non-immersive Virtual Reality Serious Game for Healthcare Fire Safety Training

In a healthcare context, the success of a fire safety procedure in a real-life emergency mainly depends on staff decisions and actions. One of the factors influencing staff decision-making is their training. In most healthcare facilities, safety educators use slide-based lectures as a training tool. Virtual Reality (VR) is gaining fire safety community attention for being an interesting training tool. However, few studies have assessed the effectiveness of VR-based fire safety training simulators compared with a slide-based lecture. The present research proposes a novel non-immersive VR-based training for healthcare fire safety education. This paper describes the prototyping steps required to develop a non-immersive VR serious game (SG) to train the staff of Vincent Van Gogh (VVG) hospital in Belgium. The paper finally validates the VR SG comparing its effectiveness against slide-based lecture training. 78 staff from VVG hospital in Belgium participated in this study. They were divided into two groups: Group A was trained using a slide-based lecture, and Group B was trained using the VR SG. The results indicated that the VR SG was more effective than the slide-based lecture in terms of knowledge acquisition and retention and in terms of self-efficacy increment in short and long terms than the slide-based lecture.
Deep Learning Based Human Activity Recognition Using Spatio-Temporal Image Formation of Skeleton Joints

Human activity recognition has become a significant research trend in the fields of computer vision, image processing, and human–machine or human–object interaction due to cost-effectiveness, time management, rehabilitation, and the pandemic of diseases. Over the past years, several methods published for human action recognition using RGB (red, green, and blue), depth, and skeleton datasets. Most of the methods introduced for action classification using skeleton datasets are constrained in some perspectives including features representation, complexity, and performance. However, there is still a challenging problem of providing an effective and efficient method for human action discrimination using a 3D skeleton dataset. There is a lot of room to map the 3D skeleton joint coordinates into spatio-temporal formats to reduce the complexity of the system, to provide a more accurate system to recognize human behaviors, and to improve the overall performance. In this paper, we suggest a spatio-temporal image formation (STIF) technique of 3D skeleton joints by capturing spatial information and temporal changes for action discrimination. We conduct transfer learning (pretrained models- MobileNetV2, DenseNet121, and ResNet18 trained with ImageNet dataset) to extract discriminative features and evaluate the proposed method with several fusion techniques. We mainly investigate the effect of three fusion methods such as element-wise average, multiplication, and maximization on the performance variation to human action recognition. Our deep learning-based method outperforms prior works using UTD-MHAD (University of Texas at Dallas multi-modal human action dataset) and MSR-Action3D (Microsoft action 3D), publicly available benchmark 3D skeleton datasets with STIF representation. We attain accuracies of approximately 98.93%, 99.65%, and 98.80% for UTD-MHAD and 96.00%, 98.75%, and 97.08% for MSR-Action3D skeleton datasets using MobileNetV2, DenseNet121, and ResNet18, respectively.
Internet-of-Things-Enabled Data Fusion Method for Sleep Healthcare Applications

The Internet of Medical Things (IoMT) aims to exploit the Internet-of-Things (IoT) techniques to provide better medical treatment scheme for patients with smart, automatic, timely, and emotion-aware clinical services. One of the IoMT instances is applying IoT techniques to sleep-aware smartphones or wearable devices' applications to provide better sleep healthcare services. As we all know, sleep is vital to our daily health. What is more, studies have shown a strong relationship between sleep difficulties and various diseases such as COVID-19. Therefore, leveraging IoT techniques to develop a longer lifetime sleep healthcare IoMT system, with a tradeoff between data transferring/processing speed and battery energy efficiency, to provide longer time services for bad sleep condition persons, especially the COVID-19 patients or survivors, is a meaningful research topic. In this study, we propose an IoT-enabled sleep data fusion networks (SDFN) module with a star topology Bluetooth network to fuse data of sleep-aware applications. A machine learning model is built to detect sleep events through an audio signal. We design two data reprocessing mechanisms running on our IoT devices to alleviate the data jam problem and save the IoT devices' battery energy. The experiments manifest that the presented module and mechanisms can save the energy of the system and alleviate the data jam problem of the device.
Towards Coding for Human and Machine Vision: Scalable Face Image Coding

The past decades have witnessed the rapid development of image and video coding techniques in the era of big data. However, the signal fidelity-driven coding pipeline design limits the capability of the existing image/video coding frameworks to fulfill the needs of both machine and human vision. In this paper, we come up with a novel face image coding framework by leveraging both the compressive and the generative models, to support machine vision and human perception tasks jointly. Given an input image, the feature analysis is first applied, and then the generative model is employed to reconstruct image with compact structure and color features, where sparse edges are extracted to connect both kinds of vision and a key reference pixel selection method is proposed to determine the priorities of the reference color pixels for scalable coding. The compact edge map serves as the basic layer for machine vision tasks, and the reference pixels act as an enhanced layer to guarantee signal fidelity for human vision. By introducing advanced generative models, we train a decoding network to reconstruct images from compact structure and color representations, which is flexible to accept inputs in a scalable way and to control the imagery effect of the outputs between signal fidelity and visual realism. Experimental results and comprehensive performance analysis over the face image dataset demonstrate the superiority of our framework in both human vision tasks and machine vision tasks, which provide useful evidence on the emerging standardization efforts on MPEG VCM (Video Coding for Machine).
Deep Shallow Fusion for RNN-T Personalization

End-to-end models in general, and Recurrent Neural Network Transducer (RNN-T) in particular, have gained significant traction in the automatic speech recognition community in the last few years due to their simplicity, compactness, and excellent performance on generic transcription tasks. However, these models are more challenging to personalize compared to traditional hybrid systems due to the lack of external language models and difficulties in recognizing rare long-tail words, specifically entity names. In this work, we present novel techniques to improve RNN-T's ability to model rare WordPieces, infuse extra information into the encoder, enable the use of alternative graphemic pronunciations, and perform deep fusion with personalized language models for more robust biasing. We show that these combined techniques result in 15.4%-34.5% relative Word Error Rate improvement compared to a strong RNN-T baseline which uses shallow fusion and text-to-speech augmentation. Our work helps push the boundary of RNN-T personalization and close the gap with hybrid systems on use cases where biasing and entity recognition are crucial.
Privacy-Preserving Implicit Authentication Protocol Using Cosine Similarity for Internet of Things

Internet of Things provides complicated value-added services to mobile intelligent terminal users. Different sensors collect various data from the users and transmit the data to the mobile intelligent terminal for storage. Consequently, a great amount of personal and sensitive information related to these rich and colorful applications is stored in the mobile intelligent terminal. Mobile intelligent terminals have become the prominent target of network attackers. Security breach and privacy leakage severely thread the application development of the Internet of Things. We present a privacy-preserving implicit authentication framework using users' behavior features sensed by the mobile intelligent terminal based on the artificial intelligence methodology. More precisely, we first summarize the security and privacy requirements for the security authentication of the mobile intelligent terminal. Then, we present a privacy-preserving implicit authentication framework using the cosine similarity and partial homomorphic public-key encryption scheme. Finally, a performance evaluation of the proposed protocol is conducted. The result shows that the communication and computation efficiency of our protocol is more efficient than other related protocols.