Reviewerly/exHarmony huggingface dataset:
YogNet: A two-stream network for realtime multiperson yoga action recognition and posture correction

Yoga is a traditional Indian exercise. It specifies various body postures called asanas, practicing them is beneficial for the physical, mental, and spiritual well-being. To support the yoga practitioners, there is a need of an expert yoga asanas recognition system that can automatically analyze practitioner’s postures and could provide suitable posture correction instructions. This paper proposes YogNet, a multi-person yoga expert system for 20 asanas using a two-stream deep spatiotemporal neural network architecture. The first stream utilizes a keypoint detection approach to detect the practitioner’s pose, followed by the formation of bounding boxes across the subject. The model then applies time distributed convolutional neural networks (CNNs) to extract frame-wise postural features, followed by regularized long short-term memory (LSTM) networks to give temporal predictions. The second stream utilizes 3D-CNNs for spatiotemporal feature extraction from RGB videos. Finally, the scores of two streams are fused using multiple fusion techniques. A yoga asana recognition database (YAR) containing 1206 videos is collected using a single 2D web camera for 367 min with the help of 16 participants and contains four view variations i.e. front, back, left, and right sides. The proposed system is novel as this is the earliest two-stream deep learning-based system that can perform multi-person yoga asanas recognition and correction in realtime. Simulation result reveals that YogNet system achieved 77.29%, 89.29%, and 96.31% accuracies using pose stream, RGB stream, and via fusion of both streams, respectively. These results are impressive and sufficiently high for recommendation towards general adaption of the system.
Sustainable Smart Industry: A Secure and Energy Efficient Consensus Mechanism for Artificial Intelligence Enabled Industrial Internet of Things

In recent years, the Internet of Things (IoT) has been industrializing in various real-world applications, including smart industry and smart grids, to make human existence more reliable. An overwhelming volume of sensing data is produced from numerous sensor devices as the Industrial IoT (IIoT) becomes more industrialized. Artificial Intelligence (AI) plays a vital part in big data analyses as a powerful analytic tool that provides flexible and reliable information insights in real-time. However, there are some difficulties in designing and developing a useful big data analysis tool using machine learning, such as a centralized approach, security, privacy, resource limitations, and a lack of sufficient training data. On the other hand, Blockchain promotes a decentralized architecture for IIoT applications. It encourages the secure data exchange and resources among the various nodes of the IoT network, removing centralized control and overcoming the industry’s current challenges. Our proposed approach goal is to design and implement a consensus mechanism that incorporates Blockchain and AI to allow successful big data analysis. This work presents an improved Delegated Proof of Stake (DPoS) algorithm-based IIoT network that combines Blockchain and AI for real-time data transmission. To accelerate IIoT block generation, nodes use an improved DPoS to reach a consensus for selecting delegates and store block information in the trading node. The proposed approach is evaluated regarding energy consumption and transaction efficiency compared with the exciting consensus mechanism. The evaluation results reveal that the proposed consensus algorithm reduces energy consumption and addresses current security issues.
Deep learning to diagnose Hashimoto’s thyroiditis from sonographic images

Hashimoto's thyroiditis (HT) is the main cause of hypothyroidism. We develop a deep learning model called HTNet for diagnosis of HT by training on 106,513 thyroid ultrasound images from 17,934 patients and test its performance on 5051 patients from 2 datasets of static images and 1 dataset of video data. HTNet achieves an area under the receiver operating curve (AUC) of 0.905 (95% CI: 0.894 to 0.915), 0.888 (0.836-0.939) and 0.895 (0.862-0.927). HTNet exceeds radiologists' performance on accuracy (83.2% versus 79.8%; binomial test, p < 0.001) and sensitivity (82.6% versus 68.1%; p < 0.001). By integrating serologic markers with imaging data, the performance of HTNet was significantly and marginally improved on the video (AUC, 0.949 versus 0.888; DeLong's test, p = 0.004) and static-image (AUC, 0.914 versus 0.901; p = 0.08) testing sets, respectively. HTNet may be helpful as a tool for the management of HT.
C2L: Causally Contrastive Learning for Robust Text Classification

Despite the super-human accuracy of recent deep models in NLP tasks, their robustness is reportedly limited due to their reliance on spurious patterns. We thus aim to leverage contrastive learning and counterfactual augmentation for robustness. For augmentation, existing work either requires humans to add counterfactuals to the dataset or machines to automatically matches near-counterfactuals already in the dataset. Unlike existing augmentation is affected by spurious correlations, ours, by synthesizing “a set” of counterfactuals, and making a collective decision on the distribution of predictions on this set, can robustly supervise the causality of each term. Our empirical results show that our approach, by collective decisions, is less sensitive to task model bias of attribution-based synthesis, and thus achieves significant improvements, in diverse dimensions: 1) counterfactual robustness, 2) cross-domain generalization, and 3) generalization from scarce data.
Evaluating Explainable AI on a Multi-Modal Medical Imaging Task: Can Existing Algorithms Fulfill Clinical Requirements?

Being able to explain the prediction to clinical end-users is a necessity to leverage the power of artificial intelligence (AI) models for clinical decision support. For medical images, a feature attribution map, or heatmap, is the most common form of explanation that highlights important features for AI models' prediction. However, it is unknown how well heatmaps perform on explaining decisions on multi-modal medical images, where each image modality or channel visualizes distinct clinical information of the same underlying biomedical phenomenon. Understanding such modality-dependent features is essential for clinical users' interpretation of AI decisions. To tackle this clinically important but technically ignored problem, we propose the modality-specific feature importance (MSFI) metric. It encodes clinical image and explanation interpretation patterns of modality prioritization and modality-specific feature localization. We conduct a clinical requirement-grounded, systematic evaluation using computational methods and a clinician user study. Results show that the examined 16 heatmap algorithms failed to fulfill clinical requirements to correctly indicate AI model decision process or decision quality. The evaluation and MSFI metric can guide the design and selection of explainable AI algorithms to meet clinical requirements on multi-modal explanation.
Key Factors in the Implementation of E-Proctoring in the Spanish University System

Between two possible practices when supervising a remote synchronous evaluation, during the COVID-19 pandemic, the majority of Spanish universities opted for the use of videoconferences with audio and active video, instead of implementing e-proctoring. Thus, in order to analyze the reasons for this non-implementation and take measures so that its use can be extended in the Spanish university system, this study focused on identifying the critical factors in the decision of Spanish universities to accept and implement e-proctoring as a method of remote supervision. For this, a causal study was carried out using the methodology of fuzzy cognitive maps, and the data obtained were processed through the FCMappers tool. This allowed a glimpse of the key role played by students in this non-implementation (who alleged the non-possibility of having the resources necessary for the use of e-proctoring and the violation of privacy that the use of this tool entailed) and highlighted the role of the “pressure or incentives from government” factor if these allegations are to be eliminated and if e-proctoring is to be implemented in Spanish universities.
Digital-Twin-Enabled IoMT System for Surgical Simulation Using rAC-GAN

A digital twin-enabled Internet of Medical Things (IoMT) system for telemedical simulation is developed, systematically integrated with mixed reality (MR), 5G cloud computing, and a generative adversarial network (GAN) to achieve remote lung cancer implementation.Patient-specific data from 90 lung cancer with pulmonary embolism (PE)-positive patients, with 1372 lung cancer control groups, were gathered from Qujing and Dehong, and then transmitted and preprocessed using 5G.A novel robust auxiliary classifier generative adversarial network (rAC-GAN)-based intelligent network is employed to facilitate lung cancer with the PE prediction model.To improve the accuracy and immersion during remote surgical implementation, a real-time operating room perspective from the perception layer with a surgical navigation image is projected to the surgeon's helmet in the application layer using the digital twin-based MR guide clue with 5G.The accuracies of the area under the curve (AUC) of our new intelligent IoMT system were 0.92, and 0.93.Furthermore, the pathogenic features learned from our rAC-GAN model are highly consistent with the statistical epidemiological results.The proposed intelligent IoMT system generates significant performance improvement to process substantial clinical data at cloud centers and shows a novel framework for remote medical data transfer and deep learning analytics for digital twin-based surgical implementation.
Augmented Reality for Machine Setups: Task Performance and Usability Evaluation in a Field Test

Head-mounted displays (HMDs) for augmented reality (AR) have received much scholarly attention. Most studies on AR HMDs, however, rely on lab-based environments or simplistic tasks that do not reflect the complexity of industrial work processes. In this work, we conducted a field study in which professional machine operators performed a setup task for an injection molding machine. We assessed and compared task performance, perceived workload, and usability of an AR HMD versus a tablet computer. We find that the use of an AR HMD is perceived as positive overall and not mentally demanding. However, task completion times were not reduced, although participants felt they were.
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

Charts are very popular for analyzing data.When exploring charts, people often ask a variety of complex reasoning questions that involve several logical and arithmetic operations.They also commonly refer to visual features of a chart in their questions.However, most existing datasets do not focus on such complex reasoning questions as their questions are template-based and answers come from a fixedvocabulary.In this work, we present a largescale benchmark covering 9.6K human-written questions as well as 23.1K questions generated from human-written chart summaries.To address the unique challenges in our benchmark involving visual and logical reasoning over charts, we present two transformer-based models that combine visual features and the data table of the chart in a unified way to answer questions.While our models achieve the state-of-the-art results on the previous datasets as well as on our benchmark, the evaluation also reveals several challenges in answering complex reasoning questions.Q1: Which year has the most divergent opinions about Brazil's economy?
Overview of the Shared Task on Hope Speech Detection for Equality, Diversity, and Inclusion

Hope Speech detection is the task of classifying a sentence as hope speech or non-hope speech given a corpus of sentences.Hope speech is any message or content that is positive, encouraging, reassuring, inclusive and supportive that inspires and engenders optimism in the minds of people.In contrast to identifying and censoring negative speech patterns, hope speech detection focused on recognising and promoting positive speech patterns online.In this paper, we report an overview of the findings and results from the shared task on hope speech detection for Tamil, Malayalam, Kannada, English and Spanish languages conducted at the second workshop on Language Technology for Equality, Diversity and Inclusion (LT-EDI-2022), organised as a part of ACL 2022.The participants were provided with annotated training & development datasets and unlabelled test datasets in all five languages.The goal of the shared task is to classify the given sentences into one of the two hope speech classes (Hope speech, Non hope speech).A total of 126 participants registered for the shared task and 14 teams finally submitted their results.The performance of the systems submitted were evaluated in terms of micro-F1 score and weighted-F1 score.The datasets for this challenge are openly available at the competition website 1 .
Trust-Augmented Deep Reinforcement Learning for Federated Learning Client Selection

In the context of distributed machine learning, the concept of federated learning (FL) has emerged as a solution to the privacy concerns that users have about sharing their own data with a third-party server. FL allows a group of users (often referred to as clients) to locally train a single machine learning model on their devices without sharing their raw data. One of the main challenges in FL is how to select the most appropriate clients to participate in the training of a certain task. In this paper, we address this challenge and propose a trust-based deep reinforcement learning approach to select the most adequate clients in terms of resource consumption and training time. On top of the client selection mechanism, we embed a transfer learning approach to handle the scarcity of data in some regions and compensate potential lack of learning at some servers. We apply our solution in the healthcare domain in a COVID-19 detection scenario over IoT devices. In the considered scenario, edge servers collaborate with IoT devices to train a COVID-19 detection model using FL without having to share any raw confidential data. Experiments conducted on a real-world COVID-19 dataset reveal that our solution achieves a good trade-off between detection accuracy and model execution time compared to existing approaches.
Detection of Metastatic Breast Cancer from Whole-Slide Pathology Images Using an Ensemble Deep-Learning Method

Background: Metastasis is the main cause of death toll among breast cancer patients. Since current approaches for diagnosis of lymph node metastases are time-consuming, deep learning (DL) algorithms with more speed and accuracy are explored for effective alternatives. Methods: A total of 220025 whole-slide pictures from patients’ lymph nodes were classified into two cohorts: testing and training. For metastatic cancer identification, we employed hybrid convolutional network models. The performance of our diagnostic system was verified using 57458 unlabeled images that utilized criteria that included accuracy, sensitivity, specificity, and P-value. Results: The DL-based system that was automatically and exclusively capable of quantifying and identifying metastatic lymph nodes was engineered. Quantification was made with 98.84% accuracy. Moreover, the precision of VGG16 and Recall was 92.42% and 91.25%, respectively. Further experiments demonstrated that metastatic cancer differentiation levels could influence the recognition performance. Conclusion: Our engineered diagnostic complex showed an elevated level of precision and efficiency for lymph node diagnosis. Our innovative DL-based system has a potential to simplify pathological screening for metastasis in breast cancer patients.
Distributed contrastive learning for medical image segmentation

Supervised deep learning needs a large amount of labeled data to achieve high performance. However, in medical imaging analysis, each site may only have a limited amount of data and labels, which makes learning ineffective. Federated learning (FL) can learn a shared model from decentralized data. But traditional FL requires fully-labeled data for training, which is very expensive to obtain. Self-supervised contrastive learning (CL) can learn from unlabeled data for pre-training, followed by fine-tuning with limited annotations. However, when adopting CL in FL, the limited data diversity on each site makes federated contrastive learning (FCL) ineffective. In this work, we propose two federated self-supervised learning frameworks for volumetric medical image segmentation with limited annotations. The first one features high accuracy and fits high-performance servers with high-speed connections. The second one features lower communication costs, suitable for mobile devices. In the first framework, features are exchanged during FCL to provide diverse contrastive data to each site for effective local CL while keeping raw data private. Global structural matching aligns local and remote features for a unified feature space among different sites. In the second framework, to reduce the communication cost for feature exchanging, we propose an optimized method FCLOpt that does not rely on negative samples. To reduce the communications of model download, we propose the predictive target network update (PTNU) that predicts the parameters of the target network. Based on PTNU, we propose the distance prediction (DP) to remove most of the uploads of the target network. Experiments on a cardiac MRI dataset show the proposed two frameworks substantially improve the segmentation and generalization performance compared with state-of-the-art techniques.
An Integrated Real-Time Hand Gesture Recognition Framework for Human–Robot Interaction in Agriculture

Incorporating hand gesture recognition in human–robot interaction has the potential to provide a natural way of communication, thus contributing to a more fluid collaboration toward optimizing the efficiency of the application at hand and overcoming possible challenges. A very promising field of interest is agriculture, owing to its complex and dynamic environments. The aim of this study was twofold: (a) to develop a real-time skeleton-based recognition system for five hand gestures using a depth camera and machine learning, and (b) to enable a real-time human–robot interaction framework and test it in different scenarios. For this purpose, six machine learning classifiers were tested, while the Robot Operating System (ROS) software was utilized for “translating” the gestures into five commands to be executed by the robot. Furthermore, the developed system was successfully tested in outdoor experimental sessions that included either one or two persons. In the last case, the robot, based on the recognized gesture, could distinguish which of the two workers required help, follow the “locked” person, stop, return to a target location, or “unlock” them. For the sake of safety, the robot navigated with a preset socially accepted speed while keeping a safe distance in all interactions.
Tree of Science - ToS: A Web-Based Tool for Scientific Literature Recommendation. Search Less, Research More!

Tree of Science (ToS) is an integrated web-based platform for a comprehensive analysis of scientific literature. ToS is designed to facilitate finding relevant literature and can be used by students, researchers, and academics. Based on graph theory metrics, this tool visualizes the works in a knowledge field as a tree where the roots are classic articles, the trunk represents those articles that allow the area to grow, and the leaves are the recently published articles. This article provides a description of how to search, format, and upload the data and identify significant literature in a specific research area. Finally, a brief description of how the Tree of Science works.
Class-Difficulty Based Methods for Long-Tailed Visual Recognition

Abstract Long-tailed datasets are very frequently encountered in real-world use cases where few classes or categories (known as majority or head classes) have higher number of data samples compared to the other classes (known as minority or tail classes). Training deep neural networks on such datasets gives results biased towards the head classes. So far, researchers have come up with multiple weighted loss and data re-sampling techniques in efforts to reduce the bias. However, most of such techniques assume that the tail classes are always the most difficult classes to learn and therefore need more weightage or attention. Here, we argue that the assumption might not always hold true. Therefore, we propose a novel approach to dynamically measure the instantaneous difficulty of each class during the training phase of the model. Further, we use the difficulty measures of each class to design a novel weighted loss technique called ‘class-wise difficulty based weighted (CDB-W) loss’ and a novel data sampling technique called ‘class-wise difficulty based sampling (CDB-S)’. To verify the wide-scale usability of our CDB methods, we conducted extensive experiments on multiple tasks such as image classification, object detection, instance segmentation and video-action classification. Results verified that CDB-W loss and CDB-S could achieve state-of-the-art results on many class-imbalanced datasets such as ImageNet-LT, LVIS and EGTEA, that resemble real-world use cases.
A continuous authentication protocol without trust authority for zero trust architecture

Zero-trust security is a novel concept to cope with intricate access, which can not be handled by the conventional perimeter-based architecture anymore. The device-to-device continuous authentication protocol is one of the most crucial cornerstones, especially in the IoT scenario. In the zero-trust architecture, trust does not rely on any position, person or device. However, to the best of our knowledge, almost all existing device-to-device continuous authentication relies on a trust authority or a node to generate secret keys or secret values. This is betrayed by the principle of zero-trust architecture. In this paper, we employ the blockchain to eliminate the trusted node. One node is chosen to produce the public parameter and secret keys for two entities through the practical Byzantine fault tolerance consensus mechanism. Additionally, the devices are categorized into three folds: trusted device, suspected device and untrusted device. Only the first two can participate in authentication, and they have different lengths of security parameters and intervals to reach a better balance between security and efficiency. Then we prove the security of the initial authentication part in the eCK model and give an informal analysis of the continuous authentication part. Finally, we implement the proposed protocol on simulated devices. The result illustrates that our scheme is highly efficient, and the continuous authentication only costs around 0.1ms.