Reviewerly/exHarmony huggingface dataset:
Words Aren’t Enough, Their Order Matters: On the Robustness of Grounding Visual Referring Expressions

Visual referring expression recognition is a challenging task that requires natural language understanding in the context of an image.We critically examine RefCOCOg, a standard benchmark for this task, using a human study and show that 83.7% of test instances do not require reasoning on linguistic structure, i.e., words are enough to identify the target object, the word order doesn't matter.To measure the true progress of existing models, we split the test set into two sets, one which requires reasoning on linguistic structure and the other which doesn't.Additionally, we create an out-of-distribution dataset Ref-Adv by asking crowdworkers to perturb in-domain examples such that the target object changes.Using these datasets, we empirically show that existing methods fail to exploit linguistic structure and are 12% to 23% lower in performance than the established progress for this task.We also propose two methods, one based on contrastive learning and the other based on multi-task learning, to increase the robustness of ViLBERT, the current state-ofthe-art model for this task.
Methodological and Empirical Platform of Triangulation in Strategic Management

The methodological and applied foundations of the application of triangulation in strategic management are defined in the article. The role of triangulation in the structure of a qualitative study of strategic processes and the feasibility of its inclusion in the design of a qualitative study from the point of view of current goals and objectives within the framework of development strategies of the object have been substantiated. The methodological foundations of the interpretation and validation of the results of the application of triangulation in strategic management are studied, author's recommendations on the use of triangulation in the coordinates of strategic management are provided. The empirical study has been conducted highlighting the findings and main recommendations.
LSAN: Modeling Long-term Dependencies and Short-term Correlations with Hierarchical Attention for Risk Prediction

Risk prediction using electronic health records (EHR) is a challenging data mining task due to the two-level hierarchical structure of EHR data. EHR data consist of a set of time-ordered visits, and within each visit, there is a set of unordered diagnosis codes. Existing approaches focus on modeling temporal visits with deep neural network (DNN) techniques. However, they ignore the importance of modeling diagnosis codes within visits, and a lot of task-unrelated information within visits usually leads to unsatisfactory performance of existing approaches. To minimize the effect caused by noise information of EHR data, in this paper, we propose a novel DNN for risk prediction termed as LSAN, which consists of a Hierarchical Attention Module (HAM) and a Temporal Aggregation Module (TAM). Particularly, LSAN applies HAM to model the hierarchical structure of EHR data. Using the attention mechanism in the hierarchy of diagnosis code, HAM is able to retain diagnosis details and assign flexible attention weights to different diagnosis codes by their relevance to corresponding diseases. Moreover, the attention mechanism in the hierarchy of visit learns a comprehensive feature throughout the visit history by paying greater attention to visits with higher relevance. Based on the foundation laying by HAM, TAM uses a two-pathway structure to learn a robust temporal aggregation mechanism among all visits for LSAN. It extracts long-term dependencies by a Transformer encoder and short-term correlations by a parallel convolutional layer among different visits. With the construction of HAM and TAM, LSAN achieves the state-of-the-art performance on three real-world datasets with larger AUCs, recalls and F1 scores. Furthermore, the model analysis results demonstrate the effectiveness of the network construction with good interpretability and robustness of decision making by LSAN.
Label Confusion Learning to Enhance Text Classification Models

Representing the true label as one-hot vector is the common practice in training text classification models. However, the one-hot representation may not adequately reflect the relation between the instance and labels, as labels are often not completely independent and instances may relate to multiple labels in practice. The inadequate one-hot representations tend to train the model to be over-confident, which may result in arbitrary prediction and model overfitting, especially for confused datasets (datasets with very similar labels) or noisy datasets (datasets with labeling errors). While training models with label smoothing can ease this problem in some degree, it still fails to capture the realistic relation among labels. In this paper, we propose a novel Label Confusion Model (LCM) as an enhancement component to current popular text classification models. LCM can learn label confusion to capture semantic overlap among labels by calculating the similarity between instance and labels during training and generate a better label distribution to replace the original one-hot label vector, thus improving the final classification performance. Extensive experiments on five text classification benchmark datasets reveal the effectiveness of LCM for several widely used deep learning classification models. Further experiments also verify that LCM is especially helpful for confused or noisy datasets and superior to the label smoothing method.
A Systematic Review of Virtual Reality Interfaces for Controlling and Interacting with Robots

There is a significant amount of synergy between virtual reality (VR) and the field of robotics. However, it has only been in approximately the past five years that commercial immersive VR devices have been available to developers. This new availability has led to a rapid increase in research using VR devices in the field of robotics, especially in the development of VR interfaces for operating robots. In this paper, we present a systematic review on VR interfaces for robot operation that utilize commercially available immersive VR devices. A total of 41 papers published between 2016–2020 were collected for review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Papers are discussed and categorized into five categories: (1) Visualization, which focuses on displaying data or information to operators; (2) Robot Control and Planning, which focuses on connecting human input or movement to robot movement; (3) Interaction, which focuses on the development of new interaction techniques and/or identifying best interaction practices; (4) Usability, which focuses on user experiences of VR interfaces; and (5) Infrastructure, which focuses on system architectures or software to support connecting VR and robots for interface development. Additionally, we provide future directions to continue development in VR interfaces for operating robots.
Higher Education in the First Year of COVID-19: Thoughts and Perspectives for the Future

In the last year a new virus (SARS‑CoV‑2) and the disease caused by it (COVID-19) has quickly spread around the world, leading the World Health Organization to declare a public health emergency and, then, a global pandemic status. The strategies adopted by many countries to reduce the impact of the pandemic were mainly based on social distancing rules and on stay-at-home measures or lockdowns. These strategies had severe disruptive consequences on many sectors, including all levels of education. While the “traditional” (face-to-face) Higher Education (HE) system was unprepared for the lockdown (e.g., no plans for a massive shift to online teaching were available/ready), it reacted in an extremely quick and effective way, replacing face-to-face teaching with online teaching. While COVID-19 has been extremely challenging for education, the experience has undoubtedly provided positive inputs for the digitalization of the HE system. The question is however, if whether after the COVID-19 emergency everything will go back to the previous situation or instead if the pandemic has irreversibly changed HE. While we are still in the middle of the crisis, it is in our view beneficial to start to reflect on the challenges and open issues that emerged during this period and the lessons learned for the “new normal” (as it is often referred to). In this conceptual paper we seek to start this discussion by focusing on the following relevant aspects that should be considered to succeed in the digital transformation: broadband network infrastructure and hardware devices; e-learning software; organization of teaching activities; pedagogical issues; diversity and inclusivity; and a number of other issues. We conclude that the COVID-19 pandemic will irreversibly change HE and probably for the better.
A Sequence-to-Set Network for Nested Named Entity Recognition

Named entity recognition (NER) is a widely studied task in natural language processing. Recently, a growing number of studies have focused on the nested NER. The span-based methods, considering the entity recognition as a span classification task, can deal with nested entities naturally. But they suffer from the huge search space and the lack of interactions between entities. To address these issues, we propose a novel sequence-to-set neural network for nested NER. Instead of specifying candidate spans in advance, we provide a fixed set of learnable vectors to learn the patterns of the valuable spans. We utilize a non-autoregressive decoder to predict the final set of entities in one pass, in which we are able to capture dependencies between entities. Compared with the sequence-to-sequence method, our model is more suitable for such unordered recognition task as it is insensitive to the label order. In addition, we utilize the loss function based on bipartite matching to compute the overall training loss. Experimental results show that our proposed model achieves state-of-the-art on three nested NER corpora: ACE 2004, ACE 2005 and KBP 2017. The code is available at https://github.com/zqtan1024/sequence-to-set.
A Novel Occlusion-aware Vote Cost for Light Field Depth Estimation

Capturing the directions of light by light field cameras powers next-generation immersive multimedia applications. A critical problem in taking advantage of the rich visual information in light field images is depth estimation. Conventional light field depth estimation methods build a cost volume that measures the photo-consistency of pixels refocused to a range of depths, and the highest consistency indicates the correct depth. This strategy works well in most regions but usually generates blurry edges in the estimated depth map due to occlusions. Recent work shows that integrating occlusion models to light field depth estimation can largely reduce blurry edges. However, existing occlusion handling methods rely on complex edge-aided processing and post-refinement, and this reliance limits the resultant depth accuracy and impacts on the computational performance. In this paper, we propose a novel occlusion-aware vote cost (OAVC) which is able to accurately preserve edges in the depth map. Instead of using photo-consistency as an indicator of the correct depth, we construct a novel cost from a new perspective that counts the number of refocused pixels whose deviations from the central-view pixel are less than a small threshold, and utilizes that number to select the correct depth. The pixels from occluders are thus excluded in determining the correct depth. Without the use of any explicit occlusion handling methods, the proposed method can inherently preserve edges and produces high-quality depth estimates. Experimental results show that the proposed OAVC outperforms state-of-the-art light field depth estimation methods in terms of depth estimation accuracy and computational complexity.
Scattering-Keypoint-Guided Network for Oriented Ship Detection in High-Resolution and Large-Scale SAR Images

Ship detection in synthetic aperture radar (SAR) images is a significant and challenging task.Recently, deep convolutional neural networks have been applied to solve the detection problem and made a great breakthrough.Previous works mostly rely on the manually designed anchor boxes to search for the region of interests, which is less flexible and suffers from a heavy computational load.Moreover, these detectors have limited performance in large-scale and complex scenes due to the strong interference of inshore background and the variability of object imaging characteristics.In this paper, a novel ship detection method based on the scattering keypoints guided network (SKG-Net) is proposed to remedy these problems.First, an anchor-free network is built to eliminate the effect of anchor boxes, in which a more robust representation scheme is designed for the arbitrary oriented objects.Second, a contextaware feature selection module is introduced to dynamically learn both local and context features.In this process, the semantic information of objects can be enhanced while suppressing the background interference.Third, according to the SAR imaging mechanism, a set of scattering keypoints is defined to describe the local scattering regions and reflect the discriminative structural characteristics of ships.Based on this conception, a novel feature adaption method is proposed with the purpose of dealing with the imaging variability issue.Furthermore, to demonstrate the effectiveness of the proposed improvements, we build the Gaofen-3 ship detection data set (GF3SDD).Meanwhile, the public SAR ship detection data set (SSDD) is introduced to verify the robustness and generalization ability of the detector.Experimental results on these two data sets show that the proposed method achieves the state-of-the-art performance.
Convolutional Recurrent Neural Networks for Text Classification

Recurrent neural network (RNN) and convolutional neural network (CNN) are two prevailing architectures used in text classification. Traditional approaches combine the strengths of these two networks by straightly streamlining them or linking features extracted from them. In this article, a novel approach is proposed to maintain the strengths of RNN and CNN to a great extent. In the proposed approach, a bi-directional RNN encodes each word into forward and backward hidden states. Then, a neural tensor layer is used to fuse bi-directional hidden states to get word representations. Meanwhile, a convolutional neural network is utilized to learn the importance of each word for text classification. Empirical experiments are conducted on several datasets for text classification. The superior performance of the proposed approach confirms its effectiveness.
Autonomous Vehicles Perception (AVP) Using Deep Learning: Modeling, Assessment, and Challenges

Perception is the fundamental task of any autonomous driving system, which gathers all the necessary information about the surrounding environment of the moving vehicle.The decision-making system takes the perception data as input and makes the optimum decision given that scenario, which maximizes the safety of the passengers.This paper surveyed recent literature on autonomous vehicle perception (AVP) by focusing on two primary tasks: Semantic Segmentation and Object Detection.Both tasks play an important role as a vital component of the vehicle's navigation system.A comprehensive overview of deep learning for perception and its decision-making process based on images and LiDAR point clouds is discussed.We discussed the sensors, benchmark datasets, and simulation tools widely used in semantic segmentation and object detection tasks, especially for autonomous driving.This paper acts as a road map for current and future research in AVP, focusing on models, assessment, and challenges in the field.
Deep Learning-Based Mapping of Tumor Infiltrating Lymphocytes in Whole Slide Images of 23 Types of Cancer

The role of tumor infiltrating lymphocytes (TILs) as a biomarker to predict disease progression and clinical outcomes has generated tremendous interest in translational cancer research. We present an updated and enhanced deep learning workflow to classify 50x50 um tiled image patches (100x100 pixels at 20x magnification) as TIL positive or negative based on the presence of 2 or more TILs in gigapixel whole slide images (WSIs) from the Cancer Genome Atlas (TCGA). This workflow generates TIL maps to study the abundance and spatial distribution of TILs in 23 different types of cancer. We trained three state-of-the-art, popular convolutional neural network (CNN) architectures (namely VGG16, Inception-V4, and ResNet-34) with a large volume of training data, which combined manual annotations from pathologists (strong annotations) and computer-generated labels from our previously reported first-generation TIL model for 13 cancer types (model-generated annotations). Specifically, this training dataset contains TIL positive and negative patches from cancers in additional organ sites and curated data to help improve algorithmic performance by decreasing known false positives and false negatives. Our new TIL workflow also incorporates automated thresholding to convert model predictions into binary classifications to generate TIL maps. The new TIL models all achieve better performance with improvements of up to 13% in accuracy and 15% in F-score. We report these new TIL models and a curated dataset of TIL maps, referred to as TIL-Maps-23, for 7983 WSIs spanning 23 types of cancer with complex and diverse visual appearances, which will be publicly available along with the code to evaluate performance. Code Available at: https://github.com/ShahiraAbousamra/til_classification.
The Metaverse in Cardiovascular Medicine: Applications, Challenges, and the Role of Non-Fungible Tokens

The recent pandemic triggered an acceleration in adopting innovative technologies by the health care industry. One of the technological breakthroughs that flourished during this period—and has rapidly gained popularity—is the trending and novel concept of the metaverse. The metaverse represents a blend of digital environments that use augmented reality (AR) and virtual reality (VR) to create an immense online world, running on blockchain, in which users can interact using avatars and perform various activities from shopping to attending virtual events.
<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si10.svg"><mml:mrow><mml:mi>ω</mml:mi></mml:mrow></mml:math>-net: Dual supervised medical image segmentation with multi-dimensional self-attention and diversely-connected multi-scale convolution

Although U-Net and its variants have achieved some great successes in medical image segmentation tasks, their segmentation performances for small objects are still unsatisfactory. Therefore, in this work, a new deep model, ω-Net, is proposed to achieve more accurate medical image segmentations. The advancements of ω-Net are mainly threefold: First, it incorporates an additional expansive path into U-Net to import an extra supervision signal and obtain a more effective and robust image segmentation by dual supervision. Then, a multi-dimensional self-attention mechanism is further developed to highlight salient features and suppress irrelevant ones consecutively in both spatial and channel dimensions. Finally, to reduce semantic disparity between the feature maps of the contracting and expansive paths, we further propose to integrate diversely-connected multi-scale convolution blocks into the skip connections, where several multi-scale convolutional operations are connected in both series and parallel. Extensive experimental results on three abdominal CT segmentation tasks show that (i) ω-Net greatly outperforms the state-of-the-art image segmentation methods in medical image segmentation tasks; (ii) the proposed three advancements are all effective and essential for ω-Net to achieve the superior performances; and (iii) the proposed multi-dimensional self-attention (resp., diversely-connected multi-scale convolution) is more effective than the state-of-the-art attention mechanisms (resp., multi-scale solutions) for medical image segmentations. The code will be released online after this paper is formally accepted.
Real-Time Vehicle Classification and Tracking Using a Transfer Learning-Improved Deep Learning Network

Accurate vehicle classification and tracking are increasingly important subjects for intelligent transport systems (ITSs) and for planning that utilizes precise location intelligence. Deep learning (DL) and computer vision are intelligent methods; however, accurate real-time classification and tracking come with problems. We tackle three prominent problems (P1, P2, and P3): the need for a large training dataset (P1), the domain-shift problem (P2), and coupling a real-time multi-vehicle tracking algorithm with DL (P3). To address P1, we created a training dataset of nearly 30,000 samples from existing cameras with seven classes of vehicles. To tackle P2, we trained and applied transfer learning-based fine-tuning on several state-of-the-art YOLO (You Only Look Once) networks. For P3, we propose a multi-vehicle tracking algorithm that obtains the per-lane count, classification, and speed of vehicles in real time. The experiments showed that accuracy doubled after fine-tuning (71% vs. up to 30%). Based on a comparison of four YOLO networks, coupling the YOLOv5-large network to our tracking algorithm provided a trade-off between overall accuracy (95% vs. up to 90%), loss (0.033 vs. up to 0.036), and model size (91.6 MB vs. up to 120.6 MB). The implications of these results are in spatial information management and sensing for intelligent transport planning.
COVID-19 Pandemic and Student Reading Achievement: Findings From a School Panel Study

Since 2020, the COVID-19 pandemic had an impact on education worldwide. There is increased discussion of possible negative effects on students' learning outcomes and the need for targeted support. We examined fourth graders' reading achievement based on a school panel study, representative on the student level, with N = 111 elementary schools in Germany (total: N = 4,290 students, age: 9-10 years). The students were tested with the Progress in International Reading Literacy Study instruments in 2016 and 2021. The analysis focused on (1) total average differences in reading achievement between 2016 and 2021, (2) average differences controlling for student composition, and (3) changes in achievement gaps between student subgroups (i.e., immigration background, socio-cultural capital, and gender). The methodological approach met international standards for the analysis of large-scale assessments (i.e., multiple multi-level imputation, plausible values, and clustered mixed-effect regression). The results showed a substantial decline in mean reading achievement. The decline corresponds to one-third of a year of learning, even after controlling for changes in student composition. We found no statistically significant changes of achievement gaps between student subgroups, despite numerical tendencies toward a widening of achievement gaps between students with and without immigration background. It is likely that this sharp achievement decline was related to the COVID-19 pandemic. The findings are discussed in terms of further research needs, practical implications for educating current student cohorts, and educational policy decisions regarding actions in crises such as the COVID-19 pandemic.