Reviewerly/exHarmony huggingface dataset:
Virtual reality applications for the built environment: Research trends and opportunities

With its advanced capabilities of immersive and interactive visualization, virtual reality (VR) has been advocated to facilitate design, engineering, construction, and management for the built environment. Substantial efforts have thus been devoted to VR-related applications in the architecture, engineering, and construction (AEC) industry over the last decades, resulting in a vast, diverse, and fragmented body of knowledge. The objective of this research is to systematically gain an in-depth understanding of research trends as well as reveal challenges and opportunities for future research in the area. To achieve this objective, this research explores the state-of-the-art in VR applications for the built environment by a mixed quantitative-qualitative review method. A total of 229 journal articles are collected using a structured data acquisition approach from Scopus and then fed into a bibliometric analysis to construct science maps. By doing so, the main research outlets, articles, and themes of this research field are quantitatively identified. Subsequently, an in-depth qualitative discussion is presented to provide deeper insights into the challenges and opportunities of main research topics. Furthermore, future research directions are proposed as follows: 1) user-centered adaptive design, 2) attention-driven virtual reality information systems, 3) construction training systems incorporating human factors, 4) occupant-centered facility management, and 5) industry adoption. This research contributes to the body of knowledge by synthesizing the state of the art of VR technology for the built environment and exposing its research needs, which can serve both academia and industry in terms of promoting VR applications for the built environment.
Blockchain and smart contracts in supply chain management: A game theoretic model

This paper proposes a supply chain game that is composed of two firms, one supplier and one retailer. The supplier sells goods to the retailer and sets the service strategy efforts. The retailer determines the optimal quantity to purchase and the selling price. The supply chain can be managed through either a traditional online platform or a blockchain. In the former, firms face business risks due to delivery and service and also pay high transaction costs. In the latter, they work on a blockchain platform, remove all risks over the supply chain, and save the transaction costs. However, the blockchain requires initial implementation investments as well as variable costs. Furthermore, the firms gain in terms of visibility, transparency, and security, which are summarized in tokens. We identify the conditions and the stochastic cases in which the blockchain is not worth implementing. Then, we investigate the suitability of a smart wholesale price contract and a smart revenue sharing contract to better coordinate firms’ relationships and negotiations. We highlight all cases in which the use of smart contracts makes blockchain applications more operationally convenient and economically appealing.
Self-supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance

Self-supervised monocular depth estimation presents a powerful method to obtain 3D scene information from single camera images, which is trainable on arbitrary image sequences without requiring depth labels, e.g., from a LiDAR sensor. In this work we present a new self-supervised semantically-guided depth estimation (SGDepth) method to deal with moving dynamic-class (DC) objects, such as moving cars and pedestrians, which violate the static-world assumptions typically made during training of such models. Specifically, we propose (i) mutually beneficial cross-domain training of (supervised) semantic segmentation and self-supervised depth estimation with task-specific network heads, (ii) a semantic masking scheme providing guidance to prevent moving DC objects from contaminating the photometric loss, and (iii) a detection method for frames with non-moving DC objects, from which the depth of DC objects can be learned. We demonstrate the performance of our method on several benchmarks, in particular on the Eigen split, where we exceed all baselines without test-time refinement.
End-to-end privacy preserving deep learning on multi-institutional medical imaging

Using large, multi-national datasets for high-performance medical imaging AI systems requires innovation in privacy-preserving machine learning so models can train on sensitive data without requiring data transfer. Here we present PriMIA (Privacy-preserving Medical Image Analysis), a free, open-source software framework for differentially private, securely aggregated federated learning and encrypted inference on medical imaging data. We test PriMIA using a real-life case study in which an expert-level deep convolutional neural network classifies paediatric chest X-rays; the resulting model’s classification performance is on par with locally, non-securely trained models. We theoretically and empirically evaluate our framework’s performance and privacy guarantees, and demonstrate that the protections provided prevent the reconstruction of usable data by a gradient-based model inversion attack. Finally, we successfully employ the trained model in an end-to-end encrypted remote inference scenario using secure multi-party computation to prevent the disclosure of the data and the model. Gaining access to medical data to train AI applications can present problems due to patient privacy or proprietary interests. A way forward can be privacy-preserving federated learning schemes. Kaissis, Ziller and colleagues demonstrate here their open source framework for privacy-preserving medical image analysis in a remote inference scenario.
Mobile-Former: Bridging MobileNet and Transformer

We present Mobile-Former, a parallel design of MobileNet and transformer with a two-way bridge in between. This structure leverages the advantages of MobileNet at local processing and transformer at global interaction. And the bridge enables bidirectional fusion of local and global features. Different from recent works on vision transformer, the transformer in Mobile-Former contains very few tokens (e.g. 6 or fewer tokens) that are randomly initialized to learn global priors, resulting in low computational cost. Combining with the proposed light-weight cross attention to model the bridge, Mobile-Former is not only computationally efficient, but also has more representation power. It outperforms MobileNetV3 at low FLOP regime from 25M to 500M FLOPs on ImageNet classification. For instance, Mobile-Former achieves 77.9% top-1 accuracy at 294M FLOPs, gaining 1.3% over MobileNetV3 but saving 17% of computations. When transferring to object detection, Mobile-Former outperforms MobileNetV3 by 8.6 AP in RetinaNet framework. Furthermore, we build an efficient end-to-end detector by replacing backbone, encoder and decoder in DETR with Mobile-Former, which outperforms DETR by 1.3 AP but saves 52% of computational cost and 36% of parameters. Code will be released at https://github.com/aaboys/mobileformer.
CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation

Convolutional neural networks (CNNs) have been the de facto standard for nowadays 3D medical image segmentation. The convolutional operations used in these networks, however, inevitably have limitations in modeling the long-range dependency due to their inductive bias of locality and weight sharing. Although Transformer was born to address this issue, it suffers from extreme computational and spatial complexities in processing high-resolution 3D feature maps. In this paper, we propose a novel framework that efficiently bridges a Convolutional neural network and a Transformer (CoTr) for accurate 3D medical image segmentation. Under this framework, the CNN is constructed to extract feature representations and an efficient deformable Transformer (DeTrans) is built to model the long-range dependency on the extracted feature maps. Different from the vanilla Transformer which treats all image positions equally, our DeTrans pays attention only to a small set of key positions by introducing the deformable self-attention mechanism. Thus, the computational and spatial complexities of DeTrans have been greatly reduced, making it possible to process the multi-scale and high-resolution feature maps, which are usually of paramount importance for image segmentation. We conduct an extensive evaluation on the Multi-Atlas Labeling Beyond the Cranial Vault (BCV) dataset that covers 11 major human organs. The results indicate that our CoTr leads to a substantial performance improvement over other CNN-based, transformer-based, and hybrid methods on the 3D multi-organ segmentation task. Code is available at: https://github.com/YtongXie/CoTr.
Meta-Learning

Meta-learning, or learning to learn, is the science of systematically observing how different machine learning approaches perform on a wide range of learning tasks, and then learning from this experience, or meta-data, to learn new tasks much faster than otherwise possible. Not only does this dramatically speed up and improve the design of machine learning pipelines or neural architectures, it also allows us to replace hand-engineered algorithms with novel approaches learned in a data-driven way. In this chapter, we provide an overview of the state of the art in this fascinating and continuously evolving field.
Acceptance of immersive head-mounted virtual reality in older adults

Abstract Immersive virtual reality has become increasingly popular to improve the assessment and treatment of health problems. This rising popularity is likely to be facilitated by the availability of affordable headsets that deliver high quality immersive experiences. As many health problems are more prevalent in older adults, who are less technology experienced, it is important to know whether they are willing to use immersive virtual reality. In this study, we assessed the initial attitude towards head-mounted immersive virtual reality in 76 older adults who had never used virtual reality before. Furthermore, we assessed changes in attitude as well as self-reported cybersickness after a first exposure to immersive virtual reality relative to exposure to time-lapse videos. Attitudes towards immersive virtual reality changed from neutral to positive after a first exposure to immersive virtual reality, but not after exposure to time-lapse videos. Moreover, self-reported cybersickness was minimal and had no association with exposure to immersive virtual reality. These results imply that the contribution of VR applications to health in older adults will neither be hindered by negative attitudes nor by cybersickness.
A Novel Robust Gaussian–Student's <i>t</i> Mixture Distribution Based Kalman Filter

In this paper, a novel Gaussian-Student's t mixture (GSTM) distribution is proposed to model non-stationary heavy-tailed noises. The proposed GSTM distribution can be formulated as a hierarchical Gaussian form by introducing a Bernoulli random variable, based on which a new hierarchical linear Gaussian state-space model is constructed. A novel robust GSTM distribution based Kalman filter is proposed based on the constructed hierarchical linear Gaussian state-space model using the variational Bayesian approach. The Kalman filter and robust Student's t based Kalman filter (RSTKF) with fixed distribution parameters are two existing special cases of the proposed filter. The novel GSTM distributed Kalman filter has the important advantage over the RSTKF that the adaptation of the mixing parameter is much more straightforward than learning the degrees of freedom parameter. Simulation results illustrate that the proposed filter has better estimation accuracy than those of the Kalman filter and RSTKF for a linear state-space model with non-stationary heavy-tailed noises.
Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning

Learning is an inherently continuous phenomenon. When humans learn a new task there is no explicit distinction between training and inference. As we learn a task, we keep learning about it while performing the task. What we learn and how we learn it varies during different stages of learning. Learning how to learn and adapt is a key property that enables us to generalize effortlessly to new settings. This is in contrast with conventional settings in machine learning where a trained model is frozen during inference. In this paper we study the problem of learning to learn at both training and test time in the context of visual navigation. A fundamental challenge in navigation is generalization to unseen scenes. In this paper we propose a self-adaptive visual navigation method (SAVN) which learns to adapt to new environments without any explicit supervision. Our solution is a meta-reinforcement learning approach where an agent learns a self-supervised interaction loss that encourages effective navigation. Our experiments, performed in the AI2-THOR framework, show major improvements in both success rate and SPL for visual navigation in novel scenes. Our code and data are available at: https://github.com/allenai/savn.
Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defenses

Research on adversarial examples in computer vision tasks has shown that small, often imperceptible changes to an image can induce misclassification, which has security implications for a wide range of image processing systems. Considering L2 norm distortions, the Carlini and Wagner attack is presently the most effective white-box attack in the literature. However, this method is slow since it performs a line-search for one of the optimization terms, and often requires thousands of iterations. In this paper, an efficient approach is proposed to generate gradient-based attacks that induce misclassifications with low L2 norm, by decoupling the direction and the norm of the adversarial perturbation that is added to the image. Experiments conducted on the MNIST, CIFAR-10 and ImageNet datasets indicate that our attack achieves comparable results to the state-of-the-art (in terms of L2 norm) with considerably fewer iterations (as few as 100 iterations), which opens the possibility of using these attacks for adversarial training. Models trained with our attack achieve state-of-the-art robustness against white-box gradient-based L2 attacks on the MNIST and CIFAR-10 datasets, outperforming the Madry defense when the attacks are limited to a maximum norm.
Day-ahead power forecasting in a large-scale photovoltaic plant based on weather classification using LSTM

Photovoltaic (PV) solar power generation is always associated with uncertainties due to weather parameters intermittency. This poses difficulties in grid management as solar penetration rate rise continuously. Thus, accurate Photovoltaic (PV) power prediction is required for the successful integration of solar energy into the power grid, and short-term forecasting (minutes-1 day ahead) is significant for real-time power dispatching. Day-ahead power output time-series forecasting methods are proposed in this paper, in which ideal weather type and non-ideal weather types have been separately discussed. For ideal weather conditions, a forecasting method is proposed based on meteorology data of next day for ideal weather condition, using long short term memory (LSTM) networks. For non-ideal weather conditions, time-series relevance and specific non-ideal weather type characteristic are considered in LSTM model by introducing adjacent day time-series and typical weather type information. Specifically, daily total power, which is obtained by discrete grey model (DGM), is regarded as input variables and applied to correct power output time-series prediction. Prediction performance comparison between proposed methods with traditional algorithms reveal that the RMSE accuracy of forecasting methods based on LSTM networks can reach 4.62% for ideal weather condition. For non-ideal weather condition, the dynamic characteristic is effectively described by proposed methods and the proposed methods obtained superior prediction accuracy.
IN-YOLO: Real-Time Detection of Outdoor High Voltage Insulators Using UAV Imaging

The high voltage insulator requires continuous monitoring and inspection to prevent failures and emergencies. Manual inspections are costly as it requires covering a large geographical area where insulators are often subjected to harsh weather conditions. Automatic detection of insulators from aerial images is the first step towards performing real-time classification of insulator conditions using Unmanned Aerial Vehicle (UAV). In this paper, we provide a cost-effective solution for detecting insulators under the conditions of an uncluttered background, varied object resolution and illumination conditions using You Only Look Once (YOLO) deep learning neural network model from aerial images. We apply data augmentation to avoid overfitting with a training set size of 56000 image samples. It is demonstrated experimentally that this method can accurately locate insulator on UAV based real-time image data. The detected insulator images are then successfully subjected to insulator surface condition assessment for the presence of ice, snow and water using different classifiers.
Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

Many vision and language models suffer from poor visual grounding -- often falling back on easy-to-learn language priors rather than basing their decisions on visual concepts in the image. In this work, we propose a generic approach called Human Importance-aware Network Tuning (HINT) that effectively leverages human demonstrations to improve visual grounding. HINT encourages deep networks to be sensitive to the same input regions as humans. Our approach optimizes the alignment between human attention maps and gradient-based network importances -- ensuring that models learn not just to look at but rather rely on visual concepts that humans found relevant for a task when making predictions. We apply HINT to Visual Question Answering and Image Captioning tasks, outperforming top approaches on splits that penalize over-reliance on language priors (VQA-CP and robust captioning) using human attention demonstrations for just 6% of the training data.
HACS: Human Action Clips and Segments Dataset for Recognition and Temporal Localization

This paper presents a new large-scale dataset for recognition and temporal localization of human actions collected from Web videos. We refer to it as HACS (Human Action Clips and Segments). We leverage consensus and disagreement among visual classifiers to automatically mine candidate short clips from unlabeled videos, which are subsequently validated by human annotators. The resulting dataset is dubbed HACS Clips. Through a separate process we also collect annotations defining action segment boundaries. This resulting dataset is called HACS Segments. Overall, HACS Clips consists of 1.5M annotated clips sampled from 504K untrimmed videos, and HACS Segments contains 139K action segments densely annotated in 50K untrimmed videos spanning 200 action categories. HACS Clips contains more labeled examples than any existing video benchmark. This renders our dataset both a large-scale action recognition benchmark and an excellent source for spatiotemporal feature learning. In our transfer learning experiments on three target datasets, HACS Clips outperforms Kinetics-600, Moments-In-Time and Sports1M as a pretraining source. On HACS Segments, we evaluate state-of-the-art methods of action proposal generation and action localization, and highlight the new challenges posed by our dense temporal annotations.
Efficient Classification of White Blood Cell Leukemia with Improved Swarm Optimization of Deep Features

Abstract White Blood Cell (WBC) Leukaemia is caused by excessive production of leukocytes in the bone marrow, and image-based detection of malignant WBCs is important for its detection. Convolutional Neural Networks (CNNs) present the current state-of-the-art for this type of image classification, but their computational cost for training and deployment can be high. We here present an improved hybrid approach for efficient classification of WBC Leukemia. We first extract features from WBC images using VGGNet, a powerful CNN architecture, pre-trained on ImageNet. The extracted features are then filtered using a statistically enhanced Salp Swarm Algorithm (SESSA). This bio-inspired optimization algorithm selects the most relevant features and removes highly correlated and noisy features. We applied the proposed approach to two public WBC Leukemia reference datasets and achieve both high accuracy and reduced computational complexity. The SESSA optimization selected only 1 K out of 25 K features extracted with VGGNet, while improving accuracy at the same time. The results are among the best achieved on these datasets and outperform several convolutional network models. We expect that the combination of CNN feature extraction and SESSA feature optimization could be useful for many other image classification tasks.
Evaluation of deep learning detection and classification towards computer-aided diagnosis of breast lesions in digital X-ray mammograms

Deep learning detection and classification from medical imagery are key components for computer-aided diagnosis (CAD) systems to efficiently support physicians leading to an accurate diagnosis of breast lesions. In this study, an integrated CAD system of deep learning detection and classification is proposed aiming to improve the diagnostic performance of breast lesions. First, a deep learning YOLO detector is adopted and evaluated for breast lesion detection from entire mammograms. Then, three deep learning classifiers, namely regular feedforward CNN, ResNet-50, and InceptionResNet-V2, are modified and evaluated for breast lesion classification. The proposed deep learning system is evaluated over 5-fold cross-validation tests using two different and widely used databases of digital X-ray mammograms: DDSM and INbreast. The evaluation results of breast lesion detection show the capability of the YOLO detector to achieve overall detection accuracies of 99.17% and 97.27% and F1-scores of 99.28% and 98.02% for DDSM and INbreast datasets, respectively. Meanwhile, the YOLO detector could predict 71 frames per second (FPS) at the testing time for both DDSM and INbreast datasets. Using detected breast lesions, the classification models of CNN, ResNet-50, and InceptionResNet-V2 achieve promising average overall accuracies of 94.50%, 95.83%, and 97.50%, respectively, for the DDSM dataset and 88.74%, 92.55%, and 95.32%, respectively, for the INbreast dataset. The capability of the YOLO detector boosted the classification models to achieve a promising breast lesion diagnostic performance. Such prediction results should help to develop a feasible CAD system for practical breast cancer diagnosis.