Reviewerly/exHarmony huggingface dataset:
A Preliminary Work: Mixed Reality-Integrated Computer-Aided Surgical Navigation System for Paranasal Sinus Surgery Using Microsoft HoloLens 2

Paranasal sinus surgery has high demands for minimal invasion and safety. Computer-aided surgical navigation (CSN) applications have been recognized as the standard of the surgical practice; the operations of a user can be guided with visually complementary data such as preoperative medical imaging. The introduction of new innovation from mixed reality head mounted display (MR-HMD) technologies is a promising research direction for enhanced usability of paranasal sinus CSN applications. The combined use of MR-HMD with CSN provides a physically unified environment where a user's field of view in intraoperative sites can be augmented with complementary preoperative data, thereby enhancing their situational awareness. In this study, we present an early phase of the MR introduction for paranasal sinus surgery. We developed an alpha version of a commercial paranasal sinus CSN application using 3D Slicer, a dominant open-source clinical software development platform, and then implemented a scene sharing extension module. We refer to it as MR-CSN system. It enables a user wearing MR-HMD networked to equip with their MR-enhanced navigation; their navigation using surgical instruments in the intraoperative sites can be aided with the real-time information from the CSN application. The feasibility of our MR-CSN system was evaluated by experimenting a paranasal sinus surgical simulation with a phantom model.
Computationally-Efficient Roadmap-based Inspection Planning via Incremental Lazy Search

The inspection-planning problem calls for computing motions for a robot that allow it to inspect a set of points of interest (POIs) while considering plan quality (e.g., plan length). This problem has applications across many domains where robots can help with inspection, including infrastructure maintenance, construction, and surgery. Incremental Random Inspection-roadmap Search (IRIS) is an asymptotically-optimal inspection planner that was shown to compute higher-quality inspection plans orders of magnitudes faster than the prior state-of-the-art method. In this paper, we significantly accelerate the performance of IRIS to broaden its applicability to more challenging real-world applications. A key computational challenge that IRIS faces is effectively searching roadmaps for inspection plans—a procedure that dominates its running time. In this work, we show how to incorporate lazy edge-evaluation techniques into IRIS’s search algorithm and how to reuse search efforts when a roadmap undergoes local changes. These enhancements, which do not compromise IRIS’s asymptotic optimality, enable us to compute inspection plans much faster than the original IRIS. We apply IRIS with the enhancements to simulated bridge inspection and surgical inspection tasks and show that our new algorithm for some scenarios can compute similar-quality inspection plans 570× faster than prior work.
Region of Interest Extraction Based on Unsupervised Cross-Domain Adaptation for Remote Sensing Images

Extracting region of interest (ROI) plays an important role in many computer vision tasks. Recently, deep methods have shown excellent performance, however, when it comes to remote sensing image (RSI) domain, which lacks pixel-level annotations, training often leads to under-fitting and low-accuracy. In this paper, we propose a novel ROI extraction model based on unsupervised cross-domain adaptation for RSIs. Firstly, we pretrain the network, RS- RoINet, by large-scale natural datasets to learn general features. Through top-down propagation mechanism, we combine global and local information to generate the accurate edge of extraction maps. Then, we introduce domain adaptation module to reduce the difference between natural domain and RSI domain. Data from both domains is transferred into Reproducing Kernel Hilbert Space to measure the domain distribution distance. Finally, the model is adaptive for RSIs and extracts ROI more accurately. Compared with recent fully-supervised state-of-the-arts, our unsupervised method shows outstanding performance.
Improving Key Human Features for Pose Transfer

It is still a great challenge in the Pose Transfer task to generate visually coherent images, to preserve the texture of clothes, to maintain the source identity and to realistically generate key human features such as the face or the hands. To tackle these challenges, we first conduct a study to obtain the most robust conditioning labels for this task and the baseline method [44] that we choose. We then improve upon the baseline by including deep source features from an Auto-encoder through an Attention mechanism. Finally we add region discriminators that are focused on key human features, thus obtaining results competitive with the state-of-the-art.
A multi-scale patch-wise algorithm for multi-exposure image fusion

Multi-exposure fusion is a high dynamic range image generation method. The challenge it faces is to reconstruct artifact-free high dynamic range images for dynamic scenes containing moving objects. This paper proposes a structural patch decomposition algorithm based on the Laplacian pyramid to obtain high dynamic range images without artifacts. This paper discusses the relationship between the total exposure quality score and the number of poorly exposed pixels and proposes a robust reference image selection method. It avoided the local exposure distortion in the fused image caused by the improper selection of reference images and improved the algorithm's fusion effect in dynamic scenes. More importantly, this paper inject a new fusion mechanism into the conventional Laplacian pyramid fusion framework, namely the non-standardized structural patch decomposition algorithm. While retaining its original advantages, it reduces the computational complexity of the algorithm. Experimental results on dynamic and static scenes show that the proposed algorithm can generate fused images with a clear structure and almost no halo artifacts.
A Self-supervised Learning System for Object Detection in Videos Using Random Walks on Graphs

This paper presents a new self-supervised system for learning to detect novel and previously unseen categories of objects in images. The proposed system receives as input several unlabeled videos of scenes containing various objects. The frames of the videos are segmented into objects using depth information, and the segments are tracked along each video. The system then constructs a weighted graph that connects sequences based on the similarities between the objects that they contain. The similarity between two sequences of objects is measured by using generic visual features, after automatically re-arranging the frames in the two sequences to align the viewpoints of the objects. The graph is used to sample triplets of similar and dissimilar examples by performing random walks. The triplet examples are finally used to train a siamese neural network that projects the generic visual features into a low-dimensional manifold. Experiments on three public datasets, YCB-Video, CORe50 and RGBD-Object, show that the projected low-dimensional features improve the accuracy of clustering unknown objects into novel categories, and outperform several recent unsupervised clustering techniques.
Declarative Approach to UAVs Mission Contingency Planning in Dynamic Environments

This paper presents a novel approach to the joint proactive and reactive planning of the deliveries by a UAVs’ fleet. We develop a receding horizon based approach to a contingency planning for the UAVs’ fleet mission. We considered the delivery of goods to spatially dispersed customers, over assumed time horizon. In order to take into account forecasted weather changes which affect the energy consumption of UAVs and limit their range we propose a set of reaction rules that can be encountered during delivery in a highly dynamic and unpredictable environment. These rules are used in course of the contingency plans design related to the need to implement an emergency return of the UAV to the base or handling of ad hoc ordered deliveries. Due to nonlinearity of environment’s characteristics a constraint programming paradigm has been implemented and due to the NP-difficult nature of the considered planning problem, conditions have been developed that allow for the acceleration of calculations. The computational experiments have shown that the developed model is capable of providing feasible plans contingency plans of UAVs’ mission performed in dynamic environment.
Label Propagation Across Graphs: Node Classification Using Graph Neural Tangent Kernels

Graph neural networks (GNNs) have achieved superior performance on node classification tasks in the last few years. Commonly, this is framed in a transductive semi-supervised learning setup wherein the entire graph – including the target nodes to be labeled – is available for training. Driven in part by scalability, recent works have focused on the inductive case where only the labeled portion of a graph is available for training. In this context, our current work considers a challenging inductive setting where a set of labeled graphs are available for training while the unlabeled target graph is completely separate, i.e., there are no connections between labeled and unlabeled nodes. Under the implicit assumption that the testing and training graphs come from similar distributions, our goal is to develop a labeling function that generalizes to unobserved connectivity structures. To that end, we employ a graph neural tangent kernel (GNTK) that corresponds to infinitely wide GNNs to find correspondences between nodes in different graphs based on both the topology and the node features. We augment the capabilities of the GNTK with residual connections and empirically illustrate its performance gains on standard benchmarks.
Self-supervised extractive text summarization for biomedical literatures

In this study, we propose a self-supervised approach to extractive text summarization for biomedical literature. The approach uses abstracts to find the most informative content in the article, then generate a summary for training a classification model. The Sentences in the abstract and literature were first embedded using BERT. A similarity-based model was then applied to label the informative sentences for training the classifier. We used logistic regression as our classification model and used the features of sentence embedding for the classification. The results showed the feasibility of employing the abstract to perform self-supervised training of a classification model to generate extractive summarization. This approach can enable automatic generation of one or two-page executive summaries of biomedical literature to keep clinicians and biomedical researchers up to date with the latest development
Improving Model Generalization by Agreement of Learned Representations from Data Augmentation

Data augmentation reduces the generalization error by forcing a model to learn invariant representations given different transformations of the input image. In computer vision, on top of the standard image processing functions, data augmentation techniques based on regional dropout such as CutOut, MixUp, and CutMix and policy-based selection such as AutoAugment demonstrated state-of-the-art (SOTA) results. With an increasing number of data augmentation algorithms being proposed, the focus is always on optimizing the input-output mapping while not realizing that there might be an untapped value in the transformed images with the same label. We hypothesize that by forcing the representations of two transformations to agree, we can further reduce the model generalization error. We call our proposed method Agreement Maximization or simply AgMax. With this simple constraint applied during training, empirical results show that data augmentation algorithms can further improve the classification accuracy of ResNet50 on ImageNet by up to 1.5%, WideResNet40-2 on CIFAR10 by up to 0.7%, WideResNet40-2 on CIFAR100 by up to 1.6%, and LeNet5 on Speech Commands Dataset by up to 1.4%. Experimental results further show that unlike other regularization terms such as label smoothing, AgMax can take advantage of the data augmentation to consistently improve model generalization by a significant margin. On downstream tasks such as object detection and segmentation on PascalVOC and COCO, AgMax pre-trained models outperforms other data augmentation methods by as much as 1.0mAP (box) and 0.5mAP (mask). Code is available at https://github.com/roatienza/agmax.
Deep Semi-Supervised Algorithm for Learning Cluster-Oriented Representations of Medical Images Using Partially Observable DICOM Tags and Images

The task of automatically extracting large homogeneous datasets of medical images based on detailed criteria and/or semantic similarity can be challenging because the acquisition and storage of medical images in clinical practice is not fully standardised and can be prone to errors, which are often made unintentionally by medical professionals during manual input. In this paper, we propose an algorithm for learning cluster-oriented representations of medical images by fusing images with partially observable DICOM tags. Pairwise relations are modelled by thresholding the Gower distance measure which is calculated using eight DICOM tags. We trained the models using 30,000 images, and we tested them using a disjoint test set consisting of 8000 images, gathered retrospectively from the PACS repository of the Clinical Hospital Centre Rijeka in 2017. We compare our method against the standard and deep unsupervised clustering algorithms, as well as the popular semi-supervised algorithms combined with the most commonly used feature descriptors. Our model achieves an NMI score of 0.584 with respect to the anatomic region, and an NMI score of 0.793 with respect to the modality. The results suggest that DICOM data can be used to generate pairwise constraints that can help improve medical images clustering, even when using only a small number of constraints.
"There Should Be More Than One Voice in A Healthy Society": Infrastructural Violence and Totalitarian Computing in China

During the COVID-19 pandemic, Chinese citizens were caught up within a public health crisis and an expansive censorship and propaganda infrastructure built by the Chinese government over the past few decades. In March 2020, we interviewed 24 Chinese citizens to understand how they experienced censorship and propaganda during the first few months of the pandemic. Drawing from the lens of infrastructural violence, we document state violence inflicted on the Chinese people at the infrastructural scale, penetrating ordinary citizens' minds and daily lives. We examine three dimensions of infrastructural violence: epistemic violence of truth distortion, emotional violence of state retaliation, and interpersonal violence of relationship deterioration. Based on our empirical findings, we reflect on how infrastructural violence is constituted through totalitarian computing technologies. We stress the need to study the geopolitics of technology and propose a research agenda for studying totalitarian computing.
Deep Unsupervised 3D SfM Face Reconstruction Based on Massive Landmark Bundle Adjustment

We address the problem of reconstructing 3D human face from multi-view facial images using Structure-from-Motion (SfM) based on deep neural networks. While recent learning-based monocular view methods have shown impressive results for 3D facial reconstruction, the single-view setting is easily affected by depth ambiguities and poor face pose issues. In this paper, we propose a novel unsupervised 3D face reconstruction architecture by leveraging the multi-view geometry constraints to train accurate face pose and depth maps. Facial images from multiple perspectives of each 3D face model are input to train the network. Multi-view geometry constraints are fused into unsupervised network by establishing loss constraints from spatial and spectral perspectives. To make the trained 3D face have more details, facial landmark detector is explored to acquire massive facial information to constrain face pose and depth estimation. Through minimizing massive landmark displacement distance by bundle adjustment, an accurate 3D face model can be reconstructed. Extensive experiments demonstrate the superiority of our proposed approach over other methods.
Effect of water flow on energy matrices and life-cycle cost analysis of urban BiSPVT system

The Energy Payback Time (EPBT) and Life-cycle Cost Analysis (LCCA) of a building integrated semi-transparent photovoltaic thermal (BiSPVT) system are carried out for different climatic and geographic zones. These two performance criteria are adversely affected by the shadow of the neighborhood's buildings and obstacles. For countering the shadow effect, the BiSPVT system is proposed to install with optimum tilt and azimuth angle. The relative height, plan orientation, plan area, position, and horizontal distance of neighboring buildings are critical parameters in determining these angles to generate maximum energy. The proposed thermal model accurately calculates these angles and the output energy under various weather conditions at different places. The model assumes a water flow over the BIPV panel from an overhead tank, in view of increasing the efficiency of PV panel. The periodic nature of relevant parameters is considered in the thermal modeling of the BiSPVT system. The cost of energy produced by the proposed system ranges between 1.12 and 2.03 US$/ kWh depending on the climatic conditions of the place, and the EPBT of the system is found to be ranging between 11.6 years to 20.1 years which is lower than the expected service life (30 years) of the modules.
Detecting Internet-Scale Surveillance Devices Using RTSP Recessive Features

In recent years, fingerprinting online surveillance devices has been a hot research topic. However, large-scale devices still can not be identified their brands in previous studies and mainstream search engines. In this work, we propose a novel neural network-based approach for automatically discovering surveillance devices and identifying their brands in cyberspace. Moreover, by using the deep semi-supervised learning algorithm, the most unlabeled samples with new-explored recessive features can be learned of RTSP protocol. In the global IPv4 space, we implement an evaluation on 3, 123, 489 active RTSP-hosts for training and testing. The experimental results demonstrate our approach can discover 2, 803, 406 surveillance devices, which are eight times and three times more than those discovered by Shodan and Zoomeye. Moreover, the number of identified brand-level devices by our approach is 2, 457, 661 devices with their brands, which is at least four times more than existing methods. The performance of these results with precision and recall can both achieve \(93\%\).
An efficient FPGA-based co-processor for feature point detection and tracking

The use of mobile agents is propagating throughout various industries. Nevertheless, the success of novel applications relies on the utilization of novel computing platforms and algorithms, including acceleration technology and onboard localization. We propose an FPGA-based sparse optical flow computing accelerator based on the FAST feature detection and BRIEF feature descriptor. The correspondences are found by splitting the image into static regions, where for each region, the feature points are tracked in-between the frames. The accelerator is fully pipelined and achieves a performance of 300 fps with VGA resolution images. The experimentation with the default configuration of the accelerator shows to support a reliable measurement of frame-to-frame image plane rotation of 9 degrees and translation of 24 pixels, with the total error below 0.4 degrees and 0.16 pixels.
Efficient Reinforcement Learning Development with RLzoo

Many multimedia developers are exploring for adopting Deep Reinforcement Learning (DRL) techniques in their applications. They however often find such an adoption challenging. Existing DRL libraries provide poor support for prototyping DRL agents (i.e., models), customising the agents, and comparing the performance of DRL agents. As a result, the developers often report low efficiency in developing DRL agents. In this paper, we introduce RLzoo, a new DRL library that aims to make the development of DRL agents efficient. RLzoo provides developers with (i) high-level yet flexible APIs for prototyping DRL agents, and further customising the agents for best performance, (ii) a model zoo where users can import a wide range of DRL agents and easily compare their performance, and (iii) an algorithm that can automatically construct DRL agents with custom components (which are critical to improve agent's performance in custom applications). Evaluation results show that RLzoo can effectively reduce the development cost of DRL agents, while achieving comparable performance with existing DRL libraries.
Offline Handwritten MODI Character Recognition Using GoogLeNet and AlexNet

“MODI lipi” is one of the scripts used to write religious scriptures of Maharashtra in Western India and it was also the official script for the Maratha administration from the 17th century to the middle of the 20th century. This cultural treasure, “MODI-manuscript,” speaks about the history of its time. Although it has immense importance as a source of inspiration and information to the present generation, very few people know this “lipi”. The field of Handwritten Character Recognition offers a scope to develop a recognition system for MODI to make it easy to learn. However its structural characteristics demand a special approach. Deep Convolutional Neural Networks (DCNN) has shown their remarkable potential in distinct feature extraction and classification of characters. So, in this paper we are focusing primarily on the performance evaluation of DCNN and their comparative study for MODI handwritten character recognition. Networks are evaluated based on trainable parameters, training time and memory consumption. Later, the tuned networks are also tested for transformed MODI dataset. The result shows the effectiveness of deep learning approach on Handwritten MODI character recognition.
SMT-Based Optimal Deployment of Mobile Rechargers

Efficient recharging is an essential requirement for autonomous mobile robots. In an indoor robotic application, charging stations can be installed offline. However, frequent trips to the charging stations cause inefficiency in the performance of the mobile robots. In an outdoor environment, a charging station cannot even be installed easily. We propose a framework and algorithms for enabling a group of mobile wireless rechargers to fulfill the energy requirement of autonomous mobile robots in a workspace efficiently. Our algorithm finds the optimal trajectories for the mobile rechargers in such a way that once there is a need for a recharge, the robots do not need to spend significant time and energy to get access to a recharger. Our algorithm is based on a reduction of the problems to Satisfiability Modulo Theory (SMT) solving problems. We present extensive experimental results to show that the optimal trajectories for mobile rechargers can be generated successfully for different types of robots and workspaces within a reasonable time. Moreover, a comparison with the performance of static charging stations establishes that mobile rechargers are more effective in terms of allowing the autonomous robot to continue their work for a longer time.