Reviewerly/exHarmony huggingface dataset:
Deep 360$$^\circ $$ Optical Flow Estimation Based on Multi-projection Fusion

Optical flow computation is essential in the early stages of the video processing pipeline. This paper focuses on a less explored problem in this area, the 360 $$^\circ $$ optical flow estimation using deep neural networks to support increasingly popular VR applications. To address the distortions of panoramic representations when applying convolutional neural networks, we propose a novel multi-projection fusion framework that fuses the optical flow predicted by the models trained using different projection methods. It learns to combine the complementary information in the optical flow results under different projections. We also build the first large-scale panoramic optical flow dataset to support the training of neural networks and the evaluation of panoramic optical flow estimation methods. The experimental results on our dataset demonstrate that our method outperforms the existing methods and other alternative deep networks that were developed for processing 360 $$^\circ $$ content.
REX: Reasoning-aware and Grounded Explanation

Effectiveness and interpretability are two essential properties for trustworthy AI systems. Most recent studies in visual reasoning are dedicated to improving the accuracy of predicted answers, and less attention is paid to explaining the rationales behind the decisions. As a result, they commonly take advantage of spurious biases instead of actually reasoning on the visual-textual data, and have yet developed the capability to explain their decision making by considering key information from both modalities. This paper aims to close the gap from three distinct perspectives: first, we define a new type of multi-modal explanations that explain the decisions by progressively traversing the reasoning process and grounding keywords in the images. We develop a functional program to sequentially ex-ecute different reasoning steps and construct a new dataset with 1,040,830 multi-modal explanations. Second, we iden-tify the critical need to tightly couple important components across the visual and textual modalities for explaining the decisions, and propose a novel explanation generation method that explicitly models the pairwise correspon-dence between words and regions of interest. It improves the visual grounding capability by a considerable margin, resulting in enhanced interpretability and reasoning performance. Finally, with our new data and method, we perform extensive analyses to study the effectiveness of our explanation under different settings, including multi-task learning and transfer learning. Our code and data are available at https://github.com/szzexpoi/rex.
Lazy Rearrangement Planning in Confined Spaces

Object rearrangement is important for many applications but remains challenging, especially in confined spaces, such as shelves, where objects cannot be accessed from above and they block reachability to each other. Such constraints require many motion planning and collision checking calls, which are computationally expensive. In addition, the arrangement space grows exponentially with the number of objects. To address these issues, this work introduces a lazy evaluation framework with a local monotone solver and a global planner. Monotone instances are those that can be solved by moving each object at most once. A key insight is that reachability constraints at the grasps for objects' starts and goals can quickly reveal dependencies between objects without having to execute expensive motion planning queries. Given that, the local solver builds lazily a search tree that respects these reachability constraints without verifying that the arm paths are collision free. It only collision checks when a promising solution is found. If a monotone solution is not found, the non-monotone planner loads the lazy search tree and explores ways to move objects to intermediate locations from where monotone solutions to the goal can be found. Results show that the proposed framework can solve difficult instances in confined spaces with up to 16 objects, which state-of-the-art methods fail to solve. It also solves problems faster than alternatives, when the alternatives find a solution. It also achieves high-quality solutions, i.e., only 1.8 additional actions on average are needed for non-monotone instances.
Multi-View Transformer for 3D Visual Grounding

The 3D visual grounding task aims to ground a natural language description to the targeted object in a 3D scene, which is usually represented in 3D point clouds. Previous works studied visual grounding under specific views. The vision-language correspondence learned by this way can easily fail once the view changes. In this paper, we propose a Multi-View Transformer (MVT) for 3D visual grounding. We project the 3D scene to a multi-view space, in which the position information of the 3D scene under different views are modeled simultaneously and aggregated together. The multi-view space enables the network to learn a more robust multi-modal representation for 3D visual grounding and eliminates the dependence on specific views. Extensive experiments show that our approach significantly outperforms all state-of-the-art methods. Specifically, on Nr3D and Sr3D datasets, our method outperforms the best competitor by 11.2% and 7.1% and even surpasses recent work with extra 2D assistance by 5.9% and 6.6%. Our code is available at https://github.com/sega-hsj/MVT-3DVG.
Open Vocabulary Object Detection with Pseudo Bounding-Box Labels

Despite great progress in object detection, most existing methods work only on a limited set of object categories, due to the tremendous human effort needed for bounding-box annotations of training data. To alleviate the problem, recent open vocabulary and zero-shot detection methods attempt to detect novel object categories beyond those seen during training. They achieve this goal by training on a pre-defined base categories to induce generalization to novel objects. However, their potential is still constrained by the small set of base categories available for training. To enlarge the set of base classes, we propose a method to automatically generate pseudo bounding-box annotations of diverse objects from large-scale image-caption pairs. Our method leverages the localization ability of pre-trained vision-language models to generate pseudo bounding-box labels and then directly uses them for training object detectors. Experimental results show that our method outperforms the state-of-the-art open vocabulary detector by 8% AP on COCO novel categories, by 6.3% AP on PASCAL VOC, by 2.3% AP on Objects365 and by 2.8% AP on LVIS. .
Building Predictive Model by Using Data Mining and Feature Selection Techniques on Academic Dataset

In the field of education, every institution stores a significant amount of data in digital form on the academic performance of students.If this data is correctly analysed to discover any pattern related to student learning, it can assist the institution in achieving a favorable outcome in the future.Because of this, the use of data mining techniques makes it much simpler to unearth previously concealed information or detect patterns in student data.We use a variety of data mining methods, such as Naive Bayes, Random Forest, Decision Tree, Multilayer Perceptron, and Decision Table, to predict the academic performance of individual students.In the real world, a dataset may contain many features, yet the mining process may only place significance on some of those aspects.The correlation attribute evaluator, the information gain attribute evaluator, and the gain ratio attribute evaluator are some of the feature selection methods that are used in data mining to remove features that are not important for the mining process.Other feature selection methods include the gain ratio attribute evaluator and the gain ratio attribute evaluator.In conclusion, each classification algorithm that is designed using some feature selection methods enhances the overall predictive performance of the algorithms, which in turn improves the performance of the algorithms overall.
Reporting and Interpreting Multivariate Analysis of Variance (MANOVA): Adopting the Best Practices in Educational Research

There is seemingly anonymity and observation among consumers of educational research that multivariate analysis of variance (MANOVA) is one of the most predominant and commonly used statistical models for the analysis of data in the discipline of education. Despite the large use of MANOVA in research studies, most research practitioners within the educational research trail still face difficulties in reporting and meaningfully interpreting MANOVA results. In analysis, multivariate analysis of variance tests for two or more independent variables and two or more dependent variables. In practice, the one-way multivariate analysis of variance is used to determine whether there are any differences between independent groups on more than one continuous dependent variable. This paper provides a data driven example of reporting and interpreting presentation of multivariate analysis of variance for consumers of research by capitalizing on how it can be reported and interpreted using the APA format. In the paper, the researcher used an example data-driven set throughout the write up to illustrate how MANOVA is reported and interpreted in educational studies. The paper emphasized that if researchers do not run the statistical tests on assumptions of using MANOVA correctly, the accrued results that will be might not be valid. These could therefore have structural effects on the conclusions and implications that will be drawn from the analysis. The paper concludes with remarks that throws more lights and emphasis on the relevant and consequent implications of using MANOVA in educational studies.
Visual Search in Augmented Reality: Effect of Target Cue Type and Location

The current experiment examined the relative benefit of different cueing aids during a visual target search task, and the tradeoff between reduced information access effort and increased overlay clutter. Using an augmented reality head-mounted display (AR-HMD), participants completed a 180-degree visual search task with three different cue types (world-referenced arrow, screen-referenced icon target image, and screen-referenced minimap) compared to a control condition (no cueing aid). Target cues differed in terms of display proximity and where they were presented on the AR-HMD (the central field of view or approximately 15 degrees downward from the center). We found an overall performance benefit when searching for an object in the far domain with a target cue compared to searching with no cue, and the arrow cue (highest display proximity) showed the greatest overall benefit. We also found a performance benefit for cues located at the center of the AR-HMD compared to the downward location, but this benefit was offset by the higher clutter of the icon image and the minimap. These findings suggest that target cues with higher display proximity that also reduce information access effort (scanning) may be more suitable cueing aids when searching for an object in the far domain.
Rearrangement-Based Manipulation via Kinodynamic Planning and Dynamic Planning Horizons

Robot manipulation in cluttered environments of-ten requires complex and sequential rearrangement of multiple objects in order to achieve the desired reconfiguration of the target objects. Due to the sophisticated physical interactions involved in such scenarios, rearrangement-based manipulation is still limited to a small range of tasks and is especially vulnerable to physical uncertainties and perception noise. This paper presents a planning framework that leverages the efficiency of sampling-based planning approaches, and closes the manipulation loop by dynamically controlling the planning horizon. Our approach interleaves planning and execution to progressively approach the manipulation goal while correcting any errors or path deviations along the process. Meanwhile, our framework allows the definition of manipulation goals without requiring explicit goal configurations, enabling the robot to flexibly interact with all objects to facilitate the manipulation of the target ones. With extensive experiments both in simulation and on a real robot, we evaluate our framework on three manipulation tasks in cluttered environments: grasping, relocating, and sorting. In comparison with two baseline approaches, we show that our framework can significantly improve planning efficiency, robustness against physical uncertainties, and task success rate under limited time budgets.
Voice Expression System of Visual Environment for a Guide Dog Robot

In this paper, a voice expression system of visual environment for a guide dog robot is proposed. This system enables the robot to recognize objects and scenes in front of the robot by CNN (Convolutional Neural Network) and generate captions of the scenes by LSTM (Long-Short-Term Memory) network. Then the robot expresses the recognized visual scene by voice, which is generated by speech synthesis. The guide dog robot can guide the visually impaired person safely to the desired destination, as well as entertain the user by expressing various visual information through voice. The system is composed of object recognition, scene caption, and speech synthesis. The effectiveness of this system is confirmed through experiments conducted with our guide dog robot.
Language Matters: A Weakly Supervised Vision-Language Pre-training Approach for Scene Text Detection and Spotting

Recently, Vision-Language Pre-training (VLP) techniques have greatly benefited various vision-language tasks by jointly learning visual and textual representations, which intuitively helps in Optical Character Recognition (OCR) tasks due to the rich visual and textual information in scene text images. However, these methods cannot well cope with OCR tasks because of the difficulty in both instance-level text encoding and image-text pair acquisition (i.e. images and captured texts in them). This paper presents a weakly supervised pre-training method, oCLIP, which can acquire effective scene text representations by jointly learning and aligning visual and textual information. Our network consists of an image encoder and a character-aware text encoder that extract visual and textual features, respectively, as well as a visual-textual decoder that models the interaction among textual and visual features for learning effective scene text representations. With the learning of textual features, the pre-trained model can attend texts in images well with character awareness. Besides, these designs enable the learning from weakly annotated texts (i.e. partial texts in images without text bounding boxes) which mitigates the data annotation constraint greatly. Experiments over the weakly annotated images in ICDAR2019-LSVT show that our pre-trained model improves F-score by +2.5% and +4.8% while transferring its weights to other text detection and spotting networks, respectively. In addition, the proposed method outperforms existing pre-training techniques consistently across multiple public datasets (e.g., +3.2% and +1.3% for Total-Text and CTW1500).
Exact matching of random graphs with constant correlation

This paper deals with the problem of graph matching or network alignment for Erdős–Rényi graphs, which can be viewed as a noisy average-case version of the graph isomorphism problem. Let G and $$G'$$ be G(n, p) Erdős–Rényi graphs marginally, identified with their adjacency matrices. Assume that G and $$G'$$ are correlated such that $${\mathbb {E}}[G_{ij} G'_{ij}] = p(1-\alpha )$$ . For a permutation $$\pi $$ representing a latent matching between the vertices of G and $$G'$$ , denote by $$G^\pi $$ the graph obtained from permuting the vertices of G by $$\pi $$ . Observing $$G^\pi $$ and $$G'$$ , we aim to recover the matching $$\pi $$ . In this work, we show that for every $$\varepsilon \in (0,1]$$ , there is $$n_0>0$$ depending on $$\varepsilon $$ and absolute constants $$\alpha _0, R > 0$$ with the following property. Let $$n \ge n_0$$ , $$(1+\varepsilon ) \log n \le np \le n^{\frac{1}{R \log \log n}}$$ , and $$0< \alpha < \min (\alpha _0,\varepsilon /4)$$ . There is a polynomial-time algorithm F such that $${\mathbb {P}}\{F(G^\pi ,G')=\pi \}=1-o(1)$$ . This is the first polynomial-time algorithm that recovers the exact matching between vertices of correlated Erdős–Rényi graphs with constant correlation with high probability. The algorithm is based on comparison of partition trees associated with the graph vertices.
A STUDY ON GENETIC ALGORITHM AND ITS APPLICATIONS

We need a way to differentiate the best and worse solutions in order to get the best outcomes.A statistical model or simulation could be the metric, or it could be a subjective one where we favor better results over worse ones.In addition, the fitness function chooses a top solution for a particular issue, which is then used by the GA to direct the evolution of other top solutions.The use of genetic algorithms in many fields is demonstrated in this study along with how they may be integrated with a number of other approaches to arrive at the best possible solution and speed up information retrieval systems.
Success probability and performance optimization in non-Gaussian continuous-variable quantum teleportation

Non-Gaussian operations have been shown to enhance the fidelity of continuous-variable quantum teleportation. However, a disadvantage of these non-Gaussian operations is that they are probabilistic in nature. In this article we study the trade-off between teleportation fidelity and success probability for optimal performance of the teleportation protocol. To this end, we first derive a unified expression for the Wigner characteristic function describing three non-Gaussian states: photon-subtracted, photon-added, and photon-catalyzed two-mode squeezed vacuum states. We then utilize it to obtain the fidelity of teleportation for input coherent and squeezed vacuum states using the aforementioned non-Gaussian resource states. We optimize the product of the relative enhancement in fidelity and the probability of state preparation by tuning the transmissivity of the beam splitters involved in implementing non-Gaussian operations. This leads to a scenario that can be effectively implemented in a laboratory to enhance fidelity. It turns out that among all the non-Gaussian resource states considered, the symmetric one-photon-subtracted two-mode squeezed vacuum state is the most advantageous. We provide the associated optimal squeezing and beam splitter transmissivity values for the non-Gaussian resource states considered, which will be of significant interest to the experimental community. We also consider the effect of imperfect photon detectors on teleportation fidelity. Further, we expect the derived Wigner characteristic function to be useful in state characterization and other quantum information processing protocols.
Prostate lesion segmentation based on a 3D end-to-end convolution neural network with deep multi-scale attention

Prostate cancer is one of the deadest cancers among human beings. To better diagnose the prostate cancer, prostate lesion segmentation becomes a very important work, but its progress is very slow due to the prostate lesions small in size, irregular in shape, and blurred in contour. Therefore, automatic prostate lesion segmentation from mp-MRI is a great significant work and a challenging task. However, the most existing multi-step segmentation methods based on voxel-level classification are time-consuming, may introduce errors in different steps and lead to error accumulation. To decrease the computation time, harness richer 3D spatial features, and fuse the multi-level contextual information of mp-MRI, we present an automatic segmentation method in which all steps are optimized conjointly as one step to form our end-to-end convolutional neural network. The proposed end-to-end network DMSA-V-Net consists of two parts: (1) a 3D V-Net is used as the backbone network, it is the first attempt in employing 3D convolutional neural network for CS prostate lesion segmentation, (2) a deep multi-scale attention mechanism is introduced into the 3D V-Net which can highly focus on the ROI while suppressing the redundant background. As a merit, the attention can adaptively re-align the context information between the feature maps at different scales and the saliency maps in high-levels. We performed experiments based on five cross-fold validation with data including 97 patients. The results show that the Dice and sensitivity are 0.7014 and 0.8652 respectively, which demonstrates that our segmentation approach is more significant and accurate compared to other methods.
Enhancement Technique Based on the Breast Density Level for Mammogram for Computer-Aided Diagnosis

Mass detection in mammograms has a limited approach to the presence of a mass in overlapping denser fibroglandular breast regions. In addition, various breast density levels could decrease the learning system’s ability to extract sufficient feature descriptors and may result in lower accuracy performance. Therefore, this study is proposing a textural-based image enhancement technique named Spatial-based Breast Density Enhancement for Mass Detection (SbBDEM) to boost textural features of the overlapped mass region based on the breast density level. This approach determines the optimal exposure threshold of the images’ lower contrast limit and optimizes the parameters by selecting the best intensity factor guided by the best Blind/Reference-less Image Spatial Quality Evaluator (BRISQUE) scores separately for both dense and non-dense breast classes prior to training. Meanwhile, a modified You Only Look Once v3 (YOLOv3) architecture is employed for mass detection by specifically assigning an extra number of higher-valued anchor boxes to the shallower detection head using the enhanced image. The experimental results show that the use of SbBDEM prior to training mass detection promotes superior performance with an increase in mean Average Precision (mAP) of 17.24% improvement over the non-enhanced trained image for mass detection, mass segmentation of 94.41% accuracy, and 96% accuracy for benign and malignant mass classification. Enhancing the mammogram images based on breast density is proven to increase the overall system’s performance and can aid in an improved clinical diagnosis process.