Reviewerly/exHarmony huggingface dataset:
Temporal Structure Mining for Weakly Supervised Action Detection

Different from the fully-supervised action detection problem that is dependent on expensive frame-level annotations, weakly supervised action detection (WSAD) only needs video-level annotations, making it more practical for real-world applications. Existing WSAD methods detect action instances by scoring each video segment (a stack of frames) individually. Most of them fail to model the temporal relations among video segments and cannot effectively characterize action instances possessing latent temporal structure. To alleviate this problem in WSAD, we propose the temporal structure mining (TSM) approach. In TSM, each action instance is modeled as a multi-phase process and phase evolving within an action instance, \emph{i.e.}, the temporal structure, is exploited. Meanwhile, the video background is modeled by a background phase, which separates different action instances in an untrimmed video. In this framework, phase filters are used to calculate the confidence scores of the presence of an action's phases in each segment. Since in the WSAD task, frame-level annotations are not available and thus phase filters cannot be trained directly. To tackle the challenge, we treat each segment's phase as a hidden variable. We use segments' confidence scores from each phase filter to construct a table and determine hidden variables, i.e., phases of segments, by a maximal circulant path discovery along the table. Experiments conducted on three benchmark datasets demonstrate the state-of-the-art performance of the proposed TSM.
ConvPath: A software tool for lung adenocarcinoma digital pathological image analysis aided by a convolutional neural network

The spatial distributions of different types of cells could reveal a cancer cell's growth pattern, its relationships with the tumor microenvironment and the immune response of the body, all of which represent key "hallmarks of cancer". However, the process by which pathologists manually recognize and localize all the cells in pathology slides is extremely labor intensive and error prone.In this study, we developed an automated cell type classification pipeline, ConvPath, which includes nuclei segmentation, convolutional neural network-based tumor cell, stromal cell, and lymphocyte classification, and extraction of tumor microenvironment-related features for lung cancer pathology images. To facilitate users in leveraging this pipeline for their research, all source scripts for ConvPath software are available at https://qbrc.swmed.edu/projects/cnn/.The overall classification accuracy was 92.9% and 90.1% in training and independent testing datasets, respectively. By identifying cells and classifying cell types, this pipeline can convert a pathology image into a "spatial map" of tumor, stromal and lymphocyte cells. From this spatial map, we can extract features that characterize the tumor micro-environment. Based on these features, we developed an image feature-based prognostic model and validated the model in two independent cohorts. The predicted risk group serves as an independent prognostic factor, after adjusting for clinical variables that include age, gender, smoking status, and stage.The analysis pipeline developed in this study could convert the pathology image into a "spatial map" of tumor cells, stromal cells and lymphocytes. This could greatly facilitate and empower comprehensive analysis of the spatial organization of cells, as well as their roles in tumor progression and metastasis.
Weakly Supervised Object Detection With Segmentation Collaboration

Weakly supervised object detection aims at learning precise object detectors, given image category labels. In recent prevailing works, this problem is generally formulated as a multiple instance learning module guided by an image classification loss. The object bounding box is assumed to be the one contributing most to the classification among all proposals. However, the region contributing most is also likely to be a crucial part or the supporting context of an object. To obtain a more accurate detector, in this work we propose a novel end-to-end weakly supervised detection approach, where a newly introduced generative adversarial segmentation module interacts with the conventional detection module in a collaborative loop. The collaboration mechanism takes full advantages of the complementary interpretations of the weakly supervised localization task, namely detection and segmentation tasks, forming a more comprehensive solution. Consequently, our method obtains more precise object bounding boxes, rather than parts or irrelevant surroundings. Expectedly, the proposed method achieves an accuracy of 53.7% on the PASCAL VOC 2007 dataset, outperforming the state-of-the-arts and demonstrating its superiority for weakly supervised object detection.
Cooperative Perception for 3D Object Detection in Driving Scenarios Using Infrastructure Sensors

3D object detection is a common function within the perception system of an autonomous vehicle and outputs a list of 3D bounding boxes around objects of interest. Various 3D object detection methods have relied on fusion of different sensor modalities to overcome limitations of individual sensors. However, occlusion, limited field-of-view and low-point density of the sensor data cannot be reliably and cost-effectively addressed by multi-modal sensing from a single point of view. Alternatively, cooperative perception incorporates information from spatially diverse sensors distributed around the environment as a way to mitigate these limitations. This article proposes two schemes for cooperative 3D object detection using single modality sensors. The early fusion scheme combines point clouds from multiple spatially diverse sensing points of view before detection. In contrast, the late fusion scheme fuses the independently detected bounding boxes from multiple spatially diverse sensors. We evaluate the performance of both schemes, and their hybrid combination, using a synthetic cooperative dataset created in two complex driving scenarios, a T-junction and a roundabout. The evaluation shows that the early fusion approach outperforms late fusion by a significant margin at the cost of higher communication bandwidth. The results demonstrate that cooperative perception can recall more than 95% of the objects as opposed to 30% for single-point sensing in the most challenging scenario. To provide practical insights into the deployment of such system, we report how the number of sensors and their configuration impact the detection performance of the system.
Attention-Driven Loss for Anomaly Detection in Video Surveillance

Recent video anomaly detection methods focus on reconstructing or predicting frames. Under this umbrella, the long-standing inter-class data-imbalance problem resorts to the imbalance between foreground and stationary background objects in video anomaly detection and this has been less investigated by existing solutions. Naively optimizing the reconstructing loss yields a biased optimization towards background reconstruction rather than the objects of interest in the foreground. To solve this, we proposed a simple yet effective solution, termed attention-driven loss to alleviate the foreground-background imbalance problem in anomaly detection. Specifically, we compute a single mask map that summarizes the frame evolution of moving foreground regions and suppresses the background in the training video clips. After that, we construct an attention map through the combination of the mask map and background to give different weights to the foreground and background region respectively. The proposed attention-driven loss is independent of backbone networks and can be easily augmented in most existing anomaly detection models. Augmented with attention-driven loss, the model is able to achieve AUC 86.0% on Avenue, 83.9% on Ped1, 96% on Ped2 datasets. Extensive experimental results and ablation studies further validate the effectiveness of our model.
BlockQNN: Efficient Block-Wise Neural Network Architecture Generation

Convolutional neural networks have gained a remarkable success in computer vision. However, most popular network architectures are hand-crafted and usually require expertise and elaborate design. In this paper, we provide a block-wise network generation pipeline called BlockQNN which automatically builds high-performance networks using the Q-Learning paradigm with epsilon-greedy exploration strategy. The optimal network block is constructed by the learning agent which is trained to choose component layers sequentially. We stack the block to construct the whole auto-generated network. To accelerate the generation process, we also propose a distributed asynchronous framework and an early stop strategy. The block-wise generation brings unique advantages: (1) it yields state-of-the-art results in comparison to the hand-crafted networks on image classification, particularly, the best network generated by BlockQNN achieves 2.35 percent top-1 error rate on CIFAR-10. (2) it offers tremendous reduction of the search space in designing networks, spending only 3 days with 32 GPUs. A faster version can yield a comparable result with only 1 GPU in 20 hours. (3) it has strong generalizability in that the network built on CIFAR also performs well on the larger-scale dataset. The best network achieves very competitive accuracy of 82.0 percent top-1 and 96.0 percent top-5 on ImageNet.
High-Utility Itemset Mining with Effective Pruning Strategies

High-utility itemset mining is a popular data mining problem that considers utility factors, such as quantity and unit profit of items besides frequency measure from the transactional database. It helps to find the most valuable and profitable products/items that are difficult to track by using only the frequent itemsets. An item might have a high-profit value which is rare in the transactional database and has a tremendous importance. While there are many existing algorithms to find high-utility itemsets (HUIs) that generate comparatively large candidate sets, our main focus is on significantly reducing the computation time with the introduction of new pruning strategies. The designed pruning strategies help to reduce the visitation of unnecessary nodes in the search space, which reduces the time required by the algorithm. In this article, two new stricter upper bounds are designed to reduce the computation time by refraining from visiting unnecessary nodes of an itemset. Thus, the search space of the potential HUIs can be greatly reduced, and the mining procedure of the execution time can be improved. The proposed strategies can also significantly minimize the transaction database generated on each node. Experimental results showed that the designed algorithm with two pruning strategies outperform the state-of-the-art algorithms for mining the required HUIs in terms of runtime and number of revised candidates. The memory usage of the designed algorithm also outperforms the state-of-the-art approach. Moreover, a multi-thread concept is also discussed to further handle the problem of big datasets.
OCR-VQA: Visual Question Answering by Reading Text in Images

The problem of answering questions about an image is popularly known as visual question answering (or VQA in short). It is a well-established problem in computer vision. However, none of the VQA methods currently utilize the text often present in the image. These "texts in images" provide additional useful cues and facilitate better understanding of the visual content. In this paper, we introduce a novel task of visual question answering by reading text in images, i.e., by optical character recognition or OCR. We refer to this problem as OCR-VQA. To facilitate a systematic way of studying this new problem, we introduce a large-scale dataset, namely OCRVQA-200K. This dataset comprises of 207,572 images of book covers and contains more than 1 million question-answer pairs about these images. We judiciously combine well-established techniques from OCR and VQA domains to present a novel baseline for OCR-VQA-200K. The experimental results and rigorous analysis demonstrate various challenges present in this dataset leaving ample scope for the future research. We are optimistic that this new task along with compiled dataset will open-up many exciting research avenues both for the document image analysis and the VQA communities.
Multipopulation cooperative particle swarm optimization with a mixed mutation strategy

The traditional particle swarm optimization algorithm learns from the two best experiences: the best position previously learned by the particle itself and the best position learned by the entire population to date. This learning strategy is simple and ordinary, but when addressing high-dimensional optimization problems, it is unable to quickly find the global optimal solution due to its low efficiency. This paper proposes a multipopulation cooperative particle swarm optimization (MPCPSO) algorithm with a dynamic segment-based mean learning strategy and a multidimensional comprehensive learning strategy. In MPCPSO, the dynamic segment-based mean learning strategy (DSMLS), which is employed to construct learning exemplars, achieves information sharing and coevolution between populations. The multidimensional comprehensive learning strategy (MDCLS) is employed to speed up convergence and improve the accuracy of MPCPSO solutions. Additionally, a differential mutation operator is introduced to increase the population diversity and enhance the global exploration ability of MPCPSO. Sixteen benchmark functions and seven well-known PSO variants are employed to verify the advantages of MPCPSO. The comparison results indicate that MPCPSO has a faster convergence speed, obtains more accurate solutions, and is more robust.
Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy

Since deep learning models have been implemented in many commercial applications, it is important to detect out-of-distribution (OOD) inputs correctly to maintain the performance of the models, ensure the quality of the collected data, and prevent the applications from being used for other-than-intended purposes. In this work, we propose a two-head deep convolutional neural network (CNN) and maximize the discrepancy between the two classifiers to detect OOD inputs. We train a two-head CNN consisting of one common feature extractor and two classifiers which have different decision boundaries but can classify in-distribution (ID) samples correctly. Unlike previous methods, we also utilize unlabeled data for unsupervised training and we use these unlabeled data to maximize the discrepancy between the decision boundaries of two classifiers to push OOD samples outside the manifold of the in-distribution (ID) samples, which enables us to detect OOD samples that are far from the support of the ID samples. Overall, our approach significantly outperforms other state-of-the-art methods on several OOD detection benchmarks and two cases of real-world simulation.
Privacy-Preserving Authentication Scheme for Connected Electric Vehicles Using Blockchain and Zero Knowledge Proofs

With the increasing interest in connected vehicles along with electrification opportunities, there is an ongoing effort to automate the charging process of electric vehicles (EVs) through their capabilities to communicate with the infrastructure and each other. However, charging EVs takes time and thus in-advance scheduling is needed. As this process is done frequently due to limited mileage per charge on EVs, it may expose the locations and charging pattern of the EV to the service providers, raising privacy concerns for their users. Nevertheless, the EV still needs to be authenticated to charging providers, which means some information will need to be provided anyway. While there have been many studies to address the problem of privacy-preserving authentication for vehicular networks, such solutions will be void if charging payments are made through traditional means. In this paper, we tackle this problem by utilizing distributed applications enabled by Blockchain and smart contracts. We adapt zero-knowledge proofs to Blockchain for enabling privacy-preserving authentication while removing the need for a central authority. We introduce two approaches, one using a token-based mechanism and another utilizing the Pederson Commitment scheme to realize anonymous authentication. We also describe a protocol for the whole process which includes scheduling and charging operations. The evaluation of the proposed approaches indicates that the overhead of this process is affordable to enable real-time charging operations for connected EVs.
Integrating advanced technologies to uphold security of payment: Data flow diagram

Security of payment (SOP) issues still persist in the construction industry despite numerous investigations and incremental reforms. Various solutions and policies have been proposed and analysed in-depth in previous studies. However, limited studies have focused on the integration of advanced technologies to address SOP issues. The aim of this research is to develop a comprehensive framework that integrates practical advanced technologies to address SOP issues in the construction industry. A concurrent mixed-method design was adopted to (a) identify the industry's perspective on what advanced technologies can be accepted to address SOP issues through a questionnaire survey, and (b) identify the use of advanced technologies through a live construction project as a case study. Subsequently, a data flow diagram framework was developed to articulate the whole process flow of how the system delivers automatic payments to subcontractors upon the completion of their contractual obligations and work done. This research contributes new and practical insights into the application and integration of smart sensors, oracles, BIM, blockchain technology and smart contracts in addressing SOP issues in the construction industry.
Norm-Aware Embedding for Efficient Person Search

Person Search is a practically relevant task that aims to jointly solve Person Detection and Person Re-identification (re-ID). Specifically, it requires to find and locate all instances with the same identity as the query person in a set of panoramic gallery images. One major challenge comes from the contradictory goals of the two sub-tasks, i.e., person detection focuses on finding the commonness of all persons while person re-ID handles the differences among multiple identities. Therefore, it is crucial to reconcile the relationship between the two sub-tasks in a joint person search model. To this end, We present a novel approach called Norm-Aware Embedding to disentangle the person embedding into norm and angle for detection and re-ID respectively, allowing for both effective and efficient multi-task training. We further extend the proposal-level person embedding to pixel-level, whose discrimination ability is less affected by mis-alignment. We outperform other one-step methods by a large margin and achieve comparable performance to two-step methods on both CUHK-SYSU and PRW. Also, Our method is easy to train and resource-friendly, running at 12 fps on a single GPU.
Using Augmented Reality to Stimulate Students and Diffuse Escape Game Activities to Larger Audiences

In a world engaged in a perpetual race for progress, augmented reality (AR) is a new frontier that has attracted much research attention in recent years. The generalization of smartphones and the miniaturization and democratization of connected gadgets are leading to new uses and new expectations. In this article, we provide an AR application to the trending educational activity of escape games in order to diffuse AR to a large audience. As the application of an educational escape game, which is a good tool to improve the motivation and communication skills of students, requires considerable time for design, manufacturing, and operating, a numeric, mobile AR version is proposed here. For this purpose, an original paper-based version of the scenario is presented, principally turning around the Solvay process for a high school/college audience. In this scenario, the students are led to follow in the footsteps of a young Belgian physicist, Ernest Solvay. Through enigmas, participants discover a secret room containing Solvay’s secrets in Brussels. Puzzles about the periodic table of elements, the CPK color code, chemical reaction balancing, the notion of molar mass, the Solvay process, or the philanthropic conferences organized by Solvay and the many other scientists involved are used to illustrate the chemical process discovered. This scenario and the Leblanc process scenario have been adapted to an AR experience with the open application Metaverse. Unlike conventional escape game or escape lab activities, our AR escape games can be easily diffused to large audience classrooms without the presence of any teacher being required and without any systematic preparation. The advantages and limits of such tools are discussed. The feedback received from students that participated in a test exercise with the tool corroborates the increase of motivation through such activities.
Long-Range Indoor Navigation With PRM-RL

Long-range indoor navigation requires guiding robots with noisy sensors and controls through cluttered environments along paths that span a variety of buildings. We achieve this with PRM-RL, a hierarchical robot navigation method in which reinforcement learning (RL) agents that map noisy sensors to robot controls learn to solve short-range obstacle avoidance tasks, and then sampling-based planners map where these agents can reliably navigate in simulation; these roadmaps and agents are then deployed on robots, guiding them along the shortest path where the agents are likely to succeed. In this article, we use probabilistic roadmaps (PRMs) as the sampling-based planner, and AutoRL as the RL method in the indoor navigation context. We evaluate the method with a simulation for kinematic differential drive and kinodynamic car-like robots in several environments, and on differential-drive robots at three physical sites. Our results show that PRM-RL with AutoRL is more successful than several baselines, is robust to noise, and can guide robots over hundreds of meters in the face of noise and obstacles in both simulation and on robots, including over 5.8 km of physical robot navigation.
SEUSS

This paper presents a system-level method for achieving the rapid deployment and high-density caching of serverless functions in a FaaS environment. For reduced start times, functions are deployed from unikernel snapshots, bypassing expensive initialization steps. To reduce the memory footprint of snapshots we apply page-level sharing across the entire software stack that is required to run a function. We demonstrate the effects of our techniques by replacing Linux on the compute node of a FaaS platform architecture. With our prototype OS, the deployment time of a function drops from 100s of milliseconds to under 10 ms. Platform throughput improves by 51x on workload composed entirely of new functions. We are able to cache over 50,000 function instances in memory as opposed to 3,000 using standard OS techniques. In combination, these improvements give the FaaS platform a new ability to handle large-scale bursts of requests.