Reviewerly/exHarmony huggingface dataset:
Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Driving

3D human pose estimation (HPE) in autonomous vehicles (AV) differs from other use cases in many factors, including the 3D resolution and range of data, absence of dense depth maps, failure modes for LiDAR, relative location between the camera and LiDAR, and a high bar for estimation accuracy. Data collected for other use cases (such as virtual reality, gaming, and animation) may therefore not be usable for AV applications. This necessitates the collection and annotation of a large amount of 3D data for HPE in AV, which is time-consuming and expensive.In this paper, we propose one of the first approaches to alleviate this problem in the AV setting. Specifically, we propose a multi-modal approach which uses 2D labels on RGB images as weak supervision to perform 3D HPE. The proposed multi-modal architecture incorporates LiDAR and camera inputs with an auxiliary segmentation branch. On the Waymo Open Dataset [27], our approach achieves a ∼ 22% relative improvement over camera-only 2D HPE baseline, and ∼ 6% improvement over LiDAR-only model. Finally, careful ablation studies and parts based analysis illustrate the advantages of each of our contributions.
Artificial neural network in diagnostic cytology

The artificial neural network (ANN) is a computer software design or model that simulates the biological neural network of the human brain. Instead of biological neurons, ANN is composed of many layers of nodes that carry the signal and process it to make the final decision. ANN is a modern technology that is widely used in different fields of science. The ANN is reshaping the medical system and the various areas of pathology. In this paper, the basic concept and applications of ANN in cytology have been discussed. In this paper, the various articles published on ANN in the field of cytology have been systemically reviewed. The ANN is relatively less used in cytology. After introducing convolutional neural network and whole slide scanners in the commercial market, it is now essential to have thorough knowledge in this field to start diagnostic application of ANN.
Trustworthy Machine Learning

Machine learning (ML) techniques have numerous applications in many fields, including healthcare, medicine, finance, marketing, and cyber security. For example, ML techniques are being applied to determine whether to give a loan to a customer or whether the computing system has been attacked. However, the ML techniques themselves may be subject to attacks and may discriminate when determining who should get the loan. Therefore, the ML techniques have to be secure, ensure privacy of the individuals, incorporate fairness and be accurate. Such collection of ML techniques has come to be known as trustworthy machine learning (trustworthy ML). This article describes an architecture to support scalable trustworthy ML and describes the features that have to be incorporated into the ML techniques to ensure that they are trustworthy.
Quantifying Explainability in Outcome-Oriented Predictive Process Monitoring

Abstract The growing interest in applying machine and deep learning algorithms in an Outcome-Oriented Predictive Process Monitoring (OOPPM) context has recently fuelled a shift to use models from the explainable artificial intelligence (XAI) paradigm, a field of study focused on creating explainability techniques on top of AI models in order to legitimize the predictions made. Nonetheless, most classification models are evaluated primarily on a performance level, where XAI requires striking a balance between either simple models (e.g. linear regression) or models using complex inference structures (e.g. neural networks) with post-processing to calculate feature importance. In this paper, a comprehensive overview of predictive models with varying intrinsic complexity are measured based on explainability with model-agnostic quantitative evaluation metrics. To this end, explainability is designed as a symbiosis between interpretability and faithfulness and thereby allowing to compare inherently created explanations (e.g. decision tree rules) with post-hoc explainability techniques (e.g. Shapley values) on top of AI models. Moreover, two improved versions of the logistic regression model capable of capturing non-linear interactions and both inherently generating their own explanations are proposed in the OOPPM context. These models are benchmarked with two common state-of-the-art models with post-hoc explanation techniques in the explainability-performance space.
OptZConfig: Efficient Parallel Optimization of Lossy Compression Configuration

Lossless compressors have very low compression ratios that do not meet the needs of today’s large-scale scientific applications that produce vast volumes of data. Error-bounded lossy compression (EBLC) is considered a critical technique for the success of scientific research. Although EBLC allows users to set an error bound for the compression, users have been unable to specify the requirements on the compression quality, limiting practical use. Our contributions are: (1) We formulate the problem of configuring EBLC to preserve a user-defined metric as an optimization problem. This allows many classes of new metrics to be preserved, which improves over current practices. (2) We present a framework, OptZConfig, that can adapt to improvements in the search algorithm, compressor, and metrics with minimal changes, enabling future advancements in this area. (3) We demonstrate the advantages of our approach against the leading methods to configure compressors to preserve specific metrics. Our approach improves compression ratios against a specialized compressor by up to <inline-formula><tex-math notation="LaTeX">$3\times$</tex-math></inline-formula> , has a 56× speedup over FRaZ, 1000× speedup over MGARD-QOI post tuning, and 110× speedup over systematic approaches which had not been bounded by compressors before.
Old but Gold: Prospecting TCP to Engineer and Live Monitor DNS Anycast

DNS latency is a concern for many service operators: CDNs exist to reduce service latency to end-users but must rely on global DNS for reachability and load-balancing. Today, DNS latency is monitored by active probing from distributed platforms like RIPE Atlas, with Verfploeter, or with commercial services. While Atlas coverage is wide, its 10k sites see only a fraction of the Internet. In this paper we show that passive observation of TCP handshakes can measure live DNS latency, continuously, providing good coverage of current clients of the service. Estimating RTT from TCP is an old idea, but its application to DNS has not previously been studied carefully. We show that there is sufficient TCP DNS traffic today to provide good operational coverage (particularly of IPv6), and very good temporal coverage (better than existing approaches), enabling near-real time evaluation of DNS latency from real clients. We also show that DNS servers can optionally solicit TCP to broaden coverage. We quantify coverage and show that estimates of DNS latency from TCP is consistent with UDP latency. Our approach finds previously unknown, real problems: DNS polarization is a new problem where a hypergiant sends global traffic to one anycast site rather than taking advantage of the global anycast deployment. Correcting polarization in Google DNS cut its latency from 100 ms to 10 ms; and from Microsoft Azure cut latency from 90 ms to 20 ms. We also show other instances of routing problems that add 100–200 ms latency. Finally, real-time use of our approach for a European country-level domain has helped detect and correct a BGP routing misconfiguration that detoured European traffic to Australia. We have integrated our approach into several open source tools: ENTRADA, our open source data warehouse for DNS, a monitoring tool (Anteater), which has been operational for the last 2 years on a country-level top-level domain, and a DNS anonymization tool in use at a root server since March 2021.
Text Detection and Recognition Using Augmented Reality and Deep Learning

In recent years, the detection and recognition of text in natural images has become a very attractive and important subject for researchers. Many applications were developed for text detection and recognition and the majority of them are based on deep learning (DL) and augmented reality (AR). In this article, we propose a perfect solution based on both deep learning and augmented reality in order to make the text reading process more efficient, clear and safer. The system purpose is to help visually impaired people read a text from natural images. First of all, the user has to hover his smartphone's camera over the image of the text present in his environment. Then, the system executes the detection and recognition module using the DL model. Finally, the system displays the associated graphical data augmented on the identified text on the screen of the smartphone using the AR method. AR method is used to improve the visualization of the detected and recognized word so that the user can read that text more efficiently. This mobile application has the highest-level visual features to improve the reading process of the detected and recognized text. To validate the system performance, the application is tested on a group of people who answer a questionnaire that reflects their experience with our proposed approach. In addition, user study test is performed to test user friendliness and satisfaction.
Real-Time Multi-Modal Human–Robot Collaboration Using Gestures and Speech

Abstract As artificial intelligence and industrial automation are developing, human–robot collaboration (HRC) with advanced interaction capabilities has become an increasingly significant area of research. In this paper, we design and develop a real-time, multi-model HRC system using speech and gestures. A set of 16 dynamic gestures is designed for communication from a human to an industrial robot. A data set of dynamic gestures is designed and constructed, and it will be shared with the community. A convolutional neural network is developed to recognize the dynamic gestures in real time using the motion history image and deep learning methods. An improved open-source speech recognizer is used for real-time speech recognition of the human worker. An integration strategy is proposed to integrate the gesture and speech recognition results, and a software interface is designed for system visualization. A multi-threading architecture is constructed for simultaneously operating multiple tasks, including gesture and speech data collection and recognition, data integration, robot control, and software interface operation. The various methods and algorithms are integrated to develop the HRC system, with a platform constructed to demonstrate the system performance. The experimental results validate the feasibility and effectiveness of the proposed algorithms and the HRC system.
A Noncontact Control Strategy for Circular Peg-in-Hole Assembly Guided by the 6-DOF Robot Based on Hybrid Vision

Facing some special operating environments or conditions, existing control methods for the peg-in-hole assembly guided by robots always have their own disadvantages, for example, low efficiency or poor adaptability. For the above problem, in this article, a new circular peg-in-hole assembly control strategy is proposed for the 6-Degree of Freedom (DOF) robot based on hybrid visual measurements, avoiding peg-in-hole contacts during the robotic operation. In the strategy, the pose of the monocular camera mounted at the end-effector is adaptively adjusted to improve the image quality through an algorithm based on the rough pose measurement of the target hole by the binocular camera; the accurate 3-D pose of the hole is determined by an algorithm based on processing of high-quality images and the compensation of the orientation error. Combined with the robotic collision-free path planning, the automatic peg-in-hole assembly can be implemented in real setting. The assembly precision of the robotic system based on the proposed method is validated and discussed based on experimental results. Then, the minimal peg-in-hole interval relative to the alignment error is modeled through the spatial relation analysis to analyze the applicable condition of the robotic system with the control strategy. Also, the reliability of the proposed strategy is verified through experimental tests under some applicable conditions. Finally, suggestions and plans of future works are discussed for further extension of the application area of the proposed strategy, such as fields of precision and ultraprecision manufacturing. This contribution has the major significance on the automatic peg-in-hole assembly under 3-D operating environment.
DLRP: Learning Deep Low-Rank Prior for Remotely Sensed Image Denoising

Remotely sensed images degraded by additive white Gaussian noise (AWGN) are not beneficial for the analysis of their contents. Such a phenomenon is usually modeled as an inverse problem which can be solved by model-based optimization methods or discriminative learning approaches. The former pursue their pleasing performance at the cost of a highly computational burden while the latter are impressive for their fast testing speed but are limited by their application range. To join their merits, this letter proposes a nonlocal self-similar (NSS) block-based deep image denoising scheme, namely deep low-rank prior (DLRP), which includes the following key points: First, the low-rank property of the neighboring NSS patches ordered lexicographically is utilized to model a global objective function (GOF). Second, with the aid of an alternative iteration strategy, the GOF can be easily decomposed into two independent subproblems. One is a quadratic optimization problem, and has a closed-form solution. While the other is a low-rank minimization denoising problem and is learned by deep convolutional neural network (DCNN). Then, the deep denoiser, acted as a modular part, is plugged into the model-based optimization method with adaptive noise level estimation to solve the inverse problem. In the experiments, we first discuss parameter setting and the convergence. Then, quantitative/qualitative comparisons of experimental results validate that the DLRP is a flexible and powerful denoising method to achieve competitive performance which even outperforms those produced by state-of-the-arts.
A Survey of Convolutional Neural Networks for Image Classification: Models and Datasets

Convolutional Neural Networks (CNN) are a form of feedforward neural network with a deep structure and convolution computations. It is one of the most well-known deep learning algorithms. Convolutional neural networks were first studied in the 1980s and 1990s. After the advent of deep learning theory in the twenty-first century, time delay networks and LeNet-5 were the first convolutional neural networks. Convolutional neural networks have rapidly evolved as numerical computing technology has improved, and they are now utilized in computer vision, natural language processing, and other disciplines. This essay will mostly focus on CNN’s use in image categorization. Many researchers have been researching CNN in recent years, and the CNN model is at its heart, so we'll start there.
NLFNet: Non-Local Fusion Towards Generalized Multimodal Semantic Segmentation across RGB-Depth, Polarization, and Thermal Images

In recent years, intelligent driving navigation has made considerable progress, and semantic segmentation is one of the most advanced scene perception methods. At present, traditional semantic segmentation methods can use RGB images for detection of obstacles that are clearly visible in outdoor scenes. However, in the face of complex realistic driving scenes, RGB images cannot provide sufficient information. We need some other modal information to supplement the RGB information. In this paper, we propose Non-Local Fusion Network (NLFNet), which is a semantic segmentation network that can selectively fuse multimodal input information in an adaptive manner. It can use complementary information collected by different optical sensors to extract effective features for fusion. Thereby, it improves the segmentation accuracy of the network and solves the problem of object recognition in various challenging real-world scenes. We conduct comprehensive experiments to verify the effectiveness and generalization ability of the framework across RGB-Depth, RGB-Polarization, and RGB-Thermal image semantic segmentation, which is especially suitable for autonomous driving and robot vision applications.
Embryo Grading With Unreliable Labels Due to Chromosome Abnormalities by Regularized PU Learning With Ranking

We propose a method for human embryo grading with its images. This grading has been achieved by positive-negative classification (i.e., live birth or non-live birth). However, negative (non-live birth) labels collected in clinical practice are unreliable because the visual features of negative images are equal to those of positive (live birth) images if these non-live birth embryos have chromosome abnormalities. For alleviating an adverse effect of these unreliable labels, our method employs Positive-Unlabeled (PU) learning so that live birth and non-live birth are labeled as positive and unlabeled, respectively, where unlabeled samples contain both positive and negative samples. In our method, this PU learning on a deep CNN is improved by a learning-to-rank scheme. While the original learning-to-rank scheme is designed for positive-negative learning, it is extended to PU learning. Furthermore, overfitting in this PU learning is alleviated by regularization with mutual information. Experimental results with 643 time-lapse image sequences demonstrate the effectiveness of our framework in terms of the recognition accuracy and the interpretability. In quantitative comparison, the full version of our proposed method outperforms positive-negative classification in recall and F-measure by a wide margin (0.22 vs. 0.69 in recall and 0.27 vs. 0.42 in F-measure). In qualitative evaluation, visual attentions estimated by our method are interpretable in comparison with morphological assessments in clinical practice.
Evaluating Accuracy of the Tobii Eye Tracker 5

Eye-tracking sensors are a relatively new technology and currently has use as an accessibility method to allow those with disabilities to use technology with greater independence. This study evaluates the general accuracy and precision of Tobii eye-tracking software and hardware, along with the efficacy of training a neural network to improve both aspects of the eye-tracker itself. With three human testers observing a grid of data points, the measured and known point locations are recorded and analyzed using over 250 data points. The study was conducted over two days, with each participant performing four trials each. In this study, we use basic statistics and a k-means clustering algorithm to examine the data in depth and give insights into the performance of the Tobii-5 eye-tracker. In addition to evaluating performance, this study also attempts to improve the accuracy of the Tobii-5 eye-tracker by using a Multi-Layer Perceptron Regressor to reassign gaze locations to better line up with the expected gaze location. Potential future developments are also discussed.
Effects of fishery complementary photovoltaic power plant on near-surface meteorology and energy balance

Solar energy plays an essential role in achieving carbon goals and mitigating climate change. Therefore, solar power plants are rapidly developing in the renewable energy sector. However, many reports of solar power plants are on land, and extremely limited observational research has been conducted on the impacts of fishery complementary photovoltaic power plants (FPVs) on near-surface meteorology and surface energy. This study selected two adjacent eddy covariance observational towers at the fishery complementary photovoltaic power plant (FPV) in Yangzhong, Jiangsu Province, China, to explore this topic. The results indicated that the percent frequency of east wind (<4 m s−1) at 2 m decreased by 25.3% in the FPV site compared with the reference site. The FPV array has not an obvious heating effect on the ambient environment. The average air temperature difference in the two sites at 2 m and 10 m was 0.3 K and 0.1 K, respectively, during the heating period. The net radiation increased by 47.8 W m−2 in the two sites. The sensible heat increased by 7.9 W m−2 due to the heating effect. The latent heat in the two sites is contrary to the sensible heat (−13.0 W m−2). The difference in the water storage heat is 32.18 W m−2, which implies that the water absorbs higher heat in the FPV site than in the reference site. This work illustrated the importance of observational experiments to animate process-based understanding combined with FPV systems and provides a scientific basis for establishing FPV energy balance models based on observations that may be used to reveal the impact of utility-scale FPV deployment on climatic effects.
Toward Generative Adversarial Networks for the Industrial Internet of Things

Machine learning, as a viable way of conducting data analytics, has been successfully applied to a number of areas. Nonetheless, the lack of sufficient data is one critical issue for applying machine learning in Industrial Internet of Things (IIoT) systems. Insufficient data raises could negatively affect the accuracy of machine learning models. To tackle this issue, we design a framework to systematically investigate the impacts of insufficient data on model training. This framework employs the generative adversarial network (GAN) and continuous learning to generate and engage new data in model training, enabling us to study the security risks of introducing new data in the model training process and develop countermeasures to mitigate these risks. To validate the efficacy of our framework, we consider a representative IIoT scenario, in which a variety of industrial components needs to be recognized by convolutional neural networks (CNNs), and design and implement three evaluation scenarios that are based on a real-world IIoT data set. Our experimental results confirm that insufficient data can have a significant impact on the model accuracy, but that new data generated by GAN and continuous learning can greatly improve the model accuracy. Our experimental results also show that the data poisoning threat posed by the GAN can significantly reduce the model accuracy. However, our proposed defensive mechanism is capable of securing the model learning process. We conclude this article by discussing some emerging issues that need to be addressed in future work.
Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Clouds

Understanding the flow in 3D space of sparsely sampled points between two consecutive time frames is the core stone of modern geometric-driven systems such as VR/AR, Robotics, and Autonomous driving. The lack of real, non-simulated, labeled data for this task emphasizes the importance of self- or un-supervised deep architectures. This work presents a new self-supervised training method and an architecture for the 3D scene flow estimation under occlusions. Here we show that smart multi-layer fusion between flow prediction and occlusion detection outperforms traditional architectures by a large margin for occluded and non-occluded scenarios. We report state-of-the-art results on Flyingthings3D and KITTI datasets for both the supervised and self-supervised training. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>