Reviewerly/exHarmony huggingface dataset:
Infrared Image Captioning with Wearable Device

Wearable devices have garnered widespread attention as a mobile solution, and various intelligent modules based on wearable devices are increasingly being integrated. Additionally, image captioning is an important task in computer vision that maps images to text. Existing image captioning achievements are based on high-quality visible images. However, higher target complexity and insufficient light can lead to reduced captioning performance and mistakes. In this paper, we present an infrared image captioning framework designed to solve the problem of invalid visible image captioning in special conditions. Remarkably, we integrate the infrared image captioning model into the wearable device. Volunteers perform offline and real-time environmental analysis tasks in the real world to evaluate the framework's effectiveness in multiple scenarios. The results indicate that both the accuracy of infrared image captioning and the feedback from wearable device users are promising.
Focused Adaptation of Dynamics Models for Deformable Object Manipulation

In order to efficiently learn a dynamics model for a task in a new environment, one can adapt a model learned in a similar source environment. However, existing adaptation methods can fail when the target dataset contains transitions where the dynamics are very different from the source environment. For example, the source environment dynamics could be of a rope manipulated in free space, whereas the target dynamics could involve collisions and deformation on obstacles. Our key insight is to improve data efficiency by focusing model adaptation on only the regions where the source and target dynamics are similar. In the rope example, adapting the free-space dynamics requires significantly less data than adapting the free-space dynamics while also learning collision dynamics. We propose a new method for adaptation that is effective in adapting to regions of similar dynamics. Additionally, we combine this adaptation method with prior work on planning with unreliable dynamics to make a method for data-efficient online adaptation, called FOCUS. We first demonstrate that the proposed adaptation method achieves statistically significantly lower prediction error in regions of similar dynamics on simulated rope manipulation and plant watering tasks. We then show on a bimanual rope manipulation task that FOCUS achieves data-efficient online learning, in simulation and in the real world.
Transparent Objects: A Corner Case in Stereo Matching

Stereo matching is a common technique used in 3D perception, but transparent objects such as reflective and penetrable glass pose a challenge as their disparities are often estimated inaccurately. In this paper, we propose transparency-aware stereo (TA-Stereo), an effective solution to tackle this issue. TA-Stereo first utilizes a semantic segmentation or salient object detection network to identify transparent objects, and then homogenizes them to enable stereo matching algorithms to handle them as non-transparent objects. To validate the effectiveness of our proposed TA-Stereo strategy, we collect 260 images containing transparent objects from the KITTI Stereo 2012 and 2015 datasets and manually label pixel-level ground truth. We evaluate our strategy with six deep stereo networks and two types of transparent object detection methods. Our experiments demonstrate that TA-Stereo significantly improves the disparity accuracy of transparent objects. Our project webpage can be accessed at mias.group/TA-Stereo.
NanoFlowNet: Real-time Dense Optical Flow on a Nano Quadcopter

Nano quadcopters are small, agile, and cheap platforms that are well suited for deployment in narrow, cluttered environments. Due to their limited payload, these vehicles are highly constrained in processing power, rendering conventional vision-based methods for safe and autonomous navigation incompatible. Recent machine learning developments promise high-performance perception at low latency, while dedicated edge computing hardware has the potential to augment the processing capabilities of these limited devices. In this work, we present NanoFlowNet, a lightweight convolutional neural network for real-time dense optical flow estimation on edge computing hardware. We draw inspiration from recent advances in semantic segmentation for the design of this network. Additionally, we guide the learning of optical flow using motion boundary ground truth data, which improves performance with no impact on latency. Validation results on the MPI-Sintel dataset show the high performance of the proposed network given its constrained architecture. Additionally, we successfully demonstrate the capabilities of NanoFlowNet by deploying it on the ultra-low power GAP8 microprocessor and by applying it to vision-based obstacle avoidance on board a Bitcraze Crazyflie, a 34 g nano quadcopter.
Obstacle avoidance using Raycasting and Riemannian Motion Policies at kHz rates for MAVs

This paper presents a novel method for using Riemannian Motion Policies on volumetric maps, shown in the example of obstacle avoidance for Micro Aerial Vehicles (MAVs), Today, most robotic obstacle avoidance algorithms rely on sampling or optimization-based planners with volumetric maps. However, they are computationally expensive and often have inflexible monolithic architectures. Riemannian Motion Policies are a modular, parallelizable, and efficient navigation alternative but are challenging to use with the widely used voxel-based environment representations. We propose using GPU raycasting and tens of thousands of concurrent policies to provide direct obstacle avoidance using Riemannian Motion Policies in voxelized maps without needing map smoothing or pre-processing. Additionally, we present how the same method can directly plan on LiDAR scans without any intermediate map. We show how this reactive approach compares favorably to traditional planning methods and can evaluate up to 200 million rays per second. We demonstrate the planner successfully on a real MAV for static and dynamic obstacles. The presented planner is made available as an open-source package <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://github.com/ethz-asl/reactive_avoidance.
Sequence-Agnostic Multi-Object Navigation

The Multi-Object Navigation (MultiON) task requires a robot to localize an instance (each) of multiple object classes. It is a fundamental task for an assistive robot in a home or a factory. Existing methods for MultiON have viewed this as a direct extension of Object Navigation (ON), the task of localising an instance of one object class, and are pre-sequenced, i.e., the sequence in which the object classes are to be explored is provided in advance. This is a strong limitation in practical applications characterized by dynamic changes. This paper describes a deep reinforcement learning framework for sequence-agnostic MultiON based on an actor-critic architecture and a suitable reward specification. Our framework leverages past experiences and seeks to reward progress toward individual as well as multiple target object classes. We use photo-realistic scenes from the Gibson benchmark dataset in the AI Habitat 3D simulation environment to experimentally show that our method performs better than a pre-sequenced approach and a state of the art ON method extended to MultiON.
Deep Unsupervised Visual Odometry Via Bundle Adjusted Pose Graph Optimization

Unsupervised visual odometry as an active topic has attracted extensive attention, benefiting from its label-free practical value and robustness in real-world scenarios. However, the performance of camera pose estimation and tracking through deep neural network is still not as ideal as most other tasks, such as detection, segmentation and depth estimation, due to the lack of drift correction in the estimated trajectory and map optimization in the recovered 3D scenes. In this work, we introduce pose graph and bundle adjustment optimization to our network training process, which iteratively updates both the motion and depth estimations from the deep learning network, and enforces the refined outputs to further meet the unsupervised photometric and geometric constraints. The integration of pose graph and bundle adjustment is easy to implement and significantly enhances the training effectiveness. Experiments on KITTI dataset demonstrate that the introduced method achieves a significant improvement in motion estimation compared with other recent unsupervised monocular visual odometry algorithms.
Learning a Single Near-hover Position Controller for Vastly Different Quadcopters

This paper proposes an adaptive near-hover position controller for quadcopters, which can be deployed to quadcopters of very different mass, size and motor constants, and also shows rapid adaptation to unknown disturbances during runtime. The core algorithmic idea is to learn a single policy that can adapt online at test time not only to the disturbances applied to the drone, but also to the robot dynamics and hardware in the same framework. We achieve this by training a neural network to estimate a latent representation of the robot and environment parameters, which is used to condition the behaviour of the controller, also represented as a neural network. We train both networks exclusively in simulation with the goal of flying the quadcopters to goal positions and avoiding crashes to the ground. We directly deploy the same controller trained in the simulation without any modifications on two quadcopters in the real world with differences in mass, size, motors, and propellers with mass differing by 4.5 times. In addition, we show rapid adaptation to sudden and large disturbances up to one-third of the mass of the quadcopters. We perform an extensive evaluation in both simulation and the physical world, where we outperform a state-of-the-art learning-based adaptive controller and a traditional PID controller specifically tuned to each platform individually. Video results can be found at https://youtu.be/U-c-LbTfvoA.
Learning Depth Completion of Transparent Objects using Augmented Unpaired Data

We propose a technique for depth completion of transparent objects using augmented data captured directly from real environments with complicated geometry. Using cyclic adversarial learning we train translators to convert between painted versions of the objects and their real transparent counterpart. The translators are trained on unpaired data, hence datasets can be created rapidly and without any manual labeling. Our technique does not make any assumptions about the geometry of the environment, unlike SOTA systems that assume easily observable occlusion and contact edges, such as ClearGrasp. We show how our technique outperforms ClearGrasp in a dishwasher environment, in which occlusion and contact edges are difficult to observe. We also show how the technique can be used to create an object manipulation application with a humanoid robot. Supplementary URI: https://ftorise.github.io/faking_depth_web/.
Unsupervised Road Anomaly Detection with Language Anchors

Road anomaly detection is critical to safe autonomous driving, because current road scene understanding models are usually trained in a closed-set manner and fail to identify unknown objects. What's worse, it is difficult, if not impossible, to collect a large-scale dataset with anomaly annotations. So this paper studies unsupervised anomaly detection which finds out anomaly regions using scene parsing logits solely. While former methods depend on the weights learned from the closed training set as anchors for logit generation, we resort to language anchors that are learned from enormous paired vision and language data. Thanks to rich open-set semantic information contained in these language anchors, our method performs better than former unsupervised counterparts while maintaining the advantage of training without accessing any out-of-distribution data. We delve into this new paradigm and identify the superiority of using pair-wise binary logits, which we credit to a better understanding of the negation language anchor. Last but not least, we find that the former top-1 selection of semantic labels for uncertainty measurement is problematic in many cases and a new blended standardization strategy brings clear improvements to our solution. We report state-of-the-art performance on FS LostAndFound, LostAndFound and RoadAnomaly datasets among comparable methods. The codes are publicly available at https://github.com/TB5z035/URAD-LA.git
VP-STO: Via-point-based Stochastic Trajectory Optimization for Reactive Robot Behavior

Achieving reactive robot behavior in complex dynamic environments is still challenging as it relies on being able to solve trajectory optimization problems quickly enough, such that we can replan the future motion at frequencies which are sufficiently high for the task at hand. We argue that current limitations in Model Predictive Control (MPC) for robot manipulators arise from inefficient, high-dimensional trajectory representations and the negligence of time-optimality in the trajectory optimization process. Therefore, we propose a motion optimization framework that optimizes jointly over space and time, generating smooth and timing-optimal robot trajectories in joint-space. While being task-agnostic, our formulation can incorporate additional task-specific requirements, such as collision avoidance, and yet maintain real-time control rates, demonstrated in simulation and real-world robot experiments on closed-loop manipulation. For additional material, please visit https://sites.google.com/oxfordrobotics.institute/vp-sto.
Fast Region of Interest Proposals on Maritime UAVs

Unmanned aerial vehicles assist in maritime search and rescue missions by flying over large search areas to autonomously search for objects or people. Reliably detecting objects of interest requires fast models to employ on embedded hardware. Moreover, with increasing distance to the ground station only part of the video data can be transmitted. In this work, we consider the problem of finding meaningful region of interest proposals in a video stream on an embedded GPU. Current object or anomaly detectors are not suitable due to their slow speed, especially on limited hardware and for large image resolutions. Lastly, objects of interest, such as pieces of wreckage, are often not known a priori. Therefore, we propose an end-to-end future frame prediction model running in real-time on embedded GPUs to generate region proposals. We analyze its performance on large-scale maritime data sets and demonstrate its benefits over traditional and modern methods.
Blockchain and Federated Learning Enabled Smart Traffic Management System for Smart Cities

In recent years, traffic management across smart cities has become a global concern due to the increasing number of vehicles and data breaches. Addressing and resolving various security challenges associated with smart cities has become a difficult process as the user connects smart traffic infrastructures in a variety of ways such as smartphones, connected devices, etc. Even a slight compromise in real-time traffic data can lead to adverse consequences as they are highly confidential. Blockchain remains to be an efficient technique that can identify and detect security threats and appropriately manage traffic systems. In this paper, we define a blockchain and federated learning-enabled smart traffic management system for smart cities that effectively manages the real-time traffic status and prevent security vulnerabilities in a decentralized manner. The use of the federated learning technique improves the system's efficiency through the use of local and global models. It is observed from the experimental results that the proposed approach remains resistant to byzantine attack and Sybil attacks providing more resilient services to the end users.
Implementation of Hand Written based Signature Verification Technology using Deep Learning Approach

There have been several iterations of biometric techniques released for use in personal identification. Vision-based technologies include facial recognition, fingerprint scanning, iris scanning, and retina scanning. The most well-known non-visual methods are voice recognition and signature verification. As long as signatures remain an important element of financial, economic, and legal procedures, truly secure authentication is becoming more important. Signatures from authorized individuals continue to be the most trusted form of verification and are sometimes referred to as “seals of approval.” This research describes a method that begins with the processing of an image, continues with the extraction of geometric characteristics, and ends with the training and verification of a neural network. The authenticity of a test signature may be determined by feeding its extracted features into a neural network that has been trained as part of the verification process.
Framework for Implementation of Android Automation Tool in Agro Business Sector

As we know, India is an Agricultural dominated country. According to survey of National Statistical Office's Situation Assessment of Agricultural Households (SAAH) report for 2018–19 India officially has anywhere from 90 million-plus to almost 150 million farmers. But the fact is that the concept of smart farming is on the low scale and farmers are not implementing in their land areas. Because they do not know the power of automation. If the breadwinner of the world is happy then all the country will become happy, isn't it? So that is why we need to come one step forward to do something smart in this agricultural area with some innovations that are truly farmer friendly. More than 90% farmers using submersible & open well motors. This paper represents the implementation of farmer friendly android automation tool which is capable to do many tasks according to the workflow of farmer. This android system is designed to turn on/off motor, schedule on/off, timer, gathering of electrical information from panel box and many more applications. This device will reduce the manual operation of motors for which farmer need to go to the farm field.
Framework for Design and Implementation of Chat Support System using Natural Language Processing

Companies have increasingly started adopting chatbots as a customer service support system to provide more personalized service experience to customers. This technology is catching attention in the customer service context because chatbots can support multiple customers at one, have the ability to provide multi-lingual support, they are available <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$24^*7$</tex> with no extra cost, relieves humans from repetitive task and can respond to a variety of troubleshooting queries without any latency. This research paper aims at developing a customer support chatbot called ‘Anjore’ for the Chhattisgarh State Power Distribution Company Limited. The word Anjore (pronounced as <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\Lambda \mathrm{n}.\mathrm{d}_3\mathrm{o}:\mathrm{r}$</tex> ) is derived from chhattisgarhi language which means ‘light’. The chatbot is developed in django web frame work and spaCy NLP library for Python has been employed to understand utterances in English language.
Artificial Intelligence based Chatbot for promoting Equality in High School Advising

The sustainable development goal 4 (SDG4) aims to “ensure inclusive and equitable quality education and promote lifelong learning opportunities for all”, Therefore., researchers are inspired to study the fairness and equality in different aspects of education. Some studies are focused on social and academic initiatives and others on developing state-of-the-art technology to enhance the education between students equally. Henceforth., in this paper a novel affordable chatbot implemented by using a neural network model and natural language processing (NLP) to assist students in high schools particularly., since high school is one of the most essential stages in students” lives., as in this stage., students have the option to select their academic streams and advanced courses that can shape their career with their passions and interests. The dataset in this study collected from different academic resources such as schools & universities websites., schools” advisers., parents and students. It includes (968) pairs of enquiries and tags. The first column represents the student's enquiry; the second column indicates the tag or the class of each sentence. The model built by connecting the input data into embedding layer., and then the data fed into the LSTM layer with different number of neurons., then authors used sigmoid function for the output layer. The result in this study shows that the performance of the chatbot is improved by increasing the number of neurons from 5 to 8, the model achieved high accuracy ratio with score (96.5 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">%</sup> ). In future the model will be developed with stacked LSTM layers with using softmax activation function in the output layer, as different classes will be added as well in the dataset.
Developing an Open Domain Arabic Question Answering System Using a Deep Learning Technique

A question-answering system (QAS) retrieves a relevant response to user queries.The existing QASs are limited in performance in satisfying the users' intention.In recent times, researchers have focused on developing Arabic QASs.However, a significant number of QASs are based on specific domains.Therefore, the study intends to develop an open-domain QAS using a deep learning technique.The proposed QAS comprises three phases: data pre-processing, name entity relationship, and response retrieval.The researchers apply de-diacritization, minimizing orthographic ambiguity, tokenization, and morphological analysis to extract the key terms from the Arabic content.This phase supports the QAS in overcoming the challenges of understanding the Arabic content.Multinomial Naïve Bayes algorithm is applied to uncover the relationship among the Arabic terms.In addition, the authors employ the Embeddings from Language Models approach with a quaternion long-short-term memory neural network (QLSTM) for constructing the QAS with limited resources.The Arabic reading comprehension dataset (ARCD) and TyDiQA are utilized to evaluate the performance.The experimental outcome reveals that the proposed QAS achieve accuracy, precision, recall, F1-score, MCC and Kappa of 96.23, 97, 96.95, 97, 95.98, 95.7 and 95.35, 94.8, 94.68, 94.73, 92.98, and 93.6 for ARCD and TyDiQA, respectively.The structure of the proposed QAS is lightweight and can be implemented in real-world applications.
Enforcing Fairness With Blockchain Support: Proposals for Multi-Two-Party Contract Signing

In this paper, we present two proposals for multi-two-party contract signing with blockchain support. In the scenario in this paper, a customer wants to purchase products/services from multiple vendors with the requirement that she wants all or none of the products. Our proposals satisfy the security requirements for contract signing: fairness, timeliness, nonrepudiation, effectiveness and confidentiality. Furthermore, one of the two proposals requires the execution of blockchain functions only in the case of dishonest behaviour (bc-optimistic). The only previous solution found in the literature for the multi-two-party contract signing scenario requires the use of a trusted third party (TTP); however, the approach is highly inefficient, requiring the exchange of many messages. Our proposals are much more efficient than the TTP-based proposal, as our proposal reduces the number of exchanged messages required to complete multi-two-party contract signing. As the use of blockchain incurs a fee, we prove the technological and economic viability of our proposals on different blockchains.