Reviewerly/exHarmony huggingface dataset:
Did chatbots miss their “Apollo Moment”? Potential, gaps, and lessons from using collaboration assistants during COVID-19

<h2>Summary</h2> Artificial intelligence (AI) technologies have long been positioned as a tool to provide crucial data-driven decision support to people. In this survey paper, I look at how collaboration assistants (chatbots for short), a type of AI that allows people to interact with them naturally (such as using speech, gesture, and text), have been used during a true global exigency—the COVID-19 pandemic. The key observation is that chatbots missed their "Apollo Moment" when at the time of need, they could have provided people with useful and life-saving contextual, personalized, and reliable decision support at a scale that the state-of-the-art makes possible. By "Apollo Moment", I refer to the opportunity for a technology to attain the pinnacle of its impact. I review the chatbot capabilities that are feasible with existing methods, identify the potential that chatbots could have met, and highlight the use-cases they were deployed on, the challenges they faced, and gaps that persisted. Finally, I draw lessons that, if implemented, would make them more relevant in future health emergencies.
From Gradient Leakage To Adversarial Attacks In Federated Learning

Deep neural networks (DNN) are widely used in real-life applications despite the lack of understanding on this technology and its challenges. Data privacy is one of the bottlenecks that is yet to be overcome and more challenges in DNN arise when researchers start to pay more attention to DNN vulnerabilities. In this work, we aim to cast the doubts towards the reliability of the DNN with solid evidence particularly in Federated Learning environment by utilizing an existing privacy breaking algorithm which inverts gradients of models to reconstruct the input data. By performing the attack algorithm, we exemplify the data reconstructed from inverting gradients algorithm as a potential threat and further reveal the vulnerabilities of models in representation learning. Pytorch implementation are provided at https://github.com/Jiaqi0602/adversarial-attack-from-leakage/
Enhanced Broad Siamese Network for Facial Emotion Recognition in Human–Robot Interaction

Robotics is one important domain of artificial intelligence applications. Proliferation of robotics makes our lives more convenient and colorful. Emotion recognition is one important task for a robot to communicate with human beings. Although different works have been proposed to complete the task of emotion recognition, conventional methods often require a lot of time and memory resources. Since a robot is often required to react timely and equipped with limited resources, we propose an enhanced Siamese network algorithm to resolve such a problem. We combine Siamese network with broad learning system while further improving mechanism of similarity metric. Experiment results show that our method can achieve a comparable performance to conventional deep learning method while reducing consumption of computing time and memory resources.
A deep dense residual network with reduced parameters for volumetric brain tissue segmentation from MR images

Deep convolutional neural networks (DCNN) have proven to be the state-of-the-art methods for brain tissue segmentation; however, their complex architectures, and the large number of parameters make them computationally expensive and difficult to optimize. In this paper, a novel 3D DCNN architecture, which is built upon the U-Net structure, is presented for compact feature representation and efficient parameter reduction in order to segment the brain tissues into white matter, gray matter, and cerebrospinal fluid (Code is available at: https://github.com/basnetr/U-DenseResNet). The basic idea in the proposed method is to use densely connected convolutional layers and residual skip-connections in order to increase the representation capacity, improve the gradient flow, facilitate easier and better learning, and reduce the number of parameters of the network. The loss functions, cross-entropy, dice similarity, and a combination of the two are used for the training of the proposed network. Experimental results show that the proposed approach provides the best performance on the test dataset of the single-modality IBSR18 dataset containing MR scans of diverse age groups and competitive performance on the multi-modality brain tissue segmentation challenge, iSeg-2017, containing MR scans of infants while reducing, for both the datasets, the parameters ranging from 40% to 98% compared to that of the other deep-learning based architectures. The proposed method significantly reduces the number of parameters of DCNNs while still providing high degree of accuracy. The proposed method can be used for the study of brain structure and development, in detecting a wide range of abnormal tissues, to aid diagnosis, and for guiding surgical procedures.
PhotoCube at the Lifelog Search Challenge 2021

The Lifelog Search Challenge (LSC) is a venue where retrieval system researchers compete in solving tasks to retrieve the correct image from a lifelog collection. At LSC 2021, we introduce the PhotoCube system as a new competitor. PhotoCube is an interactive media retrieval system that considers media items to exist in a hypercube in multidimensional metadata space. To solve tasks, users explore the contents of the hypercube by dynamically (a) applying a variety of filters and (b) projecting the hypercube to a three-dimensional cube that is visualised on screen.
Variational quantum eigensolver techniques for simulating carbon monoxide oxidation

Abstract Variational Quantum Eigensolver (VQE) methods aim to maximize the resources of existing noisy devices. However, they encounter difficulties in simulating molecules of industrially-relevant sizes, such as constructing the efficient ansatz. Adaptive variational algorithms (ADAPT-VQE) can solve this problem but with a significant increase in the number of measurements. Here, we reduce the measurement overhead of ADAPT-VQE via adding operator batches to the ansatz while keeping it compact. We reformulate the previously proposed qubit pool completeness criteria for the tapered qubit space and propose an automated pool construction procedure. Our numerical results indicate that reducing the qubit pool size from polynomial to linear conversely increases the number of measurements. We simulate a set of molecules, participating in the carbon monoxide oxidation processes using the statevector simulator and compare the results with VQE-UCCSD and classical methods. Our results pave the way towards usage of variational approaches for solving practically relevant chemical problems.
Sentiment analysis of social media data based on chaotic coyote optimization algorithm based time weight‐<scp>AdaBoost</scp> support vector machine approach

Summary Sentiment analysis or opinion mining is exploited in business, customer services, and so forth, where people provide their opinions in the form of reviews. However, the people's opinions are in a perplexing form such as, sarcasm, irony, and implied meaning which can cause an impact on sentiment analysis. The only way to analyze these words is through context. Nevertheless, there still exist some issues, to tackle those issues, a lot of research has been conducted by focusing the feature engineering. However, the optimized output has not been acquired yet. Hence, we propose a novel method known as chaotic coyote optimization algorithm (COA) based time weight‐AdaBoost support vector machine (SVM) approach which can be used to attain the precise classifications in context. The proposed time weight‐AdaBoost SVM can be used to circumvent the drift concept issues and can be utilized for the perfect learning of data for further classifications. Further, the class imbalance issues can be overcome by adopting a modified CO algorithm, that is, the chaotic COA. Furthermore, the proposed work performance is analyzed with state‐of‐art works such as DICE, ABCDM, and SVM approaches. The comparative analysis shows that our proposed work classifies the social media content acquired from Twitter more accurately than the other works. Thus our work outperforms all the existing approaches in terms of accuracy, precision, recall, and F1 score.
Patterns of Adults with Low Literacy Skills Interacting with an Intelligent Tutoring System

Abstract A common goal of Intelligent Tutoring Systems (ITS) is to provide learning environments that adapt to the varying abilities and characteristics of users. This type of adaptivity is possible only if the ITS has information that characterizes the learning behaviors of its users and can adjust its pedagogy accordingly. This study investigated an intelligent tutoring system with computer agents ( AutoTutor ) designed to improve comprehension skills in adults with low reading literacy. One goal of this study was to classify adults into different clusters based on their behavioral patterns (accuracy and response time to answer questions) while they interacted with AutoTutor to help them improve their reading comprehension skills. A second goal was to investigate whether adults’ behaviors were associated with different reading components. A third goal was to assess improvements in reading comprehension skills, based on psychometric tests, of different clusters of readers. Performance on AutoTutor was collected in a targeted 100-hour hybrid intervention for adult readers ( n = 252) that included both human teachers and the AutoTutor system. The adults’ average accuracy and response time in AutoTutor were used to cluster the adults into four categories: higher performers (comparatively fast and accurate), conscientious readers (slow but accurate), under-engaged readers (fast at the expense of somewhat lower accuracy) and struggling readers (slow and inaccurate). Two psychometric tests of comprehension were used to assess comprehension. Gains in comprehension scores were highest for conscientious readers, lowest for struggling readers, with higher performing readers and under-engaged readers in between. The results provide guidance to enhance the adaptivity of AutoTutor.
On the Local Planners in the RRT* for Dynamical Systems and Their Reusability for Compound Cost Functionals

In this article, we study properties of local planners for nonholonomic dynamical systems to achieve asymptotic global optimality in a sampling-based planner, namely, the RRT*. Formal results are presented concerning length optimal trajectories under a nonholonomic metric, more specifically, an analysis about necessary or sufficient conditions for the local planners is provided. The analysis makes use of the case of finding time-optimal trajectories for differential drive robots. Later, relevant concepts for complex robotic systems such as detachability and time dominance are introduced. Detachability is a property over the cost functional such that, if respected, it allows the reusability of previously designed local planners for simpler systems into more complex ones. The introduced concepts are further illustrated through a robotic system comprised of a mobile manipulator robot equipped with a camera in the arm&#x2019;s end-effector, whose task is to move between states maintaining a desired object within its field of view. Experiments in a physical mobile manipulator robot are shown, which validate the proposed theoretical modeling.
Pull request latency explained: an empirical overview

Pull request latency evaluation is an essential application of effort evaluation in the pull-based development scenario. It can help the reviewers sort the pull request queue, remind developers about the review processing time, speed up the review process and accelerate software development. There is a lack of work that systematically organizes the factors that affect pull request latency. Also, there is no related work discussing the differences and variations in characteristics in different scenarios and contexts. In this paper, we collected relevant factors through a literature review approach. Then we assessed their relative importance in five scenarios and six different contexts using the mixed-effects linear regression model. The most important factors differ in different scenarios. The length of the description is most important when pull requests are submitted. The existence of comments is most important when closing pull requests, using CI tools, and when the contributor and the integrator are different. When there exist comments, the latency of the first comment is the most important. Meanwhile, the influence of factors may change in different contexts. For example, the number of commits in a pull request has a more significant impact on pull request latency when closing than submitting due to changes in contributions brought about by the review process. Both human and bot comments are positively correlated with pull request latency. In contrast, the bot’s first comments are more strongly correlated with latency, but the number of comments is less correlated. Future research and tool implementation needs to consider the impact of different contexts. Researchers can conduct related studies based on our publicly available datasets and replication scripts.
Multi-Modal Pedestrian Detection with Large Misalignment Based on Modal-Wise Regression and Multi-Modal IoU

The combined use of multiple modalities enables accurate pedestrian detection under poor lighting conditions by using the high visibility areas from these modalities together. The vital assumption for the combination use is that there is no or only a weak misalignment between the two modalities. In general, however, this assumption often breaks in actual situations. Due to this assumption's breakdown, the position of the bounding boxes does not match between the two modalities, resulting in a significant decrease in detection accuracy, especially in regions where the amount of misalignment is large. In this paper, we propose a multimodal Faster-RCNN that is robust against large misalignment. The keys are 1) modal-wise regression and 2) multi-modal IoU for mini-batch sampling. To deal with large misalignment, we perform bounding box regression for both the RPN and detection-head with both modalities. We also propose a new sampling strategy called “multi-modal mini-batch sampling” that integrates the IoU for both modalities. We demonstrate that the proposed method's performance is much better than that of the state-of-the-art methods for data with large misalignment through actual image experiments.
Descending stairs and floors classification as control reference in autonomous smart wheelchair

Disability is a disruption or limitation of a person’s body functions in carrying out daily activities. A person with physical disabilities needs an assistive device such as a wheelchair. The latest wheelchair development is the smart wheelchair. Smart wheelchairs require a control system to detect obstacles quickly. This aims to provide safety, especially for users. One of the obstacles that are quite dangerous is descending stairs. Therefore the researchers propose a descending stairs detection system for smart wheelchairs. The proposed method in this study is the gray level co-occurrence matrix (GLCM) as the feature extraction algorithm, learning vector quantization (LVQ) as the classification algorithm, and sequential forward selection (SFS) for feature selection. Based on the simulation result, the SFS feature selection gets two selected GLCM features. The best accuracy is 94.5% with the selected features, namely contrast and dissimilarity. This result has an increase in accuracy when compared to using the GLCM 6 feature method with the LVQ classification that does not use feature selection, where the method gets 92.5% accuracy in off-time testing. Accuracy decreased to 78.21% when detecting floors and 89.06% when detecting descending stairs in real-time system testing.
Low-Power Ultra-Small Edge AI Accelerators for Image Recognition with Convolution Neural Networks: Analysis and Future Directions

Edge AI accelerators have been emerging as a solution for near customers’ applications in areas such as unmanned aerial vehicles (UAVs), image recognition sensors, wearable devices, robotics, and remote sensing satellites. These applications require meeting performance targets and resilience constraints due to the limited device area and hostile environments for operation. Numerous research articles have proposed the edge AI accelerator for satisfying the applications, but not all include full specifications. Most of them tend to compare the architecture with other existing CPUs, GPUs, or other reference research, which implies that the performance exposé of the articles are not comprehensive. Thus, this work lists the essential specifications of prior art edge AI accelerators and the CGRA accelerators during the past few years to define and evaluate the low power ultra-small edge AI accelerators. The actual performance, implementation, and productized examples of edge AI accelerators are released in this paper. We introduce the evaluation results showing the edge AI accelerator design trend about key performance metrics to guide designers. Last but not least, we give out the prospect of developing edge AI’s existing and future directions and trends, which will involve other technologies for future challenging constraints.
Automated resection planning for bone tumor surgery

Planning for bone tumor resection surgery is a technically demanding and time-consuming task, reliant on manual positioning of cutting planes (CPs). This work describes an automated approach for generating bone tumor resection plans, where the volume of healthy bone collaterally resected with the tumor is minimized through optimized placement of CPs. Particle swarm optimization calculates the optimal position and orientation of the CPs by introducing a single new CP to an existing resection, then optimizing all CPs to find the global minima. The bone bounded by all CPs is collaterally resected with the tumor. The approach was compared to manual resection plans from an experienced surgeon for 20 tumor cases. It was found that a greater number of CPs reduce the collaterally resected healthy bone, with diminishing returns on this improvement after five CPs. The algorithm-generated resection plan with equivalent number of CPs resulted in a statistically significant improvement over manual plans (paired t-test, p < 0.001). The described approach has potential to improve patient outcomes by reducing loss of healthy bone in tumor surgery while offering a surgeon multiple resection plan options.
Slow Breathing Exercise with Multimodal Virtual Reality: A Feasibility Study

Many studies have shown that slow breathing training is beneficial for human health. However, several factors might discourage beginners from continuing their training. For example, a long training period is generally required for benefit realization, and there is no real-time feedback to trainees to adjust their breathing control strategy. To raise the user’s interest in breathing exercise training, a virtual reality system with multimodal biofeedback is proposed in this work. In our system, a realistic human model of the trainee is provided in virtual reality (VR). At the same time, abdominal movements are sensed, and the breathing rate can be visualized. Being aware of the breathing rate, the trainee can regulate his or her breathing to achieve a slower breathing rate. An additional source of tactile feedback is combined with visual feedback to provide a more immersive experience for the trainees. Finally, the user’s satisfaction with the proposed system is reported through questionnaires. Most of the users find it enjoyable to use such a system for mediation training.
Exploring rich intermediate representations for reconstructing 3D shapes from 2D images

Recovering 3D voxelized shapes with fine details from single-view 2D images is an extremely challenging and ill-conditioned problem. Most of the existing methods learn the 3D reconstruction process by encoding the 3D shapes and the 2D images into the same low-dimensional latent vector, which lacks the capacity to capture detailed features in the surface of the 3D object shapes. To address this issue, we propose to explore rich intermediate representation for 3D shape reconstruction by using a newly designed network architecture. We first use a two-steam network to infer the depth map and the topology-specific mean shape from the given 2D image, which forms the intermediate representation prediction branch. The intermediate representations capture the global spatial structure and the visible surface geometric structure, which are important for reconstructing high-quality 3D shapes. Based on the obtained intermediate representation, a novel shape transformation network is then proposed to reconstruct the fine details of the whole 3D object shapes. The experimental results on the challenging ShapeNet and Pix3D datasets show that our approach outperforms the existing state-of-the-art methods.
Video Description: Datasets &amp; Evaluation Metrics

Rapid expansion and the novel phenomenon of deep learning have manifested a variety of proposals and concerns in the area of video description, particularly in the recent past. Automatic event localization and textual alternatives generation for the complex and diverse visual data supplied in a video can be articulated as video description, bridging the two leading realms of computer vision and natural language processing. Several sequence-to-sequence algorithms are being proposed by splitting the task into two segments, namely encoding, i.e., getting and learning the insights of the visual representations, and decoding, i.e., transforming the learned representations to a sequence of words, one at a time. Implemented deep learning approaches have gained a lot of recognition for the reason of their superior computing capabilities and tremendous performance. However, the accomplishment of these algorithms strongly depends on the nature, diversity, and amount of data they are trained, validated and tested on. Techniques applied on insufficient and inadequate train/test data cannot deliver promising conclusions, consequently making it complicated to evaluate the quality of generated results. This survey focuses explicitly on the benchmark datasets, and evaluation metrics developed and deployed for video description tasks and their capabilities and limitations. Finally, we concluded with the need for essential enhancements and encouraging research directions on the topic.
(AD)<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si1.svg"><mml:msup><mml:mrow /><mml:mn>2</mml:mn></mml:msup></mml:math>: Adversarial domain adaptation to defense with adversarial perturbation removal

Deep Neural Networks (DNNs) are demonstrated to be vulnerable to adversarial examples, which are crafted by adding adversarial perturbations to the legitimate examples. To address this issue, some defense methods have been proposed. Among them, the adversarial training (AT) is a popular method to improve the robustness of DNNs. However, theory analysis has shown that in the adversarial training framework, the improvement of the robustness will lead to a decline of standard accuracy. In this paper, we propose a modularized defense framework, namely Adversarial Domain Adaptation to Defense ((AD)2). Different from all adversarial training methods, (AD)2 detects adversarial example using a generative algorithm and applies the adversarial domain adaptation method to remove adversarial perturbation. Experimental results show that (AD)2 is effective to remove the adversarial perturbation and mitigate the odds between the robustness and standard accuracy for DNNs.