Reviewerly/exHarmony huggingface dataset:
Compute to Tell the Tale: Goal-Driven Narrative Generation

Man is by nature a social animal. One important facet of human evolution is through narrative imagination, be it fictional or factual, and to tell the tale to other individuals. The factual narrative, such as news, journalism, field report, etc., is based on real-world events and often requires extensive human efforts to create. In the era of big data where video capture devices are commonly available everywhere, a massive amount of raw videos (including life-logging, dashcam or surveillance footage) are generated daily. As a result, it is rather impossible for humans to digest and analyze these video data. This paper reviews the problem of computational narrative generation where a goal-driven narrative (in the form of text with or without video) is generated from a single or multiple long videos. Importantly, the narrative generation problem makes itself distinguished from the existing literature by its focus on a comprehensive understanding of user goal, narrative structure and open-domain input. We tentatively outline a general narrative generation framework and discuss the potential research problems and challenges in this direction. Informed by the real-world impact of narrative generation, we then illustrate several practical use cases in Video Logging as a Service platform which enables users to get more out of the data through a goal-driven intelligent storytelling AI agent.
A Combination of Visual-Semantic Reasoning and Text Entailment-based Boosting Algorithm for Cheapfake Detection

Misuse of real photographs with conflicting image captions in news items is one case of out-of-context (OOC) misuse of media. To detect out-of-context given pair of news (i.e., captions) and attached image, people should determine the truthfulness of the statement and evaluate whether the triplet talks to the same event. This paper presents a new method to detect the OOC media challenge introduced in ACMMM'22 Grand Challenge on Detecting Cheapfakes. For $Task_1$ (i.e., detect conflicting image-caption triplets), our approach uses bottom-up attention with visual semantic reasoning to extract global features of the image to perform comprehensive image-text matching, multiple models of natural language processing to extract semantic relation of the caption and utilize boosting to improve the accuracy of the performance. Our method achieved an 85.5% accuracy score on 80% of the testing dataset and has 4.5% higher accuracy than baseline. For Task2 (i.e., determine whether a given (Image, Caption) pair is real or fake), we detect the veracity of captioned image base semantic features and correlation between image/caption and achieve 76% accuracy. Our source code is available at https://github.com/latuanvinh1998/Cheapfakes_detection_acmmm. Docker for submission is available at https://hub.docker.com/repository/docker/latuanvinh1998/acmmmcheapfakes.
Blockchain-enabled secure communication for unmanned aerial vehicle (UAV) networks

While 5G can provide high-speed Internet connectivity and over-the-horizon control for Unmanned Aerial Vehicles (UAVs), authentication becomes a key security component in 5G-enabled UAVs. This is due to fact that the communicating entities in the network mostly uses unsecured communication channel to exchange critical surveillance data. Authentication thus plays a crucial role in the 5G-enabled UAV network, providing a range of security services such as credential privacy, Session-Key (SK) security, and secure mutual authentication. However, transparency, anonymity, traceability and centralized control are few major security requirements that cannot be fulfilled by the traditional authentication schemes. One of the upcoming technologies that can provide a solution for present centralized 5G-enabled UAV network is blockchain-based authentication scheme. Motivated from aforementioned discussion, this paper presents a Permissioned Blockchain empowered Secure Authentication and Key Agreement framework in 5G-enabled UAVs. In this framework, first an authentication phase between UAV-to-UAV, UAV-to-Edge Server (ES) and Edge-to-Cloud Server (CS) supporting mutual authentication and key agreement is proposed. The authenticated surveillance data collected from UAV is used by the peer-to-peer CS for transaction verification, block creation and addition using smart contract-based consensus mechanism. The practical implementation of framework shows the effectiveness of the proposed approach.
Cross-lingual Word Sense Disambiguation Using Multilingual Co-occurrence Graphs

Contextual representations of words built using neural language models have been shown to successfully store minor differences between multiple meanings of the same word. However, because these representations are not linked to a semantic network, they leave word meanings undefined, and hence overlook information obtained from the knowledge base. Word sense disambiguation (WSD), or the problem of assigning senses to words in context, has seen a surge in interest with the advent of neural models, along with a considerable improvement in performance. Using syntagmatic information to bridge performance gap between knowledge-based and supervised WSD) is a potentially fruitful research path to pursue. The great majority of languages in the world are believed to be too under-resourced for deep learning technology to be used successfully. Graph learning is a successful improvement to meta-learning methods that allows for better and more consistent learning, as well as setting a new state of the art for specific languages, performing on par with other methods while using just a minimal amount of labeled data. We can therefore use graph learning for WSD. This chapter offers a comprehensive review of multilingual and cross-lingual graph-based methods.
COVID-19 in Canada—The Fourth Through Seventh Waves

This Viewpoint discusses Canada’s experience and response to COVID-19 pandemic–related events during the fourth through seventh waves.
An Effective Hybrid Symbolic Regression–Deep Multilayer Perceptron Technique for PV Power Forecasting

The integration of Photovoltaic (PV) systems requires the implementation of potential PV power forecasting techniques to deal with the high intermittency of weather parameters. In the PV power prediction process, Genetic Programming (GP) based on the Symbolic Regression (SR) model has a widespread deployment since it provides an effective solution for nonlinear problems. However, during the training process, SR models might miss optimal solutions due to the large search space for the leaf generations. This paper proposes a novel hybrid model that combines SR and Deep Multi-Layer Perceptron (MLP) for one-month-ahead PV power forecasting. A case study analysis using a real Australian weather dataset was conducted, where the employed input features were the solar irradiation and the historical PV power data. The main contribution of the proposed hybrid SR-MLP algorithm are as follows: (1) The training speed was significantly improved by eliminating unimportant inputs during the feature selection process performed by the Extreme Boosting and Elastic Net techniques; (2) The hyperparameters were preserved throughout the training and testing phases; (3) The proposed hybrid model made use of a reduced number of layers and neurons while guaranteeing a high forecasting accuracy; (4) The number of iterations due to the use of SR was reduced. The presented simulation results demonstrate the higher forecasting accuracy (reductions of more than 20% for Root Mean Square Error (RMSE) and 30 % for Mean Absolute Error (MAE) in addition to an improvement in the R2 evaluation metric) and robustness (preventing the SR from converging to local minima with the help of the ANN branch) of the proposed SR-MLP model as compared to individual SR and MLP models.
Exploring the perceptions of former ICU patients and clinical staff on barriers and facilitators to the implementation of virtual reality exposure therapy: A qualitative study

Abstract Background Virtual reality (VR) as a digital technology has developed rapidly, becoming more realistic, portable, sensory and easier to navigate. Although studies have found VR to be effective for many clinical applications, patients and clinicians have described several barriers to the successful implementation of this technology. To remove barriers for implementation of VR in health care, a greater understanding is needed of how VR can integrate into clinical environments, particularly complex settings such as an intensive care unit. Aim This study aimed to explore the perceived barriers and facilitators for the implementation of VR exposure therapy for intensive care patients and clinical staff. Study Design A qualitative study using an Interpretative Description approach was undertaken. Semi‐structured focus groups were conducted with 13 participants: nine patients and four health care professionals. Focus groups explored barriers and facilitators of using virtual reality (VR) exposure therapy in intensive care. Thematic analysis was employed to produce codes and themes. Results In total, eight themes describing the perceived barriers and facilitators to implementing VR exposure therapy were identified. Four themes related to the perceived barriers of implementing VR exposure therapy in intensive care were identified: psychological, sensory, environmental and staff competency and confidence. There were a further four themes related to the perceived facilitators to the implementation of VR exposure therapy: staff training, patient capacity, orientation to technology and support during the intervention. Conclusions This study identified novel barriers and facilitators that could be expected when implementing VR exposure therapy for patients' post‐intensive care unit stay. The findings suggest that psychological barriers of fear and apprehension were expected to provoke patient avoidance of exposure therapy. Perceived barriers for staff focused on preparedness to deliver the VR exposure therapy and a lack of technological competence. Both patients and staff stated that a comprehensive induction, orientation and training could facilitate VR exposure therapy, improving engagement. Relevance to Clinical Practice This study has identified that with appropriate staff training, resources, and integration into current patient care pathways, VR exposure therapy may be a valuable intervention to support patient recovery following critical illness. Prior to undertaking VR exposure therapy, patients often need reassurance that side‐effects can be managed, and that they can easily control their virtual exposure experience.
Circular fringe projection profilometry and 3D sensitivity analysis based on extended epipolar geometry

For fringe projection profilometry, the phase-shifting direction needs to fit the position layout of a camera and a projector, or the quality of 3D reconstruction will be poor. Two-direction fringe projection profilometry, i.e. scanning along both horizontal and vertical directions, achieves robust performance, while the number of patterns is doubled comparing with one-direction scanning. In this paper, based on the extended epipolar geometry, we propose circular fringe projection profilometry to overcome the direction issue without doubling the number of patterns. First, by introducing the farthest and nearest points to the epipole in projector space, we can robustly control the spatial frequency of circular fringes in patterns. Second, by analytically relating the circularly-shaped phase to 3D world coordinate, we naturally reduce the number of the equations from 6 down to 3 for the proposed algorithm, and compute 3D point clouds in real time. Finally, we propose 3D sensitivity to conveniently analyze the error of reconstruction point clouds. The experimental results show that, compared with two-direction fringe projection profilometry which is robust in accuracy, the proposed is of a little better quality while the number of the patterns is half.
Deep learning-based system for automatic prediction of triple-negative breast cancer from ultrasound images

Abstract To develop a deep-learning system for the automatic identification of triple-negative breast cancer (TNBC) solely from ultrasound images. A total of 145 patients and 831 images were retrospectively enrolled at Peking Union College Hospital from April 2018 to March 2019. Ultrasound images and clinical information were collected accordingly. Molecular subtypes were determined from immunohistochemical (IHC) results. A CNN with VGG-based architecture was then used to predict TNBC. The model’s performance was evaluated using randomized k-fold stratified cross-validation. A t-SNE analysis and saliency maps were used for model visualization. TNBC was identified in 16 of 145 (11.03%) patients. One hundred fifteen (80%) patients, 15 (10%) patients, and 15 (10%) patients formed the train, validation, and test set respectively. The deep learning system exhibits good efficacy, with an AUC of 0.86 (95% CI: 0.64, 0.95), an accuracy of 85%, a sensitivity of 86%, a specificity of 86%, and an F1-score of 0.74. In addition, the internal representation features learned by the model showed clear differentiation across molecular subtype groups. Such a deep learning system can automatically predict triple-negative breast cancer preoperatively and accurately. It may help to get to more precise and comprehensive management. Graphical Abstract
Privacy‐preserving blockchain‐based contract signing with multi‐party supervision

Abstract With the advancement of information technology, electronic contracts are gradually replacing paper contracts. Blockchain can serve as a decentralized trusted third party to help signatories solve the fairness of electronic contract signing. However, most of the existing blockchain‐based contract signing protocols are only applicable to two‐party signing, and lack the supervision of the contract and the protection of its confidentiality under supervision. This article proposes a privacy‐preserving blockchain‐based contract signing system with multi‐party supervision (CSS), in which multiple supervisors are set up to supervise contracts by voting. Before supervision, an identity‐based multi‐receiver re‐encryption algorithm is used to encrypt the contract, which takes less computation and can control the access rights of different supervisors, thus ensuring the privacy of the contract. Security analysis shows that the proposed CSS system enjoys unforgeability, non‐repudiation and fairness of electronic contract signing, and realizes privacy protection and access control under multi‐party supervision. Performance analysis demonstrates the high efficiency of our construction.
Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks

Light-weight super-resolution (SR) models have received considerable attention for their serviceability in mobile devices. Many efforts employ network quantization to compress SR models. However, these methods suffer from severe performance degradation when quantizing the SR models to ultra-low precision (e.g., 2-bit and 3-bit) with the low-cost layer-wise quantizer. In this paper, we identify that the performance drop comes from the contradiction between the layer-wise symmetric quantizer and the highly asymmetric activation distribution in SR models. This discrepancy leads to either a waste on the quantization levels or detail loss in reconstructed images. Therefore, we propose a novel activation quantizer, referred to as Dynamic Dual Trainable Bounds (DDTB), to accommodate the asymmetry of the activations. Specifically, DDTB innovates in: 1) A layer-wise quantizer with trainable upper and lower bounds to tackle the highly asymmetric activations. 2) A dynamic gate controller to adaptively adjust the upper and lower bounds at runtime to overcome the drastically varying activation ranges over different samples. To reduce the extra overhead, the dynamic gate controller is quantized to 2-bit and applied to only part of the SR networks according to the introduced dynamic intensity. Extensive experiments demonstrate that our DDTB exhibits significant performance improvements in ultra-low precision. For example, our DDTB achieves a 0.70 dB PSNR increase on Urban100 benchmark when quantizing EDSR to 2-bit and scaling up output images to $$\times $$ 4. Code is at https://github.com/zysxmu/DDTB .
Less is More: Generating Grounded Navigation Instructions from Landmarks

We study the automatic generation of navigation instructions from 360° images captured on indoor routes. Existing generators suffer from poor visual grounding, causing them to rely on language priors and hallucinate objects. Our Marky-mt5 system addresses this by focusing on visual landmarks; it comprises a first stage landmark detector and a second stage generator-a multimodal, multilingual, multi-task encoder-decoder. To train it, we bootstrap grounded landmark annotations on top of the Room-across-Room (RxR) dataset. Using text parsers, weak supervision from RxR's pose traces, and a multilingual image-text encoder trained on 1.8b images, we identify 971k English, Hindi and Telugu landmark descriptions and ground them to specific regions in panoramas. On Room-to-Room, human wayfind-ers obtain success rates (SR) of 71% following Marky-mt5's instructions, just shy of their 75% SR following human instructions-and well above SRs with other genera-tors. Evaluations on RxR's longer, diverse paths obtain 61-64% SRs on three languages. Generating such high-quality navigation instructions in novel environments is a step to-wards conversational navigation tools and could facilitate larger-scale training of instruction-following agents.
Folk Theories of Avoiding Content Moderation: How Vaccine-Opposed Influencers Amplify Vaccine Opposition on Instagram

This study analyzes how vaccine-opposed users on Instagram share anti-vaccine content despite facing growing moderation attempts by the platform. Through a thematic analysis of Instagram content (in-feed and ephemeral “stories”) of a sample of vaccine-opposed Instagram users, we explore the observable tactics deployed by vaccine-opposed users in their attempts to avoid content moderation and amplify anti-vaccination content. Tactics range from lexical variations to encode vaccine-related keywords, to the creative use of Instagram features and affordances. The emergence of such tactics exists as a type of “folk theorization”—the cultivation of non-professional knowledge of how visibility on the platform works. Findings highlight the complications of content moderation as a route to minimizing misinformation, the consequences of algorithmic opacity and knowledge-building within problematic online communities.
Dual Contrastive Learning with Anatomical Auxiliary Supervision for Few-Shot Medical Image Segmentation

Few-shot semantic segmentation is a promising solution for scarce data scenarios, especially for medical imaging challenges with limited training data. However, most of the existing few-shot segmentation methods tend to over rely on the images containing target classes, which may hinder its utilization of medical imaging data. In this paper, we present a few-shot segmentation model that employs anatomical auxiliary information from medical images without target classes for dual contrastive learning. The dual contrastive learning module performs comparison among vectors from the perspectives of prototypes and contexts, to enhance the discriminability of learned features and the data utilization. Besides, to distinguish foreground features from background features more friendly, a constrained iterative prediction module is designed to optimize the segmentation of the query image. Experiments on two medical image datasets show that the proposed method achieves performance comparable to state-of-the-art methods. Code is available at: https://github.com/cvszusparkle/AAS-DCL_FSS .
Putting Human-Robot Interaction Research into Design Practice

As the Human-Robot Interaction (HRI) research field is relatively young and interdisciplinary, we spend a good deal of time and effort on communicating with each other, sharing ideas and empirical findings across disciplinary boundaries. While this practice is important and needs to continue, there is an increasingly pressing need for us to also communicate outside of this research community. I have met way too many designers, engineers, product managers, and roboticists, who tell me that they were either unaware that there was an HRI research community or that they have found our research community and have been disappointed by our work. That is unfortunate and it is fixable. Like the Human-Computer Interaction research community, we need to tackle this research-practice gap [1]. In this talk, I will make the case for why and how we might get started. There is so much to be gained by taking a Translational Science approach to HRI research. There are opportunities to do work that is grounded in real world problems, real people, and real robots. Furthermore, there are opportunities to make a difference in the design and deployment of robotic products and services right now. As just one example, in spending time studying robot deployments in service industries, at sea, and in the sky, we kept meeting professional Robot Wranglers. In many of our research projects, graduate students take on the Robot Wrangler role, which we claim will eventually disappear when the robots become “fully” autonomous. Looking across many industries, I believe that Robot Wranglers are here to stay. We could and should study them as real stakeholders and people. As HRI researchers, we have an opportunity to use our work to more directly inform the design of technologies. We can also find inspiration from the study of empirical [2]. I think it is time to roll up our sleeves and get better at communicating and working with the people who are building the future of robotics now. Publishing and presenting peer-reviewed papers is not where our work ends. We have much more work to do in translating our work in ways that will help others outside of our research community actually make use of our insights, engaging in collaborations with them, and coming back here to share and learn from stories about those experiences.
Relieving Long-tailed Instance Segmentation via Pairwise Class Balance

Long-tailed instance segmentation is a challenging task due to the extreme imbalance of training samples among classes. It causes severe biases of the head classes (with majority samples) against the tailed ones. This renders “how to appropriately define and alleviate the bias” one of the most important issues. Prior works mainly use label distribution or mean score information to indicate a coarse-grained bias. In this paper, we explore to excavate the confusion matrix, which carries the fine-grained misclassification details, to relieve the pairwise biases, generalizing the coarse one. To this end, we propose a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences. PCB generates fightback soft labels for regularization during training. Besides, an iterative learning paradigm is developed to support a progressive and smooth regularization in such debiasing. PCB can be plugged and played to any existing method as a complement. Experimental results on LVIS demonstrate that our method achieves state-of-the-art performance without bells and whistles. Superior results across various architectures show the generalization ability. The code and trained models are available at https://github.com/megvii-research/PCB.