Reviewerly/exHarmony huggingface dataset:
Keeping the Questions Conversational: Using Structured Representations to Resolve Dependency in Conversational Question Answering

Having an intelligent dialogue agent that can engage in conversational question answering (ConvQA) is now no longer limited to Sci-Fi movies only and has, in fact, turned into a reality. These intelligent agents are required to understand and correctly interpret the sequential turns provided as the context of the given question. However, these sequential questions are sometimes left implicit and thus require the resolution of some natural language phenomena such as anaphora and ellipsis. The task of question rewriting has the potential to address the challenges of resolving dependencies amongst the contextual turns by transforming them into intent-explicit questions. Nonetheless, the solution of rewriting the implicit questions comes with some potential challenges such as resulting in verbose questions and taking conversational aspect out of the scenario by generating the self-contained questions. In this paper, we propose a novel framework, CONVSR (CONVQA using Structured Representations) for capturing and generating intermediate representations as conversational cues to enhance the capability of the QA model to better interpret the incomplete questions. We also deliberate how the strengths of this task could be leveraged in a bid to design more engaging and more eloquent conversational agents. We test our model on the QuAC and CANARD data sets and illustrate by experimental results that our proposed framework achieves a better Fl score than the standard question rewriting model.
Seam Carving for Content-Aware Image Resizing

Effective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called seam carving that supports content-aware image resizing for both reduction and expansion. A seam is an optimal 8-connected path of pixels on a single image from top to bottom, or left to right, where optimality is defined by an image energy function. By repeatedly carving out or inserting seams in one direction we can change the aspect ratio of an image. By applying these operators in both directions we can retarget the image to a new size. The selection and order of seams protect the content of the image, as defined by the energy function. Seam carving can also be used for image content enhancement and object removal. We support various visual saliency measures for defining the energy of an image, and can also include user input to guide the process. By storing the order of seams in an image we create multi-size images, that are able to continuously change in real time to fit a given size.
Out-of-Distribution Detection in Deep Learning Models: A Feature Space-Based Approach

The deployment of Deep Learning models requires attention to certain aspects not typically considered during the training phase. One of these is identifying and labeling samples from unknown classes. This is the goal of out-of-distribution (OOD) detection, which enhances the robustness of models in open-world scenarios. There are numerous methods for addressing this problem, using different feature spaces to distinguish between in-distribution and OOD data. One such method is the Open Principal Component Score (OpenPCS), a technique developed for open-set semantic segmentation using intermediate features of a fully convolutional network. This article introduces an extension of OpenPCS for multi-class classification, called OpenPCS-Class. We evaluate our approach in image and text classification tasks using various benchmark datasets and OOD detection methods. We also assess the effect of intermediate layers and network architectures on the OOD detection task. Our method outperformed other methods by up to 6.2% in terms of AUROC and 91% in terms of FPR95.
Content-Preserving Warps for 3D Video Stabilization

We describe a technique that transforms a video from a hand-held video camera so that it appears as if it were taken with a directed camera motion. Our method adjusts the video to appear as if it were taken from nearby viewpoints, allowing 3D camera movements to be simulated. By aiming only for perceptual plausibility, rather than accurate reconstruction, we are able to develop algorithms that can effectively recreate dynamic scenes from a single source video. Our technique first recovers the original 3D camera motion and a sparse set of 3D, static scene points using an off-the-shelf structure-frommotion system. Then, a desired camera path is computed either automatically (e.g., by fitting a linear or quadratic path) or interactively. Finally, our technique performs a least-squares optimization that computes a spatially-varying warp from each input video frame into an output frame. The warp is computed to both follow the sparse displacements suggested by the recovered 3D structure, and avoid deforming the content in the video frame. Our experiments on stabilizing challenging videos of dynamic scenes demonstrate the effectiveness of our technique.
Recovering High Dynamic Range Radiance Maps from Photographs

We present a method of recovering high dynamic range radiance maps from photographs taken with conventional imaging equipment. In our method, multiple photographs of the scene are taken with different amounts of exposure. Our algorithm uses these differently exposed photographs to recover the response function of the imaging process, up to factor of scale, using the assumption of reciprocity. With the known response function, the algorithm can fuse the multiple photographs into a single, high dynamic range radiance map whose pixel values are proportional to the true radiance values in the scene. We demonstrate our method on images acquired with both photochemical and digital imaging processes. We discuss how this work is applicable in many areas of computer graphics involving digitized photographs, including image-based modeling, image compositing, and image processing. Lastly, we demonstrate a few applications of having high dynamic range radiance maps, such as synthesizing realistic motion blur and simulating the response of the human visual system.
Hack your corpus analysis: How AI can assist corpus linguists deal with messy social media data

This short reflection considers the role of Artificial Intelligence (AI) language models in supporting the work of corpus linguists. The focus is on how AI can be used in the analysis of complex and 'noisy' data commonly found in social media corpora. These datasets are characterized by unconventional language usage, shorthand, hashtags, emoji, and various other forms of imaginative expression. The discussion will be grounded in the author's ongoing research into emoji which offers an interesting example of the challenges of working with AI in the context of digital discourse analysis. The primary focus will be on how ChatGPT can support the creation and implementation of regular expressions and code for searching, matching, and manipulating text in corpora. The main objective is to guide the reader through the author's reasoning process as a way of reflecting on the integration of AI technology into the academic practice of corpus linguists.
Acquiring the Reflectance Field of a Human Face

We present a method to acquire the reflectance field of a human face and use these measurements to render the face under arbitrary changes in lighting and viewpoint. We first acquire images of the face from a small set of viewpoints under a dense sampling of incident illumination directions using a light stage. We then construct a reflectance function image for each observed image pixel from its values over the space of illumination directions. From the reflectance functions, we can directly generate images of the face from the original viewpoints in any form of sampled or computed illumination. To change the viewpoint, we use a model of skin reflectance to estimate the appearance of the reflectance functions for novel viewpoints. We demonstrate the technique with synthetic renderings of a person's face under novel illumination and viewpoints.
High Dynamic Range Display Systems

The dynamic range of many real-world environments exceeds the capabilities of current display technology by several orders of magnitude. In this paper we discuss the design of two different display systems that are capable of displaying images with a dynamic range much more similar to that encountered in the real world. The first display system is based on a combination of an LCD panel and a DLP projector, and can be built from off-the-shelf components. While this design is feasible in a lab setting, the second display system, which relies on a custom-built LED panel instead of the projector, is more suitable for usual office workspaces and commercial applications. We describe the design of both systems as well as the software issues that arise. We also discuss the advantages and disadvantages of the two designs and potential applications for both systems.
Enabling affordances of blockchain in agri-food supply chains: A value-driver framework using Q-methodology

The application of blockchain beyond cryptocurrencies has received increasing attention from industry and scholars alike. Given predicted looming food crises, some of the most impactful deployments of blockchains are likely to concern food supply chains. This study outlined how blockchain adoption can result in positive affordances in the food supply chain. Using Q methodology, this study explored the current status of the agri-food supply chain and how blockchain technology could be useful in addressing existing challenges. This theorization leads to the proposition of the 3TIC value-driver framework for determining the enabling affordances of blockchain that would increase shared value for stakeholders. First, we propose a framework based on the most promising features of blockchain technology to overcome current challenges in the agri-food industry. Our value-driver framework is driven by the Q-study findings of respondents closely associated with the agri-food supply chain. This framework can provide supply chain stakeholders with a clear perception of blockchain affordances and serve as a guideline for utilizing appropriate features of technology that match organizations’ capabilities, core competencies, goals, and limitations. Therefore, it could assist top-level decision-makers in systematically evaluating parts of the organization to focus on and improve the infrastructure for successful blockchain implementation along the agri-food supply chain. We conclude by noting certain significant challenges that must be carefully addressed to successfully adopt blockchain technology.
Block-gram: Mining knowledgeable features for efficiently smart contract vulnerability detection

Smart contracts are widely used on the blockchain to implement complex transactions, such as decentralized applications on Ethereum. Effective vulnerability detection of large-scale smart contracts is critical, as attacks on smart contracts often cause huge economic losses. Since it is difficult to repair and update smart contracts, it is necessary to find the vulnerabilities before they are deployed. However, code analysis, which requires traversal paths, and learning methods, which require many features to be trained, are too time-consuming to detect large-scale on-chain contracts. Learning-based methods will obtain detection models from a feature space compared to code analysis methods such as symbol execution. But the existing features lack the interpretability of the detection results and training model, even worse, the large-scale feature space also affects the efficiency of detection. This paper focuses on improving the detection efficiency by reducing the dimension of the features, combined with expert knowledge. In this paper, a feature extraction model Block-gram is proposed to form low-dimensional knowledge-based features from bytecode. First, the metadata is separated and the runtime code is converted into a sequence of opcodes, which are divided into segments based on some instructions (jumps, etc.). Then, scalable Block-gram features, including 4-dimensional block features and 8-dimensional attribute features, are mined for the learning-based model training. Finally, feature contributions are calculated from SHAP values to measure the relationship between our features and the results of the detection model. In addition, six types of vulnerability labels are made on a dataset containing 33,885 contracts, and these knowledge-based features are evaluated using seven state-of-the-art learning algorithms, which show that the average detection latency speeds up 25× to 650×, compared with the features extracted by N-gram, and also can enhance the interpretability of the detection model.
A semi-supervised hierarchical ensemble clustering framework based on a novel similarity metric and stratified feature sampling

Recently, both ensemble clustering and semi-supervised clustering have emerged as important paradigms of traditional clustering. Ensemble clustering seeks to integrate multiple clustering results from different methods or the same methods with different parameters. Semi-supervised clustering involves using a small amount of class membership information in some samples for the learning process. Meanwhile, Semi-Supervised Ensemble Clustering (SSEC) has attracted increasing attention due to its high performance. However, most SSEC algorithms are configured based on partitional clustering techniques, and there are few attempts on hierarchical clustering techniques. Even in existing hierarchy-based SSEC algorithms, prior knowledge is not sufficiently used and is often applied to create primary partitions. To address these problems, we propose a Semi-supervised Hierarchical Ensemble Clustering framework based on a novel Similarity metric and stratified feature Sampling, which we call SHECSS. SHECSS uses the information of all primary partitions according to their strength to calculate the similarity between samples. Also, SHECSS is equipped with a stratified feature sampling mechanism that can improve the diversity of primary partitions and deal with high-dimensional data. Here, the primary partitions are created based on multiple hierarchical clustering techniques, and the target partition is configured by a consensus function based on the clusters clustering policy. Experimental results show the effectiveness and efficiency of SHECSS compared to representative clustering methods.
Operations of data centers with onsite renewables considering greenhouse gas emissions

In a highly technology-dependent society, processing large amounts of data has become essential, resulting in a growing number of data centers. The global data center industry accounts for a significant proportion of the world’s energy consumption. A partial or total transition of the energy sources that power the operation of data centers to cleaner energy, particularly onsite renewable technologies, will help to achieve the greenhouse gas emission reduction goals established in international agreements. However, integrating such technologies is difficult due to the intermittency and availability of renewable sources. Nevertheless, temporal and geographical flexibility for processing part of the data centers’ workload provides an opportunity to efficiently integrate these renewable energy sources. In this paper, we develop an operational scheme for a network of interconnected data centers with a significant electricity supply from onsite technologies consisting of (i) an optimal job migration and scheduling policy for data centers to match their operation and the availability of renewable energy under an emissions cap-and-trade program; (ii) an energy management system to control the data centers’ onsite renewable and grid energy resources; and (iii) a robust optimization approach to account for the intermittency of the renewables and to identify the optimal migration and scheduling policies under the worst-case scenario. A case study is carried out to assess the economic and environmental impacts. The results indicate that the installation of onsite technologies and an optimal migration policy reduce the operational costs and the greenhouse gas emissions of the networks of data centers.
Landmark-based Partial Multi-label Learning with Noise Processing

Multi-label learning (MLL) assumes that all labels are ground-truth, which can be costly or difficult to implement in practice. Partial multi-label learning (PML) provides an alternative by recognizing that each instance corresponds to a set of candidate labels, with only one subset representing the ground-truth label set. However, accurately identifying the ground-truth labels from the candidate set, which may be contaminated with noise, is the main challenge of PML. To address this challenge, we propose a landmark-based PML approach called LbPML, which incorporates noise recognition and space structure processing. We evaluate LbPML against three PML algorithms and three MLL algorithms using datasets from six different domains, and assess its performance using five commonly used evaluation criteria. Our extensive experimental results provide compelling evidence of the effectiveness of our proposed method.
Two-Stream Heterogeneous Graph Network with Dynamic Interactive Learning for Video Question Answering

Video question answering (VideoQA) challenges the joint learning of visual and linguistic knowledge. Whilst the dynamic video-question interaction is not well explored in previous methods, the search for answer clues from textual semantics in the interaction is not valued. To address these issues, this paper proposes a novel Two-Stream Heterogeneous Graph Network (TSHGNet) using Dynamic Interactive Learning (DIL) to accomplish effective reasoning between videos and questions. Inspired by the way people answer questions, the two-stream architecture of TSHGNet is designed to extract question-driven visual cues and video-driven textual semantics, respectively. Therein, for each stream, DIL gradually refines the comprehension of question semantics and the extraction of visual representations through heterogeneous graph interactions. Extensive experiments and qualitative analyses demonstrate improved performances of the proposed TSHGNet on three benchmark datasets in comparison with previous state-of-the-art methods, and the effectiveness of different components of our method.
Explaining Identity-aware Graph Classifiers through the Language of Motifs

Most methods for explaining black-box classifiers (e.g., on tabular data, images, or time series) rely on measuring the impact that removing/perturbing features has on the model output. This forces the explanation language to match the classifier's feature space. However, when dealing with graph data, in which the basic features correspond to the edges describing the graph structure, this matching between features space and explanation language might not be appropriate. Decoupling the feature space (edges) from a desired high-lever explanation language (such as motifs) is thus a major challenge towards developing actionable explanations for graph classification tasks. In this paper we introduce Graphshap, a Shapley-based approach able to provide motif-based explanations for identityaware graph classifiers, assuming no knowledge whatsoever about the model or its training data: the only requirement is that the classifier can be queried as a black-box at will. For the sake of computational efficiency we explore a progressive approximation strategy and show how a simple kernel can efficiently approximate explanation scores, thus allowing Graphshap to scale on scenarios with a large explanation space (i.e., large number of motifs). We showcase Graphshap on a real-world brain-network dataset consisting of patients affected by Autism Spectrum Disorder and a control group. Our experiments highlight how the classification provided by a black-box model can be effectively explained by few connectomics patterns.
PT3: A Transformer-based Model for Sepsis Death Risk Prediction via Vital Signs Time Series

Sepsis is a life-threatening systemic syndrome with a high mortality rate. It is critical for doctors to identify sepsis patients at high risk of death in real time, saving patients and reducing in-hospital mortality. However, current clinical methods use traditional machine learning models to predict the death risk of sepsis patients based on the vital signs and lab test results, which is difficult to achieve real-time prediction. In this work, we propose a novel Transformer-based model named PT3 to predict the death risk of sepsis patients within <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$k$</tex> hours (k = 6, 24, 48) in the future based only on the vital signs time series that can be collected in real time. In clinical settings, the collection intervals of vital signs time series are usually irregular. To address this challenge, we design a time-aware mechanism by using a time decay function to explore temporal correlations between records at different moments. We further introduce an auto-imputing mechanism to our model by using the masked prediction pre-training task. To enhance the representation learning ability, we propose a similarity prediction task, a self-supervised pre-training method, to pre-train our model with triplet-loss function. We validate the effectiveness of PT3 on two public clinical databases, MIMIC-IV and eICU. Experiments results show that our model has effective prediction performance, whose AUC on MIMIC-IV and eICU datasets achieve 0.9067 and 0.8733 respectively, especially in the next 6 hours, outperforming other state-of-the-art deep learning methods.
Generating Questions via Unexploited OCR Texts: Prompt-Based Data Augmentation for TextVQA

Text-based Visual Question Answering (TextVQA) tasks rely on Optical Character Recognition (OCR) text to answer. There have been many models successfully exploring multi-modal features fusing and knowledge reasoning. However, current TextVQA datasets are few and the cost of using manual annotation is too high. So generating pseudo-labeled data is a better choice. In this paper, a prompt-based data augmentation method is proposed. The problems of current data augmentation are solved: 1) the distribution of the number of answer words in the pseudo-labeled data is not consistent with the real dataset. 2) the question forms in the pseudo-labeled data are not diverse. Specifically, prompt words are first matched to the constraints in the questions by finding the same words in the vocabulary. So, our generating model can generate different types of questions when the different prompt words are input. Experiments show that our method is significantly better than other state-of-the-art methods on TextVQA.