Reviewerly/exHarmony huggingface dataset:
Boosting Resolution and Recovering Texture of 2D and 3D Micro‐CT Images with Deep Learning

Abstract Simulation of flow directly at the pore scale depends on high‐quality digital rock images but is constrained by detector hardware. A trade‐off between the image field of view (FOV) and image resolution is made. This can be compensated for with superresolution (SR) techniques that take a wide FOV, low‐resolution (LR) image, and superresolve a high resolution (HR). The Enhanced Deep Super Resolution Generative Adversarial Network (EDSRGAN) is trained on the DeepRock‐SR, a diverse compilation of raw and processed micro‐computed tomography ( CT) images in 2D and 3D. The 2D and 3D trained networks show comparable performance of 50% to 70% reduction in relative error over bicubic interpolation with minimal computational cost during usage. Texture regeneration with EDSRGAN shows superior visual similarity versus Super Resolution Convolutional Neural Network (SRCNN) and other methods. Difference maps show SRCNN recovers large‐scale edge features while EDSRGAN regenerates perceptually indistinguishable high‐frequency texture. Physical accuracy is measured by permeability and phase topology on consistently segmented images, showing EDSRGAN results achieving the closest match. Performance is generalized with augmentation, showing high adaptability to noise and blur. HR images are fed into the network, generating HR‐SR images to extrapolate network performance to subresolution features present in the HR images themselves. Underresolution features are regenerated despite operating outside of trained specifications. Comparison with scanning electron microscopy (SEM) images shows details are consistent with the underlying geometry. Images that are normally constrained by the mineralogy of the rock, by fast transient imaging, or by the energy of the source can be superresolved accurately for further analysis.
MIME: MIMicking Emotions for Empathetic Response Generation

Current approaches to empathetic response generation view the set of emotions expressed in the input text as a flat structure, where all the emotions are treated uniformly.We argue that empathetic responses often mimic the emotion of the user to a varying degree, depending on its positivity or negativity and content.We show that the consideration of these polaritybased emotion clusters and emotional mimicry results in improved empathy and contextual relevance of the response as compared to the state-of-the-art.Also, we introduce stochasticity into the emotion mixture that yields emotionally more varied empathetic responses than the previous work.We demonstrate the importance of these factors to empathetic response generation using both automatic-and human-based evaluations.The implementation of MIME is publicly available at https: //github.com/declare-lab/MIME.
Inter-Slice Context Residual Learning for 3D Medical Image Segmentation

Automated and accurate 3D medical image segmentation plays an essential role in assisting medical professionals to evaluate disease progresses and make fast therapeutic schedules. Although deep convolutional neural networks (DCNNs) have widely applied to this task, the accuracy of these models still need to be further improved mainly due to their limited ability to 3D context perception. In this paper, we propose the 3D context residual network (ConResNet) for the accurate segmentation of 3D medical images. This model consists of an encoder, a segmentation decoder, and a context residual decoder. We design the context residual module and use it to bridge both decoders at each scale. Each context residual module contains both context residual mapping and context attention mapping, the formal aims to explicitly learn the inter-slice context information and the latter uses such context as a kind of attention to boost the segmentation accuracy. We evaluated this model on the MICCAI 2018 Brain Tumor Segmentation (BraTS) dataset and NIH Pancreas Segmentation (Pancreas-CT) dataset. Our results not only demonstrate the effectiveness of the proposed 3D context residual learning scheme but also indicate that the proposed ConResNet is more accurate than six top-ranking methods in brain tumor segmentation and seven top-ranking methods in pancreas segmentation.
Digital Transformation in Banking: A Managerial Perspective on Barriers to Change

The digitalisation of banks is seen as the omnipresent challenge which the banking industry is currently facing. In this digital change process, banks are facing disruptive innovation that requires adaptation of almost all cooperative processes. Digital transformation in the financial industry is associated with obstacles that seem to hinder smooth implementation of digital approaches. This issue has not been adequately addressed in the current academic literature. The main purpose of this qualitative exploratory study is to identify the main perceived obstacles to digital transformation in both the private and commercial banking sectors from a managerial point of view and to analyse them accordingly. The methodology is based on a methodological approach using a combination of contextual interviews with German board members of banks, inductive content analysis, and the exploration of best-practice approaches. The findings revealed that elements of strategy and management, technology and regulation, customers, and employees receive a high level of attention within the digital transformation. The other main barriers can be found in the areas of market knowledge and products, employee and customer participation, and public benefit. Each main barrier is characterised by several sub-barriers of varying importance for the digital transformation of banks and is described in detail.
RAFT: Recurrent All-Pairs Field Transforms for Optical Flow (Extended Abstract)

We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field through a recurrent unit that performs lookups on the correlation volumes. RAFT achieves state-of-the-art performance on the KITTI and Sintel datasets. In addition, RAFT has strong cross-dataset generalization as well as high efficiency in inference time, training speed, and parameter count.
BEVT: BERT Pretraining of Video Transformers

This paper studies the BERT pretraining of video transformers. It is a straightforward but worth-studying extension given the recent success from BERT pretraining of image transformers. We introduce BEVT which decouples video representation learning into spatial representation learning and temporal dynamics learning. In particular, BEVT first performs masked image modeling on image data, and then conducts masked image modeling jointly with masked video modeling on video data. This design is motivated by two observations: 1) transformers learned on image datasets provide decent spatial priors that can ease the learning of video transformers, which are often times computationally-intensive if trained from scratch; 2) discriminative clues, i.e., spatial and temporal information, needed to make correct predictions vary among different videos due to large intra-class and inter-class variations. We conduct extensive experiments on three challenging video benchmarks where BEVT achieves very promising results. On Kinetics 400, for which recognition mostly relies on discriminative spatial representations, BEVT achieves comparable results to strong supervised baselines. On Something-Something-V2 and Diving 48, which contain videos relying on temporal dynamics, BEVT outperforms by clear margins all alternative baselines and achieves state-of-the-art performance with a 71.4% and 87.2% Top-1 accuracy respectively. Code is available at https://github.com/xyzforever/BEVT.
Analyzing the Targets of Hate in Online Social Media

Social media systems allow Internet users a congenial platform to freely express their thoughts and opinions. Although this property represents incredible and unique communication opportunities, it also brings along important challenges. Online hate speech is an archetypal example of such challenges. Despite its magnitude and scale, there is a significant gap in understanding the nature of hate speech on social media. In this paper, we provide the first of a kind systematic large scale measurement study of the main targets of hate speech in online social media. To do that, we gather traces from two social media systems: Whisper and Twitter. We then develop and validate a methodology to identify hate speech on both these systems. Our results identify online hate speech forms and offer a broader understanding of the phenomenon, providing directions for prevention and detection approaches.
Multiview Clustering Based on Non-Negative Matrix Factorization and Pairwise Measurements

As we all know, multiview clustering has become a hot topic in machine learning and pattern recognition. Nonnegative matrix factorization (NMF) has been one popular tool in multiview clustering due to its competitiveness and interpretation. However, the existing multiview clustering methods based on NMF only consider the similarity of intra-view, while neglecting the similarity of inter-view. In this paper, we propose a novel multiview clustering algorithm, named multiview clustering based on NMF and pairwise measurements, which incorporates pairwise co-regularization and manifold regularization with NMF. In the proposed algorithm, we consider the similarity of the inter-view via pairwise co-regularization to obtain the more compact representation of multiview data space. We can also obtain the part-based representation by NMF and preserve the locally geometrical structure of the data space by utilizing the manifold regularization. Furthermore, we give the theoretical proof that the objective function of the proposed algorithm is convergent for multiview clustering. Experimental results show that the proposed algorithm outperforms the state-of-the-arts for multiview clustering.
Trajectory Generation for Multiagent Point-To-Point Transitions via Distributed Model Predictive Control

This paper introduces a novel algorithm for multiagent offline trajectory generation based on distributed model predictive control (DMPC).Central to the algorithm's scalability and success is the development of an on-demand collision avoidance strategy.By predicting future states and sharing this information with their neighbours, the agents are able to detect and avoid collisions while moving towards their goals.The proposed algorithm can be implemented in a distributed fashion and reduces the computation time by more than 85% compared to previous optimization approaches based on sequential convex programming (SCP), while only having a small impact on the optimality of the plans.The approach was validated both through extensive simulations and experimentally with teams of up to 25 quadrotors flying in confined indoor spaces.
GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognition

The dominant approaches for named entity recognitionm (NER) mostly adopt complex recurrent neural networks (RNN), e.g., long-short-term-memory (LSTM). However, RNNs are limited by their recurrent nature in terms of computational efficiency. In contrast, convolutional neural networks (CNN) can fully exploit the GPU parallelism with their feedforward architectures. However, little attention has been paid to performing NER with CNNs, mainly owing to their difficulties in capturing the long-term context information in a sequence. In this paper, we propose a simple but effective CNN-based network for NER, i.e., gated relation network (GRN), which is more capable than common CNNs in capturing long-term context. Specifically, in GRN we firstly employ CNNs to explore the local context features of each word. Then we model the relations between words and use them as gates to fuse local context features into global ones for predicting labels. Without using recurrent layers that process a sentence in a sequential manner, our GRN allows computations to be performed in parallel across the entire sentence. Experiments on two benchmark NER datasets (i.e., CoNLL2003 and Ontonotes 5.0) show that, our proposed GRN can achieve state-of-the-art performance with or without external knowledge. It also enjoys lower time costs to train and test.
SCN: Switchable Context Network for Semantic Segmentation of RGB-D Images

Context representations have been widely used to profit semantic image segmentation. The emergence of depth data provides additional information to construct more discriminating context representations. Depth data preserves the geometric relationship of objects in a scene, which is generally hard to be inferred from RGB images. While deep convolutional neural networks (CNNs) have been successful in solving semantic segmentation, we encounter the problem of optimizing CNN training for the informative context using depth data to enhance the segmentation accuracy. In this paper, we present a novel switchable context network (SCN) to facilitate semantic segmentation of RGB-D images. Depth data is used to identify objects existing in multiple image regions. The network analyzes the information in the image regions to identify different characteristics, which are then used selectively through switching network branches. With the content extracted from the inherent image structure, we are able to generate effective context representations that are aware of both image structures and object relationships, leading to a more coherent learning of semantic segmentation network. We demonstrate that our SCN outperforms state-of-the-art methods on two public datasets.
Drawings of real-world scenes during free recall reveal detailed object and spatial information in memory

Abstract Understanding the content of memory is essential to teasing apart its underlying mechanisms. While recognition tests have commonly been used to probe memory, it is difficult to establish what specific content is driving performance. Here, we instead focus on free recall of real-world scenes, and quantify the content of memory using a drawing task. Participants studied 30 scenes and, after a distractor task, drew as many images in as much detail as possible from memory. The resulting memory-based drawings were scored by thousands of online observers, revealing numerous objects, few memory intrusions, and precise spatial information. Further, we find that visual saliency and meaning maps can explain aspects of memory performance and observe no relationship between recall and recognition for individual images. Our findings show that not only is it possible to quantify the content of memory during free recall, but those memories contain detailed representations of our visual experiences.
Single infrared image enhancement using a deep convolutional neural network

In this paper, we propose a deep learning method for single infrared image enhancement. A fully convolutional neural network (CNN) is used to produce images with enhanced contrast and details. The conditional generative adversarial networks are incorporated into the optimization framework to avoid the background noise being amplified and further enhance the contrast and details. The existing convolutional neural network architectures, such as residual architectures and encoder–decoder architectures, fail to achieve the best results both in terms of network performance and application scope for infrared image enhancement task. To address this problem, we specifically design a new refined convolutional neural architecture that produces visually very appealing results with higher contrast and sharper details compared to other network architectures. Visible images are used for training since there are fewer infrared images. Proper training samples are generated to ensure that the network trained on visible images can be well applied to infrared images. Experiments demonstrate that our approach outperforms existing image enhancement algorithms in terms of contrast and detail enhancement. Code is available at https://github.com/Kuangxd/IE-CGAN.
Auto-Embedding Generative Adversarial Networks For High Resolution Image Synthesis

Generating images via a generative adversarial network (GAN) has attracted much attention recently. However, most of the existing GAN-based methods can only produce lowresolution images of limited quality. Directly generating highresolution images using GANs is nontrivial, and often produces problematic images with incomplete objects. To address this issue, we develop a novel GAN called auto-embedding generative adversarial network, which simultaneously encodes the global structure features and captures the fine-grained details. In our network, we use an autoencoder to learn the intrinsic high-level structure of real images and design a novel denoiser network to provide photo-realistic details for the generated images. In the experiments, we are able to produce 512 × 512 images of promising quality directly from the input noise. The resultant images exhibit better perceptual photo-realism, that is, with sharper structure and richer details, than other baselines on several datasets, including Oxford-102 Flowers, Caltech-UCSD Birds (CUB), High-Quality Large-scale CelebFaces Attributes (CelebAHQ), Large-scale Scene Understanding (LSUN), and ImageNet.
CAPTCHA recognition based on deep convolutional neural network

Aiming at the problems of low efficiency and poor accuracy of traditional CAPTCHA recognition methods, we have proposed a more efficient way based on deep convolutional neural network (CNN). The Dense Convolutional Network (DenseNet) has shown excellent classification performance which adopts cross-layer connection. Not only it effectively alleviates the vanishing-gradient problem, but also dramatically reduce the number of parameters. However, it also has caused great memory consumption. So we improve and construct a new DenseNet for CAPTCHA recognition (DFCR). Firstly, we reduce the number of convolutional blocks and build corresponding classifiers for different types of CAPTCHA images. Secondly, we input the CAPTCHA images of TFrecords format into the DFCR for model training. Finally, we test the Chinese or English CAPTCHAs experimentally with different numbers of characters. Experiments show that the new network not only keeps the primary performance advantages of the DenseNets but also effectively reduces the memory consumption. Furthermore, the recognition accuracy of CAPTCHA with the background noise and character adhesion is above 99.9%.
Look across Elapse: Disentangled Representation Learning and Photorealistic Cross-Age Face Synthesis for Age-Invariant Face Recognition

Despite the remarkable progress in face recognition related technologies, reliably recognizing faces across ages still remains a big challenge. The appearance of a human face changes substantially over time, resulting in significant intraclass variations. As opposed to current techniques for ageinvariant face recognition, which either directly extract ageinvariant features for recognition, or first synthesize a face that matches target age before feature extraction, we argue that it is more desirable to perform both tasks jointly so that they can leverage each other. To this end, we propose a deep Age-Invariant Model (AIM) for face recognition in the wild with three distinct novelties. First, AIM presents a novel unified deep architecture jointly performing cross-age face synthesis and recognition in a mutual boosting way. Second, AIM achieves continuous face rejuvenation/aging with remarkable photorealistic and identity-preserving properties, avoiding the requirement of paired data and the true age of testing samples. Third, we develop effective and novel training strategies for end-to-end learning the whole deep architecture, which generates powerful age-invariant face representations explicitly disentangled from the age variation. Extensive experiments on several cross-age datasets (MORPH, CACD and FG-NET) demonstrate the superiority of the proposed AIM model over the state-of-the-arts. Benchmarking our model on one of the most popular unconstrained face recognition datasets IJB-C additionally verifies the promising generalizability of AIM in recognizing faces in the wild.
A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors

Computer simulations often involve both qualitative and numerical inputs. Existing Gaussian process (GP) methods for handling this mainly assume a different response surface for each combination of levels of the qualitative factors and relate them via a multiresponse cross-covariance matrix. We introduce a substantially different approach that maps each qualitative factor to underlying numerical latent variables (LVs), with the mapped values estimated similarly to the other correlation parameters, and then uses any standard GP covariance function for numerical variables. This provides a parsimonious GP parameterization that treats qualitative factors the same as numerical variables and views them as affecting the response via similar physical mechanisms. This has strong physical justification, as the effects of a qualitative factor in any physics-based simulation model must always be due to some underlying numerical variables. Even when the underlying variables are many, sufficient dimension reduction arguments imply that their effects can be represented by a low-dimensional LV. This conjecture is supported by the superior predictive performance observed across a variety of examples. Moreover, the mapped LVs provide substantial insight into the nature and effects of the qualitative factors. Supplementary materials for the article are available online.
Rate-Utility Optimized Streaming of Volumetric Media for Augmented Reality

Volumetric media, popularly known as holograms, need to be delivered to users using both on-demand and live streaming, for new augmented reality and virtual reality experiences. As in video streaming, hologram streaming must support network adaptivity and fast startup, but must also handle large bandwidths, multiple simultaneously streaming objects, and frequent user interaction, which requires a low delay. In this paper, we introduce the first system to our knowledge designed specifically for streaming volumetric media. The system reduces bandwidth by introducing 3D tiles and culling them or reducing their level of detail depending on their relation to the user's view frustum and distance to the user. Our system reduces latency by introducing a window-based buffer, which in contrast to a queue-based buffer allows insertions near the head of the buffer rather than only at the tail of the buffer, to respond quickly to user interaction. To allocate bits between different tiles across multiple objects, we introduce a simple greedy yet provably optimal algorithm for rate-utility optimization. We introduce utility measures based not only on the underlying quality of the representation, but on the level of detail relative to the user's viewpoint and device resolution. The simulation results show that the proposed algorithm provides a superior quality compared with the existing video-streaming approaches adapted to hologram streaming, in terms of utility and user experience over variable, throughput-constrained networks.