Reviewerly/exHarmony huggingface dataset:
Segmentation of nucleus based on dynamic convolution and deep features of stain distribution

Abstract Analysis of the pathology image is important to diagnose cancer of lung, breast and stomach. Segmenting the nucleus is a key step for quantitative analysis, and has significance to the pathology researches and computer aided diagnosis systems. The inconsistency of colour, fuzzy boundary of nucleus and overlapping of cells are the universally acknowledged challenges. To solve these problems, the difference between the inside and outside of nucleus is enhanced by obtaining the distribution of the haematoxylin based on Lambert–Beer's law and the optical characteristics of stains. An inferior encoder, which is supervised by the inferior decoder, is proposed to extract the deep features of the distribution of stains. And these features are fed into the primary encoder to improve the accuracy of segmentation. To relieve the problem that some nuclei are segmented as background because the deep feature is inapparent, dynamic convolution is introduced into the encoders. The experiments show that the proposed model can segment the nucleus in the pathological images more precisely than the compared models. The Dice similarity coefficient (DSC) and panoptic quality (PQ) are 0.810 and 0.512, respectively.
From Detection to Action: a Human-in-the-loop Toolkit for Anomaly Reasoning and Management

Anomalies are often indicators of malfunction or inefficiency in various systems such as manufacturing, healthcare, finance, etc. While the literature is abundant in effective detection algorithms due to this practical relevance, autonomous anomaly detection is rarely used in real-world scenarios. Especially in high-stakes applications, a human-in-the-loop is often involved in processes beyond detection such as sense-making and troubleshooting. Motivated by the financial fraud verification problem, we introduce ALARM (for Analyst-in-the-Loop Anomaly Reasoning and Management); a comprehensive end-to-end framework that supports the anomaly mining cycle from detection to action and is applicable more broadly to domains beyond finance. Besides unsupervised detection of emerging anomalies, it offers anomaly explanations and an interactive GUI for human-in-the-loop processes—visual exploration, sense-making, and ultimately action-taking via designing new detection rules—that help close “the loop” as the new rules complement rule-based supervised detection, typical of many deployed systems in practice. We demonstrate ALARM’s efficacy quantitatively and qualitatively through a series of case with fraud analysts from the financial industry.
Quantifying Outlierness of Funds from their Categories using Supervised Similarity

Mutual fund categorization has become a standard tool for the investment management industry and is extensively used by allocators for portfolio construction and manager selection, as well as by fund managers for peer analysis and competitive positioning. As a result, a (unintended) miscategorization or lack of precision can significantly impact allocation decisions and investment fund managers. Here, we aim to quantify the effect of miscategorization of funds utilizing a machine learning based approach. We formulate the problem of miscategorization of funds as a distance-based outlier detection problem, where the outliers are the data-points that are far from the rest of the data-points in the given feature space. We implement and employ a Random Forest (RF) based method of distance metric learning, and compute the so-called class-wise outlier measures for each data-point to identify outliers in the data. We test our implementation on various publicly available data sets, and then apply it to mutual fund data. We show that there is a strong relationship between the outlier measures of the funds and their future returns and discuss the implications of our findings.
Graph Denoising Networks: A Deep Learning Framework for Equity Portfolio Construction

Graph-based deep learning is a rapidly evolving and practical field due to the ubiquity of graph data and its flexible topology. Although many graph learning frameworks show impressive capabilities, their outputs begin to deteriorate for sufficiently noisy data. In this paper, we look to overcome this shortcoming by introducing the Graph Denoising Network, which combines denoising diffusion methods with graph models in a compounding manner. We prove under certain conditions that this can be construed as an MCMC approach to learning and sampling from the true data distribution. When testing on a graph built from financial returns, we obtain Sharpe Ratios of up to 4.4, and consistently above 2. Compared to a baseline graph convolutional network, we find noticeable improvement and statistical evidence to conclude that graph denoising networks improve performance and attain significant economic benefits. Our findings are applicable to other domains that employ noisy graph-based data, potentially in a time-dependent context.
From random-walks to graph-sprints: a low-latency node embedding framework on continuous-time dynamic graphs

Many real-world datasets have an underlying dynamic graph structure, where entities and their interactions evolve over time. Machine learning models should consider these dynamics in order to harness their full potential in downstream tasks. Previous approaches for graph representation learning have focused on either sampling k-hop neighborhoods, akin to breadth-first search, or random walks, akin to depth-first search. However, these methods are computationally expensive and unsuitable for real-time, low-latency inference on dynamic graphs. To overcome these limitations, we propose graph-sprints a general purpose feature extraction framework for continuous-time-dynamic-graphs (CTDGs) that has low latency and is competitive with state-of-the-art, higher latency models. To achieve this, a streaming, low latency approximation to the random-walk based features is proposed. In our framework, time-aware node embeddings summarizing multi-hop information are computed using only single-hop operations on the incoming edges. We evaluate our proposed approach on three open-source datasets and two in-house datasets, and compare with three state-of-the-art algorithms (TGN-attn, TGN-ID, Jodie). We demonstrate that our graph-sprints features, combined with a machine learning classifier, achieve competitive performance (outperforming all baselines for the node classification tasks in five datasets). Simultaneously, graph-sprints significantly reduce inference latencies, achieving close to an order of magnitude speed-up in our experimental setting.
TGEditor: Task-Guided Graph Editing for Augmenting Temporal Financial Transaction Networks

Recent years have witnessed a growth of research interest in designing powerful graph mining algorithms to discover and characterize the structural pattern of interests from financial transaction networks, motivated by impactful applications including anti-money laundering, identity protection, product promotion, and service promotion. However, state-of-the-art graph mining algorithms often suffer from high generalization errors due to data sparsity, data noisiness, and data dynamics. In the context of mining information from financial transaction networks, the issues of data sparsity, noisiness, and dynamics become particularly acute. Ensuring accuracy and robustness in such evolving systems is of paramount importance. Motivated by these challenges, we propose a fundamental transition from traditional mining to augmentation in the context of financial transaction networks. To navigate this paradigm shift, we introduce TGEditor, a versatile task-guided temporal graph augmentation framework. This framework has been crafted to concurrently preserve the temporal and topological distribution of input financial transaction networks, whilst leveraging the label information from pertinent downstream tasks, denoted as , inclusive of crucial downstream tasks like fraudulent transaction classification. In particular, to efficiently conduct task-specific augmentation, we propose two network editing operators that can be seamlessly optimized via adversarial training, while simultaneously capturing the dynamics of the data: Add operator aims to recover the missing temporal links due to data sparsity, and Prune operator is formulated to remove irrelevant/noisy temporal links due to data noisiness. Extensive results on financial transaction networks demonstrate that TGEditor 1) well preserves the data distribution of the original graph and 2) notably boosts the performance of the prediction models in the tasks of vertex classification and fraudulent transaction detection.
Fine-Tuning Pretrained Language Models to Enhance Dialogue Summarization in Customer Service Centers

The application of pretrained language models in real-world business domains has gained significant attention. However, research on the practical use of generative artificial intelligence (AI) to address real-world downstream tasks is limited. This study aims to enhance the routine tasks of customer service (CS) representatives, particularly in the finance domain, by applying a fine-tuning method to dialogue summarization in CS centers. KakaoBank handles an average of 15,000 CS calls daily. By employing a fine-tuning method using real-world CS dialogue data, we can reduce the time required to summarize CS dialogues and standardize summarization skills. To ensure effective dialogue summarization in the finance domain, pretrained language models should acquire additional knowledge and skills, such as specific knowledge of financial products, problem-solving abilities, and the capacity to handle emotionally charged customers. In this study, we developed a reference fine-tuned model using Polyglot-Ko (5.8B) as the baseline PLM and a dataset containing a wide range of zero-shot instructions and partially containing summarization instructions. We compared this reference model with another model fine-tuned using KakaoBank’s CS dialogues and summarization data as the instruct dataset. The results demonstrated that the fine-tuned model based on KakaoBank’s internal datasets outperformed the reference model, showing a 199% and 12% improvement in ROUGE-L and RDASS, respectively. This study emphasizes the significance of task-specific fine-tuning using appropriate instruct datasets for effective performance in specific downstream tasks. Considering its practical use, we suggest that fine-tuning using real-world instruct datasets is a powerful and cost-effective technique for developing generative AI in the business domain.
FinBERT-FOMC: Fine-Tuned FinBERT Model with Sentiment Focus Method for Enhancing Sentiment Analysis of FOMC Minutes

In this research project, we used the financial texts published by the Federal Open Market Committee (FOMC), known as the FOMC Minutes, for sentiment analysis. The pre-trained FinBERT model, a state-of-the-art transformer-based model trained for NLP tasks in finance, was utilized for that. The focus of this research has been on improving the predictive performance of complex financial sentences, as our problem analysis has shown that such sentences pose a significant challenge to existing models. To accomplish this objective the original FinBERT model was fine-tuned for domain-specific sentiment analysis. A strategy, referred to as Sentiment Focus (SF) was utilized to reduce the complexity of sentences, making them more amenable to accurate sentiment predictions.
NFT Primary Sale Price and Secondary Sale Prediction via Deep Learning

Non Fungible Tokens (NFTs) are blockchain-based unique digital assets defining ownership deeds. They can characterize various different objects such as collectible, art, and in-game items. In general, NFTs are encoded by blockchains smart contracts, and they are traded via cryptocurrencies. Their price and investors attention on them has remarkably increased especially in 2021, making them promising alternative class of investment. Surprisingly, predicting their prices has only recently started to be analyzed systematically.
Aerial Diffusion: Text Guided Ground-to-Aerial View Synthesis from a Single Image using Diffusion Models

We present a novel method, Aerial Diffusion, for generating aerial views from a single ground-view image using text guidance. Aerial Diffusion leverages a pretrained text-image diffusion model for prior knowledge. We address two main challenges corresponding to domain gap between the ground-view and the aerial view and the two views being far apart in the text-image embedding manifold. Our approach uses a homography inspired by inverse perspective mapping prior to finetuning the pretrained diffusion model. Aerial Diffusion uses an alternating sampling strategy to compute the optimal solution on complex high-dimensional manifold and generate a high-fidelity (w.r.t. ground view) aerial image. We demonstrate the quality and versatility of Aerial Diffusion on a plethora of images and prove the effectiveness of our method with extensive ablations and comparisons. To the best of our knowledge, Aerial Diffusion is the first approach that performs single image ground-to-aerial translation in an unsupervised manner. The full paper and code can be found at https://arxiv.org/abs/2303.11444.
Training Orchestral Conductors in Beating Time

Orchestral conducting involves a rich vocabulary of gestures and so training conductors is challenging. We discuss how virtual reality and gesture detection could be used to aid this process. We describe our pilot interface for training conductors in basic beat patterns, using gestural input and virtual reality output. We investigated both positional and acceleration-based detection of beats, concluding that that the best way to detect beats reliably is to identify maxima in acceleration, that is, those moments in a beat that would appear as a flick to a human player. This practical evidence supports Wöllner’s theory of how human players detect beats. We trialled our system with novices and experts, with a range of beating styles.
Improving the Robustness of Financial Models through Identification of the Minimal Vulnerable Feature Set

Research in adversarial robustness has primarily focused on neural networks in domains like computer vision, neglecting heterogeneous tabular datasets prevalent in finance. The financial domain, in particular, faces a heightened risk, where malicious actors may exploit ML model vulnerabilities to manipulate transactions and gain unauthorized access. To address this gap, we aim to simulate the adversaries’ intentions using heterogeneous tabular data and focus on identifying a minimal vulnerable set of features that are most susceptible to an external attack. Identifying such features can enable developers to safeguard their models against adversarial attacks by updating or refining the rules of deployed models. To this effect, a GAN-based architecture, termed as the Feature Selector Network, has been proposed for learning the minimal vulnerable feature set. Experimental evaluation shows a significant reduction in attack generation time and the number of queries. The proposed method is tested using attack imperceptibility performance metrics, the number of queries, and the time to generate attacks using existing state-of-the-art attack algorithms. We observed up to a reduction in the number of queries and in overall attack generation time, along with a significant improvement in imperceptibility metrics like the Norm of perturbations and distance to closest neighbor while achieving a good success rate. Further experimental analysis suggested that the model trained on the adversaries generated using the proposed pipeline resulted in more than a decrease in the adversarial attack success rate on the test set, thus allowing developers a robust technique for safeguarding ML models.
Decision-Aware Conditional GANs for Time Series Data

We introduce the decision-aware time-series conditional generative adversarial network (DAT-CGAN), a method for the generation of time-series data that is designed to support decision-making. The framework adopts a multi-Wasserstein loss on decision-related quantities and an overlapped block-sampling approach for sample efficiency. We characterize the generalization properties of DAT-CGAN and in application to a multi-period portfolio choice problem and financial time series data, we demonstrate better training stability and generative quality in regard to both raw data and decision-related quantities than strong GAN-based baselines.
CLIP-Head: Text-Guided Generation of Textured Neural Parametric 3D Head Models

We propose CLIP-Head, a novel approach towards text-driven neural parametric 3D head model generation. Our method takes simple text prompts in natural language, describing the appearance & facial expressions, and generates 3D neural head avatars with accurate geometry and high-quality texture maps. Unlike existing approaches, which use conventional parametric head models with limited control and expressiveness, we leverage Neural Parametric Head Models (NPHM), offering disjoint latent codes for the disentangled encoding of identities and expressions. To facilitate the text-driven generation, we propose two weakly-supervised mapping networks to map the CLIP’s encoding of input text prompt to NPHM’s disjoint identity and expression vector. The predicted latent codes are then fed to a pre-trained NPHM network to generate 3D head geometry. Since NPHM mesh doesn’t support textures, we propose a novel aligned parametrization technique, followed by text-driven generation of texture maps by leveraging a recently proposed controllable diffusion model for the task of text-to-image synthesis. Our method is capable of generating 3D head meshes with arbitrary appearances and a variety of facial expressions, along with photoreal texture details. We show superior performance with existing state-of-the-art methods, both qualitatively & quantitatively, and demonstrate potentially useful applications of our method. We have released our code at https://raipranav384.github.io/clip_head.
Footstep Detection for Film Sound Production

In this paper, we presented a footstep detection method, which could assist sound editors in positioning the character’s footsteps on the timeline of the film. Based on it, a footstep detection system was designed for film sound production. Considering the characteristics of human motion and the needs of film sound production, our method included two parts: data preprocessing and footstep detection modeling. Experiments on various types of shots showed the good generalization and high accuracy of our method. The application evaluation demonstrated the high efficiency of the system.
Portfolio Optimization via Credal Probabilistic Circuits

Portfolio optimization is a crucial part of many investment approaches and is arguably employed by almost all traders in a way or another. We introduce novel approaches for determining optimal weights for portfolios using a class of robust probabilistic generative models. Specifically, we utilize credal probabilistic circuits, a type of generative model known for their ability to perform efficient exact probabilistic inferences and to handle uncertainty in a sound manner. To account for model or epistemic uncertainty, these models use the theory of imprecise probability. Sets of parameter values represent perturbations of the probabilistic circuit model and can be interpreted as an uncertainty-aware correction of the parameters of an underlying portfolio. We call the result as credal portfolio. We propose a method for determining the amount of perturbation that well-captures the uncertainty of the problem, which is employed for the analysis of investments with real-world daily stock market data, showing promising results when compared to usual approaches.
Monte Carlo Denoising via Multi-scale Auxiliary Feature Fusion Guided Transformer

Deep learning-based single-frame Monte Carlo denoising techniques have demonstrated remarkable results in photo-realistic rendering research. However, the current state-of-the-art methods relying on self-attention mechanisms underutilize auxiliary features and struggle to preserve intricate high-frequency details in complex scenes. Employing a generative adversarial architecture, we present a transformer-based denoising network guided by multi-scale auxiliary feature. The proposed U-shaped denoising network extracts multi-scale texture and geometric features from auxiliaries, modulating them to guide the improved transformer module’s denoising process. The improved transformer module employs cross-channel self-attention to capture non-local relationships with near-linear computational complexity. Additionally, a gating mechanism is introduced in the transformer module’s feed-forward network, enhancing information flow. Extensive experiments on noisy images with varied per-pixel sampling rates demonstrate the method’s superiority in quantitative metrics and visual perception compared with state-of-the-art methods. Our method excels notably in intricate scenes with complex hair and texture details, which are historically challenging to denoise.
SynthDa: Exploiting Existing Real-World Data for Usable and Accessible Synthetic Data Generation

Acquiring real-world data for computer vision presents challenges such as data scarcity, high costs, and privacy concerns. We introduce SynthDa, an automated approach for usable synthetic data generation (SDG) that empowers users with varying expertise to create diverse synthetic data from existing real-world datasets. It combines pose estimation, synthetic scene creation, and domain randomization to offer data variants. Ease of SDG through SynthDa enables different permutations and combinations of synthetic data that allow users to explore efficacy of various data configurations in relation to their specific AI tasks. Our experiments across multiple existing datasets and models demonstrate the utility of SynthDa in challenging nuances such as the “more data, the better” paradigm; revealing that excessive synthetic data may degrade performance and vice versa. In a pilot user study with 24 participants, we show the perceived usefulness of SynthDa as a promising SDG tool for overcoming challenges related to real-world data acquisition.
Reinforcement Learning Based Automated Path Planning in Garden Environment using Depth - RAPiG-D

Path planning by employing Reinforcement Learning is a versatile implementation that can account for the ability of a robot to autonomously map any unknown environment. In this paper, such a hardware implementation is proposed and tested by making use of the SARSA algorithm for path planning and by utilizing stereovision for depth estimation based obstacle detection. The robot is tested in a cell-based environment – 3x3 with 2 obstacles. The goal is to map the environment by detecting and mapping the obstacles and finding the ideal route to the destination. The robot starts at one end of the environment and runs through it for a specified number of episodes, and it is observed that the robot can accurately identify and map obstacles and find the shortest path to the destination in under 10 episodes. Currently, the destination is a fixed point and is taken as the other diagonal end of the environment.