OpenRaiser/Intern-Atlas huggingface dataset:
The style transformer with common knowledge optimization for image-text retrieval
Venue: arXiv cs.CV 2023
Year: 2023

Image-text retrieval which associates different modalities has drawn broad attention due to its excellent research value and broad real-world application. However, most of the existing methods haven't taken the high-level semantic relationships ("style embedding") and common knowledge from multi-modalities into full consideration. To this end, we introduce a novel style transformer network with common knowledge optimization (CKSTN) for image-text retrieval. The main module is the common knowledge adaptor (CKA) with both the style embedding extractor (SEE) and the common knowledge optimization (CKO) modules. Specifically, the SEE uses the sequential update strategy to effectively connect the features of different stages in SEE. The CKO module is introduced to dynamically capture the latent concepts of common knowledge from different modalities. Besides, to get generalized temporal common knowledge, we propose a sequential update strategy to effectively integrate the features of different layers in SEE with previous common feature units. CKSTN demonstrates the superiorities of the state-of-the-art methods in image-text retrieval on MSCOCO and Flickr30K datasets. Moreover, CKSTN is constructed based on the lightweight transformer which is more convenient and practical for the application of real scenes, due to the better performance and lower parameters.
Unsupervised Pathology Detection: A Deep Dive Into the State of the Art
Venue: arXiv cs.CV 2023
Year: 2023

Deep unsupervised approaches are gathering increased attention for applications such as pathology detection and segmentation in medical images since they promise to alleviate the need for large labeled datasets and are more generalizable than their supervised counterparts in detecting any kind of rare pathology. As the Unsupervised Anomaly Detection (UAD) literature continuously grows and new paradigms emerge, it is vital to continuously evaluate and benchmark new methods in a common framework, in order to reassess the state-of-the-art (SOTA) and identify promising research directions. To this end, we evaluate a diverse selection of cutting-edge UAD methods on multiple medical datasets, comparing them against the established SOTA in UAD for brain MRI. Our experiments demonstrate that newly developed feature-modeling methods from the industrial and medical literature achieve increased performance compared to previous work and set the new SOTA in a variety of modalities and datasets. Additionally, we show that such methods are capable of benefiting from recently developed self-supervised pre-training algorithms, further increasing their performance. Finally, we perform a series of experiments in order to gain further insights into some unique characteristics of selected models and datasets. Our code can be found under https://github.com/iolag/UPD_study/.
Impact of Channel Aging on Dual-Function Radar-Communication Systems: Performance Analysis and Resource Allocation
Venue: arXiv cs.IT 2023
Year: 2023

In conventional dual-function radar-communication (DFRC) systems, the radar and communication channels are routinely estimated at fixed time intervals based on their worst-case operation scenarios. Such situation-agnostic repeated estimations cause significant training overhead and dramatically degrade the system performance, especially for applications with dynamic sensing/communication demands and limited radio resources. In this paper, we leverage the channel aging characteristics to reduce training overhead and to design a situation-dependent channel re-estimation interval optimization-based resource allocation for performance improvement in a multi-target tracking DFRC system. Specifically, we exploit the channel temporal correlation to predict radar and communication channels for reducing the need of training preamble retransmission. Then, we characterize the channel aging effects on the Cramer-Rao lower bounds (CRLBs) for radar tracking performance analysis and achievable rates with maximum ratio transmission (MRT) and zero-forcing (ZF) transmit beamforming for communication performance analysis. In particular, the aged CRLBs and achievable rates are derived as closed-form expressions with respect to the channel aging time, bandwidth, and power. Based on the analyzed results, we optimize these factors to maximize the average total aged achievable rate subject to individual target tracking precision demand, communication rate requirement, and other practical constraints. Since the formulated problem belongs to a non-convex problem, we develop an efficient one-dimensional search based optimization algorithm to obtain its suboptimal solutions. Finally, simulation results are presented to validate the correctness of the derived theoretical results and the effectiveness of the proposed allocation scheme.
Algorithmic Governance for Explainability: A Comparative Overview of Progress and Trends
Venue: arXiv cs.CY 2023
Year: 2023

The explainability of AI has transformed from a purely technical issue to a complex issue closely related to algorithmic governance and algorithmic security. The lack of explainable AI (XAI) brings adverse effects that can cross all economic classes and national borders. Despite efforts in governance, technical, and policy exchange have been made in XAI by multiple stakeholders, including the public sector, enterprises, and international organizations, respectively. XAI is still in its infancy. Future applications and corresponding regulatory instruments are still dependent on the collaborative engagement of all parties.
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
Venue: arXiv cs.LG 2023
Year: 2023

ML models are ubiquitous in real world applications and are a constant focus of research. At the same time, the community has started to realize the importance of protecting the privacy of ML training data. Differential Privacy (DP) has become a gold standard for making formal statements about data anonymization. However, while some adoption of DP has happened in industry, attempts to apply DP to real world complex ML models are still few and far between. The adoption of DP is hindered by limited practical guidance of what DP protection entails, what privacy guarantees to aim for, and the difficulty of achieving good privacy-utility-computation trade-offs for ML models. Tricks for tuning and maximizing performance are scattered among papers or stored in the heads of practitioners. Furthermore, the literature seems to present conflicting evidence on how and whether to apply architectural adjustments and which components are "safe" to use with DP. This work is a self-contained guide that gives an in-depth overview of the field of DP ML and presents information about achieving the best possible DP ML model with rigorous privacy guarantees. Our target audience is both researchers and practitioners. Researchers interested in DP for ML will benefit from a clear overview of current advances and areas for improvement. We include theory-focused sections that highlight important topics such as privacy accounting and its assumptions, and convergence. For a practitioner, we provide a background in DP theory and a clear step-by-step guide for choosing an appropriate privacy definition and approach, implementing DP training, potentially updating the model architecture, and tuning hyperparameters. For both researchers and practitioners, consistently and fully reporting privacy guarantees is critical, and so we propose a set of specific best practices for stating guarantees.
A Systematic Analysis of Vocabulary and BPE Settings for Optimal Fine-tuning of NMT: A Case Study of In-domain Translation
Venue: arXiv cs.CL 2023
Year: 2023

The effectiveness of Neural Machine Translation (NMT) models largely depends on the vocabulary used at training; small vocabularies can lead to out-of-vocabulary problems -- large ones, to memory issues. Subword (SW) tokenization has been successfully employed to mitigate these issues. The choice of vocabulary and SW tokenization has a significant impact on both training and fine-tuning an NMT model. Fine-tuning is a common practice in optimizing an MT model with respect to new data. However, new data potentially introduces new words (or tokens), which, if not taken into consideration, may lead to suboptimal performance. In addition, the distribution of tokens in the new data can differ from the distribution of the original data. As such, the original SW tokenization model could be less suitable for the new data. Through a systematic empirical evaluation, in this work we compare different strategies for SW tokenization and vocabulary generation with the ultimate goal to uncover an optimal setting for fine-tuning a domain-specific model. Furthermore, we developed several (in-domain) models, the best of which achieves 6 BLEU points improvement over the baseline.
Coordination of Multiple Robots along Given Paths with Bounded Junction Complexity
Venue: arXiv cs.RO 2023
Year: 2023

We study a fundamental NP-hard motion coordination problem for multi-robot/multi-agent systems: We are given a graph $G$ and set of agents, where each agent has a given directed path in $G$. Each agent is initially located on the first vertex of its path. At each time step an agent can move to the next vertex on its path, provided that the vertex is not occupied by another agent. The goal is to find a sequence of such moves along the given paths so that each reaches its target, or to report that no such sequence exists. The problem models guidepath-based transport systems, which is a pertinent abstraction for traffic in a variety of contemporary applications, ranging from train networks or Automated Guided Vehicles (AGVs) in factories, through computer game animations, to qubit transport in quantum computing. It also arises as a sub-problem in the more general multi-robot motion-planning problem. We provide a fine-grained tractability analysis of the problem by considering new assumptions and identifying minimal values of key parameters for which the problem remains NP-hard. Our analysis identifies a critical parameter called vertex multiplicity (VM), defined as the maximum number of paths passing through the same vertex. We show that a prevalent variant of the problem, which is equivalent to Sequential Resource Allocation (concerning deadlock prevention for concurrent processes), is NP-hard even when VM is 3. On the positive side, for VM $\le$ 2 we give an efficient algorithm that iteratively resolves cycles of blocking relations among agents. We also present a variant that is NP-hard when the VM is 2 even when $G$ is a 2D grid and each path lies in a single grid row or column. By studying highly distilled yet NP-hard variants, we deepen the understanding of what makes the problem intractable and thereby guide the search for efficient solutions under practical assumptions.
UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers
Venue: arXiv cs.IR 2023
Year: 2023

Many information retrieval tasks require large labeled datasets for fine-tuning. However, such datasets are often unavailable, and their utility for real-world applications can diminish quickly due to domain shifts. To address this challenge, we develop and motivate a method for using large language models (LLMs) to generate large numbers of synthetic queries cheaply. The method begins by generating a small number of synthetic queries using an expensive LLM. After that, a much less expensive one is used to create large numbers of synthetic queries, which are used to fine-tune a family of reranker models. These rerankers are then distilled into a single efficient retriever for use in the target domain. We show that this technique boosts zero-shot accuracy in long-tail domains and achieves substantially lower latency than standard reranking methods.
Of Degens and Defrauders: Using Open-Source Investigative Tools to Investigate Decentralized Finance Frauds and Money Laundering
Venue: arXiv cs.CR 2023
Year: 2023

Fraud across the decentralized finance (DeFi) ecosystem is growing, with victims losing billions to DeFi scams every year. However, there is a disconnect between the reported value of these scams and associated legal prosecutions. We use open-source investigative tools to (1) investigate potential frauds involving Ethereum tokens using on-chain data and token smart contract analysis, and (2) investigate the ways proceeds from these scams were subsequently laundered. The analysis enabled us to (1) uncover transaction-based evidence of several rug pull and pump-and-dump schemes, and (2) identify their perpetrators' money laundering tactics and cash-out methods. The rug pulls were less sophisticated than anticipated, money laundering techniques were also rudimentary and many funds ended up at centralized exchanges. This study demonstrates how open-source investigative tools can extract transaction-based evidence that could be used in a court of law to prosecute DeFi frauds. Additionally, we investigate how these funds are subsequently laundered.
Planning for Attacker Entrapment in Adversarial Settings
Venue: arXiv cs.AI 2023
Year: 2023

In this paper, we propose a planning framework to generate a defense strategy against an attacker who is working in an environment where a defender can operate without the attacker's knowledge. The objective of the defender is to covertly guide the attacker to a trap state from which the attacker cannot achieve their goal. Further, the defender is constrained to achieve its goal within K number of steps, where K is calculated as a pessimistic lower bound within which the attacker is unlikely to suspect a threat in the environment. Such a defense strategy is highly useful in real world systems like honeypots or honeynets, where an unsuspecting attacker interacts with a simulated production system while assuming it is the actual production system. Typically, the interaction between an attacker and a defender is captured using game theoretic frameworks. Our problem formulation allows us to capture it as a much simpler infinite horizon discounted MDP, in which the optimal policy for the MDP gives the defender's strategy against the actions of the attacker. Through empirical evaluation, we show the merits of our problem formulation.
An Improved Christofides Mechanism for Local Differential Privacy Framework
Venue: arXiv cs.CR 2023
Year: 2023

The development of Internet technology enables an analysis on the whole population rather than a certain number of samples, and leads to increasing requirement for privacy protection. Local differential privacy (LDP) is an effective standard of privacy measurement; however, its large variance of mean estimation causes challenges in application. To address this problem, this paper presents a new LDP approach, an improved Christofides mechanism. It compared four statistical survey methods for conducting surveys on sensitive topics -- modified Warner, Simmons, Christofides, and the improved Christofides mechanism. Specifically, Warner, Simmons and Christofides mechanisms have been modified to draw a sample from the population without replacement, to decrease variance. Furthermore, by drawing cards without replacement based on modified Christofides mechanism, we introduce a new mechanism called the improved Christofides mechanism, which is found to have the smallest variance under certain assumption when using LDP as a measurement of privacy leakage. The assumption is do satisfied usually in the real world. Actually, we decrease the variance to 28.7% of modified Christofides mechanism's variance in our experiment based on the HCOVANY dataset -- a real world dataset of IPUMS USA. This means our method gets a more accurate estimate by using LDP as a measurement of privacy leakage. This is the first time the improved Christofides mechanism is proposed for LDP framework based on comparative analysis of four mechanisms using LDP as the same measurement of privacy leakage.
AMIGO: Sparse Multi-Modal Graph Transformer with Shared-Context Processing for Representation Learning of Giga-pixel Images
Venue: arXiv cs.CV 2023
Year: 2023

Processing giga-pixel whole slide histopathology images (WSI) is a computationally expensive task. Multiple instance learning (MIL) has become the conventional approach to process WSIs, in which these images are split into smaller patches for further processing. However, MIL-based techniques ignore explicit information about the individual cells within a patch. In this paper, by defining the novel concept of shared-context processing, we designed a multi-modal Graph Transformer (AMIGO) that uses the celluar graph within the tissue to provide a single representation for a patient while taking advantage of the hierarchical structure of the tissue, enabling a dynamic focus between cell-level and tissue-level information. We benchmarked the performance of our model against multiple state-of-the-art methods in survival prediction and showed that ours can significantly outperform all of them including hierarchical Vision Transformer (ViT). More importantly, we show that our model is strongly robust to missing information to an extent that it can achieve the same performance with as low as 20% of the data. Finally, in two different cancer datasets, we demonstrated that our model was able to stratify the patients into low-risk and high-risk groups while other state-of-the-art methods failed to achieve this goal. We also publish a large dataset of immunohistochemistry images (InUIT) containing 1,600 tissue microarray (TMA) cores from 188 patients along with their survival information, making it one of the largest publicly available datasets in this context.
Evolving Deep Neural Network by Customized Moth Flame Optimization Algorithm for Underwater Targets Recognition
Venue: arXiv cs.NE 2023
Year: 2023

This chapter proposes using the Moth Flame Optimization (MFO) algorithm for finetuning a Deep Neural Network to recognize different underwater sonar datasets. Same as other models evolved by metaheuristic algorithms, premature convergence, trapping in local minima, and failure to converge in a reasonable time are three defects MFO confronts in solving problems with high-dimension search space. Spiral flying is the key component of the MFO as it determines how the moths adjust their positions in relation to flames; thereby, the shape of spiral motions can regulate the transition behavior between the exploration and exploitation phases. Therefore, this chapter investigates the efficiency of seven spiral motions with different curvatures and slopes in the performance of the MFO, especially for underwater target classification tasks. To assess the performance of the customized model, in addition to benchmark Sejnowski & Gorman's dataset, two experimental sonar datasets, i.e., the passive sonar and active datasets, are exploited. The results of MFO and its modifications are compared to four novel nature-inspired algorithms, including Heap-Based Optimizer (HBO), Chimp Optimization Algorithm (ChOA), Ant Lion Optimization (ALO), Stochastic Fractals Search (SFS), as well as the classic Particle Swarm Optimization (PSO). The results confirm that the customized MFO shows better performance than the other state-of-the-art models so that the classification rates are increased 1.5979, 0.9985, and 2.0879 for Sejnowski & Gorman, passive, and active datasets, respectively. The results also approve that time complexity is not significantly increased by using different spiral motions.
On the Role of Reviewer Expertise in Temporal Review Helpfulness Prediction
Venue: arXiv cs.CL 2023
Year: 2023

Helpful reviews have been essential for the success of e-commerce services, as they help customers make quick purchase decisions and benefit the merchants in their sales. While many reviews are informative, others provide little value and may contain spam, excessive appraisal, or unexpected biases. With the large volume of reviews and their uneven quality, the problem of detecting helpful reviews has drawn much attention lately. Existing methods for identifying helpful reviews primarily focus on review text and ignore the two key factors of (1) who post the reviews and (2) when the reviews are posted. Moreover, the helpfulness votes suffer from scarcity for less popular products and recently submitted (a.k.a., cold-start) reviews. To address these challenges, we introduce a dataset and develop a model that integrates the reviewer's expertise, derived from the past review history of the reviewers, and the temporal dynamics of the reviews to automatically assess review helpfulness. We conduct experiments on our dataset to demonstrate the effectiveness of incorporating these factors and report improved results compared to several well-established baselines.
Evolutionary Computation in Action: Feature Selection for Deep Embedding Spaces of Gigapixel Pathology Images
Venue: arXiv cs.CV 2023
Year: 2023

One of the main obstacles of adopting digital pathology is the challenge of efficient processing of hyperdimensional digitized biopsy samples, called whole slide images (WSIs). Exploiting deep learning and introducing compact WSI representations are urgently needed to accelerate image analysis and facilitate the visualization and interpretability of pathology results in a postpandemic world. In this paper, we introduce a new evolutionary approach for WSI representation based on large-scale multi-objective optimization (LSMOP) of deep embeddings. We start with patch-based sampling to feed KimiaNet , a histopathology-specialized deep network, and to extract a multitude of feature vectors. Coarse multi-objective feature selection uses the reduced search space strategy guided by the classification accuracy and the number of features. In the second stage, the frequent features histogram (FFH), a novel WSI representation, is constructed by multiple runs of coarse LSMOP. Fine evolutionary feature selection is then applied to find a compact (short-length) feature vector based on the FFH and contributes to a more robust deep-learning approach to digital pathology supported by the stochastic power of evolutionary algorithms. We validate the proposed schemes using The Cancer Genome Atlas (TCGA) images in terms of WSI representation, classification accuracy, and feature quality. Furthermore, a novel decision space for multicriteria decision making in the LSMOP field is introduced. Finally, a patch-level visualization approach is proposed to increase the interpretability of deep features. The proposed evolutionary algorithm finds a very compact feature vector to represent a WSI (almost 14,000 times smaller than the original feature vectors) with 8% higher accuracy compared to the codes provided by the state-of-the-art methods.
Encoding of data sets and algorithms
Venue: arXiv cs.LG 2023
Year: 2023

In many high-impact applications, it is important to ensure the quality of output of a machine learning algorithm as well as its reliability in comparison with the complexity of the algorithm used. In this paper, we have initiated a mathematically rigorous theory to decide which models (algorithms applied on data sets) are close to each other in terms of certain metrics, such as performance and the complexity level of the algorithm. This involves creating a grid on the hypothetical spaces of data sets and algorithms so as to identify a finite set of probability distributions from which the data sets are sampled and a finite set of algorithms. A given threshold metric acting on this grid will express the nearness (or statistical distance) from each algorithm and data set of interest to any given application. A technically difficult part of this project is to estimate the so-called metric entropy of a compact subset of functions of \textbf{infinitely many variables} that arise in the definition of these spaces.