pwc-archive/papers-with-abstracts huggingface dataset:
A Siamese CNN for Image Steganalysis
Authors: Weike You, Hong Zhang, Xianfeng Zhao
Date: 2020-07-30 00:00:00
Proceeding: journal 2020 7
Tasks: Steganalysis

Image steganalysis is a technique for detecting data hidden in images. Recent research has shown the powerful capabilities of using convolutional neural networks (CNN) for image steganalysis. However, due to the particularity of steganographic signals, there are still few reliable CNN-based methods for applying steganalysis to images of arbitrary size. In this paper, we address this issue by exploring the possibility of exploiting a network for steganalyzing images of varying sizes without retraining its parameters. On the assumption that natural image noise is similar between different image sub-regions, we propose an end-to-end, deep learning, novel solution for distinguishing steganography images from normal images that provides satisfying performance. The proposed network first takes the image as the input, then identifies the relationships between the noise of different image sub-regions, and, finally, outputs the resulting classification based upon them. Our algorithm adopts a Siamese, CNN-based architecture, which consists of two symmetrical subnets with shared parameters, and contains three phases: preprocessing, feature extraction, and fusion/classification. To validate the network, we generated datasets composed of steganography images with multiple sizes and their corresponding normal images sourced from BOSSbase 1.01 and ALASKA #2. Experimental results produced by the data generated by various methods show that our proposed network is well-generalized and robust.
Mosaic: An Architecture for Scalable & Interoperable Data Views
Authors: Jeffrey Heer, Dominik Moritz
Date: 2023-10-26 00:00:00
Proceeding: IEEE VIS 2023 10

Mosaic is an architecture for greater scalability, extensibility, and
interoperability of interactive data views. Mosaic decouples data processing
from specification logic: clients publish their data needs as declarative
queries that are then managed and optimized by a coordinator that proxies access
to a scalable data store. Mosaic generalizes Vega-Lite's selection abstraction
to enable rich integration and linking of both standalone components (including
menus, text search, and tables) and visualizations created using vgplot, a
grammar of interactive graphics in which graphical marks act as Mosaic clients.
We demonstrate Mosaic's expressiveness, extensibility, and interoperability
through examples that compose diverse visualization, interaction, and
optimization techniques. To evaluate scalability, we present benchmark studies
with order-of-magnitude performance improvements over existing web-based
visualization systems -- enabling flexible, real-time visual exploration of
billion+ record datasets. We conclude by discussing Mosaic's potential as an
open platform that bridges visualization languages, scalable visualization, and
interactive data systems more broadly.
DEU-Net: Dual-Encoder U-Net for Automated Skin Lesion Segmentation
Authors: Ali Karimi, Karim Faez, Soheila Nazari
Date: 2023-11-28 00:00:00
Proceeding: IEEE Access 2023 11
Tasks: Decoder, Diversity, Image Segmentation, Lesion Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation, Skin Lesion Segmentation

The computer-aided diagnosis (CAD) of skin diseases relies heavily on automated skin lesion segmentation, albeit presenting considerable challenges due to lesion diversity in shape, size, color, and texture, as well as potential blurry boundaries with surrounding tissues. Traditional Convolutional Neural Networks (CNN) typically underperform in this domain, given their inherent constraints in global context information capture. In the present study, we present a new U-shaped network, Dual-Encoder U-Net (DEU-Net), which is based on an encoder-decoder architecture. DEU-Net integrates a dual-encoder branch comprising a convolutional encoder and a transformer encoder, thereby facilitating the concurrent extraction of local features and global contextual information. Additionally, in order to enhance the performance of DEU-Net, we employ an integrated test-time augmentation technique. To ascertain the efficiency and superiority of our proposed methodology, we performed comprehensive experiments across four widely accessible skin lesion datasets, namely ISIC 2016, ISIC 2017, ISIC 2018, and PH2. The Dice coefficients achieved on these datasets were 92.90%, 87.16%, 90.81%, and 95.65%, respectively. These results demonstrate superior performance compared to most current state-of-the-art methods. The source code is released at https://github.com/alikm6/DEU-Net .
Deriving intrinsic images from image sequences
Authors: Yair Weiss
Date: 2001-07-14 00:00:00
Proceeding: International Conference on Computer vision 2001 7

Intrinsic images are a useful midlevel description of scenes proposed by H.G. Barrow and J.M. Tenenbaum (1978). An image is de-composed into two images: a reflectance image and an illumination image. Finding such a decomposition remains a difficult problem in computer vision. We focus on a slightly, easier problem: given a sequence of T images where the reflectance is constant and the illumination changes, can we recover T illumination images and a single reflectance image? We show that this problem is still imposed and suggest approaching it as a maximum-likelihood estimation problem. Following recent work on the statistics of natural images, we use a prior that assumes that illumination images will give rise to sparse filter outputs. We show that this leads to a simple, novel algorithm for recovering reflectance images. We illustrate the algorithm's performance on real and synthetic image sequences.
RepViT-SAM: Towards Real-Time Segmenting Anything
Authors: Ao Wang, Hui Chen, Zijia Lin, Jungong Han, Guiguang Ding
Date: 2023-12-10 00:00:00

Segment Anything Model (SAM) has shown impressive zero-shot transfer performance for various computer vision tasks recently. However, its heavy computation costs remain daunting for practical applications. MobileSAM proposes to replace the heavyweight image encoder in SAM with TinyViT by employing distillation, which results in a significant reduction in computational requirements. However, its deployment on resource-constrained mobile devices still encounters challenges due to the substantial memory and computational overhead caused by self-attention mechanisms. Recently, RepViT achieves the state-of-the-art performance and latency trade-off on mobile devices by incorporating efficient architectural designs of ViTs into CNNs. Here, to achieve real-time segmenting anything on mobile devices, following MobileSAM, we replace the heavyweight image encoder in SAM with RepViT model, ending up with the RepViT-SAM model. Extensive experiments show that RepViT-SAM can enjoy significantly better zero-shot transfer capability than MobileSAM, along with nearly $10\times$ faster inference speed. The code and models are available at \url{https://github.com/THU-MIG/RepViT}.
MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning
Authors: Zeyuan Ma, Hongshu Guo, Jiacheng Chen, Zhenrui Li, Guojun Peng, Yue-Jiao Gong, Yining Ma, Zhiguang Cao
Date: 2023-10-12 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Benchmarking

Recently, Meta-Black-Box Optimization with Reinforcement Learning (MetaBBO-RL) has showcased the power of leveraging RL at the meta-level to mitigate manual fine-tuning of low-level black-box optimizers. However, this field is hindered by the lack of a unified benchmark. To fill this gap, we introduce MetaBox, the first benchmark platform expressly tailored for developing and evaluating MetaBBO-RL methods. MetaBox offers a flexible algorithmic template that allows users to effortlessly implement their unique designs within the platform. Moreover, it provides a broad spectrum of over 300 problem instances, collected from synthetic to realistic scenarios, and an extensive library of 19 baseline methods, including both traditional black-box optimizers and recent MetaBBO-RL methods. Besides, MetaBox introduces three standardized performance metrics, enabling a more thorough assessment of the methods. In a bid to illustrate the utility of MetaBox for facilitating rigorous evaluation and in-depth analysis, we carry out a wide-ranging benchmarking study on existing MetaBBO-RL methods. Our MetaBox is open-source and accessible at: https://github.com/GMC-DRL/MetaBox.
Implicit variance regularization in non-contrastive SSL
Authors: Manu Srinath Halvagal, Axel Laborieux, Friedemann Zenke
Date: 2022-12-09 00:00:00
Proceeding: NeurIPS 2023 11

Non-contrastive SSL methods like BYOL and SimSiam rely on asymmetric predictor networks to avoid representational collapse without negative samples. Yet, how predictor networks facilitate stable learning is not fully understood. While previous theoretical analyses assumed Euclidean losses, most practical implementations rely on cosine similarity. To gain further theoretical insight into non-contrastive SSL, we analytically study learning dynamics in conjunction with Euclidean and cosine similarity in the eigenspace of closed-form linear predictor networks. We show that both avoid collapse through implicit variance regularization albeit through different dynamical mechanisms. Moreover, we find that the eigenvalues act as effective learning rate multipliers and propose a family of isotropic loss functions (IsoLoss) that equalize convergence rates across eigenmodes. Empirically, IsoLoss speeds up the initial learning dynamics and increases robustness, thereby allowing us to dispense with the EMA target network typically used with non-contrastive methods. Our analysis sheds light on the variance regularization mechanisms of non-contrastive SSL and lays the theoretical grounds for crafting novel loss functions that shape the learning dynamics of the predictor's spectrum.
A GAN Approach for Node Embedding in Heterogeneous Graphs Using Subgraph Sampling
Authors: Hung-Chun Hsu, Bo-Jun Wu, Ming-Yi Hong, Che Lin, Chih-Yu Wang
Date: 2023-12-11 00:00:00
Tasks: Generative Adversarial Network, Graph Neural Network

Graph neural networks (GNNs) face significant challenges with class imbalance, leading to biased inference results. To address this issue in heterogeneous graphs, we propose a novel framework that combines Graph Neural Network (GNN) and Generative Adversarial Network (GAN) to enhance classification for underrepresented node classes. The framework incorporates an advanced edge generation and selection module, enabling the simultaneous creation of synthetic nodes and edges through adversarial learning. Unlike previous methods, which predominantly focus on homogeneous graphs due to the difficulty of representing heterogeneous graph structures in matrix form, this approach is specifically designed for heterogeneous data. Existing solutions often rely on pre-trained models to incorporate synthetic nodes, which can lead to optimization inconsistencies and mismatches in data representation. Our framework avoids these pitfalls by generating data that aligns closely with the inherent graph topology and attributes, ensuring a more cohesive integration. Evaluations on multiple real-world datasets demonstrate the method's superiority over baseline models, particularly in tasks focused on identifying minority node classes, with notable improvements in performance metrics such as F-score and AUC-PRC score. These findings highlight the potential of this approach for addressing critical challenges in the field.
UIEDP:Underwater Image Enhancement with Diffusion Prior
Authors: Dazhao Du, Enhan Li, Lingyu Si, Fanjiang Xu, Jianwei Niu, Fuchun Sun
Date: 2023-12-11 00:00:00
Tasks: Image Enhancement, Image Generation, Image Quality Assessment, No-Reference Image Quality Assessment, UIE

Underwater image enhancement (UIE) aims to generate clear images from low-quality underwater images. Due to the unavailability of clear reference images, researchers often synthesize them to construct paired datasets for training deep models. However, these synthesized images may sometimes lack quality, adversely affecting training outcomes. To address this issue, we propose UIE with Diffusion Prior (UIEDP), a novel framework treating UIE as a posterior distribution sampling process of clear images conditioned on degraded underwater inputs. Specifically, UIEDP combines a pre-trained diffusion model capturing natural image priors with any existing UIE algorithm, leveraging the latter to guide conditional generation. The diffusion prior mitigates the drawbacks of inferior synthetic images, resulting in higher-quality image generation. Extensive experiments have demonstrated that our UIEDP yields significant improvements across various metrics, especially no-reference image quality assessment. And the generated enhanced images also exhibit a more natural appearance.
GOGGLE: Generative Modelling for Tabular Data by Learning Relational Structure
Authors: Tennison Liu, Zhaozhi Qian, Jeroen Berrevoets, Mihaela van der Schaar
Date: 2023-10-27 00:00:00
Proceeding: ICLR 2023 10

Deep generative models learn highly complex and non-linear representations to generate realistic synthetic data. While they have achieved notable success in computer vision and natural language processing, similar advances have been less demonstrable in the tabular domain. This is partially because generative modelling of tabular data entails a particular set of challenges, including heterogeneous relationships, limited number of samples, and difficulties in incorporating prior knowledge. Additionally, unlike their counterparts in image and sequence domain, deep generative models for tabular data almost exclusively employ fully-connected layers, which encode weak inductive biases about relationships between inputs. Real-world data generating processes can often be represented using relational structures, which encode sparse, heterogeneous relationships between variables. In this work, we learn and exploit relational structure underlying tabular data to better model variable dependence, and as a natural means to introduce regularization on relationships and include prior knowledge. Specifically, we introduce GOGGLE, an end-to-end message passing scheme that jointly learns the relational structure and corresponding functional relationships as the basis of generating synthetic samples. Using real-world datasets, we provide empirical evidence that the proposed method is effective in generating realistic synthetic data and exploiting domain knowledge for downstream tasks.
M3SOT: Multi-frame, Multi-field, Multi-space 3D Single Object Tracking
Authors: Jiaming Liu, Yue Wu, Maoguo Gong, Qiguang Miao, Wenping Ma, Can Qin
Date: 2023-12-11 00:00:00
Tasks: 3D Single Object Tracking, Autonomous Driving, Object Tracking

3D Single Object Tracking (SOT) stands a forefront task of computer vision, proving essential for applications like autonomous driving. Sparse and occluded data in scene point clouds introduce variations in the appearance of tracked objects, adding complexity to the task. In this research, we unveil M3SOT, a novel 3D SOT framework, which synergizes multiple input frames (template sets), multiple receptive fields (continuous contexts), and multiple solution spaces (distinct tasks) in ONE model. Remarkably, M3SOT pioneers in modeling temporality, contexts, and tasks directly from point clouds, revisiting a perspective on the key factors influencing SOT. To this end, we design a transformer-based network centered on point cloud targets in the search area, aggregating diverse contextual representations and propagating target cues by employing historical frames. As M3SOT spans varied processing perspectives, we've streamlined the network-trimming its depth and optimizing its structure-to ensure a lightweight and efficient deployment for SOT applications. We posit that, backed by practical construction, M3SOT sidesteps the need for complex frameworks and auxiliary components to deliver sterling results. Extensive experiments on benchmarks such as KITTI, nuScenes, and Waymo Open Dataset demonstrate that M3SOT achieves state-of-the-art performance at 38 FPS. Our code and models are available at https://github.com/ywu0912/TeamCode.git.
TabCaps: A Capsule Neural Network for Tabular Data Classification with BoW Routing
Authors: Jintai Chen, Kuanlun Liao, Yanwen Fang, Danny Chen, Jian Wu
Date: 2023-10-17 00:00:00
Proceeding: ICLR 2023 10

Records in a table are represented by a collection of heterogeneous scalar features. Previous work often made predictions for records in a paradigm that processed each feature as an operating unit, which requires to well cope with the heterogeneity. In this paper, we propose to encapsulate all feature values of a record into vectorial features and process them collectively rather than have to deal with individual ones, which directly captures the representations at the data level and benefits robust performances. Specifically, we adopt the concept of "capsules" to organize features into vectorial features, and devise a novel capsule neural network called "TabCaps" to process the vectorial features for classification. In TabCaps, a record is encoded into several vectorial features by some optimizable multivariate Gaussian kernels in the primary capsule layer, where each vectorial feature represents a specific "profile" of the input record and is transformed into senior capsule layer under the guidance of a new straightforward routing algorithm. The design of routing algorithm is motivated by the Bag-of-Words (BoW) model, which performs capsule feature grouping straightforwardly and efficiently, in lieu of the computationally complex clustering of previous routing algorithms. Comprehensive experiments show that TabCaps achieves competitive and robust performances in tabular data classification tasks. Codes are available at https://github.com/WhatAShot/TabCaps.
Proxy-based Item Representation for Attribute and Context-aware Recommendation
Authors: Jinseok Seol, Minseok Gang, Sang-goo Lee, Jaehui Park
Date: 2023-12-11 00:00:00
Tasks: Attribute, Recommendation Systems

Neural network approaches in recommender systems have shown remarkable success by representing a large set of items as a learnable vector embedding table. However, infrequent items may suffer from inadequate training opportunities, making it difficult to learn meaningful representations. We examine that in attribute and context-aware settings, the poorly learned embeddings of infrequent items impair the recommendation accuracy. To address such an issue, we propose a proxy-based item representation that allows each item to be expressed as a weighted sum of learnable proxy embeddings. Here, the proxy weight is determined by the attributes and context of each item and may incorporate bias terms in case of frequent items to further reflect collaborative signals. The proxy-based method calculates the item representations compositionally, ensuring each representation resides inside a well-trained simplex and, thus, acquires guaranteed quality. Additionally, that the proxy embeddings are shared across all items allows the infrequent items to borrow training signals of frequent items in a unified model structure and end-to-end manner. Our proposed method is a plug-and-play model that can replace the item encoding layer of any neural network-based recommendation model, while consistently improving the recommendation performance with much smaller parameter usage. Experiments conducted on real-world recommendation benchmark datasets demonstrate that our proposed model outperforms state-of-the-art models in terms of recommendation accuracy by up to 17% while using only 10% of the parameters.
Why do tree-based models still outperform deep learning on typical tabular data?
Authors: Leo Grinsztajn, Edouard Oyallon, Gael Varoquaux
Date: 2022-11-28 00:00:00
Proceeding: NeurIPS 2022 11
Tasks: Benchmarking

While deep learning has enabled tremendous progress on text and image datasets, its superiority on tabular data is not clear. We contribute extensive benchmarks of standard and novel deep learning methods as well as tree-based models such as XGBoost and Random Forests, across a large number of datasets and hyperparameter combinations. We define a standard set of 45 datasets from varied domains with clear characteristics of tabular data and a benchmarking methodology accounting for both fitting models and finding good hyperparameters. Results show that tree-based models remain state-of-the-art on medium-sized data (~10K samples) even without accounting for their superior speed. To understand this gap, we conduct an empirical investigation into the differing inductive biases of tree-based models and neural networks. This leads to a series of challenges which should guide researchers aiming to build tabular-specific neural network: 1) be robust to uninformative features, 2) preserve the orientation of the data, and 3) be able to easily learn irregular functions. To stimulate research on tabular architectures, we contribute a standard benchmark and raw data for baselines: every point of a 20\,000 compute hours hyperparameter search for each learner.
Photorealistic Video Generation with Diffusion Models
Authors: Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, José Lezama
Date: 2023-12-11 00:00:00
Tasks: Super-Resolution, Text-to-Video Generation, Video Generation, Video Super-Resolution

We present W.A.L.T, a transformer-based approach for photorealistic video generation via diffusion modeling. Our approach has two key design decisions. First, we use a causal encoder to jointly compress images and videos within a unified latent space, enabling training and generation across modalities. Second, for memory and training efficiency, we use a window attention architecture tailored for joint spatial and spatiotemporal generative modeling. Taken together these design decisions enable us to achieve state-of-the-art performance on established video (UCF-101 and Kinetics-600) and image (ImageNet) generation benchmarks without using classifier free guidance. Finally, we also train a cascade of three models for the task of text-to-video generation consisting of a base latent video diffusion model, and two video super-resolution diffusion models to generate videos of $512 \times 896$ resolution at $8$ frames per second.
Building Domain-Specific LLMs Faithful To The Islamic Worldview: Mirage or Technical Possibility?
Authors: Shabaz Patel, Hassan Kane, Rayhan Patel
Date: 2023-12-11 00:00:00
Tasks: Natural Language Understanding

Large Language Models (LLMs) have demonstrated remarkable performance across numerous natural language understanding use cases. However, this impressive performance comes with inherent limitations, such as the tendency to perpetuate stereotypical biases or fabricate non-existent facts. In the context of Islam and its representation, accurate and factual representation of its beliefs and teachings rooted in the Quran and Sunnah is key. This work focuses on the challenge of building domain-specific LLMs faithful to the Islamic worldview and proposes ways to build and evaluate such systems. Firstly, we define this open-ended goal as a technical problem and propose various solutions. Subsequently, we critically examine known challenges inherent to each approach and highlight evaluation methodologies that can be used to assess such systems. This work highlights the need for high-quality datasets, evaluations, and interdisciplinary work blending machine learning with Islamic scholarship.