Express What You See: Can Multimodal LLM s Decode Visual Ciphers with Intuitive Semiosis Comprehension? (2025)

Abstract: Bridging the gap between visual and language remains a pivotal challenge for the multimodal community. Traditional VQA benchmarks encounter a modality gap and over-reliance on language priors, whereas human cognition excels at intuitive semiosis, associating abstract visual symbols to linguistic semantics. Inspired by this neurocognitive mechanism, we focus on emojis, the visual cipher conveying abstract textual semantics. Specifically, we propose a novel task of generating abstract linguistics from emoji sequence images, where such reasoning underpins critical applications in cryptography, thus challenging MLLMs’ reasoning of decoding complex semantics of visual ciphers. We introduce eWe-bench (Express What you SeE), assessing MLLMs’ capability of intuitive semiosis like humans. Our data construction framework ensures high visual sensitivity and data quality, which can be extended to future data enhancement. Evaluation results on advanced MLLMs highlight critical deficiencies in visual intuitive symbolic reasoning. We believe our interesting insights for advancing visual semiosis in MLLMs will pave the way for cryptographic analysis and high-level intuitive cognition intelligence of MLLMs.

## introduction multimodal large language models (mllms) have achieved remarkable progress in recent years @xcite @xcite @xcite @xcite , particularly in their ability to integrate visual and linguistic information for more natural humancomputer interaction @xcite @xcite . despite these advances, a central challenge remains: how can mllms truly bridge the gap between 1 all our data are available in ewe-bench. human cognition presents a different paradigm. neuroscientific studies reveal that our brains reflect intuitive semiosis capability, which spontaneously maps abstract visual symbols to semantic representations through rapid thalamocortical interactions @xcite @xcite , with symbol recognition occurring 200-300 milliseconds faster than object categorization. this visual symbol intuitive semiosis enables instantaneous understanding of abstract symbols that convey complex meanings through shared cultural codes, including traffic signs, mathematical notations, and emojis @xcite . such visual symbol reasoning underpins critical applications in cryptography and information analysis @xcite , where successful decoding requires direct symbolto-semantic mapping without explanatory prompts. motivated by this mechanism, we propose a transformative benchmark to evaluate mllms' visual intuitive semiosis comprehension in multilingual scenarios. departing from the vqa paradigm that uses textual questions as cognitive scaffolds, our approach employs emojis as visual ciphers, challenging models to decode abstract visual sym- ## conclusion we propose the ewe-bench benchmark, containing emoji visual ciphers decoding tasks including chinese idioms, english words, and english idioms, which provides a novel way to measure the visual intuitive semiosis ability of mllms to directly associate the image with its abstract linguistic semantics. we design a data engineering framework that performs data collection, data filtering, and evaluation, contributing to validating the unified high-level vision-language understanding and reasoning like human intuitive cognition. we evaluate advanced open-source and closed-source mllms with our ewe-bench, analyze the performance, conduct several explorations, and highlight future research directions with further case study.