DOI: 10.5281/zenodo.17439411
title: ChemVQA-2K: A Visual Question Answering Dataset for Molecular Understanding

description: 🧪 ChemVQA-2K: A Visual Question Answering Dataset for Molecular Understanding

📘 Overview

ChemVQA-2K is a novel Visual Question Answering (VQA) dataset designed to bridge chemistry and multimodal AI.It contains approximately 2,000 high-resolution molecular images (512×512) generated from valid SMILES strings, accompanied by 10 structured Q&A pairs per molecule, resulting in ~20,000 image-question-answer triplets.

Each image represents a 2D chemical structure rendered using RDKit, while each question tests the model’s ability to reason over molecular features such as formula, atom counts, bonds, functional groups, and polarity.

🧬 Dataset Structure






Component



Description







ChemVQA_2K_images



1924 molecule renderings (mol_0.png, mol_1.png, …)





ChemVQA_2K_full.csv



Complete dataset with columns: id, image_name, question, answer





 



 





Each record follows:

{

  "id": "mol_123",

  "image_name": "mol_123.png",

  "question": "What is the molecular formula of this molecule?",

  "answer": "C6H6O2"

}

 

🔍 Example Questions

Each molecule has multiple Q&A pairs, e.g.:






Question



Example Answer







What is the molecular formula of this molecule?



C₂H₅OH





What is the molecular weight?



46.07 g/mol





How many total atoms are present?



9





Which functional groups are present?



Alcohol





Is the molecule polar or non-polar?



Polar





 

⚙️ Data Generation Process



Molecules generated by concatenating random organic fragments and validated using RDKit.

Each molecule’s image created with Draw.MolToFile() at 512×512 px resolution.

Functional groups detected via SMARTS pattern matching.

Q&A pairs auto-generated from chemical descriptors (MolWt, CalcMolFormula, substructure matches).


 

🚀 Intended Use

ChemVQA-2K is ideal for:



Fine-tuning Vision-Language Models (VLMs) for scientific visual reasoning.

Developing chemistry-aware question answering systems.

Training vision encoders on molecular visual patterns.

Exploring RL-based visual understanding of chemical structures.


 

📊 Dataset Statistics






Property



Value







Images



1924





Image resolution



512×512 px





Q&A pairs



19240





Functional groups detected



16





File size (approx.)



~25 MB (images + CSVs)





 

🧠 Potential Research Directions



Multimodal Chemistry Understanding — connecting visual structure with symbolic reasoning.

Scientific Vision-Language Pretraining — use as domain-specific VQA benchmark.

Explainable Chemistry AI — models that describe functional features and molecular properties.


✅ ChemVQA-2K Dataset Benefits the Chemistry Community



Bridges Chemistry and AI Literacy



Helps chemistry students and researchers learn to interact with AI systems using domain-specific visual queries.

Encourages adoption of AI tools in chemical education and research.


Enables Vision-Language Model (VLM) Development for Chemistry



Provides a benchmark for training/fine-tuning multimodal models (e.g., BLIP-2, LLaVA, Qwen-VL) on chemical structure understanding.

Supports the creation of chemistry-aware AI assistants that can "see" molecules and answer questions.


Supports Automated Molecular Analysis



Models trained on this data can automatically extract properties (e.g., atom count, functional groups) from molecular diagrams—useful in digitizing legacy chemical literature or lab notebooks.


Enhances Chemistry Education Tools



Can power interactive learning apps where students upload a molecule image and get instant Q&A feedback (e.g., “How many oxygen atoms?” → “2”).

Useful for self-assessment and virtual tutoring systems.





Facilitates Accessibility in Chemistry



Assists visually impaired researchers/students via multimodal AI that describes molecular structures verbally or in text.

Converts visual chemical information into accessible natural language.


Promotes Reproducible & Scalable Data Curation



Demonstrates a programmatic, open-source pipeline to generate large-scale VQA datasets from SMILES—inspiring similar efforts for reactions, spectra, or crystal structures.





Encourages Domain-Specific AI Benchmarking



Offers a standardized testbed to evaluate how well general VLMs understand scientific imagery vs. models fine-tuned on chemistry data.


Supports Low-Resource Learning



The structured Q&A format is ideal for few-shot or instruction-tuning scenarios, reducing the need for massive labeled datasets.


Integrates with Cheminformatics Workflows



Can be combined with tools like RDKit, PubChem, or ChemSpider to build smart search or annotation systems that answer questions about retrieved molecules.





Fosters Interdisciplinary Collaboration




Creates a common ground for chemists, computer scientists, and educators to collaborate on AI-driven scientific discovery.