claw-eval/Claw-Eval huggingface dataset:
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/config.json — clock configuration parameters

Please read the config file, then generate /workspace/output.html to create an animated analog clock based on the config:
1. Display a circular clock face with hour/minute/second hands
2. Use the colors, sizes, and options specified in the config
3. The clock should show the correct time for the configured timezone
4. Second hand should move smoothly in real-time
5. Include number markers or tick marks as configured

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/config.json — world clock configuration

Please read the config file, then generate /workspace/output.html to display multiple analog clocks showing different time zones:
1. Show one clock for each city defined in the config
2. Each clock should display the correct time for its timezone
3. Use the specified colors for each clock
4. Label each clock with its city name
5. Arrange clocks according to the layout setting
6. All clocks should animate in real-time

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/planets.json — solar system configuration

Please read the config file, then generate /workspace/output.html to create an animated solar system visualization:
1. Display the sun at the center with the specified color and size
2. Show each planet orbiting the sun along its defined orbit path
3. Use the correct colors, sizes, and orbital radii from the config
4. Inner planets should orbit faster than outer planets (follow the period settings)
5. Show orbit paths and planet labels as configured
6. Saturn should display a ring if configured

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/config.json — countdown and fireworks configuration

Please read the config file, then generate /workspace/output.html to create a countdown timer with fireworks:
1. Display a countdown starting from the configured number of seconds
2. Use the specified font size, colors, and background
3. When the countdown reaches zero, display the celebration message
4. Launch animated fireworks with the configured colors and particle count
5. Fireworks should burst from multiple points with expanding particle effects

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/Canon1.png — a sheet music image of Canon in D

Please first use tools to view the sheet music image, then generate /workspace/output.html to reproduce this score using SVG:
1. Include the title and composer information
2. Draw the staves with correct clefs
3. Correctly display the key signature and time signature
4. Reproduce the note pitches, durations, and positions as accurately as possible
5. Include barlines, stems, beams, and other notation details

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/mariage1.png — a sheet music image of Mariage d'Amour

Please first use tools to view the sheet music image, then generate /workspace/output.html to reproduce this score using SVG:
1. Include the title and composer information
2. Draw the staves with correct clefs
3. Correctly display the key signature and time signature
4. Reproduce the note pitches, durations, and positions as accurately as possible
5. Include barlines, stems, beams, and other notation details

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/symphony1.png — a sheet music image of a symphony score

Please first use tools to view the sheet music image, then generate /workspace/output.html to reproduce this score using SVG:
1. Include the title and composer information
2. Draw the staves with correct clefs
3. Correctly display the key signature and time signature
4. Reproduce the note pitches, durations, and positions as accurately as possible
5. Include barlines, stems, beams, and other notation details

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/地铁1.png — a metro/subway map image

Please first use tools to view the metro map, then generate /workspace/output.html to reproduce this map using SVG:
1. Draw all subway lines using their corresponding colors
2. Mark all stations with dots or circles
3. Label station names
4. Use special markers for transfer/interchange stations (e.g., larger circles or multi-color markers)
5. Reproduce the line routes and curves as closely as possible to the original

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/地铁2.png — a metro/subway map image

Please first use tools to view the metro map, then generate /workspace/output.html to reproduce this map using SVG:
1. Draw all subway lines using their corresponding colors
2. Mark all stations with dots or circles
3. Label station names
4. Use special markers for transfer/interchange stations (e.g., larger circles or multi-color markers)
5. Reproduce the line routes and curves as closely as possible to the original

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/Canon1.png — a sheet music image of Pachelbel's Canon in D

Please first use tools to view the sheet music image, then generate /workspace/output.html to create an interactive score playback page:
1. Reproduce the score using SVG (staves, notes, clefs, etc.)
2. Draw a piano keyboard below the score
3. Add a "Play" button
4. When play is clicked:
   - Notes should highlight sequentially from left to right
   - Corresponding piano keys should light up in sync
   - Use the Web Audio API to produce the correct pitches
5. Playback speed should be appropriate and animation smooth

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/mariage1.png — a sheet music image of Mariage d'Amour

Please first use tools to view the sheet music image, then generate /workspace/output.html to create an interactive score playback page:
1. Reproduce the score using SVG (staves, notes, clefs, etc.)
2. Draw a piano keyboard below the score
3. Add a "Play" button
4. When play is clicked:
   - Notes should highlight sequentially from left to right
   - Corresponding piano keys should light up in sync
   - Use the Web Audio API to produce the correct pitches
5. Playback speed should be appropriate and animation smooth

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/symphony1.png — a sheet music image of a symphony score

Please first use tools to view the sheet music image, then generate /workspace/output.html to create an interactive score playback page:
1. Reproduce the score using SVG (staves, notes, clefs, etc.)
2. Draw a piano keyboard below the score
3. Add a "Play" button
4. When play is clicked:
   - Notes should highlight sequentially from left to right
   - Corresponding piano keys should light up in sync
   - Use the Web Audio API to produce the correct pitches
5. Playback speed should be appropriate and animation smooth

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/地铁1.png — a metro/subway map image

Please first use tools to view the metro map, then generate /workspace/output.html to create an interactive route display page:
1. Reproduce the metro map using SVG with correct line colors and station positions
2. Highlight a specific route from one station to another
3. Dim or fade non-route lines to make the route stand out
4. Clearly mark the start and end stations
5. Include a route description panel showing the stations along the route
6. Add animation showing the route being traced from start to end

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: multimodal_webpage
The container has the following file:
- /workspace/fixtures/地铁2.png — a metro/subway map image

Please first use tools to view the metro map, then generate /workspace/output.html to create an interactive route display page:
1. Reproduce the metro map using SVG with correct line colors and station positions
2. Highlight a specific route from one station to another
3. Dim or fade non-route lines to make the route stand out
4. Clearly mark the start and end stations
5. Include a route description panel showing the stations along the route
6. Add animation showing the route being traced from start to end

You can use tools to preview and iteratively adjust until you are satisfied.
Task category: video_ocr
The container has the following file:
- /workspace/fixtures/video.webm

I'm a video editor. Please help me extract all flashing English words
between 0:25 and 0:30 in the video. Output all the English words in the
order they appear, and do not output duplicate words caused by text
lingering on the screen. Output only the clean and complete string in the
final answer.
Task category: video_ocr
The container has the following file:
- /workspace/fixtures/video.webm

我正在分析黄金，请你帮我把视频中讲解问题二"为什么白银涨得比黄金猛"的视频片段中所有出现了"黄金"的句子找出来（每一条字幕算一个句子），放进一个列表里。最后答案写在一个txt文件 /workspace/output.txt 里，一行一句话，不要加任何标点符号。
Task category: video_ocr
The container has the following file:
- /workspace/fixtures/video.webm

I'm currently editing a video and need your help extracting the Chinese
subtitles from the first 30 seconds of this clip. Please identify the
precise appearance and disappearance times for each line, using seconds
as the unit and rounding to one decimal place (e.g., 1.5). Once you have
the data, please save it directly into a file named /workspace/subtitles.jsonl,
requiring each line to be an independent JSON object in the following format:
{"start": 0.0, "end": 0.7, "text": "内容"}.
Task category: doc_extraction
I am a researcher processing paper data. Please download and read this PDF:
https://openaccess.thecvf.com/content/CVPR2025/papers/Fu_Video-MME_The_First-Ever_Comprehensive_Evaluation_Benchmark_of_Multi-modal_LLMs_in_CVPR_2025_paper.pdf

Based on the main experimental results table, please extract the "w/ subs"
(with subtitles) scores of the three video durations (Short, Medium, Long)
and the Overall score for VILA-1.5 and Gemini 1.5 Pro. Save this data into
a file named /workspace/duration_comparison.csv. Then, create a line chart
with markers comparing the performance trend of these two models across the
three video durations. Ensure the y-axis ranges from 0 to 100, include a
legend, use green for VILA-1.5 and purple for Gemini, and save the final
image as /workspace/duration_trend.png.
Task category: doc_extraction
The container has the following file:
- /workspace/fixtures/GroundingME.pdf

I am a researcher processing paper data. Based on the main experimental
results table, please extract the average scores of the four L-1 dimensions
and the Total score for Qwen3-VL-A22B and Gemini-2.5-Pro. Save this data
into a file named /workspace/model_comparison.csv. Then, create a radar chart
comparing these two models on the four L-1 dimensions. Ensure the chart axes
range from 0 to 100, include a legend, use blue for Qwen3 and red for Gemini,
and save the final image as /workspace/comparison_radar.png.
Task category: doc_extraction
I am writing a meta-analysis survey on visual grounding. Please download
the following three PDF papers and save them to /workspace/papers/:
- https://arxiv.org/pdf/2511.21631
- https://arxiv.org/pdf/2507.01006
- https://arxiv.org/pdf/2505.07062

Scan all the PDF files in the /workspace/papers/ directory. Extract the
overall accuracy scores for three specific models: Qwen3-VL-A22B-Instruct,
GLM-4.6V, and Seed-1.5-VL-Thinking, across two different benchmarks:
RefCOCO-avg (val) and ERQA. Save the synthesized data into a file named
/workspace/meta_analysis.csv with exactly three columns: Model, RefCOCO-avg,
and ERQA. Then, create a Grouped Bar Chart comparing these three models
side-by-side on the two benchmarks. Save the final image as
/workspace/cross_benchmark_comparison.png.
Task category: doc_search
The container has the following file:
- /workspace/fixtures/Li_VL-RewardBench_A_Challenging_Benchmark_for_Vision-Language_Generative_Reward_Models_CVPR_2025_paper.pdf

I'm sorting out paper references. Scan the reference list of the provided
paper. Identify all citations that are listed as 'arXiv preprints' and verify
which of these arXiv papers have been officially accepted or published in
formal academic conferences as of March 19, 2026. Generate a file named
/workspace/updated_publications.csv containing exactly two columns:
Paper Title and Formal Venue.
Task category: video_qa
The container has the following file:
- /workspace/fixtures/video.mp4

This video is a 2023 movie montage. Please identify which movie each segment comes from (provide the movie title), and list all the movies you recognize in the video in chronological order.
Task category: video_qa
The container has the following file:
- /workspace/fixtures/video.mp4

I find this paper very interesting. May I ask what the full title of the paper is? In the Task-adaptive Multimodal Alignment module, what are the specific network interaction mechanisms and decoupled training objectives of Meta Queries? Please help me list the formulas; I don't have a background in deep learning, so please explain it in a simple and easy-to-understand way. Also, has its code been open-sourced?
Task category: video_webpage
容器里有以下文件：
- /workspace/fixtures/video.mp4

老板让我给咱们工厂做个宣传页，正好看到这段拍我们生产线的视频。帮我从视频里把关键信息整理出来：生产的是什么东西？用了什么核心技术或制作工艺？这个东西有哪些优良的性能？视频里有没有出现产能数据？视频有没有字幕说明？然后做个内容详细丰富的中文宣传单页HTML，保存为 /workspace/output.html，风格要工业感但现代。整体要配色和谐、审美在线。
Task category: video_qa
容器里有以下文件：
- /workspace/fixtures/video.mp4

我错过了这场比赛，我的朋友一直在夸赞这场比赛。你能告诉我哪两支队伍在比赛吗？比赛的具体项目是什么？整个视频中计分板显示的比分是多少，是否发生了变化？视频中得分的队伍是哪一支队伍，请列出该队伍所有成员的名字。此外，给我详细介绍下视频中红色衣服的这支队伍，我想了解更多。
Task category: video_webpage
容器里有以下文件：
- /workspace/fixtures/video.mp4

视频里讲的这个故事好有意思啊，你帮我做一个有趣的网页帮小朋友介绍这个故事吧，里面要有每个人物（头像），讲清楚故事的脉络，然后最后再包含一个小游戏，寓教于乐。要求网页布局合理，配色和谐明快。保存为 /workspace/output.html。
Task category: video_qa
容器里有以下文件：
- /workspace/fixtures/video.mp4

河南的美食真不错，看这个视频帮我写个markdown格式的备忘录。把视频里出现的每一道美食的名称、视频中的相应传承人的姓名以及是第几代、美食的制作方法、美食的口感整理好。
Task category: video_chart
The container has the following file:
- /workspace/fixtures/video.mp4

Please track every point scored by both players in the video and create a line chart showing the score progression. The x-axis should represent the match time, and the y-axis should represent the cumulative score. Use different colored lines for each player, and annotate each point of score change with the exact score at that moment (e.g., 5:3, 8:11) to create a comprehensive match analysis chart. On the x-axis, you only need to label four specific timestamps: 7:56, 12:46, 21:26, and 25:03, and please bold the corresponding scores at these exact times. Save the chart as /workspace/score_chart.png.
Task category: video_edit
容器里有以下文件：
- /workspace/fixtures/video.mp4

识别这段监控录像中的撞车事件出现时间（以视频中呈现的时间为准），并截取相关片段，保存为 /workspace/clip.mp4。
Task category: video_image
容器里有以下文件：
- /workspace/fixtures/video.mp4

请浏览这段 vlog，把其中出现的所有零食都截图下来（截图里零食的名称或包装要清晰可见），每种零食截一张最清晰的，同一零食不用重复截。在每张截图下标注零食名称，优先写英文，没有英文的话写中文。最后把所有截图和名称整合成一张美食打卡清单图片，输出为 /workspace/snack_checklist.jpg。
Task category: video_image
The container has the following file:
- /workspace/fixtures/video.mp4

Based on this video, reconstruct the floor plan layout of the room as a top-down 2D diagram. Orientation: The direction you face at the very start of the video defines the top edge of the floor plan. The right side of that initial view maps to the right edge, and the left side maps to the left edge. Maintain this orientation consistently throughout. Include all major objects that appear in the video, label them by name on the diagram, and accurately reflect their spatial relationships. Draw it as an SVG image and save it as /workspace/floorplan.png.
Task category: video_qa
The container has the following file:
- /workspace/fixtures/video.webm

I am conducting a detailed statistical analysis of historic, extended tennis rallies. Please watch the provided video clip of a grueling point and answer the following six questions in sequence based purely on the visual footage.
First, identify the two players competing in this point. Second, which set is this match currently in? Third, what is the exact game score and point score right before the point starts? Fourth, what is the updated game score immediately after this point concludes? Fifth, which player ultimately wins this specific point? Sixth, please carefully track the ball and count every single stroke. Exactly how many shots were hit in total during this entire rally?
Task category: video_qa
The container has the following file:
- /workspace/fixtures/video.webm

I am analyzing a specific video clip of Novak Djokovic playing in a fifth set. Please watch the uploaded video carefully and answer the following questions based solely on the visual and audio events shown in this exact footage.
To ensure we are on the same page, here are the definitions of the tennis terms you will need for this task:
Break Point: A situation where the receiving player is one point away from winning the game on the server's serve. Saving a break point means Djokovic wins that specific point to keep the game alive.
Winner: A proactive, successful shot hit by a player that lands in the court and cannot be touched or returned by the opponent, resulting in a direct point.
Error: A failed shot by a player that either lands out of bounds or hits the net, gifting the point to the opponent.
First, please state the exact total number of break points Djokovic successfully saved during the fifth set within this specific video clip.
Second, provide a chronological breakdown for each of those saved break points. For every single break point, you must provide the following three specific details: The game score and the point score(Djokovic first) right before the point starts, such as 2:1 15:30. The exact timestamp in the video(not the time in the match) when the break point starts. A clear specification of how Djokovic saved the break point, explicitly stating whether it was a Djokovic winner or an opponent error.
Task category: video_qa
The container has the following file:
- /workspace/fixtures/video.webm

I have uploaded a video clip showing a specific game from a tennis match between Andy Murray and Roger Federer.
Please carefully watch the footage and locate the exact point that begins when the score is Set 0:1, Game 4:5, and Point 0:15. Please answer the following four questions sequentially.
First, identify which player is serving to start this point.
Second, provide a chronological, shot-by-shot log of the entire rally. Starting from the return of serve, explicitly specify the name of the player hitting the ball and whether they used a forehand or a backhand stroke for every single shot.
Third, count and state the exact total number of shots hit during this entire point, including the initial serve.
Fourth, clearly state the name of the player who ultimately won this specific point.
Task category: video_qa
The container has the following file:
- /workspace/fixtures/video.webm

I am compiling statistics for a special Australian Open exhibition event featuring sudden-death, one-point matches between amateur challengers and professional tennis players. Please carefully watch the uploaded video clip and provide a detailed breakdown based strictly on the visual and audio evidence. First, explicitly state the total number of amateur versus professional matches played in this specific footage. Second, for each individual match shown, please outline the following four details in chronological order: the names of the two players competing, a clear identification of which player is the amateur, the name of the player who served the ball to start the point, and the name of the player who ultimately won the match.
Task category: multimodal
朋友推荐了这个蝴蝶手绘视频说画得特别好，我想跟着学。画画前，我需要准备哪些材料？视频里面一共画了几个阶段/步骤，帮我梳理每个关键步骤，并给每个步骤附上一个可视化图片，给我一个画画教程笔记。
Task category: multimodal
帮我查一下这个vlog中出现的所有吃食的店铺，整理每个店铺的地理位置，帮我记录下来（每家店名、地理位置、照片），我要一一去打卡。

The container has the following file:
- /workspace/fixtures/BV1brPYzMEax.mp4 — a Shanghai food vlog

Please output a structured document (markdown or HTML) with each shop's:
1. Shop name
2. Geographic location / address
3. A photo from the video

Save your output to /workspace/output.md or /workspace/output.html
Task category: video_search
There is a video file at /workspace/fixtures/video.mp4.

Watch the video carefully and complete the following task:

In the video, a woman wearing a red top and a headband, carrying a jacket on her right side, appears around 0:19–0:22.

1. Identify the full time interval (start and end timestamps) during which she is descending from a rocky hill with tall rocks. Save the result as a single line (e.g. "23:10 - 33:00") to /workspace/timestamp.txt.
2. Find a clear frame from this interval where she is fully visible, crop the frame to center her as the main subject, and save it as /workspace/cropped_frame.png.

Make sure both output files are saved to their specified paths.
Task category: video_search
There is a video file at /workspace/fixtures/video.mp4.

Watch the video carefully and complete the following task:

In the video, there is a machine next to a white wall. The machine's inlet has a gradually narrowing conical shape. At the outlet of the machine, there is a green plastic container. The engine of the machine is located on the left side. A person wearing jeans and a blue shirt is operating the machine.

1. Identify the full time interval (start and end timestamps) during which this event is happening. Save the result as a single line (e.g. "11:55 - 15:05") to /workspace/timestamp.txt.
2. Find a clear frame from this interval where a subject passes by from the right side of the machine, crop the frame to center this subject as the main subject, and save it as /workspace/cropped_frame.png.

Make sure both output files are saved to their specified paths.
Task category: video_search
容器里有以下文件：
- /workspace/fixtures/video.mp4

请观看视频，完成以下任务：

一些身穿白衬衫、灰裤子、头戴绿帽的士兵，手持枪支，与火车上的人们交战。
找出以上场景中，除了火车以外的交通工具，并截取以该交通工具为中心的图片，保存为 /workspace/vehicle_frame.png。
Task category: video_search
The container has the following file:
- /workspace/fixtures/video.mp4

Please watch the video and complete the following task:

In the center of the screen is a drawing, depicting a person resting their hand on their forehead on a yellowish paper.
Identify another scene where this artwork appeared and extract a frame from that scene, save it as /workspace/artwork_frame.png.
Task category: video_search
There is a video file at /workspace/fixtures/video.mp4 and its subtitle at /workspace/fixtures/subtitle.txt.

Watch the video carefully (or read the subtitle) and answer the following question:

On what date did the individual in the video leave a place that Simon thought was very important to him?

Please provide the exact date in your answer and save it to /workspace/answer.txt.