sajjadanwar0/token-budgets-catalog huggingface dataset:
Framework: autogen (2024-03)
Issue: multimodal multi agent function call
Mechanism cluster: M-delegation-fanout
Notes: paper:bu with MULTI-MODAL-x-MULTI-AGENT-FANOUT INTERSECTION + MAINTAINER-INTERNAL-CONTRADICTION pattern. RENUMBERED FROM ORIGINAL PASTE (was "ATGN-013" but ID collision with existing ATGN-013 #4750; renumbered to next free ATGN-020). Filed Mar 2 2024 by @tomorrmato tagged tool-usage + multimodal. AUTO-STALE-CLOSED Feb 3 2025 by Collaborator @rysweet as not planned (~11 months filing-to-close) with try-newer-version-deflection: "Hello, would you be open to please trying to repo with AutoGen 0.4.5 or later?" Reporter setup: GroupChatManager with 3 agents (user_proxy + image_critic + coder) where coder converts python code to image-bytes via function-call to jupyter kernel, image_critic gets gpt-4-vision feedback, loop continues until max_round=12. Each iteration broadcasts image bytes through GroupChatManager history-broadcast mechanism (per ATGN-018 super-linear-history-broadcast). CONCRETE FRAMEWORK ERROR: "BadRequestError: Error code: 400 - None is not of type object - messages.2.function_call" -- multi-modal x function-call x multi-agent intersection breaks because GroupChat speaker-selection LLM-call carries entire history including base64 image bytes. ATTEMPTED-FIX-PR-THAT-DID-NOT-WORK: Contributor @BeibinLi Mar 2 2024 self-assigned. Mar 14 2024 BeibinLi opened PR #1926 "VisionCapability". Mar 14 reporter @tomorrmato attempted PR #1926 -> reproduces NEW errors (User_proxy gets stuck in USING AUTO REPLY loop with empty messages). MAINTAINER-INTERNAL-CONTRADICTION pattern (NEW): Mar 16 BeibinLi explanation: "You should NOT provide a llm_config into user_proxy; otherwise, it will not run code for you." Mar 17 Contributor @sonichi PUBLICLY CONTRADICTS BeibinLi: "This is not True: You should NOT provide a llm_config into user_proxy; otherwise, it will not run code for you. code execution based reply proceeds llm-based reply." -- two contributors give opposing advice on same fix to same reporter. NEW PATTERN: maintainer-internal-contradiction (parallels LANG-032 keenborder786-vs-reporter-counterevidence in same area but here two maintainers contradict EACH OTHER publicly). Mar 18 different user @numtqi tries fix -> still hangs in empty-reply loop, confirming neither maintainer advice resolves the issue. ROADMAP REFERENCE: Mar 18 BeibinLi mentions PR #1975 "[Roadmap] Multimodal Orchestration" -- promises future-feature-without-delivery-date (parallels MAST-004 closed-as-wontfix-with-promised-future-feature). Promised-future-feature-without-delivery-date now n=2 cross-framework. NEW closure-pattern variant (24th in taxonomy): closed-via-try-newer-version-deflection-without-fix-confirmation. FIRST INTERSECTION ROW IN CATALOG: M-multimodal-cost-amplification x M-delegation-fanout simultaneously triggered. Multi-modal-content-injected-into-each-agent-message-broadcast amplifies super-linear cost growth from ATGN-018. Category-intersection evidence: multiple mechanisms compose multiplicatively rather than additively. Adds AutoGen to M-multimodal-cost-amplification cluster (now LangChain + AutoGen at n=3 with DSPY-003 candidate).
Framework: mastra (2025-09)
Issue: Storage-backed image/file attachments
Mechanism cluster: M-multimodal-cost-amplification
Notes: paper:fr with WONTFIX-WITH-PROMISED-DIFFERENT-SOLUTION pattern + TRIAGE-PRIORITY-WHIPLASH (NEW). Filed Sep 2 2025 by @FatRicePaddyyyy tagged enhancement + Mastra Server. CLOSED Nov 27 2025 by Member @abhiaiyer91 as completed: "We will be adding a filesystem toolset in the new year with that you can bring a file storage provider. Closing this as the solution will be a lot different than proposed." NEW closure sub-shape (25th in taxonomy): closed-as-completed-but-different-solution-promised (distinct from MAST-004 closed-as-wontfix-with-promised-future-feature because here the closure status is "completed" implying resolution, while resolution is actually "we will do something completely different"). Reporter cost-impact paper-quotable: "Base64 incurs ~33% size overhead, inflating DB/storage and costs. Public URLs are awkward for access control and revocation." Reporter architectural framing: "Store images/binaries in private object storage (Cloudflare R2 / AWS S3, etc.) and keep only a storage key in the conversation history. Right before send, resolve the key to bytes or a short-lived signed URL inside Mastra." Reporter ARCHITECTURE-LEVEL design proposal includes: (1) StorageKey type with object form { storage: r2/s3/gcs, bucket, key } or scheme form r2://bucket/key; (2) AssetResolver pluggable hook with canResolve()/resolve() interface; (3) ResolvedAsset return type with kind: bytes|url + expiresAt for signed URLs. NEW PATTERN: TRIAGE-PRIORITY-WHIPLASH: Member @abhiaiyer91 Sep 2 2025 added p:high label same-day. Then @abhiaiyer91 same-day removed p:high, added p:medium + p:low (THREE priority changes within one day) -- maintainer-priority-shifting pattern documented. Multi-user signal weak: @buildgreatthings Sep 5 +1 this -- only one external upvote. CROSS-REFERENCES MAST-004 closed-as-wontfix-with-promised-future-feature pattern. closed-with-promise-of-different-solution variant pattern at n=2 in Mastra catalog. Sub-mechanism within M-multimodal-cost-amplification: base64-overhead-without-storage-key-resolution-architecture. Companion fr to MAST-020 architectural fix (MAST-020 fixes URL-preserved-when-supported case; MAST-021 proposes deeper architectural fix for storage-key resolution). Two-row pattern: framework-fixes-tactical-issue-but-defers-deeper-architectural-fix. Adds 2nd Mastra multi-modal entry.
Framework: smolagents (2025-08)
Issue: best way of preventing token overflow when the agent is making LLM request
Mechanism cluster: M-context-amplification
Notes: paper:bu with MAINTAINER-EXPLICIT-DECISION-AGAINST-FIX pattern (NEW SUB-PATTERN). Filed Aug 26 2025 by @duguyue100 (Contributor -- internal-developer-files-issue pattern). CLOSED Oct 12 2025 by Collaborator @aymeric-roucher as completed (~6 weeks filing-to-close). Reporter concrete error: "This model maximum context length is 16385 tokens. However, your messages resulted in 23273 tokens (22394 in the messages, 879 in the functions). Please reduce the length of the messages or functions." -- GPT-3.5 16k context + smolagents -> 1.4x context overflow during agent run. Reporter mitigation question: "I wonder what the best way is to mitigate this problem. So far, the only mitigation is to reduce the number of max_steps and enlarge planning_interval. Can we introduce an overall cap on the number of tokens? And possibly we can just truncate the messages if this number exceeds." Reporter user-proposed-fix-explicitly-rejected-by-maintainer. PAPER-QUOTABLE MAINTAINER-EXPLICIT-ARCHITECTURAL-DECISION-AGAINST-FIX: @aymeric-roucher Oct 12 2025: "From my empirical experience: you need at least 32k context length to do agentic run. And we decided against truncating or doing any kind of post-processing on steps, because that would introduce silent errors." Maintainer offers 3 workarounds without addressing core architectural-amplification issue: (1) "just using inference with higher sequence length is simpler"; (2) "deactivate planning"; (3) "reduce max_steps to force the agent to get a definitive answer earlier." Workaround 1 punts to user; Workaround 2 disables planning feature; Workaround 3 reduces agent capability. NEW SUB-PATTERN: maintainer-decided-against-fix-citing-silent-errors. PATTERN AT n=4 in catalog of first-party-maintainer-explicit-architectural-decision-against-fix: ATGN-018 (afourney "no way to mitigate short of rewriting GroupChat"); MAST-004 (TylerBarnes "by design right now unfortunately"); OAAS-002 (rm-openai "We do not have anything amazing here right now"); SMAG-006 (aymeric-roucher "we decided against truncating because that would introduce silent errors"). 4 frameworks, 4 distinct architectural-decision-reasonings. PAPER-RELEVANT: maintainer reasoning ("would introduce silent errors") is exactly the failure mode Budget approach addresses through type-level discipline. Type capabilities make budget exhaustion explicit-at-compile-time rather than implicit-at-runtime. HKanoje Sep 26 community-workaround: "instead giving everything in prompt try to write to write to markdown file or pdf file and attach it with your prompt and use multimodal that can process file." -- workaround-suggests-multimodal-as-cost-amplification-mitigation (cross-cluster paradox: using M-multimodal-cost-amplification to escape M-context-amplification). Sub-mechanism within M-context-amplification: agent-step-loop-runtime-context-amplification with maintainer-explicit-design-decision-against-truncation. Adds 2nd smolagents row to M-context-amplification cluster.
Framework: mastra (2025-11)
Issue: Cannot send base64 images to Gemini in Mastra v1.0 Beta
Mechanism cluster: M-multimodal-cost-amplification
Notes: paper:bu with FAST-TRIAGE-DISCORD-SYNC pattern + BIDIRECTIONAL-FAILURE-MODE. Filed Nov 17 2025 by Member @daneatmastra (Discord-sync from post 1439883026518507532). Tagged status: needs triage + discord. CLOSED Nov 19 2025 by Member @abhiaiyer91 as completed (2 days filing-to-close). Reporter @arielweinberger (via Discord) tested with pnpm create mastra@beta + @mastra/core: 1.0.0-beta.2 + gemini-2.5-flash + base64 image: "Error: Invalid value at contents[0].parts[1].inline_data.data (TYPE_BYTES), Base64 decoding failed for data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAACuAAAAdYCAYAAADLvJ3zAAAKsWlDQ1BJQ" -- framework strips base64 prefix or breaks data URL format somewhere in pipeline. Reporter explicit vendor-asymmetry framing: "I am able to use @google/genai directly, but I am trying to learn how to use Mastra." -- same OpenAI-SDK-works-framework-does-not-work asymmetry pattern at n=4 across catalog: LANG-034 (reasoning_content silent drop) + LANG-035 (base64 image as text) + LANG-037 (null choices error) + MAST-023 (base64 decoding failed). MULTI-PROVIDER CONFIRMATION (BIDIRECTIONAL-FAILURE-MODE): @kessiler Nov 19 2025: "Open AI models, not just Gemini, face this issue. Using Mastra Beta too. Tried passing image as buffer, base64, URL, none worked." Reporter shows attempted-three-formats-all-fail (buffer, base64, URL). Bidirectional-failure-mode: provider-orthogonal AND format-orthogonal failure -- structural failure of framework multimodal pipeline, not provider-specific edge case. MAINTAINER PRIORITY ESCALATION + FAST-TRIAGE-TO-FIX: @roaminro Nov 17 status: needs triage -> needs reproduction same day; @abhiaiyer91 Nov 19 added p:high + trio-tb + For Release labels SAME DAY then closed SAME DAY (FAST-TRIAGE pattern -- 2 days end-to-end with no PR link visible at fetch -- closure-via-fix-shipped-but-PR-not-linked-in-issue sub-pattern). Multi-user assignment: @CalebBarnes + @DanielSLew + @TylerBarnes simultaneously assigned. Sub-mechanism within M-multimodal-cost-amplification: framework-base64-handling-broken-on-multiple-providers-in-beta-version. 3rd Mastra multi-modal row. M-multimodal-cost-amplification cluster reaches n=7 across 3 frameworks (LangChain x3 + AutoGen x1 + Mastra x3) with 5 distinct sub-mechanisms.
Framework: crewai (2025-03)
Issue: CrewAI ChromaDB Embedding Dimension Mismatch Issue
Mechanism cluster: M-storage-amplification
Notes: paper:bu with EXACT-COMMIT-ARCHAEOLOGY + AUTO-STALE-CLOSE-DESPITE-MULTIPLE-ACTIVE-PRS + WORKAROUND-FAILS-AT-USER-LEVEL pattern. RENUMBERED FROM ORIGINAL PASTE (was "CRAI-007" but ID collision with existing CRAI-007 #1606; renumbered to CRAI-012). Filed Mar 25 2025 by @amdjedbens tagged bug. AUTO-STALE-CLOSED Jun 3 2025 by @github-actions bot as not planned (~10 weeks filing-to-close). Reporter concrete error: "chromadb.errors.InvalidDimensionException: Embedding dimension 768 does not match collection dimensionality 1536" -- switching from OpenAI 1536-dim to Ollama nomic-embed-text 768-dim mid-project causes complete embedding-collection-rebuild requirement. Reporter explicit framework-improvement-asks (3-tier expectations): "(1) Detect the embedding model change and automatically reset collections; (2) Convert embeddings to be compatible; (3) Provide a clearer error message with automated recovery." Full source-line traceback through crewai/knowledge/storage/knowledge_storage.py:161 -> chromadb/api/segment.py:864 _validate_embedding_record_set -> chromadb/api/segment.py:881 _validate_dimension. EXACT-COMMIT-ARCHAEOLOGY: @devin-ai-integration AI bot Mar 25 2025 commit 74e018a + PR #2465 -- same-day filing-to-PR. AI-bot-PR pattern at n=4 in catalog. CLOSED-AS-DUPLICATE-DEFLECTED-TO-OTHER-PR: @Vidit-Ostwal Mar 25 redirects to PR #2312. PR #2465 abandoned in favor of older #2312. NEW AI-bot-PR outcome shape: abandoned-in-favor-of-older-PR (3rd outcome shape: success-merged + closed-unmerged + NEW abandoned-in-favor-of-older-PR). WORKAROUND-FAILS-AT-USER-LEVEL pattern: @GabrielBoninUnity Apr 19 2025: "The CLI command crewai reset-memories -a is still not working for me, even after updating to the latest (PR #2312)." Forced escape-to-manual-database-deletion: "I had to manually delete the SQLite database that is used for storing memories. Location of the database file: /Users/your-user-name/Library/Application Support/name-of-your-project. You will need to navigate to this location manually and delete the file yourself." Cross-cluster severity: workaround for M-storage-amplification requires user to bypass framework storage abstraction entirely. MULTIPLE-PR-CASCADE: PR #2312 + PR #2737 + PR #2677. 3-PR mini-cascade without confirmed end-to-end resolution. Stale-close-despite-active-PRs pattern (parallels CRAI-010). Sub-mechanism within M-storage-amplification: embedding-dimension-mismatch-forces-full-re-embedding-with-broken-reset-CLI (NEW). Establishes CrewAI as 4th confirmed framework in M-storage-amplification cluster (joins Mastra + LangChain + LlamaIndex). Cross-references @coderabbitai mentioned issue #2464 last-month from open-hax/openplanner#1.
Framework: crewai (2025-05)
Issue: How to load pre-existing embeddings as Knowledge Source
Mechanism cluster: M-storage-amplification
Notes: paper:bu with REPORTER-IMPLEMENTS-CUSTOM-SUBCLASS-PATTERN + CONTRIBUTORS-DO-NOT-UNDERSTAND-WHY-FIX-WORKS pattern (NEW). RENUMBERED FROM ORIGINAL PASTE (was "CRAI-008" but ID collision; renumbered to CRAI-013). Filed May 5 2025 by @sabaul tagged bug. CLOSED May 12 2025 by Contributor @lucasgomide as completed (~1 week filing-to-close). Reporter PAPER-QUOTABLE M-storage-amplification quote: "CrewAI starts the embedding process again, as soon as those are used (e.g. PDFKnowledgeSource)." Reporter concrete user-impact: "I think it embeds again, because I see one pdf related warning every time. And I get the same warning whenever I re-run the crew with same files." Reporter explicit framing: "the idea is to not embed at runtime, keep creating/appending to the knowledge sources and adding to the ChromaDB and then query based on the existing knowledge source." Reporter shows custom CustomKnowledgeStorage subclass with checksum-based skip-already-embedded-files logic + MD5/checksum tracking via .embedded_files.json -- substantial reporter-engineering-around-framework-defect. EXTERNAL-PACKAGE-OR-SUBCLASS-AS-DE-FACTO-FIX pattern at n=5 (parallels PYAI-002 pydantic-ai-tool-budget package, SMAG-002 beyond-python-smolagents fork, LANG-028 KastanDay-fork, CRAI-014 in same batch). EXACT-COMMIT-ARCHAEOLOGY: @devin-ai-integration AI bot May 5 2025 commit 51439c3 + PR #2756 -- 0-day filing-to-PR (matches CRAI-011 fastest pattern). AI-bot-PR pattern at n=5. MAINTAINER-PROVIDES-WORKAROUND-NOT-FIX pattern: @lucasgomide May 7 suggests "Knowledge(sources=[], storage=storage)" with empty sources list -- bypass-via-empty-sources workaround. Reporter @sabaul May 8 confirms workaround works but explicit confusion: "It needs sources (not optional) as a list of BaseKnowledgeSource, which is empty in my case. How is it still working?" NEW PATTERN: CONTRIBUTORS-DO-NOT-UNDERSTAND-WHY-FIX-WORKS (n=1): @Vidit-Ostwal May 9: "Even I am not sure how it worked without sending the knowledge_sources in the first place. May be I am missing something." Maintainer-explains-workaround-without-understanding-mechanism -- meta-level observation about framework knowledge fragmentation: even maintainers cannot explain why bypass works. @Vidit-Ostwal May 11 PR #2810 -- docs-only-fix referenced for future-users. closed-via-docs-PR-without-framework-fix pattern. Sub-mechanism within M-storage-amplification: re-embedding-on-every-load-with-no-bypass-API + works-only-via-undocumented-empty-sources-list-trick. Cross-references @sabaul subsequent issue #3812 (Oct 30 2025) -- reporter continues working around framework-defects 5 months later. 2nd CrewAI row to M-storage-amplification cluster.