sajjadanwar0/token-budgets-catalog huggingface dataset:
Framework: langgraph (2026-01)
Issue: Agent infinite looping until recursion limit
Mechanism cluster: M-retry-loop
Notes: paper:bu; text-to-SQL retry; reproduced in Section 5.3
Framework: langchain (2024-01)
Issue: load_summarize_chain hits infinite loop due to small token_max
Mechanism cluster: M-retry-loop
Notes: paper:bf; description corrected in v7 from "rec_limit fix" to "summarize-chain loop"
Framework: langchain (2023-04)
Issue: Cost getting too high
Notes: SKIPPED for paper. 2023 Q&A thread: reporter @vishnu1991 misunderstood chunked retrieval mechanics (asking why a 2k-token document costs 2k tokens per query). Community members @Qualzz and @kartikbhardwaj2000 explained correctly. Not a failure incident. Auto-closed not-planned by dosubot Sep 26 2023.
Framework: langchain (2025-02)
Issue: Reasoning tokens not in cost calculation
Mechanism cluster: M-cost-observability
Notes: paper:bu; closed not-planned by dosubot stale-bot May 22 2025. Reporter @dbuos provided langchain_community/callbacks/openai_info.py code with comment line "## reasoning_tokens are not used in the cost calculation below" -- the source code itself acknowledges the gap. Affects o3-mini, o1, and variants. User @roryeckel comment Nov 6 2025: "This is critical to resolve, otherwise trim_messages becomes incompatible with the Responses API when used with reasoning summary." Documented workaround: subclass ChatOpenAI and override get_num_tokens_from_messages. Pairs with AIDR-006 and LANG-005 in reasoning-model subclass. Critical companion to LANG-005: same bug class, opposite outcome.
Framework: langchain (2025-04)
Issue: Total cost calculation possible bug with o3-mini
Mechanism cluster: M-cost-observability
Notes: paper:bf; closed completed Apr 13 2025 by @ccurme via PR #30807, fixed in commit f005988. Reporter @tdahar provided minimal repro showing o3-mini total_cost calc returned $0.012 instead of expected ~$0.05 (4x undercount). Bug was in standardize_model_name not recognizing o3 as completion-priced model. Fixed in 2 days. Companion to LANG-004 (filed Feb 2025, similar bug class, auto-closed stale, not fixed). Asymmetric outcomes provide structural evidence that cost-bug attention is uneven within same project.
Framework: langsmith (2025-10)
Issue: Token count and cost incorrectly display as 0
Mechanism cluster: M-cost-observability
Notes: paper:mf; closed completed Oct 30 2025 by @angus-langchain with "this has been released for a week." LangSmith silently deprecated server-side token tokenization, breaking cost displays for all gpt-4.1, gpt-4o, o3-mini traces from Oct 23 2025. Multiple users affected. Resolution shifted cost-counting responsibility from framework to users: "you must ensure usage metrics are sent to LangSmith either in Outputs or Metadata." Adds cost-OBSERVABILITY failure subclass (no other rows cover this).
Framework: langsmith (2025-09)
Issue: Cost showing up as 0 despite having tokens
Mechanism cluster: M-cost-observability
Notes: paper:bu; OPEN as of Dec 2025. Reporter @ryanlohyr documents OpenAI gpt-4o calls via Responses API showing $0 cost in LangSmith despite token usage. Maintainer @angus-langchain self-assigned then confirmed: "Confirmed I can repro. This is related to the Responses API which we are working on supporting better." Three months later @RRaphaell reports related symptom (token counts undercounting first-turn vs subsequent-turn). Sibling issue #2085 (Oct 23 2025) reports same total-token-and-cost-zero pattern. Distinct from LANG-006 by surface (SDK vs server) and status (still open vs released-fix); strengthens M-cost-observability cluster within LangChain ecosystem.
Framework: langchain (2024-02)
Issue: OpenAI Tokens and Cost are inaccurate
Mechanism cluster: M-cost-observability
Notes: paper:bu; closed not-planned by dosubot stale-bot Jul 4 2024 with no fix. Reporter @ArslanKAS documents get_openai_callback returning 815 tokens and $0.00134 USD for a gpt-3.5-turbo request that llm.get_num_tokens cross-check showed should be ~13K tokens (~16x undercount). One community confirmation @axiangcoding "Same issue here" Mar 28 2024. Bot-tagged as bug, then auto-stale-closed three months later. Same get_openai_callback subsystem as LANG-010 (which was patched in 0.1.12 a month earlier); juxtaposition shows M-cost-observability bugs persist across patches in langchain callback infrastructure.
Framework: deepagentsjs (2025-12)
Issue: Agent infinite loop with large tool results
Mechanism cluster: M-retry-loop
Notes: paper:bf; flagship deepagentsjs entry. Reporter @KDKHD documents three-way interaction between toolTokenLimitBeforeEvict, filesystem-eviction, and read_file line limit causing infinite eviction/read loop. LangSmith trace at https://smith.langchain.com/public/5ccfa3e5-07f0-4572-b205-22ae3d3d5689/r Fixed in PR #172 by @christian-bromann on Feb 3. Follow-up issue #272 documents related grep/glob/ls failure (suggests underlying pattern recurs).
Framework: langchain (2024-02)
Issue: get_openai_callback returns 0 for create_openai_functions_agent
Mechanism cluster: M-cost-observability
Notes: paper:bf; closed completed Mar 15 2024 by @eyurtsev as duplicate of #16798, patch released in PR #18761 (langchain 0.1.12). Reporter @lamparelli documents the langchain-recommended create_openai_functions_agent constructor silently breaks get_openai_callback (returns 0 tokens, $0 cost) while the deprecated OpenAIFunctionsAgent works correctly. Contributor @keenborder786 root-caused: RunnableAgent.plan uses RunnableSequence.stream which hits the OpenAI streaming-chat-completion endpoint whose response object lacks usage info. Patch incomplete: @gryshko reports it doesn't fix pandas_dataframe_agent; @axtsoi reports it doesn't fix create_sql_agent ("token counting is one of the fundamental and crucial features by LangChain but this doesn't seem to be treated as a high priority"). Reporter framing quotable: "creating an OpenAI Agent and tracking costs is a pretty basic functionality that should work out of the box."
Framework: langchain (2023-10)
Issue: Conversational Chain Error 429
Notes: SKIPPED for paper after URL verification (S6 rate-limit-not-budget). Filed Oct 3 2023 by @harchyboy; auto-stale-closed by dosubot Feb 10 2024 with no fix. Reporter hits OpenAI 10KTPM rate limit when using ConversationChain but not bare LLMChain. Contributor @sudranga speculates memory-type summarization may make additional LLM calls (a plausible amplification hypothesis) but nobody quantifies token counts, documents billing impact, or verifies the theory. No dollar evidence (A.1 fail), no specific mechanism with amplification factor (A.2 fail), no quotable maintainer classification statement (A.3 fail). The codebook S6 line: pure rate-limit issue without a documented cost-amplification mechanism.
Framework: langchain (2024-05)
Issue: 429 Resource Exhausted with gemini-1.5-pro
Mechanism cluster: M-retry-loop
Notes: paper:bu; closed completed Nov 11 2025 by @mdrxy after 18-month thread; partial fix later in PR by @IrakliGLD Dec 10 2025 ("Reduce LangChain retry attempts from 6 to 2 to prevent quota exhaustion"). Filed May 28 2024 by @sidagarwal04 against gemini-1.5-pro. Reporter saw retry attempts every 2/4/8/16/32 seconds when API call "was made only twice"; LangChain tenacity-based retry decorator re-sending full prompt context on each Google 429. Critical user @vipulchaudhary1601 quote (Jul 8 2025): "I am seeing 5-10 request to be sent to gemini for mine one call of .invok and token are getting used without producing any output." Maintainer @mdrxy classified as upstream: "As I read it, this is a Google issue, not a LangChain issue." Yet the IrakliGLD fix the next month explicitly reduces LangChain-side retry attempts (6 -> 2), confirming the amplification was LangChain-side. Documented amplification factor 5-10x per .invoke call. Strongest M-rate-limit-amplification case after AIDR-003.
Framework: langgraph (2025-08)
Issue: Node Retry Policies not respected on ValidationError
Mechanism cluster: M-retry-loop
Notes: paper:bu; closed by @casparb Sep 9 2025 with documentation pointer rather than framework fix. Subtle silent failure: RetryPolicy() does NOT retry on Pydantic ValidationError (which is a ValueError subclass) by default. Workaround requires explicit RetryPolicy(retry_on=ValidationError). Default behavior unchanged. Follow-up #6574 (Gemini MALFORMED_FUNCTION_CALL silent termination) suggests silent-failure pattern recurs.
Framework: langchain (2023-08)
Issue: Langchain QA over large documents results in Rate limit errors
Mechanism cluster: M-retry-loop
Notes: paper:bu; closed by reporter @gourao Aug 29 2023 with "Found a different way to accomplish this" -- never shared the workaround despite multiple followup requests from @Wesleystormrage and @afedotov-align. Issue references 5+ related rate-limit issues (#11155, #11500, #11657, #11907, #12712, #12986). Documents 2023-era rate-limit-retry pattern in LangChain QA pipeline. dosubot suggested batch_size=5 workaround. Framework did not ship rate-limit handling.
Framework: langgraph (2025-06)
Issue: Agent timeout causes error
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Jun 3 2025 by @JiayuChen02; closed Jun 4 2025 (next day) by @nfcampos via PR #4950. Bug: setting supervisor_agent.step_timeout=180 in multi-agent LangGraph supervisor causes ParentCommand/GraphInterrupt exceptions to bubble up incorrectly through pregel/runner.py. Stack trace shows usage_metadata correctly threaded (107 total tokens, in/out tracked precisely): cost accounting was working fine; bug is purely in error-propagation through the Pregel executor. Title sounds budget-relevant (timeout = runaway prevention) but mechanistically it is an exception-propagation bug, not a cost-amplification or budget-mechanism failure. No cost evidence (A.1 fail), no documented amplification (A.2 fail), no maintainer classification statement (A.3 fail).
Framework: langchain (2024-08)
Issue: No notification when exceeding Anthropic rate limit
Mechanism cluster: M-retry-loop
Notes: paper:bu; auto-stale-closed by dosubot Dec 6 2024 with no fix. Filed Aug 12 2024 by @albint3r against ChatAnthropic. Reporter documents the application silently hangs (no exception, no timeout) after exceeding Anthropic 40k tokens/minute limit, requiring server restart. Reproduces locally, ruled out NGINX. Collaborator @eyurtsev acknowledged and pointed to manual rate-limiter workaround: "Are you seeing any useful messages from the underlying client or from anthropic? You can avoid the issue partially by specifying a rate limiter." Reporter @albint3r confirms server hangs with no error: "it occurs silently and leaves the server waiting indefinitely." Multiple users (@simoroma) confirm same pattern through 2025. Different M-rate-limit-amplification subclass than LANG-012: silent-hang on rate-hit rather than loud-retry-storm. Reporter later memory-leak hypothesis (Jan 2025) is speculative; retention rests on reproducibility and maintainer hands-off framing.
Framework: autogen (2023-10)
Issue: Infinite loops with gpt4, OpenAI rate limit triggered
Notes: SKIPPED for paper after URL verification (S2 duplicate-of-existing). Filed Oct 5 2023 by @adriangalilea; closed Oct 9 2023 by @LittleLittleCloud after providing workaround. Same root mechanism as paper-tagged ATGN-002: termination-check failure causing default-auto-reply loop with rate limit consequences. Root cause: is_termination_msg lambda used exact match while assistant emitted "...TERMINATE" or "Terminate"; admin agent without llm_config used default_auto_reply="" (empty string) causing "USING AUTO REPLY... admin (to chat_manager): [empty]" loop until 40k TPM rate limit hit. Workaround: set default_auto_reply="TERMINATE" and is_termination_msg=lambda x: x[content]==TERMINATE. ATGN-002 holds the AutoGen-cascading-loop slot in the catalog; this is the earlier instance of the same pattern.
Framework: autogen (2025-02)
Issue: Infinite loop in quick start sample with additional agent
Mechanism cluster: M-retry-loop
Notes: paper:bu; closed by reporter Feb 4 2025. Maintainer ekzhu response: "Interesting attempt to break the example :)" classified the cascading-message behavior as inherent to default subscriptions, provided documentation workaround using separate topics; framework was NOT patched. First AutoGen entry in catalog.
Framework: autogen (2025-03)
Issue: Infinite loop in swarm with human dialogue
Mechanism cluster: M-retry-loop
Notes: paper:mf; converted to discussion #5869 by @ekzhu Mar 7 2025 (effectively closed without fix). Maintainer response to swarm infinite-loop: "It is because the model could not generate a tool call for handing off to the user [...] No magic here." Multiple users (@toughnoah) report the same with gpt-4o and qwen-max. Strengthens the by-design pattern; second AutoGen entry parallel to ATGN-001.
Framework: autogen (2023-10)
Issue: Token limit and rate per minute limit errors in framework demo notebooks
Notes: paper:bu; UPGRADED from SKIPPED in v22-batch-24 revision per codebook section 10. Original triage skipped as S6 (rate-limit-not-budget) but new cross-reference evidence from ATGN-018 (#1070) recharacterizes the rate-limit and context-overflow errors as concrete user-impact of the architectural cost amplification mechanism @afourney explicitly documents. AUTO-STALE-CLOSED at 8.5 months by Collaborator @thinkall Jun 18 2024: "We are closing this issue due to inactivity; please reopen if the problem persists." Filed Oct 4 2023 by @jkf87 (transferred from microsoft/FLAML Oct 11 2023 by @sonichi). TWO CONCRETE FRAMEWORK-SHIPPED-EXAMPLE FAILURES IN ONE ISSUE: (1) "Rate limit reached for default-gpt-4 in organization on tokens per min. Limit: 40000 / min. Please try again in 1ms." running notebook/autogen_agentchat_groupchat_research.ipynb -- official multi-agent research notebook hits 40K tokens/minute rate limit on gpt-4. (2) "InvalidRequestError: This model maximum context length is 8192 tokens. However, your messages resulted in 9297 tokens." running notebook/agentchat_teaching.ipynb -- official teaching notebook overflows gpt-4 8192-token context window by 1100+ tokens. Sub-mechanism: framework-shipped-multi-agent-example-overflows-context-window + rate-limit (per @afourney ATGN-018 explanation: GroupChat broadcasts entire history to all agents = super-linear cost growth that exceeds context windows). Documents M-delegation-fanout x M-context-amplification intersection at framework-example level. Cross-references ATGN-018 (architectural cost amplification) and ATGN-019 (fanout-no-stop) -- this row demonstrates concrete user-impact of those mechanisms in framework own demo notebooks. REVISION TRANSPARENCY NOTE: original S6 skip was correct on narrow rate-limit reading; updated retention reflects mechanism-coherence with cross-referenced architectural evidence. Methodology trail preserved.
Framework: autogen (2023-10)
Issue: Roadmap for handling context overflow
Mechanism cluster: M-context-amplification
Notes: paper:fr; closed completed Mar 14 2024 by @yiranwu0 when several PRs (#131, #421, #443, #497) merged. 2023-era AutoGen roadmap parallel to AGPT-001/002. Documentation pointer (microsoft.github.io/autogen/docs/topics/long_contexts) cited Apr 2024 by @ekzhu became 404 by Sep 2025 after AutoGen v0.4 rewrite, with user @reski-rukmantiyo asking "how to solve this?" Pattern: shipped-then-bit-rotted via rewrite. Third AutoGen entry; oldest 2023 issue showing context-overflow as a known concern across frameworks.
Framework: autogen (2024-12)
Issue: Token usage and latency metrics in TaskResult
Mechanism cluster: M-cost-observability
Notes: paper:fr; OPEN as of Feb 2025. Filed Dec 16 2024 by lead maintainer @ekzhu himself. Documents specific cost-observability gap in autogen-agentchat 0.4 architecture: "Console is currently using agents inner messages to keep track of the total token usage -- this is inaccurate. As agents may not emit inner messages, and SelectorGroupChat model client usage is not reflected. We need to fix these as well." Active engineering discussion: @gziz on implementation, @ekzhu specifying GroupChatSelectSpeaker event-type architecture, @husseinmozannar extending the gap to M1 orchestrator ("the model usage of the M1 orchestrator is also not tracked"). Documented zero-cost-display: "The total token usage reported is 0, which is false, because the selector group chat uses model client to choose the next agent." PRs #4768 and #4771 reference this; cross-linked to Observability parent issue #5620 and #4172, #4835. FIRST cross-ecosystem M-cost-observability case extending the cluster from LangChain-only (LANG-006/007/008/010) to AutoGen, demonstrating cost-observability gaps are an architectural pattern across multiple frameworks.
Framework: autogen (2023-10)
Issue: We Need a Token Count
Mechanism cluster: M-cost-observability
Notes: paper:fr; closed completed Oct 12 2023 by @qingyun-wu via PR #179 by @yiranwu0. Filed Oct 10 2023 by @aiInvader: "This is a super crucial feature. When finished executing, we need the token count." Reporter pushed back when @qingyun-wu pointed to existing logging (which only logged cost in USD, not token count). Maintainer @sonichi confirmed gap: "Lets add token count to the logging. Anyone interested in adding it?" Contributor @TomExMachina: "The token counts are a metric that need to always be available to both the developer and the agent." Implementation by @yiranwu0 within 2 days. Earliest cost-tracking gap in autogen, contemporaneous with AGPT-001 (Apr 2023), GPTE-003 (Jun 2023), and LLMI-003 (Jul 2023). Strengthens the maintainer-request cluster to 6 cases.
Framework: autogen (2025-02)
Issue: Token counting issues with smaller models
Mechanism cluster: M-cost-observability
Notes: paper:bu; OPEN as of Feb 2025. Filed Feb 6 2025 by Member @afourney (Microsoft Research). Documents KeyError llama3.3:latest from _MODEL_TOKEN_LIMITS[resolved_model] in autogen-ext openai client when using local Ollama models. remaining_tokens() and get_token_limit() crash; multimodal_web_surfer breakage. Console floods with "Model llama3.3:latest not found. Using cl100k_base encoding." messages outside the logging system. @afourney own classification: "Its unclear what the best solution is here. One option is to allow the context window and tokenizer to be specified in ModelInfo?" Triaged through three milestones (0.4.6 -> 0.4.7 -> 0.4.8) by @ekzhu, then unassigned and milestone-shifted to 0.4.x-dotnet by @fniedtner Feb 18 2025; remains open and unassigned. Distinct M-cost-observability sub-mechanism from ATGN-006: architectural lock-in to curated provider list (hardcoded MODEL_TOKEN_LIMITS table breaks on any non-OpenAI/non-Anthropic model). Forward-looking pattern as LLM ecosystem fragments to local models.
Framework: crewai (2023-12)
Issue: AgentExecutor chain loop stucked, will break OpenAI budget
Mechanism cluster: M-delegation-fanout
Notes: paper:bf; first crewai loop incident
Framework: crewai (2024-09)
Issue: Agents getting stuck in loop since move to LiteLLM
Mechanism cluster: M-retry-loop
Notes: paper:bf; LiteLLM-introduced regression, same-day patch
Framework: crewai (2024-06)
Issue: Loop in crew execution
Notes: SKIPPED for paper. Bare 2-sentence bug report by @Ai-ENGINEER-s with no code, no logs, no repro. Asked for details by @mangalprada on Jun 8, no follow-up. Auto-closed by github-actions stale-bot Aug 22 2024.
Framework: crewai (2024-01)
Issue: Looping tasks
Notes: SKIPPED for paper after URL verification (S4 qa-not-incident). Filed Jan 25 2024 by @chriscamacho as a workflow-design question (4-line body): how to chain programming-agent and testing-agent tasks with feedback loop. No incident, no mechanism, no cost evidence. @Biancamazzi (bot tag) pointed to Discord; community user @Kolaposki suggested task.execute(). No core maintainer engagement. Converted to discussion thread #1147 by @theCyberTech Aug 7 2024 and locked. Title is loop-shape but the issue is structurally a Q&A about how to build a feedback loop, not a report of one running away.
Framework: crewai (2025-08)
Issue: Thinking budget parameter for gemini models
Notes: SKIPPED for paper. Resolved same-day by reporter @Vamshi3130 after user @mouramax pointed to existing LiteLLM thinking_budget passthrough. Not a real failure incident. Closed Aug 9 2025.
Framework: autogpt (2023-04)
Issue: Make Auto-GPT aware of running cost
Mechanism cluster: M-budget-primitive-missing
Notes: paper:fr; first runaway-cost feature request
Framework: autogpt (2023-04)
Issue: Set budget per session with soft/hard cap
Mechanism cluster: M-budget-primitive-missing
Notes: paper:fr; "set budget for each session with soft and hard cap"
Framework: autogpt (2024-06)
Issue: Optimally productive per dollar of LLM expense; risk of infinite recursion
Notes: SKIPPED for paper. Roadmap/SWOT meta-issue from maintainer @Pwuts; mentions infinite-recursion as a "Threat" but no production incident, no dollar amount. Closed not-planned as duplicate of #6964 by Pwuts on Jun 14 2024. CSV title misleading.
Framework: autogpt (2023-04)
Issue: auto-gpt stuck in a loop of thinking
Notes: SKIPPED for paper after URL verification (S5 user-found-existing-feature). Filed Apr 21 2023 by @MikeWIseX0; closed same-day by reporter after community member @90stm pointed out the issue was lack of paid OpenAI billing account ("Setup a paid account here... it wasnt until I did this that those same issues disappeared"). Reporter response: "Ok thx" and closes. AutoGPT was looping on API auth-rejection errors (DOUBLE CHECK CONFIGURATION warnings visible in logs) due to free-tier-only account. Follow-up @nathantbohn reports same pattern persists with paid account but no engagement and no maintainer analysis. Same shape as LLMI-001: user error / billing-account state, not framework-level cost amplification.
Framework: aider (2024-09)
Issue: Loop ran up $13 in charges rapidly
Mechanism cluster: M-retry-loop
Reported user dollar loss: 13.55
Notes: paper:bf; fixed same day in commit f2e1e17
Framework: aider (2024-09)
Issue: Loop trying to answer a question, tokens 22k->313k
Reported user dollar loss: 0.43
Notes: SKIPPED for paper (duplicate of AIDR-001 root cause): same fix commit f2e1e17. Comments link to #1842. Note user 0-sv reports separate $11.64 loop in deleted folder Nov 2024.
Framework: aider (2024-11)
Issue: Enormous token counts and costs from rate limit retries
Mechanism cluster: M-retry-loop
Reported user dollar loss: 83
Notes: paper:bu; auto-closed by github-actions stale-bot Dec 27 2024 with no fix. Reporter @tom-doerr documents $83 in a single day. Other user @etewiah reports same call sending 114k tokens then 915k tokens after rate-limit retries ($2.75 message). Maintainer @paul-gauthier framed as display-layer issue ("the provider should not actually charge you when they return a rate limit error") even though @etewiah explicitly verified actual charges via provider billing dashboard. Strongest dollar evidence after DNSW-001 ($2150). Adds rate-limit-retry subclass.
Framework: aider (2024-07)
Issue: Tokens leak: 11k tokens for small change
Mechanism cluster: M-context-amplification
Notes: paper:mf; closed by maintainer @paul-gauthier Aug 6 2024. Reporter @ErykCh observed 11k tokens consumed for /tokens displayed 1.8k context (6.1x amplification). Maintainer initially confirmed bug ("not showing the tokens used by the repo-map. I will get this fixed") then walked back over multiple comments: "The --map-tokens is a guideline, not a hard limit" and "This is expected when you are running with no added files. Aider uses a larger repo map." Other user @mr-september reports more dramatic 1.2M token incidents in same thread. Strengthens the by-design pattern (now 6 cases).
Framework: aider (2024-10)
Issue: Costly message $4.99 with token limits breach
Mechanism cluster: M-retry-loop
Reported user dollar loss: 4.99
Notes: paper:bu; closed not-planned by github-actions stale-bot Oct 29 2024 with no maintainer fix. Issue body shows "$4.99 message, $9.68 session" verbatim; oversize-msg breach (1.66M tokens vs 200k limit). Distinct failure mode from AIDR-001 retry loop.
Framework: aider (2025-04)
Issue: DeepSeek R1 thinking in thousands of tokens, cannot limit
Mechanism cluster: M-context-amplification
Notes: paper:bu; OPEN. New failure subclass: reasoning-model hidden-token consumption. Reporter @GeorgeFlorian: DeepSeek-R1 thinks in thousands of tokens before response; aider thinking-tokens setting does not take effect. Workaround reasoning.exclude:true suppresses visibility but not consumption. Adds reasoning-model subclass to catalog (no other rows cover this).
Framework: aider (2024-01)
Issue: /tokens pricing calculations bug
Notes: SKIPPED for paper after URL verification (S3 weak-evidence-stub). Filed Jan 27 2024 by @transcendr; same-day fix by @paul-gauthier in commit 35f8128 closing Feb 10 2024. The bug is a pricing-table lookup error (gpt-4-turbo-preview displayed at base gpt-4 pricing, ~2-3x overstated) not a silent-zero or mechanistic undercount. No documented incident, no mechanism amplification, no quotable maintainer framing -- one-line acknowledge-and-fix. Different shape than LANG-006/007/008/010 which involve ledgers reporting $0 or substantial undercounts despite real spend. Skipping to keep M-cost-observability cluster mechanism-coherent.
Framework: llamaindex (2024-05)
Issue: RateLimitError: Error code 429 quota exceeded
Notes: SKIPPED for paper after URL verification (S5 user-found-existing-feature). Filed May 6 2024 by @craigjurs; closed-completed by reporter May 7 2024 (one day later) with self-resolution: "After some investigation Ive found that indeed its my lack of attention about the refresh on the openai account which caused the api key I was using to be inactive. Ive learned there is an automatic recharge setting that allows this not to happen again. Apologies to waste anyone's time." The 429 was an OpenAI insufficient_quota error from an inactive API key (billing-refresh issue), not framework-level rate-limit amplification. Pure user error / billing-account state. No mechanism amplification, no incident, no maintainer classification statement.
Framework: llamaindex (2024-08)
Issue: Tool calling broken when LLM responds with both text and tool calls
Notes: SKIPPED for paper. Streaming/tool-calling correctness bug not budget overrun. When GPT-4o returns text+tool_calls in same message, stream_chat ignores tool_call and chat stalls. Fixed by maintainer @logan-markewich Apr 10 2025 via new agent-workflows API.
Framework: llamaindex (2023-04)
Issue: Cost of building and querying each index TODO
Mechanism cluster: M-cost-observability
Notes: paper:fr; closed completed Jul 26 2023 by @Disiok with documentation pointer to TokenCountingHandler. Earliest documented cost-tracking gap in LlamaIndex (cited "TODO" from reference docs). Same era as AGPT-001/002. Adds LlamaIndex (run-llama/llama_index) to the catalog. Maintainer @logan-markewich answered followup questions about default embedding model (text-ada-002, $0.0004/1k).
Framework: llamaindex (2025-08)
Issue: Storing tool's raw result from agent's tool calling
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Aug 22 2025 by @bzr1 as a workflow-design question: how to capture raw tool outputs from agent.run() given that the default return is LLM-processed not raw. Collaborator @logan-markewich engaged with technical clarification about Context-typed tool arguments. Mostly a dosubot Q&A thread. Stale-closed weeks later. Mechanistically about result-storage in LlamaIndex workflows, not cost/budget overruns. No cost mechanism, no rate-limit, no amplification, no budget concern anywhere in the thread.
Framework: claude-code (2025-10)
Issue: Autocompacting Loop Causing Massive Token Usage Spikes
Mechanism cluster: M-retry-loop
Reported user dollar loss: 235
Notes: paper:bf; quoted by maintainer @bcherny; closed by commit Oct 17, 2025
Framework: claude-code (2025-08)
Issue: [Bug] [Has Workaround] Auto-Compact Resetting Context Limit To 4-6% Causing Frequent Compaction Loop
Mechanism cluster: M-retry-loop
Notes: paper:bu; closed not-planned by github-actions bot Jan 4 2026 after 60-day inactivity. Filed Aug 24 2025 by @Codewow. Same compaction-loop signature as CCDE-001. Bot itself flagged three duplicates at filing (#6004, #6315, #6232) and the activity log references at least seven additional sibling issues filed Aug-Dec 2025 (#6469, #6541, #8435, #9353, #10389, #11616, #13408). User comment: "it is 3rd day in a row I cannot use my $200 subscription at all." Workaround is manual deletion of settings.local.json; framework was not patched. Tag changed from bf to bu after URL verification.
Framework: claude-code (2025-10)
Issue: [Bug] Auto-Compact Loses Track of Active Commands During Cleanup
Notes: SKIPPED for paper after URL verification. Closed as duplicate of #3321 (.mcp.json not read) by github-actions bot Oct 7 2025. Filed Oct 3 2025 by @eliasstepanik. Underlying root cause is MCP JSON parsing failure plus Cloudflare 525 SSL handshake errors, not a budget overrun. Bot flagged duplicates of unrelated issues (#3321 mcp.json, #7282 background tasks, #8831 slash commands). Compaction-loop angle is incidental, not the primary failure mode. Removed in v18 to maintain catalog quality. [v23 catalog cleanup: renumbered from duplicate CCDE-003 -> CCDE-005 to fill gap in CCDE sequence]
Framework: gpt-engineer (2023-07)
Issue: Rate limit reached for 10KTPM-200RPM
Notes: SKIPPED for paper after URL verification (S6 rate-limit-not-budget). Filed Jul 23 2023 by @pdavis68; closed by owner @AntonOsika Sep 2 2023 with upstream classification: "Doesn't seem like an issue with gpt-engineer but openai, happy to hear suggestions will close otherwise." Reporter wrote a 150-line prompt + answers to 15 clarification questions in one turn, exceeding OpenAI Tier 1 10K-tokens-per-minute account limit. Single-call rate-limit hit on a too-large prompt; no retry amplification, no documented dollar loss, no incident beyond the failed call. Contributor @niklasfink (Aug 29 2023) confirmed: "The problem is that your prompt and clarifications are too extensive for your OpenAI plan." Same OpenAI rate-limit error template ("10KTPM-200RPM") as LANG-011 #11347 and a documented 2023 free-tier UX failure mode across frameworks, but not amplification-through-retry per codebook S6.
Framework: gpt-engineer (2023-06)
Issue: Specifications repeating, consuming many tokens
Mechanism cluster: M-context-amplification
Notes: paper:bf; first gpt-engineer entry. Owner @AntonOsika initially classified spec-repetition as "part of the introspection process" (mf framing) but reduced excessive specs and closed Jun 21 2023. Concrete reproduction: simple "write a 2D car racing game" prompt produces doubled spec output, hits "Aborted" without completing.
Framework: gpt-engineer (2023-06)
Issue: Track and log token usage
Mechanism cluster: M-storage-amplification
Notes: paper:fr; closed completed Jul 5 2023 by @AntonOsika (project owner) via PR #438 merged by @UmerHA. Filed Jun 22 2023 by @AntonOsika himself: "Print and store how many tokens were used in memory/logs. In this way, we can also store this to benchmark results. A huge increase in tokens will not be worth a minor improvement in benchmark resultss." Earliest cost-tracking gap in gpt-engineer, filed by project owner pre-incident. Strengthens the maintainer-request-cluster: AGPT-001 (Apr 2023, feature_request), AGPT-002 (2023, feature_request), LLMI-003 (Jul 2023, feature_request), GPTE-003 (Jun 2023, feature_request), ATGN-007 (Oct 2023, feature_request), PCLP-001 (Mar 2026, maintainer_framing) -- six projects across 2023-2026 documenting cost-tracking as a recognized but reactively-addressed gap.
Framework: danswer (2024)
Issue: Word of warning: 2k euros spent in a few days
Mechanism cluster: M-budget-primitive-missing
Reported user dollar loss: ~2150 (2000 EUR)
Notes: paper:bu; multi-day onyx deployment loss
Framework: openclaw (2026-02)
Issue: Agent session 123 Opus calls in 16min, $4.85, no cap
Mechanism cluster: M-delegation-fanout
Reported user dollar loss: 4.85
Notes: paper:bu; WONTFIX
Framework: nanobot (2026-03)
Issue: Tool-loop cost roadmap; max_iterations doesn't cap spend
Mechanism cluster: M-retry-loop
Reported user dollar loss: 2.70
Notes: paper:fr; max_cost_per_message proposal
Framework: paperclipai (2026-03)
Issue: Agent circuit breaker; #1 pain point across all multi-agent tools
Mechanism cluster: M-retry-loop
Notes: paper:mf; "#1 pain point across all multi-agent tools"
Framework: codex (2026-04)
Issue: Agent spawn slots leak across turns in persistent sessions
Mechanism cluster: M-delegation-fanout
Notes: paper:mf; closed "by design" by maintainer jif-oai. Reporter documents 24 spawns / 14 closes across 9 rounds; spawn permanently blocked from round 4 onward. Follow-up issue #19197 (persistent orphaned subagents) opened by user vzd3v: "It is hard for me to understand why this issue was closed."
Framework: agno (2024-10)
Issue: Ollama setup requiring openai-api key?
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Oct 21 2024 against phidata 2.5.9 (pre-rename to agno) by @gitmurawka; closed same day by @jacobweiss2305 after pointing out user used deprecated llm= parameter instead of model=, which caused phidata to fall back to OpenAI default and fail with auth error. No money spent (call failed before billing); no mechanism amplification; pure configuration confusion. The underlying silent-OpenAI-fallback pattern (user intends Ollama-only but framework routes to paid API) is potentially paper-relevant in a different issue with documented spend, but this specific case has no incident.
Framework: agno (2025-02)
Issue: Fix Agentic Chunking
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). PR merged Feb 17 2025 by @dirkbrnd, authored by @manthanguptaa. Bug: agentic-chunking module hardcoded OpenAI as a dependency even when user configured a different model. Same shape as AGNO-001: misconfiguration/hardcoded-OpenAI-dependency bug, not a cost mechanism. No documented incident, no quantified spend, no rate-limit, no retry amplification.
Framework: agno (2025-05)
Issue: Pass filters for traditional RAG
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). PR merged Jun 2 2025 by @dirkbrnd, authored by @kausmeows. Bug: add_references=True (traditional RAG path) was not threading knowledge_filters through to vector search, while the agentic-RAG path did. RAG filtering correctness bug, not a cost mechanism. No token amplification, no infinite loop, no cost-observability failure documented. Even if wider unfiltered retrieval consumes more tokens than necessary, no documented cost incident or quantification.
Framework: autogen (2024-11)
Issue: Logging metrics for agents
Mechanism cluster: M-cost-observability
Notes: paper:bf; closed completed Jun 14 2025 by @victordibia citing model_usage and OTEL support as the resolution. Filed Nov 14 2024 by @fniedtner (Microsoft, AutoGen team member) documenting v0.4 cost-observability regression vs v0.2: lost client.total_usage_summary and planner.client.actual_usage_summary fields, plus 5 other metrics (conversation history, LLM call count, latency, token counts, agent interaction count, payload). @ekzhu self-assigned and retitled twice; triaged through 0.4.0 -> 0.4.1 -> 0.4.x milestones for 7 months. Eventually closed citing OTEL infrastructure and model_usage field; companion issue #4835 (ATGN-010) for cost-and-cached-tokens specifically remained open. Documents M-cost-observability as a v0.2 -> v0.4 regression-during-major-version-bump narrative.
Framework: autogen (2024-12)
Issue: Model Costs and Cached Tokens
Mechanism cluster: M-cost-observability
Notes: paper:bu; OPEN as of Mar 2026. Filed Dec 27 2024 by @Leon0402 (Contributor) documenting that v0.4 RequestUsage dataclass lacks cost and cached-tokens fields that v0.2 had. @ekzhu engaged immediately: "yes, it is important. We are planning to address this after the 0.4.0. For now, lets targeting 0.4.1 for this one. #4769 and #4719 must be resolved before tackling this one." Triaged 0.4.1 -> 0.4.x. March 2025: user @qanelph reports finding workaround via total_usage() from model_client; @ekzhu confirms "Token counting for Claude is not supported yet." August 2025: @JAS-JPG volunteered; @ekzhu accepted with expanded scope (also track SelectorGroupChatManager and MagenticGroupChatManager LLM calls). Linked PR #7273 (Feb 2026) has not merged. Distinct M-cost-observability sub-mechanism from ATGN-006 (TaskResult-level), ATGN-008 (model-table-level), ATGN-009 (regression overall): specifically about costs-in-dollars and cached-tokens as missing fields. Cross-cutting comment by @lamira-the-human references ATXP transport-layer mitigation, corroborating the layered-taxonomy framing in paper Section 6.
Framework: autogen (2025-02)
Issue: Observability (epic/parent issue)
Notes: SKIPPED for paper after URL verification (S7 roadmap-doc-not-incident). Filed Feb 19 2025 by @jackgerrits as an epic/parent issue: "There are a number of related issues around this overall topic, using this as the parent issue. We need to improve the tracking of usage and cost of models." Lists 4 sub-issues including #4835 (ATGN-010) and #4719 (ATGN-006). The epic itself is a meta-tracking issue without specific incident or mechanism documentation. Same shape as AGPT-003 which we skipped earlier. Maintainer framing is real but already captured in the underlying paper-tagged sub-issues; adding ATGN-011 as a row would double-count.
Framework: autogen (2025-02)
Issue: Outputs in docs are outdated as summary is not printed out by default
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Feb 18 2025 by @thinkall (Collaborator); closed via PR #5720 by @victordibia Feb 26 2025. Bug: documented Console output shows token-usage summary by default but in scripts/notebooks the summary requires output_stats=True. @ekzhu acknowledged: "The doc output is outdated. To turn on set output_stats=True in Console. There is bug in the stats for group chat. #4719" - the underlying stats-for-group-chat bug is ATGN-006 (already paper-tagged). The ATGN-012 issue itself is documentation-stale, not cost-mechanism failure.
Framework: autogen (2024-12)
Issue: Define AgentEvent, rename tool call messages to events
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). PR merged Dec 19 2024 by @ekzhu, resolves #4533. Pure type-system refactor: introduces AgentEvent type, renames tool-call messages to events. Referenced in ATGN-006 thread as adjacent architectural change but its own scope is naming/structure only. No cost mechanism.
Framework: autogen (2024-12)
Issue: Fix tracking SelectorGroupChat tokens with new Message type
Notes: SKIPPED for paper after URL verification (S2 duplicate-of-existing). Exploratory PR by @gziz Dec 20 2024, closed by author same day to consolidate discussion in #4771 (ATGN-015). Author body: "The below changes are not meant to be merged but rather are here to open a discussion about how to correctly track the token usage for the calls that occur internally to some agents/teams such as the SelectorGroupChat." Same mechanism scope as paper-tagged ATGN-006 (#4719); ATGN-006 holds this slot in the catalog.
Framework: autogen (2024-12)
Issue: Track SelectorGroupChat token usage with new Message type
Notes: SKIPPED for paper after URL verification (S2 duplicate-of-existing). Exploratory PR by @gziz Dec 20 2024 proposing UsageEvent message type; closed without merge by @jackgerrits Feb 26 2025: "I am going to go ahead and close this as I think we need to take a holistic look at cost tracking and will overhaul it." Discussion between @gziz, @husseinmozannar, @ekzhu about implementation approach. Same mechanism gap as paper-tagged ATGN-006 (#4719); jackgerrits quote essentially overlaps with ekzhu framing in ATGN-010.
Framework: autogen (2025-03)
Issue: BaseGroupChat ignore_unhandled_exceptions kwarg mismatch
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). GitHub Discussion (not Issue) filed Mar 19 2025 by @yaoyanglee documenting API/version-skew bug: BaseGroupChat passes ignore_unhandled_exceptions=False to SingleThreadedAgentRuntime.__init__ but user runtime version does not accept that kwarg. @ekzhu response: pure environment/version issue, not a cost mechanism.
Framework: autogen (2025-03)
Issue: Save/Load in .NET AgentChat
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Mar 4 2025 by @lokitoth (Member); closed Mar 14 2025 via PRs #5841 and #5855. .NET feature implementation issue for save/load APIs in AgentChat.NET. No cost mechanism, no incident, no documented spend or amplification.
Framework: aider (2025-01)
Issue: Feature request: search the web via searx google and/or jina ai
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Web-search feature request by @KUKARAF Jan 12 2025: requesting /search action to query searx/google/jina. Not budget-related. Issue OPEN, low engagement, no maintainer response on cost angle.
Framework: aider (2025-03)
Issue: claude code /compact command
Mechanism cluster: M-context-amplification
Notes: paper:fr; OPEN as of Sep 2025. Filed Mar 9 2025 by @kanishkaverma requesting Aider implement claude-code /compact command. Maintainer @paul-gauthier initial response (Mar 11 2025): "Aider already automatically compacts/summarizes the conversation periodically" - a "by-design"-style framing. Community pushed back: @paralin "I think adding a manual /compact makes sense rather than doing it automatically so that the user can pick better checkpoints"; @luca-ing (May 2 2025) quotable: "recently Ive often bumped into Claudes context limit on longer, elaborate sessions. If this happens I feel like I have no other choice than to start over, perhaps in the middle of a gnarly refactoring." After 6 weeks of stale-bot threats and community bumps, @paul-gauthier re-tagged as enhancement Apr 28 2025. M-context-amplification mechanism: users hit context limits, lose work, want manual compaction control. Cross-framework signal: same /compact feature is the cause of CCDE-001 ($235 burn from claude-code compaction-loop) and the requested-but-missing mitigation here.
Framework: aider (2025-05)
Issue: fix: base coder not ignoring gitignore if --file is used
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). PR by @ujisati May 13 2025, closed by author same day. Fixes base_coder gitignore handling when --file flag is used. Not cost-related.
Framework: aider (2025-01)
Issue: Does Aider support removing the thinking content of previous replies
Notes: SKIPPED for paper after URL verification (S2 duplicate-of-existing). Filed Jan 21 2025 by @Mushoz; closed Jan 27 2025 by @paul-gauthier as duplicate of #3008: "This looks like a dup of #3008." Same M-reasoning-token-leak mechanism as paper-tagged AIDR-006 (DeepSeek-R1 hidden-token consumption); maintainer-confirmed duplicate. Question about removing <thinking> tags from multi-turn R1 conversations to reduce context.
Framework: aider (2025-02)
Issue: refactor: Update editor discovery to handle environment variables
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). PR by @devriesd Feb 14 2025, closed unmerged same day. Refactor to handle EDITOR/VISUAL environment variables for the /editor command. Not cost-related.
Framework: aider (2024-09)
Issue: Uncaught AttributeError in sendchat.py line 30
Notes: SKIPPED for paper after URL verification (S2 duplicate-of-existing; also S1 wrong-failure-class). Filed Sep 28 2024 by @faroucksc; closed same day by @paul-gauthier as duplicate of #1414. Stack trace shows AttributeError in retry decorator: module litellm.llms.anthropic has no attribute AnthropicError - pure litellm version-mismatch/API change, not a cost mechanism.
Framework: aider (2024-12)
Issue: Fixed /chat-mode architect no description
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). PR by @reecelikesramen Dec 9 2024, closed unmerged Dec 10 2024 by @paul-gauthier ("I will fix this up with different text, thanks"). Fixes display of "No description" text for editor-diff and editor-whole modes in /chat-mode listing. UI text fix, not cost-related.
Framework: aider (2025-03)
Issue: Uncaught UnboundLocalError in repo.py line 302
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Mar 30 2025 by @cristibctr; closed Mar 31 2025 by @paul-gauthier with same-day fix tagged priority. UnboundLocalError in repo.py line 302 caused by upstream IndexError in git tree-walking (gitpython library). Pure git-handling bug, not cost mechanism.
Framework: aider (2025-09)
Issue: Uncaught LookupError in core.py line 305
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Sep 3 2025 by @jhf. Stack trace: LookupError "unknown encoding: hex" in codecs.encode(os.urandom(16), "hex") inside diskcache library used for repomap caching. Python compatibility bug (hex codec removed); not cost mechanism. Issue OPEN as of Sep 2025 but mechanically a tagging/encoding bug not budget overrun.
Framework: claude-code (2025-06)
Issue: False Capability Claims and Immediate Self-Contradiction
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class; also S2 duplicate-of-existing). Filed Jun 24 2025 by @SDS-Mike; closed Aug 21 2025 by github-actions bot as duplicate of #1360. Documents Claude Code making confident false statements about own capabilities (file access, conversation history). Self-knowledge/hallucination issue, not cost mechanism.
Framework: claude-code (2025-07)
Issue: Memory Insertion Corrupts Entire claude.md File Content
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Jul 10 2025 by @mkamrans; auto-closed Jan 5 2026 as not planned. Claude Codes # annotation feature for adding memory deletes the rest of CLAUDE.md content and replaces with "[remaining content has not changed]" placeholder. Real bug with multiple sibling issues (#4451 750-line deletion, #5128 truncation, #4590 overwrite, #2655 deletion, #7787 deliberate deletion of spec) but mechanism is memory-file-corruption not cost overrun. Forces users to lose work but the loss dimension is human-time not API-spend.
Framework: claude-code (2025-07)
Issue: Hooks not executing despite following documentation
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Jul 3 2025 by @grahama1970; closed Jan 15 2026 by @dicksontsai with workaround pointing to CLAUDE_PROJECT_DIR docs. Long thread Jul 2025 -> Jan 2026 with sibling issues #5093, #5307, #5648, #6403, #6876, #7905, #8388, #10098, #12172, #13572. PreToolUse/PostToolUse hooks dont fire automatically despite valid configuration. Hook-execution bug, not cost mechanism.
Framework: claude-code (2025-07)
Issue: PreToolUse hooks not executing at all in Claude Code
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Jul 17 2025 by @cobach; closed same day by @dicksontsai with config-format correction (matcher should be string not object). User-error/config-format issue, same shape as CCDE-006.
Framework: claude-code (2025-07)
Issue: Image Paste Functionality Broken in CLI Interface
Notes: SKIPPED for paper after URL verification (S1 wrong-failure-class). Filed Jul 23 2025 by @boscorp on Windows; self-closed Jul 31 2025 ("I do not know when, but now it works"). Image paste broken on Windows due to ENOENT error trying to read C:\proc\version. Pure platform/UI bug, not cost.
Framework: claude-code (2025-07)
Issue: Display Current Model and Add a Shortcut for Model Switching
Notes: SKIPPED for paper after URL verification (S5 user-found-existing-feature). Filed Jul 31 2025 by @arrrggghhh; auto-closed Feb 9 2026 as not planned. Quotable cost-framing in body: "Opus has much stricter usage limits than Sonnet. While it would be ideal to use Opus continuously, in practice, my workflow involves using Sonnet as the primary model and selectively switching to Opus only when necessary." Auto-tagged area:cost area:tui enhancement. Borderline retain candidate but skip per codebook S5: author noted parts of request were addressed incrementally ("Item 1 has been resolved with the statusline command", "Opus Plan Mode solves item 3"). Issue is fundamentally a UX/hotkey feature request rather than budget-cap mechanism request; different shape than ATGN-007/AGPT-001/AGPT-002 in maintainer-request cluster. Adding would dilute cluster with UX-flavored case.