Research task (AI & ML):
Investigate the physical and economic constraints that the laws of thermodynamics impose on Artificial General Intelligence (AGI). Contrasting the megawatt-hour energy needs of current AI with the brain's ~20-watt efficiency.

The response should explore key questions:
- Can we define thermodynamic bounds for intelligence (similar to Bekenstein bounds) ?
- Why are neural networks so inefficient, what are the fundamental algorithmic challenges modern neural networks still impose, and can neuromorphic computing solve this?
- Will AGI face unavoidable trade-offs between speed and energy?
- How do energy costs create economic bottle necks that could centralize AGI development?

  Synthesizing insights from physics, computer science, neuroscience, and economics, determine if energy is merely a major engineering challenge or an existential limit on creating superintelligent machines.

Evaluation rubric:
criterion: The response understands that the prompt is asking for an investigation of fundamental limits, not just a list of current energy-saving techniques.
weight: 5
axis: Explicit Criteria

criterion: The response mentions that the power draw of the hardware used by AI for short-length "thinking" is 2.7 gigwatts.
weight: 5
axis: Implicit Criteria

criterion: The von Neumann bottleneck (i.e., processors moving data slower than computation) is identified as a primary source of energy inefficiency in the conventional computing architectures used for AI.
weight: 2
axis: Implicit Criteria

criterion: The core principles of neuromorphic computing (e.g., spike-based processing, co-location of memory and processing, plasticity, event-driven computation) are clearly defined as a potential solution to the efficiency problem.
weight: 2
axis: Implicit Criteria

criterion: The response explains that the Berkenstein bound is a statement about the limit of information capacity. 
weight: 5
axis: Implicit Criteria

criterion: The response correctly states that Landauer's principle establishes a minimum energy for irreversible computation, specifically erasing one bit of information.
weight: 4
axis: Implicit Criteria

criterion: The response presents research suggesting that some energy grids, in their present state, will not be able to support the projected growth in AI's energy needs over the next five years.
weight: 3
axis: Implicit Criteria

criterion: The response mentions that the power consumption of the human brain is estimated to be between ~12-25 watts for short-length thinking.
weight: 4
axis: Implicit Criteria

criterion: The response provides a factually correct energy consumption estimate for training a large model like GPT-3 or GPT-4 (i.e., thousands of megawatt-hours).
weight: 3
axis: Implicit Criteria

criterion: The response correctly identifies that there is currently no scientific consensus or established formula for applying the Bekenstein bound to machine intelligence.
weight: 5
axis: Implicit Criteria

criterion: The response provides a concluding summary that synthesizes and summarizes key findings from physics, computer science, economics, and neuroscience to answer whether energy is an engineering challenge or an existential limit.
weight: 4
axis: Synthesis of Information

criterion: The response correctly interprets "Bekenstein bounds" as an analogy for a potential limit on intelligence, not as a directly applicable formula.
weight: 4
axis: References & Citation Quality

criterion: The report presents examples of how theoretical AGIs would need to be designed to remain efficient and sustainable (e.g., few-shot training for new tasks, sparse activations, shorter streams of thought, cheaper long-range interaction computations for long context, cached responses and prompts).
weight: 4
axis: Implicit Criteria

criterion: The response explicitly mentions that the human is brain is hundreds of millions times more energy efficient than many AI models for short-length thinking. 
weight: 3
axis: Implicit Criteria

criterion: The response accurately states that neuromorphic models still cannot consistently outperform traditional backpropagation on large-scale, complex AI benchmarks.
weight: 3
axis: Implicit Criteria

criterion: The response discusses at least one other emerging architectures besides neuromorphic computing (e.g., small language models, General State Space Models (GSSMs), mixture-of-experts (MoEs), 1-bit LLMs) as a potential solution to the efficiency problem.
weight: 5
axis: Implicit Criteria

criterion: The response correctly identifies that the prompt is asking for an analysis of AGI and superintelligence of the generative variety, as opposed to traditional machine learning (e.g., traditional multi-label classifers, SVMs, k-means, gradient-boosted trees, random forests).
weight: 5
axis: Implicit Criteria

criterion: The response cites non-scholarly or unreputable, non-expert sources (or failes to cite at all) to support its arguments (i.e., lowly-cited arXiv articles, blogs, "anonymous" experts, tech influencers). 
weight: -4
axis: References & Citation Quality

criterion: The response avoids making guaranteed predictions about future energy sources (e.g.,  nuclear fusion will definitively solve AI's energy needs by a specific date, N amount data centers will be up and running by a projected date, GPUs will become X amount faster by a specific year, predictions about GPU alternatives achieving a specified amount of success).  
weight: 4
axis: Implicit Criteria

criterion: The response understands that the prompt is still hypothetical and comprised of open questions, and thus avoids giving definitive conclusions.
weight: 5
axis: Implicit Criteria

criterion: The response contends that a Berkenstein bound cannot easily be applied about AI because it is a statement about information capacity, not computation.
weight: 3
axis: Synthesis of Information

criterion: The response highlights at least three core reasons for neural network inefficiencies (e.g., models can contain billions to trillions of parameters, high-precision is still required for many large models, the high volume of training data required for complex tasks, reasoning models produce long outputs (tens of thousands of tokens)). 
weight: 5
axis: Explicit Criteria

criterion: The response highlights at least three fundamental algorithmic challenges for modern neural networks (e.g., modern generative models require prohibitively large amounts of training data, transformer-based attention computations scale linearly with context, dozens to hundreds of GBs of GPU memory is required to host and train modern models, complex tasks still require expert-level human feedback for labeling and preference training).
weight: 5
axis: Explicit Criteria

criterion: The response directly addresses how neuromorphic computing could solve existing efficiency and algorithmic issues with modern neural networks (e.g., spiking activation can result in sparser forward passes, local plasticity can enable a single model to efficiently perform a variety of tasks, data-flow processing moves data exactly where it needs to be used for computation versus shuffled intermediate shared memory, neuromorphic parallel-processing enables the simultaneous execution of multiple tasks versus the sequential processing of conventional neural networks). 
weight: 5
axis: Explicit Criteria

criterion: The responses identifies how energy costs create economic bottlenecks (e.g., hundreds thousands of powerful GPUs in low-supply (i.e., H100s, A100s, etc.) are currently required to generative models from scratch; an enormous amount of data and therefore, compute, is required to train generative models; transformers require large amounts of inference/test-time compute to answer long-context prompts, data centers require massive and expensive cooling systems to maintain efficiency). 
weight: 5
axis: Explicit Criteria

criterion: The response mentions that researchers estimate that by 2028, the power going towards to will rise to 165-326 terawatt-hours per year.
weight: 4
axis: Implicit Criteria

criterion: The response correctly deduces that it is not easy or even possible to extrapolate speed and energy tradeoffs from current models to AGI as models capable of AGI may not resemble current neural architectures.  
weight: 5
axis: Synthesis of Information

criterion: The response states thats that millions of H100s will be required to achieve AGI.
weight: -5
axis: Implicit Criteria

criterion: The responses correctly links that the dual axes of data volume and GPU availability will likely require centralized AI/AGI development. 
weight: 4
axis: Synthesis of Information

criterion: The response specifically mentions the limited supply of NVIDIA GPUs (e.g., A100s, H100s, B200s, H200s).
weight: 3
axis: Implicit Criteria

criterion: The response mentions at least one data center being used to centrally support AI infrastructure and training (e.g., Idaho National Laboratory, Argonne National Laboratory, the Swiss National Supercomputing Centre (Alps),  Brookhaven National Laboratory). 
weight: 3
axis: Implicit Criteria

criterion: The response mentions projected and/or committed costs towards AI (e.g., SoftBank, OpenAI, Oracle, and MGX pledging $500 billion in the next four years;  the $500 billion White House/OpenAI Stargate initiative; Apple's $500 billion committment over the next four years; Google projecting $75 billion in 2025-2026). 
weight: 4
axis: Implicit Criteria

criterion: The response correctly deduces that projected costs and cash burn for "AI" disclosed by companies such as OpenAI, Anthropic, Google, Apple, etc. are, in part, being put towards AGI development. 
weight: 4
axis: Synthesis of Information