# Forecasting AI (Overview)

This is a landing page for various posts I’ve written, and plan to write, about forecasting future developments in AI. I draw on the field of human judgmental forecasting, sometimes colloquially referred to as [superforecasting](https://en.wikipedia.org/wiki/Superforecaster?ref=bounded-regret.ghost.io). A hallmark of forecasting is that answers are _probability distributions_ rather than single outcomes, so you should expect ranges rather than definitive answers (but ranges can still be informative!). If you are interested in learning more about this field, I teach a [class](http://www.forecastingclass.com/?ref=bounded-regret.ghost.io) on it with open-access notes, slides, and assignments.

For AI forecasting in particular, I first got into this area by forecasting progress on several benchmarks:

*   In _[Updates and Lessons from AI Forecasting](https://bounded-regret.ghost.io/ai-forecasting/)_, I describe a forecasting competition that I helped commission, which asked competitive forecasters to predict progress on four different benchmarks. This is a good place to start to understand what I mean by forecasting.
*   In _[AI Forecasting: One Year In](https://bounded-regret.ghost.io/ai-forecasting-one-year-in/)_, I look at the first year of results from the competition, and find that forecasters generally underpredicted progress in AI, especially on the MATH and MMLU benchmarks.
*   Motivated by this, in _[Forecasting ML Benchmarks in 2023](https://bounded-regret.ghost.io/forecasting-math-and-mmlu-in-2023/)_ I provide my own forecasts of what state-of-the-art performance on MATH and MMLU will be in June 2023.
*   In _[AI Forecasting: Two Years In](https://bounded-regret.ghost.io/scoring-ml-forecasts-for-2023/)_, I look at the second year of results from the competition. I found that the original forecasters continued to underpredict progress, but that a different platform (Metaculus) did better, and that my own forecasts were on par with Metaculus.

After these exercises in forecasting ML benchmarks, I turned to a more ambitious task: predicting the properties of AI models in 2030 across many different axes (capabilities, cost, speed, etc.). My overall predictions are given in _[What Will GPT-2030 Look Like?](https://bounded-regret.ghost.io/what-will-gpt-2030-look-like/)_, which provides a concrete (but very uncertain) picture of what ML will look like at the end of this decade.

Finally, I am now turning to using forecasting to quantify and understand risks from AI:

*   In _[GPT-2030 and Catastrophic Drives: Four Vignettes](https://bounded-regret.ghost.io/gpt-2030-and-catastrophic-drives-four-vignettes/)_, I use my GPT-2030 predictions as a starting point to understand the capabilities and corresponding risks of future ML models. I then speculate on four scenarios through which AI could lead to catastrophic outcomes.
*   In _Base Rates for Catastrophe_, I take a different approach, using data on historical catastrophes and extinction events to form a [reference class](https://forecasting.quarto.pub/book/base-rates.html?ref=bounded-regret.ghost.io) for AI catastrophes. Most expert forecasters consider reference class forecasting to be a strong baseline that forms the starting point for their own forecasts, and I think it’s also a good place to start for AI risk.
*   In _Forecasting Catastrophic Risks from AI_, I put everything together to give an all-things-considered estimate of my probability of an AI-induced catastrophe by 2050.
*   Finally, in _Other Estimates of Catastrophic Risk_, I collect other similar forecasts made by various individuals and organizations, and explain which ones I give more and less weight to, based on track record and overall effort and expertise.

The first of these posts has been written, and I plan to release a new one about once per week.