trentmkelly/LessWrong-43k huggingface dataset:
Progress links and tweets, 2023-06-14
by jasoncrawford — 2023-06-14T16:30:12.346Z
Link: https://rootsofprogress.org/links-and-tweets-2023-06-14

Announcements & opportunities

The Long Now Foundation is hiring an Executive Director
Terraform Industries (solar-powered carbon-neutral natural gas synthesis) is hiring
A class on how politics works, so you can get involved productively
Read Something Wonderful, writing that has stood the test of time (including my pieces on iron and smallpox)
Turpentine Media, a new media network covering tech, business, & culture

News

Ted Kaczynski, the “Unabomber,” has died in prison at 81. See Kevin Kelly’s summary of and rebuttal to his philosophy

Links

Marc Andreessen on AI safety. Note what I highlighted and where I disagree
Jacob Steinhardt predicts what will GPT look like in 2030
What Lant Pritchett is for: productivity, state capability, education, labor mobility
The Illusion of Moral Decline (via @a_m_mastroianni). One of many decline illusions
The untold story of the precursors of the steam engine (by @antonhowes)
Holden Karnofsky suggests “a playbook for AI risk reduction”
Mark Lutter interview on his Caribbean charter city project (via @MarkLutter)
Trailer for Nuclear Now, which is now on streaming platforms (via @oklo)
Casey Handmer on why we don’t build underground. Building a road tunnel costs “$100,000 per meter, or equivalent to a stack of Hamiltons of the same length.” (!) Although I don’t know if Casey used Norway or Seattle figures
Also Casey: 1 gram of stratospheric SO2 offsets 1 ton of CO2 for 1 year (!)
“Existential Crunch” synthesizes research about social collapse (via @mattsclancy)
Turnspit dogs (via @RebeccaRideal via @antonhowes)

Requests for books/sources on…

How Moore’s Law actually happened?
Related, the modern history of chips/AI?
West Midlands in the Industrial Revolution and today?
Programming design for Apollo 11?
The history of commodities markets?
The S&L crisis?

Other queries

Who are the virtuosos of the ChatGPT form?
What societies have successfully gone through degrowth? (If any…)
Anecdotes of scientists eschewing management or calling for more autonomy?
Term for the age you would have died at, if not for modern medicine?
Best treatment of why actions often produce the opposite of the intended effect?
How to square Cruise’s claim of 1 injury in first 1M driverless miles with SF’s claims? (Maybe three are actually Waymo, and none seem at-fault?)

Quotes

“The flying machine is one of God’s most gracious and precious gifts”
Admiral Rickover on academic rea

...
Philosophical Cyborg (Part 1)
by ukc10014 — 2023-06-14T16:20:40.317Z

This post is part of the output from AI Safety Camp 2023’s Cyborgism track, run by Nicholas Kees Dupuis - thank you to AISC organizers & funders for their support. Thank you for comments from Peter Hroššo; and the helpful background of conversations about the possibilities (and limits) of LLM-assisted cognition with Julia Persson, Kyle McDonnell, and Daniel Clothiaux.

Epistemic status: this is not a rigorous or quantified study, and much of this might be obvious to people experienced with LLMs, philosophy, or both. It is mostly a writeup of my (ukc10014) investigations during AISC and is a companion to The Compleat Cybornaut.

TL;DR

This post documents research into using LLMs for domains such as culture, politics, or philosophy (which arguably are different - from the perspective of research approach - from science or running a business, the more common suggested use-cases for LLMs/AIs i.e. Conjecture’s CoEm, Ajey Cotra’s

scientist model

, or Andrew Critch’s

production web

).

As a case study, I (ukc10014) explore using LLMs to respond to a speculative essay by Paul Christiano: the response is posted here. The current post is more about the process of LLM-assisted, cyborgist reasoning, and follows on from

The Compleat Cybornaut

.

The takeaway is not surprising: base models are useful for generating ideas, surveying an unfamiliar space, and gathering further avenues for research. RLHF-tuned models like ChatGPT are able to write summaries of existing content, often in considerable detail, but this requires human skill in generating (often, chains of) prompts that tease out the model's latent knowledge, and specifically requires the human to know enough about the topic to ask pointed questions. There is a constant risk of hallucination, particularly when using a chat-type interface, where previous portions of a conversation can ‘infect’ (as well as usefully inform) the current query.

Models are not very helpful in planning the overall research direction or developing arguments, though when asked leading questions the tuned models seem designed to respond in a supportive manner. But it seems harder to prompt them to ‘push back’ or usefully critique the human’s approach.[1]

Introduction

Can LLMs be useful for doing philosophy?

This post approaches the question by: a) exploring how ...
Is the confirmation bias really a bias?
by Lionel — 2023-06-14T14:06:06.315Z
Link: https://lionelpage.substack.com/p/reassessing-the-confirmation-bias

The confirmation bias is a widely discussed cognitive phenomenon that has drawn considerable attention in psychology and behavioural economics.

Traditionally, the search for confirmatory information has been portrayed as a bias, suggesting a systematic deviation from rational information processing. However, upon closer examination, this characterisation lacks a solid foundation in a precise understanding of what truly constitutes the best information processing strategy.

Optimal information acquisition involves making the best use of limited time and resources. In this context, the optimal strategy to acquire information can require looking for confirmatory evidence that aligns with one’s pre-existing beliefs. This point has been made by several researchers, and recently by Zhong (Optimal Dynamic Information Acquisition, Econometrica, 2022) in a general framework where a decision-maker can fully flexibly decide the type of information to collect under two constraints: more informative signals are more costly and waiting longer to collect information is costly. Zhong’s result is that the decision-maker’s optimal strategy is to look for confirmatory information in the form of a Poisson process that gives from time to time strong signals confirming the decision-maker’s prior belief (when this prior belief is right).

I develop this general point in the Substack post linked. Another interesting post on this topic is Klein’s The Curious Case of the Confirmation Bias, that presents older criticisms about the psychological evidence about the confirmation bias.

In the end, it looks like the notion of “confirmation bias” has been at best overused, at worst it may be a pure misconception to think of it as a bias.
Lightcone Infrastructure/LessWrong is looking for funding
by habryka — 2023-06-14T04:45:53.425Z

Lightcone Infrastructure is looking for funding and are working on the following projects:

We run LessWrong, the AI Alignment Forum, and have written a lot of the code behind the Effective Altruism Forum.
During 2022 and early 2023 we ran the Lightcone Offices, and are now building out a campus at the Rose Garden Inn in Berkeley, where we've been doing repairs and renovations for the past few months.
We've also been substantially involved in the Survival and Flourishing Fund's S-Process (having written the app that runs the process) and are now running Lightspeed Grants.
We also pursue a wide range of other smaller projects in the space of "community infrastructure" and "community crisis management". This includes running events, investigating harm caused by community institutions and actors, supporting programs like SERI MATS, and maintaining various small pieces of software infrastructure.

If you are interested in funding us, please shoot me an email at habryka@lesswrong.com (or if you want to give smaller amounts, you can donate directly via PayPal here).

Funding is quite tight since the collapse of FTX, and I do think we work on projects that have a decent chance of reducing existential risk and generally making humanity's future go a lot better, though this kind of stuff sure is hard to tell. We are looking to raise around $3M to $6M for our operations in the next 12 months.

[1]

Edit (June 23): I've now given a lot more details on how we operate and what we work on in the comments. I would recommend checking them out if you want to more context on our work.

^

Two draft readers of this post expressed confusion that Lightcone needs money, given that we just announced a funding process that is promising to give away $5M in the next two months. The answer to that is that we do not own the money moved via Lightspeed Grants and are only providing grant recommendations to Jaan Tallinn and other funders.

We do separately apply for funding from the Survival and Flourishing Fund, through which Jaan has been our second biggest funder. We also continue to actively fundraise from both SFF and Open Philanthropy (our largest funder).
Anthropic | Charting a Path to AI Accountability
by Gabe M — 2023-06-14T04:43:33.563Z
Link: https://www.anthropic.com/index/charting-a-path-to-ai-accountability

Below is the text copied from the linked post (since it's short). I'm not affiliated with Anthropic but wanted to link-post this here for discussion.

This week, Anthropic submitted a response to the National Telecommunications and Information Administration’s (NTIA) Request for Comment on AI Accountability. Today, we want to share our recommendations as they capture some of Anthropic’s core AI policy proposals.

There is currently no robust and comprehensive process for evaluating today’s advanced artificial intelligence (AI) systems, let alone the more capable systems of the future. Our submission presents our perspective on the processes and infrastructure needed to ensure AI accountability. Our recommendations consider the NTIA’s potential role as a coordinating body that sets standards in collaboration with other government agencies like the National Institute of Standards and Technology (NIST).

In our recommendations, we focus on accountability mechanisms suitable for highly capable and general-purpose AI models. Specifically, we recommend:

Fund research to build better evaluations

Increase funding for AI model evaluation research. Developing rigorous, standardized evaluations is difficult and time-consuming work that requires significant resources. Increased funding, especially from government agencies, could help drive progress in this critical area.
Require companies in the near-term to disclose evaluation methods and results. Companies deploying AI systems should be mandated to satisfy some disclosure requirements with regard to their evaluations, though these requirements need not be made public if doing so would compromise intellectual property (IP) or confidential information. This transparency could help researchers and policymakers better understand where existing evaluations may be lacking.
Develop in the long term a set of industry evaluation standards and best practices. Government agencies like NIST could work to establish standards and benchmarks for evaluating AI models’ capabilities, limitations, and risks that companies would comply with.

Create risk-responsive assessments based on model capabilities

Develop standard capabilities evaluations for AI systems. Governments should fund and participate in the development of rigorous capability and safety evaluations targeted at critical risks from advanced AI, such as deception and autonomy. These eval

...
Demystifying Born's rule
by Christopher King — 2023-06-14T03:16:20.941Z

Under the many-worlds interpretation (MWI) of quantum physics, born probabilities are a bit mysterious and hard to deal with. How do you do Solomonoff induction when you keep splitting? There are ways to do this, but they end up equivalent to another interpretation of quantum mechanics that is much simpler than all that: the pilot wave theory. Not only does it include an interpretation of Born probabilities, but it becomes a consequence of statistical mechanics. We also do not need to adapt our epistemology to QM anymore, because the pilot wave theory is deterministic. All our uncertainty comes from initial conditions, just like in Newtonian physics. Much of the weirdness of QM just reduces to the weirdness of anthropics in general. The pilot wave theory is an interpretation, meaning choosing it is a philosophical choice instead of a scientific one.

The ontology includes two components: a wave function (like in many-worlds) and a certain configuration of particles. The configuration is called the "actual" configuration (although the wave function and this configuration are both necessary components). The wave function follows Schrödinger's equation and the configuration follows the guiding equation.

Simulation of trajectories a photon can take in the double-slit experiment. (The large gaps on the right edge are the gaps in the interference pattern.)

So far, this might seem more complicated than MWI. MWI already has the wave function (it is the same wave function ins MWI and pilot wave), why bother with the "actual configuration"? But MWI also needs to postulate Born probabilities, and if you want to actually do inference you need to figure out how this interacts with embedded agency (good luck trying to figure out how embedded agency applies to Wigner's friend in MWI). To actually interpret MWI, you end up needing to select a configuration to feed into your inference procedure, and this will be roughly as complicated as the pilot wave theory anyways, except you still needed to assume Born's rule as an axiom. In the pilot wave theory, you just select observations from the "actual" configuration.

So how does Born's rule get demystified? Well, given a small amount of uncertainty in the initial conditions, the ensemble quickly converges to be such that the configuration of particles is distributed according to Born's rule applied to the wave function. This is called...
My guess for why I was wrong about US housing
by romeostevensit — 2023-06-14T00:37:04.162Z

Edit/update: this is receiving more upvotes and I think it is worth explicitly pointing out that while housing is up 60% over this time frame, the Sp500 is up 90+%, so this is not intended as a full on refutation of all the points made in the original post.

A few years ago I wrote a post highlighting what I felt were underappreciated considerations in home buying, most of which cut against the standard advice of homes as a major part of your investment portfolio. It garnered a lot of argument, most of which, I felt, boiled down to reference class forecasting over which mean housing would regress to for price trajectory: a more current one, or a more long term one.

Since then, housing has gone up 60+%. And the reason I think I got the reference class wrong is that I missed something very important about the US housing market. Since discovering it I find most people I mention it to are surprised by it.

I was very surprised that the US had the second lowest house price to income ratio in the world. It's less surprising when you consider that the income side of the equation is quite high, but still. Since then, the US has started dropping down the list and is currently in 6th. But this means hundreds of other countries have more expensive houses relative to local incomes. I was so busy looking at reference classes that are local price trends that I missed the major reference class of the US having started out as an outlier.

Were the US to continue to regress towards the world median of around 12, prices would need to appreciate another 2-2.5x from where they are at currently, which would put the median house at $900k-1.2MM. I failed to appreciate that most of the world has a much sharper property owner class divide and it wouldn't be all that surprising to see a relatively younger nation move in that direction over time. This also renders it much less surprising that investment firms are buying huge swathes of housing stock, even at the new higher rates.
Notes from the Bank of England Talk by Giovanni Dosi on Agent-based Modeling for Macroeconomics
by PixelatedPenguin — 2023-06-13T22:25:00.769Z

Presentation 1: Exploring Microfoundations in Economic Theory by Giovanni Dosi

Key Discussion Points:

1. The main subject of the talk is Professor Dosi's forthcoming book, "The Foundations of Complex Evolving Economies", which critically explores the microfoundations of economic theory.
2. Discussion centered around how agent-based models can illuminate analysis of technology, innovation, and macroeconomic policy.
3. Patterns of development, Kuznetsian patterns of long-term growth, and the Kaldorian stylized facts were analyzed.
4. Professor Dosi employed a unique metaphor - cake-making as an example of input-output functions to highlight the limitations of unconstrained functions.
5. The talk explored organizational behaviors, heuristics, and routines in firms' behaviors, along with firms' heterogeneity.
6. The talk extended Professor David Soskice's discussion on comparative management practices, focusing on differences between British and American managers.

Questions and Responses:

1. The merits of Agent-based modeling (ABM) over Dynamic Stochastic General Equilibrium (DSGE) models were discussed, specifically in handling innovation.
2. A query about the predictive ability of ABM for complex phenomena, like whale movement, remained unanswered.
3. Professor Dosi shared his perspective on technology: while it has significantly improved, it has also ushered in severe environmental challenges.

Presentation 2: Heterogeneity, Interaction and Macroeconomic Implications

Key Discussion Points:

1. Methodological considerations for building ABMs were discussed.
2. Interpretative elements necessary for understanding ABMs were highlighted.
3. The model, according to Professor Dosi, rejects parameter scanning and emphasizes qualitative fact generation.
4. It was clarified that the model is not calibrated against DSGE.
5. Professor Dosi reviewed a series of K+S models, specifically citing Dosi et al. 2022.
6. The discussion delved into how technology is defined, specifically around innovation and efficiency.
7. The competitiveness of firms was identified as a critical factor in determining their market share in each round.

Presentation 3: Discussion

Key Discussion Points:

1. The challenge of reconciling stylized facts with microeconomic theories was discussed.
2. The presentation highlighted problems faced by economic models, with specific examples from Minnesota and Cambridge.
3. Essential gui...
Introducing The Long Game Project: Improving Decision-Making Through Tabletop Exercises and Simulated Experience
by Dan Stuart — 2023-06-13T21:45:02.265Z

“Imagine a commercial airline pilot trusted to navigate turbulent skies with no prior simulation training … why then are critical decisions in organisations often left to ‘theory’, ‘experience’, ‘intuition’, or ‘gut instinct’?" - de Suarez et al 2012.

Summary:

The Long Game Project aims to bolster organisational decision-making, team collaboration, and resilience. We're committed to a) creating tools and resources to streamline tabletop exercise accessibility; b) advising, offering thought leadership, and comprehensive consultation to organisations keen on tabletop exercises; c) fostering a culture of organisational gaming and building a community around tabletop games.

Imagine a universe where organisations are primed for the unforeseen, where decision-makers skillfully steer through crises to yield favourable outcomes amidst chaos thanks to routine simulated practice. The art of effective decision-making under uncertain conditions is a skill worth honing. Play now, so you're prepared later.

Our mission is to help organisations improve decision-making (IIDM), enhance team collaboration, and build resilience using tabletop exercises that are rules-light but experience heavy.

Rationale

The world is becoming increasingly complex and unpredictable, with organisations facing various challenges. Institutional decision-making in practice is often ill-equipped to handle such complexity, leading to sub-optimal decision quality and severe failure modes. 80,000 hours lists IIDM as one of the best ways to improve existing levers, interventions and practical suggestions for our long-term future and lists this as a neglected issue. While IIDM is a complex cause area, requiring some disentangling, there are existing levers, interventions and practical suggestions that are under-utilised in practice. The Long Game Project exists to bridge this gap by applying several levers to a diverse range of sectors, helping organisations adapt and thrive in an ever-changing landscape. Our toolkit of levers includes; tabletop exercises, role-playing, future simulations, facilitation, probabilistic thinking games, goal and value alignment, decision design and other methods that combine game design and ideas from behavioural economics, psychology, and organisational theory.

Our Aims:

1. Becoming the go-to source for tabletop scenario advice and tools, providing expert con...
Intelligence allocation from a Mean Field Game Theory perspective
by Marv K — 2023-06-13T19:52:14.260Z

Each of us now faces a choice every minute of every day:
Should I really use my brain power for this, or shoud I let ChatGPT do it?

Using your own brain has its merits. You might get more intelligent and resilient in the long run. But ChatGPT already does most things much better and quicker, so why not let it spare you stirring your own grey goo? Then again, supporting ChatGPT may increase the likelihood of an extinction event.

In this post, we’re gonna look at this from a mean field game theory perspective.

First, let’s state the assumptions:

There are N individuals, each of them having a level of intelligence described by the stochastic process X_i(t) for i=1,...,N.

Each individual can choose at each time t a control α_i(t) in [0,1], where α_i(t) = 0 means the individual uses their own intelligence to solve the task and α_i(t) = 1 means they use ChatGPT to solve the task.

The intelligence of an individual and the AGI evolve as a result of these decisions.

The payoff for each individual is the cumulative tasks solved up to time T, but if the average intelligence of ChatGPT surpasses a certain threshold before time T, everyone is extinct and the payoff is zero.

Individual Behavior (SDE)

The dynamics of the individual's intelligence could be modeled as the following stochastic differential equation:

dX_i(t) = [α_i(t) b_1(X_i(t), m(t)) + (1-α_i(t)) b_2(X_i(t), m(t))]dt + σ W_i(t)

Here, the terms b_1 and b_2 represent the drifts under the two options available to the agents - use their own intelligence or use ChatGPT to solve tasks.

Here's a possible way to define them:

b_1 could be a function that represents the rate of increase in the player's intelligence when they solve tasks on their own. For instance, we could model this as proportional to the current intelligence of the player, but with diminishing returns. This might look like: b_1(X_i(t), m(t)) = a_1 * X_i(t) / (1 + d_1 * X_i(t)) Here, a_1 is a positive constant that represents the maximum possible rate of increase in intelligence, and d_1 is another positive constant that represents the rate at which the increase in intelligence slows down as the player's intelligence increases.
b_2 could be a function that represents the rate of increase in the AGI's intelligence when it is used to solve tasks. For instance, we could model this as proportional to the average intelligence of all players (since more intelligent

...