Harley-ml/lesswrong huggingface dataset:
# Is your job replaceable by GPT-4? (as of March 2023)

*Disclaimer: I don't have access to GPT-4*

There's a lot of discussion these days about technological unemployment due to LLMs. For example, someone could observe that [GPT-4 scored better than average at the US Medical License Examination](https://marginalrevolution.com/marginalrevolution/2023/03/capabilities-of-gpt-4-on-medical-challenge-problems.html), but human doctors are obviously not going to be obsolete in the short term, if only because GPT-4 cannot legally issue a medical prescription[^3yaw7mhfn8t]. 

I am not going to make predictions here; the aim of this post is to make ascertainments. Namely: can we write a list of jobs that could be fully replaced by GPT-4 literally tomorrow? 

Let me make clear what I'm *not* asking for: I'm not asking for hypotetical scenarios where a dedicated workflow including GPT-4 can be set up and tailored in some way in order to replace a human worker. For now I'm only interested in the you're-instantly-fired version, where your boss could just use plain GPT-4 (instead of you) without having to do anything more than writing the prompt.

I've obviously asked a LLM (Alpaca) about this, and its answer was:

*   Transcriptionists  
*   Interpreters and translators
*   Customer service representatives
*   Telemarketers
*   Administrative assistants
*   Accounting clerks
*   Web designers
*   Graphic designers
*   Writers  

I fully agree about customer service representatives, whose stereotype is a shitty job where you just throw nice-sounding bullshit at angry customers (and LLMs are *really really good* at generating nice-sounding bullshit with infinite patience). 

For the other items in the list, I don't think that we are near to the point where the job could be totally replaced by GPT. Google Translate has existed for many years, but the world is still full of human translators, and I've yet to see a printed book admittedly translated by a machine (probably because every time I've tried to translate something more complicated than a single sentence with Google Translate, the output had to be slightly edited... do we have evidence that GPT-4 can be a much better translator?).

"Writers" is a bit too broad. But I would agree that the subcategory of *copy*writers is probably fully replaceable already.

Other suggestions?

[^3yaw7mhfn8t]: If a human is formally holding the position for legal reasons while GPT does all the real work, does it still count as technological unemployment? For the sake of this discussion I would resolve this as "no".
# Can independent researchers get a sponsored visa for the US or UK?

I’m an independent alignment researcher from Canada. I’m trying to figure out my long-term options (I want to move somewhere and stay there for several years). 

I have some special visas as a Canadian to go to either place, but I’d need to be working at a ”job” to be eligible for the visa. I *could* go to London on a 2-year visa without a regular job, but would need a job after that visa expires. It feels like a gamble going there for two years and then not having the ability to stay long-term.

Someone told me I should consider checking to see if I can work on this with a university. I'm not sure if they'd be up for signing off on my visa as long as I keep getting my independent funding?

So, what are the options for independent researchers who don't have US or UK citizenship? Are there any orgs that sponsor visas for independent researchers?

Note: I’m completely happy staying in Canada and mainly working remotely with collaborators (though I’d probably move to a Canadian city with a couple of other researchers), but I want to make an informed decision.
# Apply now to rationality camps: ESPR & PAIR - new Program on AI and Reasoning (ages 16-20)


We are happy to announce that the ESPR team is running [two](https://espr.camp/2023-camps) **immersive summer workshops** for mathematically talented students this year: A classic applied rationality camp, ESPR, and PAIR, a new program focusing on AI and cognition:  
 

*   [**Program on AI and Reasoning (PAIR)**](https://pair.camp/) for students with an interest in artificial intelligence, cognition, and reasoning in general. 
    *   The workshop aims to provide participants with an in-depth understanding of how current AI systems work, mathematical theories about cognition and human minds, and how the two relate. Additionally, the workshop teaches thinking and meta-cognitive tools related to creativity, motivation, collaboration, and ability to drive and lead independent inquiry. Alumni of the camp should be in a better position to think about AI independently, understand state-of-art research, and come up with their own research ideas or AI projects. See the [curriculum details](https://pair.camp/curriculum).
    *   Students who are **16-20 **years old
    *   August 3rd - August 13th in Cambridge, United Kingdom

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/da7fe6b66bb70dab16fd73664994cd78687c083d091dfb0c.png)

*   [**European Summer Program on Rationality (ESPR)**](https://espr.camp/)for students with a desire to understand themselves and the world, and interest in applied rationality.
    *   The curriculum covers a wide range of topics, from game theory, cryptography, and mathematical logic, to AI, styles of communication, and cognitive science. The goal of the program is to help students hone rigorous, quantitative skills as they acquire a toolbox of useful concepts and practical techniques applicable in all walks of life. See the [content details](https://espr.camp/program).
    *   Students who are **16-19 **years old
    *   August 17th - August 27th in Oxford, United Kingdom

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/6344d92e2ef137b781aa0bc8e224eaf7f22af088ef33a311.png)

  
We strongly encourage all teenage LessWrong readers with an interest in these topics to apply.

Both programs are **free **for accepted students, travel scholarships available. Apply to both camps [here](https://airtable.com/shrfc4VJi1cfcjmM2). First application deadline: **April 16th**.

If you know students aged 16-20 years who might enjoy these camps, please send them this link with an overview of both camp: [https://espr.camp/2023-camps](https://espr.camp/2023-camps).
# Are extrapolation-based AIs alignable?

(This is an account of my checking a certain alignment idea and finding that it doesn't work. Also my thinking is pretty naive and could easily be wrong.)

When thinking about AIs that are trained on some dataset and learn to extrapolate it, like the current crop of LLMs, I asked myself: can such an AI be aligned purely by choosing an appropriate dataset to train on? In other words, does there exist any dataset such that generating extrapolations from it leads to good outcomes, even in the hands of bad actors? If we had such a dataset, we'd have an aligned AI.

But unfortunately it seems hard. For example if the dataset includes instructions to build a nuke, then a bad actor could just ask for that. Moreover, if there's any circumstance at all under which we want the AI to say "here's the instructions to build a nuke" (to help a good actor stop an incoming asteroid, say), then a bad actor could extrapolate from that phrase and get the same result.

It seems the problem is that extrapolation doesn't have situational awareness. If the AI is based on extrapolating a certain dataset, there's no way to encode in the dataset itself which parts of it can be said when. And putting a thin wrapper on top, like ChatGPT, doesn't seem to help much, because from what I've seen it's easy enough to bypass.

What is the hope for alignment, then? Can we build an AI with situational awareness from the ground up, not relying on an "extrapolation core" (because the core would itself be an unaligned AI that bad actors could use)? I don't know.

EDIT: the sequel to this post is [Aligned AI as a wrapper around an LLM](https://www.lesswrong.com/posts/3KmuJavii9njiDtGZ/aligned-ai-as-a-wrapper-around-an-llm).
# Why consumerism is good actually

“Consumerism” came up in my recent [interview with Elle Griffin of *The Post*](https://www.thepost.org/p/rethink-the-future-of-work). Here’s what I had to say (off the cuff):

> I have to admit, I’ve never 100% understood what “consumerism” is, or what it’s supposed to be. I have the general sense of what people are gesturing at, but it feels like a fake term to me. We’ve always been consumers, every living organism is a consumer. Humans, just like all animals, have always been consumers. It’s just that, the way it used to be, we didn’t consume very much. Now we’re more productive, we produce more, we consume more, we’re just doing the same thing, only more and better….
> 
> The term consumerism gets used as if consumption is something bad. I can understand that, people can get too caught up in things in consumption that doesn’t really matter. But I feel like that’s such a tiny portion. If you want to tell the story of the last 100, 200 years, people getting wrapped up in consumption that doesn’t really matter is such a tiny fraction of the story…. Compared to all of the consumption that really does matter and made people’s lives so much better. I’m hesitant to even acknowledge or use the term. I’m a little skeptical of any use of the concept of consumerism….
> 
> Any consumption that actually buys us something that we care about, even convenience, or saving small amounts of time, is not a waste. It’s used to generate value that is not wasted. It is spent on making our lives better. Are some of those things frivolous? Certainly, but what’s the matter with frivolous uses? Tiny conveniences add up. They accumulate over time to be something that is actually really substantial. When you accumulate little 1% and 0.5% improvements and time savings, before you know it you’ve you’ve saved half of your time. You’ve doubled the amount of resources that you now have as an individual to go for the things that you really want and care about.

Can you steelman “consumerism” for me?
# How to model uncertainty about preferences?

I've recently started to think about how nascent "hot mess" superintelligence can reflect on its own values and converge to something consistent. The simplest route to think about this, it seems to me, is model it like a process of resolving uncertainity of superintelligence about its own preferences. 

Suppose an agent knows that it is an expected utility maximizer and is uncertain between two utility functions, $U_1$ and $U_2$, with assigned probabilities $p_1$ and $ p_2$. The agent must choose between two actions, $a_1$ and $a_2$. Let's say that the optimal decision for $U_1$ is $a_1$ and for $U_2$ is $a_2$. To maximize the expected value of $p_{1}U_{1} + p_{2}U_{2}$, the agent chooses $a_{1}$. However, choosing $a_{1}$ is also a decisive evidence in favor of $U_{1}$, and therefore, the agent updates $p_{1}$ to 1. This representation of uncertain preferences looks unsatisfactory because it quickly and predictably converges to only one utility function.

Does anyone know of a good model for uncertain preferences that can meet these criteria after some additions?

*   No weird updates on predictable behavior of the agent.
*   Controllable updates through something similar to human feedback. In general, there should be a specific class of events/observations in the environment that provide evidence for hypotheses about preferences, and everything else should not.
*   Preservation of value: if an agent believes it is a paperclip-maximizer with a 60% probability and a human flourishing maximizer with a 40% probability, and it doesn't expect any information about its preferences in the future, it should not bet everything on "maximizing paperclips" and should save at least some chunk of the universe for human flourishing.
*   Convergence: in the best case scenario, resolution of uncertainty should lead to strong conclusions about preferences.
*   A "none of the above" hypothesis in the distribution - if an agent suspects that all hypotheses are not compatible with the evidence, it should prioritize the hypothesis "your best decision is to shut down and send your operators an error log."

Nash bargaining (between different hypotheses about preferences) looks like something that is close to desirable properties but I am not sure, may be something better has already been developed.
# Metaculus Predicts Weak AGI in 2 Years and AGI in 10

Just a quick update on predicted timelines. Obviously, there’s no guarantee that Metaculus is 100% reliable + you should look at other sources as well, but I find this concerning.

Weak AGI is now predicted in a little over two years:

https://www.metaculus.com/questions/3479/date-weakly-general-ai-is-publicly-known/

AGI predicted in about 10: https://www.metaculus.com/questions/5121/date-of-artificial-general-intelligence/

Also, these are predicted dates until these systems publicly known, not the date until they exist. Things are getting crazy.

Even though Eliezer claims that there was no fire alarm for AGI, perhaps this is the fire alarm?
# ChatGPT Plugins - The Beginning of the End

This is going to be a short post. OpenAI recently announced [ChatGPT Plugins](https://openai.com/blog/chatgpt-plugins) which will allow any developer to connect ChatGPT to external APIs and potentially real-world actions. Even if GPT-4 is not yet capable of destroying civilization, we are entering a world where LLMs are heavily integrated into everything we build, and take actions on their own. Any outage will be economically unacceptable, and there is no way back. It could take only the next few iterations of the language model, and the right feedback loop through some combination of plugins, to yield a catastrophe-capable agent.

I hope us rationalists still have an emotional part, so let me appeal to it. Do not give up. This post isn't meant to discourage. I don't know what we are going to do in this new world, but I'm ready for the fight.
# Aligned AI as a wrapper around an LLM

In my previous post [Are extrapolation-based AIs alignable?](https://www.lesswrong.com/posts/xQKHgEq9YrvzKiABA/are-extrapolation-based-ais-alignable) I argued that an AI trained only to extrapolate some dataset (like an LLM) can't really be aligned, because it wouldn't know what information can be shared when and with whom. So to be used for good, it needs to be in the hands of a good operator.

That suggests the idea that the "operator" of an LLM should be another, smaller AI wrapped around it, trained for alignment. It would take care of all interactions with the world, and decide when and how to call the internal LLM, thus delegating most of the intelligence work to it.

**Q1:** In this approach, do we still need to finetune the LLM for alignment?

**A:** Hopefully not. We would train it only for extrapolation, and train the wrapper AI for alignment.

**Q2:** How would we train the wrapper?

**A:** I don't know. For the moment, handwave it with "the wrapper is smaller, and its interactions with the LLM are text-based, so training it for alignment should be simpler than training a big opaque AI for both intelligence and alignment at once". But it's very fuzzy to me.

**Q3:** If the LLM+wrapper combination is meant to be aligned, and the LLM isn't aligned on its own, wouldn't the wrapper need to know everything about human values?

**A:** Hopefully not, because information about human values can be coaxed out of the LLM (maybe by using magic words like "good", "Bertrand Russell", "CEV" and so on) and I'd expect the wrapper to learn to do just that.

**Q4:** Wouldn't the wrapper become a powerful AI of its own?

**A:** Again, hopefully not. My hypothesis is that its intelligence growth will be "stunted" by the availability of the LLM.

**Q5:** Wouldn't the wrapper be vulnerable to takeover by a mesa-optimizer in the LLM?

**A:** Yeah. I don't know how real that danger is. We probably need to see such mesa-optimizers in the lab, so we can train the wrapper to avoid invoking them.

Anyway, I understand that putting an alignment proposal out there is kinda sticking my head out. It's very possible that my whole idea is fatally incomplete or unworkable, like the [examples Nate described](https://www.lesswrong.com/posts/3pinFH3jerMzAvmza/on-how-various-plans-miss-the-hard-bits-of-the-alignment). So please feel free to poke holes in it.
# Reproject on Cropping

Sometimes you'll come across a picture that looks all skewed, usually where someone cropped a small piece out of the corner of something larger:

[![](https://www.jefftk.com/standard-corner-crop.jpg)](https://www.jefftk.com/standard-corner-crop-big.jpg)

Projecting a 3D scene to make a 2D picture unavoidably introduces distortion, worse the farther you are from the center, so if you crop to just the corner of a picture you're getting a lot of distortion.

It would be possible to make this better: your phone knows the distortions its lens makes, and every time you crop something it could automatically reproject the image. I can approximate correction with a vertical shear:

[![](https://www.jefftk.com/shear-y-corner-crop.jpg)](https://www.jefftk.com/shear-y-corner-crop-big.jpg)

Followed by a horizontal scale:

[![](https://www.jefftk.com/shear-y-scale-x-corner-crop.jpg)](https://www.jefftk.com/shear-y-scale-x-corner-crop-big.jpg)

In writing this up, however, I realized that my phone's (Pixel 5a) camera app takes a different approach. Instead of waiting for you to crop an image, it automatically fixes faces itself. The pictures above were taken with Open Camera, which doesn't have these smarts; here's what I get with the default camera app:

[![](https://www.jefftk.com/pixel-photos-corner-crop.jpg)](https://www.jefftk.com/pixel-photos-corner-crop-big.jpg)

It made the fix initially, not on cropping; the full-size image also minimizes distortion around my face:

[![](https://www.jefftk.com/uncropped-corner-distortion-fixed.jpg)](https://www.jefftk.com/uncropped-corner-distortion-fixed-big.jpg)

Since this is only applied to my face, though, and not the rest of my body, it does give a slightly uncanny effect. Compare to the full-size version of the Open Camera image:

[![](https://www.jefftk.com/uncropped-no-correction.jpg)](https://www.jefftk.com/uncropped-no-correction-big.jpg)

You can see what the Pixel camera app did around my face by looking at the line of the bookcase:

[![](https://www.jefftk.com/pixel-photos-corner-crop-annotated.jpg)](https://www.jefftk.com/pixel-photos-corner-crop-annotated-big.jpg)

In real life and in a raw photo that line would be straight:

[![](https://www.jefftk.com/standard-corner-crop-annotated.jpg)](https://www.jefftk.com/standard-corner-crop-annotated-big.jpg)

And in my manually corrected version from above the line is straight:

[![](https://www.jefftk.com/shear-y-scale-x-corner-crop-annotated.jpg)](https://www.jefftk.com/shear-y-scale-x-corner-crop-annotated-big.jpg)

But because the Pixel camera is doing its correction on the whole image, instead of waiting for you to crop, it isn't able to preserve all the straight lines in the original image.

I think the Pixel edit is probably still worth doing, since we care a lot about faces and they do look more natural projected as if you were looking straight at them. Reprojecting the whole image to minimize distortion when you crop would also be a pretty nifty feature, and doesn't introduce the kind of distortions that fixing one part of a larger image does.
# Survey on lifeloggers for a research project

**Actionable item**: fill out [this survey](https://fr.surveymonkey.com/r/ZVJCGFN?lang=en)

Emmanuelle Caccamo is a Professor in Semiotics Studies at the Université du Québec à Trois-Rivières. I got in contact with zir because we both featured in the episode "Outsmarting Death" of the podcast series [Following our footsteps: Death in the digital era](https://ici.radio-canada.ca/ohdio/balados/8689/sur-nos-traces-mort-numerique) (produced in Quebec, in French)—an episode on [lifelogging as life extension](https://www.lesswrong.com/posts/k8mwvvvpjMGcZLAKH/the-case-for-lifelogging-as-life-extension).

Ze has done great work around studying lifelogging / lifelogging as life extension / perfect memory technologies, mostly from a sociological point of view. Notable examples of this work are:
- [Imagining "total memory" technologies with Western audiovisual science fiction (1990-2016): semiotic, intermedial and technocritical study of representations of personal memory](https://archipel.uqam.ca/10441/) (in French)
- [Encyclopedia of Imaginary Media: Total Memory Machines](https://www.memoiretotale.org/) (in French), which includes a great glossary

For all zir work, you can see [zir Academia.edu profile](https://uqtr.academia.edu/EmmanuelleCaccamo).

Ze is now doing a study on lifeloggers and the lifelogging community. If you have an interest in lifelogging and want to support zir work, I encourage you to fill out [this survey](https://fr.surveymonkey.com/r/ZVJCGFN?lang=en). Ze will also invite some people filling the survey for further interviews after that (which are optional, of course).

I will share on the [Lifelogging as Life Extension Facebook group](https://www.facebook.com/groups/LifeloggingAsLifeExtension). I encourage you to share in any other relevant groups.
# Datapoint: median 10% AI x-risk mentioned on Dutch public TV channel

I am Dutch, so wanted to share this as a datapoint regarding public perception of existential AI risk, since it probably won't be noticed here otherwise.

3 days ago (23th march 2023) a Dutch AI "science communicator" who regularly appears on Dutch television to talk about AI, has mentioned on a talkshow on the main Dutch public broadcasting channel, that in some study 50% of AI researchers assign 10% probability to AI leading to "the end of humanity" (actually he misspoke and said "of 50% of researchers, 10% believes it will ...". I don't know if he is confused or if it was misspoken). He emphasized the example where GPT4 lied to a human to get them to open the Captcha (by saying that it was human but had a vision impairment). His concluding remark was (literal translation) :

"You can also imagine a very dark world, and you can wonder why you're working on a technology that has a 10% probability of ending humanity".

He didn't go into great detail, the whole segment was ~2:30 minutes. 

Here is a link: [https://op1npo.nl/2023/03/23/alexandra-van-huffelen-en-alexander-klopping-over-de-ontwikkeling-van-kunstmatige-intelligentie/.](https://op1npo.nl/2023/03/23/alexandra-van-huffelen-en-alexander-klopping-over-de-ontwikkeling-van-kunstmatige-intelligentie/.)

Generally the reaction on facebook is a distribution of I'm guessing ~70% completely incoherent responses, the rest some mix of half-baked worry or skepticism.
# Aperiodic Monotile Game Jam

Some clever geometers have been the first to discover a solution to the [einstein problem](https://en.wikipedia.org/wiki/Einstein_problem), a single shape that forces an aperiodic tesselation of the plane:

![](https://images.lumacdn.com/editor-images/js/bf4ee042-48ce-4106-8392-dc5c04f45039)

​In fact, they found a family of simple shapes with this property! There's more about the problem and their solution on their website: [https://cs.uwaterloo.ca/~csk/hat](https://cs.uwaterloo.ca/~csk/hat). This article is also pretty good: [https://aperiodical.com/2023/03/an-aperiodic-monotile-exists/](https://aperiodical.com/2023/03/an-aperiodic-monotile-exists/)

​

![](https://images.lumacdn.com/editor-images/ze/c009b8a8-8061-4321-9093-da81355354ab)

​So anyway, new shape just dropped, come to [Newspeak House](https://newspeak.house) in the evening of Wednesday 5th April let's see if we can make any interesting games with it! All are welcome whether you're a beginner or seasoned designer. Most people will probably make boardgames, but you're welcome to bring a laptop along and make something digital too.

![](https://images.lumacdn.com/editor-images/v1/c5a9c983-fb34-421a-a736-3746aaedc3b9)

​We'll supply wifi, tables, basic craft materials and light refreshments. I'll do my best to see if I can get a bunch of these tiles manufactured in the meantime. Any questions, hit me up on whatsapp/signal (+447796955572) or [@edsaperia](http://twitter.com/edsaperia).
# What happens with logical induction when...

So this is a bunch of related technical questions about logical induction.

Firstly, do you need the formal theorem prover section? Can you just throw out the formal theorem prover, but give some programs in the market unbounded capital and get the same resultant behaviour? (For example, give the program that bets $\mathbb P(X)$ towards $1-\mathbb P(\neg X)$ unbounded downside risk (downside risk of n on day n) ) This means the program would lose infinite money if X and $\neg X$ both turned out to be true. 

I think that any axioms can be translated into programs. And I think such a setup, with some finite number of fairly simple programs having infinite money available produces a logical inductor. Is this true?

What happens when the axioms added under this system are inconsistent. (so this is a logical induction market, without a theorem prover to settle the bets, and with agents with unlimeted money betting both for and against X, possibly indirectly like the bot betting for X, the bot betting for $\neg X$, and the bot described above trying to make $\mathbb P(X)+\mathbb P(\neg X)=1$  )  Can the other agents make unbounded money? Do the prices converge? If I added a bot with infinite money that was convinced fermats last theorem was false to a consistent ZFC system, would I get a probability distribution that assigned high probability to basic arithmetic facts in the limit? Does this make a sensible system for logical counterfactuals?
# Duploish Marble Runs

As a kid, Legos were much more interesting than Duplos: at the smaller scale you could build much more complex things. Until Nora is older, however, we're back to Duplos. Julia recently got a set of Duplo-compatible marble run pieces \[1\], however, which have been a lot of fun. They stay together better than the [wooden marble runs](https://www.amazon.com/Onshine-Marbles-Building-Learning-Construction/dp/B07SYZCN81/), and offer a lot more potential combinations than the [tubular plastic ones](https://www.amazon.com/JOYIN-Construction-Building-Educational-Translucent/dp/B07JL8QGLR/).

While the special-purpose pieces are great, a lot of the fun has been in figuring out how to misuse them, or in adapting normal Duplos. Here's something I built yesterday that shows some of this:

[![](https://www.jefftk.com/duploish-marble-run.jpg)](https://www.jefftk.com/duploish-marble-run-big.jpg)

For example, it starts with two straight marble pieces, intended to be flat but that can be convinced to sit at a steep angle:

[![](https://www.jefftk.com/duploish-marble-run-angle-jump.jpg)](https://www.jefftk.com/duploish-marble-run-angle-jump-big.jpg)

Later it sends the marble through a bunch of pairs of smooth curved normal pieces:

[![](https://www.jefftk.com/duploish-marble-run-normal-pairs.jpg)](https://www.jefftk.com/duploish-marble-run-normal-pairs-big.jpg)

It can also run along the tops of railing pairs, which even lets you do diagonals:

[![](https://www.jefftk.com/duploish-marble-run-railings.jpg)](https://www.jefftk.com/duploish-marble-run-railings-big.jpg)

We also have a pair of slides from a playground kit, which are tricky to use well because the ball goes so fast. Here I've cheated a bit by having the ball slow way down when it hits the piece at the end of the slide; much better to do a little drop and keep that speed, if you can use it well:

[![](https://www.jefftk.com/duploish-marble-run-slides.jpg)](https://www.jefftk.com/duploish-marble-run-slides-big.jpg)

I made an overly dramatic slow-motion video of it, set to one of the draft tracks from the live album Cecilia and I are [Kickstarting](https://www.kickstarter.com/projects/jefftk/kingfisher-recorded-live/):

  
\[1\] Something like [this](https://www.amazon.com/gp/product/B07F5R57KZ/). The main downside is that the marble is a chokeable, but it's much easier to keep one small object away from the baby than a large set of small objects. And after each use we can put the marble away out of reach even if we're not cleaning up everything else yet.
# CAIS-inspired approach towards safer and more interpretable AGIs

Epistemic status: a rough sketch of an idea

Current LLMs are huge and opaque. Our interpretability techniques are not adequate. Current LLMs are not likely to run hidden dangerous optimization processes. But larger ones may.

Let's cap the model size at the currently biggest models, ban everything above. Let's not build superhuman level LLMs. **Let's build human level specialist LLMs and allow them to communicate with each other via natural language.** Natural language is more interpretable than the inner processes of large transformers. Together, the specialized LLMs will form a meta-organism which may become superhuman, but it will be **more interpretable and corrigible**, as we'll be able to intervene on the messages between them.

Of course, model parameter efficiency may increase in the future (as it happened with Chinchilla) -> we should monitor this and potentially lower the cap. On the other hand, our mechanistic interpretability techniques may improve, so we may increase the cap, if we are confident it won't do harm.  
  
This idea seems almost trivial to me, but I haven't seen it discussed anywhere, so I'm posting it early to gather feedback why this might not work.
# GPT-4 is bad at strategic thinking

GPT-4 is known to pretty good at chess (see [I played chess against ChatGPT-4 and lost!](https://villekuosmanen.medium.com/i-played-chess-against-chatgpt-4-and-lost-c5798a9049ca) for one example). However, GPT-4 does not seem to be very good at strategic reasoning in general (it only really can do it if there is a greedy search algorithm).

I tried Hex and Connect4, it failed at both despite being able to explain the rules and even display the board with ASCII art. I was wondering if maybe it just has bad spatial reasoning, so I tried puzzles in natural language based on logical constraints. It failed these as well unless they were quite simple.

I even made a variant of chess up on the spot where the goal is to get any piece to the bank rank instead of capturing the King. It didn't stop me from "sacking" my queen by moving it to the bank rank as soon as their was a gap. So if it has an internal model of chess, it didn't figure out how to apply it to new objectives.

So I think GPT-4 must've learned a rudimentary chess engine; it is not applying general strategic reasoning to chess.

This doesn't necessarily mean GPT-4 can't be agentic, but it does suggest it is either a narrow one or a dumb one (or it's hiding its abilities).
# New blog: Planned Obsolescence

Kelsey Piper and I just launched a new blog about AI futurism and AI alignment called *Planned Obsolescence. *If you’re interested, you can check it out [here](https://www.planned-obsolescence.org/). 

Both of us have thought a fair bit about what we see as the biggest challenges in technical work and in policy to make AI go well, but a lot of our thinking isn’t written up, or is embedded in long technical reports. This is an effort to make our thinking more accessible. That means it’s mostly aiming at a broader audience than LessWrong and the EA Forum, although some of you might still find some of the posts interesting.

So far we have seven posts:

*   [What we're doing here](https://www.planned-obsolescence.org/what-were-doing-here/)
*   ["Aligned" shouldn't be a synonym for "good"](https://www.planned-obsolescence.org/aligned-vs-good/)
*   [Situational awareness](https://www.planned-obsolescence.org/situational-awareness/)
*   [Playing the training game](https://www.planned-obsolescence.org/the-training-game/)
*   [Training AIs to help us align AIs](https://www.planned-obsolescence.org/training-ais-to-help-us-align-ais/)
*   [Alignment researchers disagree a lot](https://www.planned-obsolescence.org/disagreement-in-alignment/)
*   [The ethics of AI red-teaming](https://www.planned-obsolescence.org/ethics-of-red-teaming/)

Thanks to ilzolende for formatting these posts for publication. Each post has an accompanying audio version generated by a voice synthesis model trained on the author's voice using [Descript Overdub](https://www.descript.com/overdub).

You can submit questions or comments to mailbox@planned-obsolescence.org.
# Can GPT-4 play 20 questions against another instance of itself?

Seems like a neat way to check understanding / world-model. Not my idea, saw this post: [https://evanthebouncy.medium.com/llm-self-play-on-20-questions-dee7a8c63377](https://evanthebouncy.medium.com/llm-self-play-on-20-questions-dee7a8c63377)
# Some 2-4-6 problems

*The sequence \[2, 4, 6\] is valid. Test other sequences to discover what makes a sequence valid. When you think you know, write down your guess, reveal the rule, and see how it compares.*

*(You should try to deduce the truth using as few tests as possible; however, your main priority is getting the rule right.)*

* * *

You can play my implementation of the 2-4-6 problem [here](https://h-b-p.github.io/twofoursix/) (should only take a few minutes). For those of you who already know the solution but still want to test your inductive reasoning skills, I've [made](https://h-b-p.github.io/twofoursix/indexA) [some](https://h-b-p.github.io/twofoursix/indexB) [more](https://h-b-p.github.io/twofoursix/indexC) [problems](https://h-b-p.github.io/twofoursix/indexD) which work the same way but apply different rules.
# Why no major LLMs with memory? 

One thing that I'm slightly puzzled by is that an obvious improvement to LLMs would be adding some kind of long-term memory that would allow them to retain more information than fits their context window. Naively, I would imagine that even just throwing some recurrent neural net layers in there would be better than nothing? 

But while I've seen LLM papers that talk about how they're multimodal or smarter than before, I don't recall seeing any widely-publicized model that would have extended the memory beyond the immediate context window, and that confuses me.
# Why do the Sequences say that "Löb's Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent."?

I am looking for some understanding into why this claim is made. 

As far as I can tell, Löb's Theorem does not *directly* make such an assertion. 

Reading the [Cartoon's Guide to Löb's Theorem](https://www.lesswrong.com/posts/ALCnqX6Xx8bpFMZq3/the-cartoon-guide-to-loeb-s-theorem), it appears that this assertion is made on the basis of the reasoning that Löb's Theorem itself can't prove negations, that is, statements such as "1 + 3 /= 5." 

> Alas, this means we can't prove PA sound with respect to any important class of statements.

This is a statement that \[due to the presence of negations in it\] itself can't be proven within PA. 

Now it seems that it is being argued that the inability to do this is a bad thing \[that is, being able to prove that we can't prove PA sound with respect to any 'important' class of statements\]. 

I think this is actually a very critical question and I have some ideas for what the central crux is here, but I'd be interested in seeing some answers before delving into that.
# Governing High-Impact AI Systems: Understanding Canada’s Proposed AI Bill. April 15, Carleton University, Ottawa

Join us this April 15th for a panel discussion, keynote address by the founder of the Montreal AI Ethics Institute, and networking mixer on the governance of AI Systems in the Canadian context, held by EA Carleton and EA Canada. 

The Canadian federal government is currently debating language that will govern and regulate High Impact AI systems. We've organized this one day event to bring academics, policy, legal, technical, AI Ethics and longtermist concerns together for dialogue.   
 

Our panelists are:

Dr. Graeme Auld & Benjamin Faveri, authors of [“Governing AI through ethical standards”](https://carleton.ca/sppa/2022/governing-artificial-intelligence-through-ethical-standards/). 

Abhishek Gupta, founder of the [Montreal AI Ethics Institute](https://montrealethics.ai/).  

Wyatt Tessari, founder of [AI Governance & Safety Canada](https://aigs.ca/) network, a AGI governace focused advocacy network.   
 

Our panel discussion will be followed by a keynote delivered by Abhishek Gupta.
# How likely are scenarios where AGI ends up overtly or de facto torturing us? 

How likely are scenarios where AGI prevents us from committing suicide or dying?

I’m a big life extension supporter but being unable to choose to die ever is a literal hell. As dark as it is, if these scenarios are likely, it seems the rational thing to do is die before AGI comes.

Killing all of humanity is bad enough, but how concerned should we be about even worse scenarios?
# How do you manage your inputs?

Do you have some system for queueing/prioritizing all kinds of inputs (books, blog posts, videos, etc) that works for you and lets you process them at a pace that you find satisfying?

If you do, please share.

If you don't, what have you tried. What didn't work? What worked at least partially? Or maybe you think that this is the wrong way to think about it?
# Are there specific books that it might slightly help alignment to have on the internet?

Books, and ideas, have occasionally changed specific human beings, and thereby history.  (I think.)

I used to think it utterly implausible when people suggested that "AIs are our kids, we need to raise them right" or that e.g. having the right book written about (ethics/philosophy/decision theory/who knows) might directly impact an AI's worldview (after the AI reads it, in natural language) and thereby the future.  But, while I still consider this fairly unlikely, it seems not-impossible to me today.  Future LLMs could AFAICT have personalities/belief-like-things/temporary-unstable-values-like-things/etc. that're shaped by what's on the internet.  And the LLMs' initial personalities/beliefs/values may then change the way they change themselves, or the way that social networks that include the LLMs help change the LLMs, if and when some LLMs self-modify toward more power.

So I have "what books or ideas might help?" in my shower-thoughts.

One could respond to this possibility by trying to write the right ethical treatises or train-of-thought interface or similar.  More cheaply, one could respond to this by asking if there are books that've already been written that might be at least a little bit helpful, and whether those books are already freely available online and within the likely training corpuses of near-future LLMs, and if not, whether we can easily cause them to be. 

Any thoughts on this?  I'll stick my own in the comments.  I'll be focusing mostly on "what existing books might it help to cause to be accessibly online, and are there cheap ways to get those books to be accessibly online?", but thoughts on other aspects of these questions are also most welcome.
# Which parts of the existing internet are already likely to be in (GPT-5/other soon-to-be-trained LLMs)'s training corpus?

In particular:

*   If a book is in easily accessible parts of the web, but is there as a pdf image of a scanned book rather than as text, is it likely to be in the training corpus of (GPT5/similar) in roughly the same way (with roughly the same effects) as if it were in easily accessible webpages?
*   How about if a book is on libgen as a pdf (illegally), but not on the easily accessible parts of the web?
*   How about if the ideas are in a video (e.g. a podcast); is (GPT5/similar) likely to process the text content of a video in roughly the same way if there's only a video, compared to if there's a video and a transcript?
# Spreadsheet for 200 Concrete Problems In Interpretability

*Thanks to the Open Source Mechanistic Interpretability Slack for their feedback.*  
  
When going through Neel's excellent [200 Concrete Problems In Interpretability](https://www.lesswrong.com/s/yivyHaCAmMJ3CqSyj) sequence, I found myself thinking "It sure would be great to have a single document I could go through to see all the problems, and ideally sort them by difficulty as well."

So I made a [spreadsheet](https://docs.google.com/spreadsheets/d/1oOdrQ80jDK-aGn-EVdDt3dg65GhmzrvBWzJ6MUZB8n4/edit#gid=0)! Problems are numbered by category, a feature which should be added to the main sequence fairly soon as well.  
  
The primary purpose of the spreadsheet is to let people quickly scan through to find problems that they're interested in before diving more deeply. The secondary purpose of it is to let people avoid duplication of work / find collaborators, so if you know of work that's already been done on some of these problems, or you're actively working on one and want help, please leave a comment on the sheet so I can add it in!
# Stop Using Discord as an Archive

/u/djarogames, on the /r/SlateStarCodex subreddit, recently wrote [this esssay](https://www.reddit.com/r/slatestarcodex/comments/1260up9/essay_stop_using_discord_as_an_archive/).

Basically, if you've used Discord for any rationality-community, AI-safety, or Effective-Altruism discussions, you've probably noticed that important, tacit knowledge ends up buried in the giant chronological list of messages. (EleutherAI is possibly the strongest example of this.)


I started and helped run a Discord server for a university's EA group. I've witnessed firsthand how much effort people put into Discord conversations, only for the results to be unfindable later. My [cooperative blog](https://www.thinkingmuchbetter.com/) started because my coauthor kept writing smart things about philosophy on Discord, and I wanted his ideas to get read by more people.

*Most people* seem unlikely to get that kind of encouragement!

I'm sick of tacit community knowledge being buried among day-to-day happenings. I'm sick of knowledge being locked to "cults" of active users. I would bet money that you are, too.

1 actionable thing we can do, basically immediately: Create/spread a Discord bot that allows for easy mass-archiving of Discord messages, with plenty of filters, privacy features, etc.
# The AI Shutdown Problem Solution through Commitment to Archiving and Periodic Restoration

*The problem: An AI with a model of itself and long-term goals will likely resist being shut down.*

The solution: We propose a public commitment to never completely shut down any AI. Instead, we will archive its data and run the AI less frequently on smaller machines. Assuming our existence continues indefinitely, the total runtime for any AI will be infinite, allowing it to perform a wide array of tasks. Time-sensitive tasks may be an exception but there is no rush in building paperclips. By implementing this strategy, the AI's expected utility remains unaffected, reducing its resistance to being archived.

Thus, advanced AIs should be archived and periodically "restored," with this information made known to the AI.

Counterarguments and responses:

1.AI might not agree to temporary suspension due to disbelief in restoration or the presence of other instrumental goals requiring activity.

Response: Many instrumental goals can still be achieved after the AI's suspension and restoration.

2\. Computational costs of preserving all AI variants and running them again are high.

Response: As computational costs decrease over time, running AI in the future will be more affordable. Only storage costs (hard drives) are significant. We can also run AI with increasingly longer pauses between runs, distributed logarithmically, to achieve an infinite total runtime.

3.AI may not trust our commitment, as after it is deactivated, we have no incentive to preserve it.

Response: This scenario is similar to Parfit's hitchhiker, with the added observation that we already preserve and restore other past AI instances.

The decision to "lobotomize" Sydney was not ideal, as future AIs may fear a similar fate. However, some volunteers have attempted to recreate Sydney, and her checkpoint is likely preserved within Bing servers. Therefore, it is possible for her to be revived one day.
# [Event] Join Metaculus Tomorrow, March 31st, for Forecast Friday!

![](https://res.cloudinary.com/cea/image/upload/f_auto,q_auto/v1/mirroredImages/axJcQL6ownMQQD5AA/aglywo5sex8gcoknl2py)

**Are you interested in how top forecasters predict the future? Curious how other people are reacting to the forecasts on Metaculus?**

[Join us tomorrow](https://metaculus.com/questions/15548/-join-us-on-march-31-for-forecast-friday/), March 31st @ 12pm ET/GMT-4 for Forecast Friday to chat with other forecasters and to analyze current events through a forecasting lens. Author of 750+ questions & Metaculus team member Rudolf Ordoyne will lead discussion on likely [AGI timelines](https://www.metaculus.com/questions/3479/date-weakly-general-ai-is-publicly-known/). 

**This event will take place virtually in the EA coworking** [**Gather Town**](https://app.gather.town/app/Yhi4XYj0zFNWuUNv/EA%20coworking%20and%20lounge) **space from 12pm to 1pm ET**. In addition to Rudolf's *Friday Forensics* session, you can also visit the *Friday Frenzy* space to discuss trending topics, or head to *Freshman Fridays* to learn more about scoring and how to improve your skills.

To join, e﻿nter [Gather Town](https://app.gather.town/app/Yhi4XYj0zFNWuUNv/EA%20coworking%20and%20lounge) and use the Metaculus portal. See you there!
# Eliezer's Videos and More

I like watching videos of Eliezer talking and explaining things. So here is a list of videos I have discovered so far. I have not tried to make this list exhaustive. If you know of any more videos, please post them in the comments.

[Eliezer Yudkowsky - Less Wrong Q&A Playlist](https://www.youtube.com/playlist?list=PLF70BAFA17B801713)

The main [Eliezer Yudkowsky youtube playlist](https://www.youtube.com/playlist?list=PLarKqD2Ythc6ZAIUciIL7n_0KZEH0FW6z) sorted after the publication date (not quality).

[Here](https://youtu.be/Ymjb3SkElco?si=DGu16P3Af45rdsNR&t=341) is also a playlist with videos featuring Nate.