trentmkelly/LessWrong-43k huggingface dataset:
Save Humanity! Breed Sapient Octopuses!
by Yair Halberstadt — 2022-04-05T18:39:07.478Z

One of the problems that make AI alignment so difficult is that we don't have any models for how non-human intelligences will think. This makes it both hard to reason about how an AGI would behave, but also makes it difficult to get people to take the threat of AI seriously.

Ideally we would have a way to create a human level AGI, hold it at that level for a few years so we can understand it deeply, then once we've solved alignment slowly crank it up a bit. Unfortunately, given trends like moors law, that's unlikely to be possible.

So what if we didn't use computers to create an AGI? What if we used animals? Those are harder to scale from human level intelligence to kill all humans level intelligence without someone being a total idiot.

It's the 21st century, so we don't need to use your grandpa's breeding techniques. If we can identify genes that strongly code for intelligence in an animal, we can select for gametes with those genes, and selectively breed them. We can even use crispr to create some gametes with All The Positive Mutationsᵀᴹ.

Done this way it seems possible we will be able to improve intelligence by as much as 5% a generation. Even if animals start off 10 times less intelligence than humans, it would take just 50 generations to reach human level intelligence.

50 years seems like a reasonable estimate till AGI appears on the scene. So all we need is an already intelligent animal with a generation time of less than one 1 year.

Enter the larger Pacific striped octopus!

yeah, I was born this good looking

These intelligent octopuses reach sexual maturity after 6 months. Even better, unlike most octopuses they don't die after mating, so they have a longer lifespan to learn things in - about 2 years.

And given that octopuses evolved intelligence through completely alien pathways to humans, this will give us as close a window into what an artificial intelligence might look like as is possible.

It's clear that our very best option to ensure human survival is to start a breeding program involving hundreds of thousands of larger Pacific striped octopuses and use cutting edge genetic research and ethically dubious techniques to get them to human level intelligence faster than computers. Anyone who says otherwise is clearly on the side of the transistors!
Ideal governance (for companies, countries and more)
by HoldenKarnofsky — 2022-04-05T18:30:19.228Z

Click lower right to download or find on Apple Podcasts, Spotify, Stitcher, etc.

I'm interested in the topic of ideal governance: what kind of governance system should you set up, if you're starting from scratch and can do it however you want?

Here "you" could be a company, a nonprofit, an informal association, or a country. And "governance system" means a Constitution, charter, and/or bylaws answering questions like: "Who has the authority to make decisions (Congress, board of directors, etc.), and how are they selected, and what rules do they have to follow, and what's the process for changing those rules?"

I think this is a very different topic from something like "How does the US's Presidential system compare to the Parliamentary systems common in Europe?" The idea is not to look at today's most common systems and compare them, but rather to generate options for setting up systems radically different from what's common today.

I don't currently know of much literature on this topic (aside from the literature on social choice theory and especially voting methods, which covers only part of the topic). This post describes the general topic and why I care, partly in the hopes that people can point me to any literature I've missed. Whether or not I end up finding any, I'm likely to write more on this topic in the future.

Outline of the rest of the piece:

I'll outline some common governance structures for countries and major organizations today, and highlight how much room there is to try different things that don't seem to be in wide use today. More

I'll discuss why I care about this question. I have a few very different reasons:

A short-term, tangible need: over the last several years, I've spoken with several (more than 3) organizations that feel no traditional corporate governance structure is satisfactory, because the stakes of their business are too great and society-wide for shareholder control to make sense, yet they are too early-stage and niche (and in need of nimbleness) to be structured like a traditional government. An example would be an artificial intelligence company that could end up with a normal commercial product, or could end up bringing about the most important century of all time for humanity. I wish I could point them to someone who was like: "I've read all of, and written much of, the literature on what your options are.

...
Non-programmers intro to AI for programmers
by Dustin — 2022-04-05T18:12:08.337Z

It occurs to me a lot of people are interested in AI but aren't programmers themselves.

This quick post describes using an AI helper called Github CoPilot. It will do so via a short scenario wherein CoPilot helps me write a bit of code. It will be almost completely non-technical (or, at least, as non-technical as is possible). The aim is to describe the experience and practicalities of using this technology.

Since this is aimed at non-programmers, a piece of background information is in order:

Programming is mostly not about remembering how to do specific things. It's more about knowing how systems work so you know what to look up when you want to do some specific thing. (Of course, the more you do a specific thing, the remembering comes along.)

On to the scenario...

Say I need to write some code to check if some content on a web page is larger than it's container on a web page. You've seen this before when some content on a page has its own scroll bars in addition to the main scroll bar on the side of your browser.

I've done this kind of check before, and I know how web browsers work, and I know basic logic. However, I don't remember off-hand the specific ways to access the information I need.

A piece of code that would tell me if some content was bigger than it's container might be called isOverflowing, so let's try that:

(Don't worry, you don't really have to read or understand any of that if you don't want to. Again, we're just describing the experience.)

The green text that appears is what CoPilot is suggesting I use as the code for the function...and it's exactly what I want. I know it is what I want because it prompts my memory for things I already knew. So, I pressed Tab and the suggested code became real code in my editor.

This is one of the benefits I get out of CoPilot: I don't have to go searching on the web for the names of the information I need, CoPilot just knows them for me. Here it probably saved me 5-10 minutes of Googlin'.

In this instance, as soon as I saw them, I remembered that scrollHeight, clientHeight, etc. are the right tools for the job I needed done. However, if I was less familiar with what I needed, I would hit up documentation and/or Google to make sure this code was correct because sometimes what I want and what CoPilot thinks I want don't match up.

CoPilot helps with other types of problems as well:

Drudge work. If there is

...
The case for Doing Something Else (if Alignment is doomed)
by Rafael Harth — 2022-04-05T17:52:21.459Z

(Related to What an Actually Pessimistic Containment Strategy Looks Like)

It seems to me like there are several approaches with an outside chance of preventing doom from AGI. Here are four:

Convince a significant chunk of the field to work on safety rather than capability

Solve the technical alignment problem

Rethink fundamental ethical assumptions and search for a simple specification of value

Establish international cooperation toward Comprehensive AI Services, i.e., build many narrow AI systems instead of something general

Furthermore, these approaches seem quite different, to the point that some have virtually no overlap in a Venn-diagram. #1 is entirely a social problem, #2 a technical and philosophical problem, #3 primarily a philosophical problem, and #4 in equal parts social and technical.

Now suppose someone comes to you and says, "Hi. I'm working on AI safety, which I think is the biggest problem in the world. There are several very different approaches for doing this. I'm extremely confident (99%+) that the approach I've worked on the most and know the best will fail. Therefore, my policy recommendation is that we all keep working on that approach and ignore the rest."

I'm not saying the above describes Eliezer, only that the ways in which it doesn't are not obvious. Presumably Eliezer thinks that the other approaches are even more doomed (or at least doomed to a degree that's sufficient to make them not worth talking about), but it's unclear why that is or why we can be confident in it given the lack of effort that has been extended so far.

Take this comment as an example:

How about if you solve a ban on gain-of-function research [before trying the policy approach], and then move on to much harder problems like AGI? A victory on this relatively easy case would result in a lot of valuable gained experience, or, alternatively, allow foolish optimists to have their dangerous optimism broken over shorter time horizons.

This reply makes sense if you are already convinced that policy is a dead end and just want to avoid wasting resources on that approach. If policy can work, it sounds like a bad plan since we probably don't have time to solve the easier problem first, especially not if one person has to do it without the combined effort of the community. (Also, couldn't we equally point to unsolved subproblems in alignment, or alignment for easier cas...
My Transhuman Dream
by Johannes C. Mayer — 2022-04-05T15:44:46.636Z

Since watching Ghost in the Shell (1995), I had the dream of self-modifying into a higher being.

The scene when Kusanagi (story spoiler)

merges with project 2501

is the most interesting scene that I have ever seen. Not because of the scene directly, but rather because the scene provides a springboard for thinking really interesting thoughts.

What if I was Kusanagi? What happens in

the merger? What would the experience of merging be like?

What would it be like afterward? Is Kusanagi still the same person afterward?

It did not take long before I was considering what could be achieved with controlled self-modification. What if you could self-modify however you wanted? I was not even thinking in the obvious direction of immortality. I was mainly thinking about how the algorithms in my brain could be changed. How could they be improved? What are the limits of optimality in this domain? Could you be never tired; always be motivated to do the best/right thing? Could you be a system that is always "well", without suffering, while still being able to perform effective optimization?

Around the same time, I came across the Suit Switch Scene from SOMA's excellent story. That triggered some very interesting thoughts and important conceptual advancements, about what it would mean to upload yourself. And if you would still be you afterward. So I wouldn't react anymore like Simon does in the ending (not sure if I ever would have). I understand that if I upload myself that then Johannes would exist two times. And "I" would not be the upload. Maybe you can replace all your neurons one step at a time with better artificial variants, or in some other way continuously transform yourself. But it is unclear if this would actually make a difference. We don't understand consciousness after all.

After that something terrible happened. I sort of gave up on this dream. Not explicitly, I was never thinking about giving up, but rather I was starting to think that it would be impossible for me to ever be able to self-modify. So eventually I stopped thinking about it almost completely.

All that was in 2017, 5 years ago. Sometime before I got interested in AI alignment.

Self-modification was something I wanted for myself. In a sense, that was a very concrete thing, easy to relate to. It was all very self-centered. I did not even think about what this would imply for other people.

Compare this t...
Are the fundamental physical constants computable?
by Yair Halberstadt — 2022-04-05T15:05:42.393Z

Fundamental physical constants are physical constants where the constant value seems unexplainable in terms of more basic physics. For example we have no idea why the relative masses of the various elementary particles are what they are - we see no reason why a muon is 206.768... times larger than an electron instead of 17.1328... times larger or 2035.97... times larger.

My initial reaction to learning about fundamental physical constants is to assume they are either not truly fundamental, and are the outcomes of more fundamental physics we don't know yet, or that they are completely arbitrary random numbers.

If you were to tell me that the fine structure constant was exactly equal to (9/1024)exp[(1/3−√3/8)⋅2/3], but not for any particular reason, it just happened to be that way, I would say you were barking mad.

A computable number is a number that can be computed to an arbitrary precision by a terminating algorithm. So there exists a Turing machine, which reads as input the desired precision you want, and outputs the binary representation of the number up to that number of digits.

There are countably infinite Turing machines, and uncountably infinite real numbers. This means almost all reals are uncomputable. In fact most real numbers aren't even definable! I would give you an example of an undefinable number right now, but unfortunately I can't...

Kolmogorov complexity is a way of defining the complexity of an object.

It asks what is the shortest input to a universal Turing machine that would produce the object as an output.

It can be used to calculate the complexity of the universe in terms of how long would the input to a universal Turing machine be that simulated the universe perfectly accurately.

Solomonoff induction is a formalization of Occams razor. Essentially it says that you should assume the universe is the one with the shortest possible Kolmogorov complexity that accurately predicts the observations you see.

The Kolmogorov complexity of the universe is equivalent to the shortest program (however inefficient) that can simulate the fundamental physical rules of the universe, given as inputs the fundamental physical constants, and the starting state if the universe.

The Kolmogorov complexity of a non-computable number is infinity (by definition). In fact the Kolmogorov complexity of an arbitrarily chosen computable number is massive.

If we knew the fun...
Case for emergency response teams
by Jan_Kulveit — 2022-04-05T12:45:08.371Z
Link:  https://forum.effectivealtruism.org/posts/sgcxDwyD2KL6BHH2C/case-for-emergency-response-teams

This project is seeking a paid coordinator. Please apply

here

.

Motivation

See Hinges and crises for expanded explanation.

The hinge of history

So far, long-termist a x-risk oriented efforts to change the trajectory of the world focus on far-off events. This is on the assumption that we foresee some important problem and influence its outcome by working on the problem for longer. We thus start working on it sooner than others, we lay the groundwork for future research, we raise awareness, and so on.

Many longtermists propose that we now live at the “hinge of history”, usually understood on the timescale of critical centuries, or critical decades. But ”hinginess” is likely not constant: some short periods will be significantly more eventful than others. It is also possible that these periods will present even more leveraged opportunities for changing the world’s trajectory.

These “maximally hingey” moments might be best influenced by sustained efforts long before them (as described above). But it seems plausible that in many cases, the best realistic chance to influence them is “while they are happening”, via a concentrated effort at that moment. Some reasons for this: there are decreased problems with cluelessness; an increase in the resources spent on the problem; actual decisions being made by powerful actors, and so on.

Crises as times of opportunity

Even if a specific crisis is not a “hinge of history”, crises often bring opportunities to change the established order. For example, policies well outside the Overton window can suddenly become live options, and intellectual development in disciplines and technologies (think of the sudden intense focus on reproducibility in epidemiology). These effects often persist for decades after the initial event (think of taking off your shoes at the airport; think of face masks in post-SARS countries) - and so shaping the response is of interest to longtermists.

How to impact hinges or crises

Acting effectively during hinges or crises may depend on factors such as "do we have a relevant policy proposal in the drawer?", "do we have a team of experts able to advise?" or “do we have a relevant network?” It is possible to prepare these!

This document proposes the creation of an “emergency response team” as one sensible preparation.

We tested the above principles during the COVID pandemic, by launching a good-sized re...
Why is Toby Ord's likelihood of human extinction due to AI so low?
by ChristianKl — 2022-04-05T12:16:00.278Z

In The Precipice: Existential Risk and the Future of Humanity Toby Ord gives 1/10 as the likelihood for humanity to die this century due to AI risk. This seems to be a very different view than that of MIRI. Is FHI in general much more optimistic or is it just Toby Ord that's this optimistic in regards to AI risk?
A paradox of existence
by Chris van Merwijk — 2022-04-05T09:45:09.620Z

Introduction

There is a question in philosophy "why is there something rather than nothing?" I have always thought of this question as completely impossible to answer: either it is a meaningless question, or at least there seems to be no way that we can ever begin to answer it. Yet it has always seemed very weird to me that there is such a thing as existence. I recently developed a paradox that has made this confusion less mysterious for me. The paradox is not about why something exists, but how it is possible that we know that we exist. Note, the question is still very confusing, but it has changed the level of confusion for me from something like "completely unsolvable ever" to merely roughly as confusing as the hard problem of consciousness. Maybe this idea already exists out there, or maybe I'm just being confused, but as far as I can tell, this paradox really seems to make my confusion around existence seem no longer completely intractable.

First of all, note that we humans have a very strong sense that there is a “reality”, that “something exists” or that “I exist”, and that we actually know this fact (i.e. it is not thought of as merely a speculative hypothesis). We have uncertainty about this reality, e.g. perhaps the world as we perceive it is an “illusion” or a simulation, but we seem to know that something like a reality exists. It is not at all obvious that this is the case, i.e. that creatures in a reality know that they exist. It could just as well have been the case that (1) there is an existing reality within which intelligent lifeforms have evolved, but (2) at no point do these lifeforms notice that they exist, or that existence is even a thing, they just do the usual stuff of building spaceships and inventing the internet without ever reflecting on or noticing or even having the concept of “existence”.

The paradox

So having said that, I'm going to state a paradox. By paradox I mean a set of observations, each of which seems (to the author and possibly the reader) to be true, but where it also seems that they cannot all be true together. There has to be a mistake somewhere, either one of the observations is wrong or confused, or the argument is wrong or confused, but I don’t know where the mistake is. The paradox is as follows:

We seem to know that there is a reality that exists. Note, the claim is not just that there ex

...
Print Books of Scott Alexander's Writing
by Dawn Drain — 2022-04-05T06:55:18.520Z

tl;dr: I made it so you can buy print versions of The Library of Scott Alexandria, Scott’s collected fiction, and Slate Star Codex Abridged. You can get the pdfs and ebooks from the resources folder of my repo here.

You can see the three books together here (I have no idea how to embed an image on LessWrong, and not for lack of trying, sorry.)

I’ve wanted a classy, print copy of SSC ever since I saw u/ekygip make such a book for his sister in law. It looks really lovely! He used the printing service Lulu, which DrunkFishBreatheAir also used to print Scott’s novel Unsong.

I wanted something similar, but more ambitious than u/ekygip’s twenty-four favorite essays. u/k185bv has scraped and uploaded all the posts from SSC, including images (although many of them seem missized). With size twelve font the entire corpus comes out to 5600 pages, which would be $250-300 to print. I figured it made more sense to start with a smaller sample. There are several community surveys of people’s favorite posts, and multiple attempts to curate a coherent ebook. Rob Bensinger collected The Library of Scott Alexandria, which also includes posts from Scott on LessWrong and his LiveJournal that predate SSC. I’ve uploaded a printable pdf here. The ancient library hasn’t been updated since 2015, however.

I flirted with the idea of curating my own list, but thankfully found SSC Abridged before getting carried away. There was already a print version of SSC Abridged that you can buy on Amazon, which is broken into three installments for $30 each. That version doesn’t have images, though, and uses a large font and small pages. I was able to comfortably pack the same chapters into a quarter as many pages.

Making the Print Books

For collecting the short stories and other fiction, I started with some lists that were floating around, and then skimmed the SSC archive since 2019 for any new stories that were missing. The cover images of the peacock and crab are from pixabay, and inspired by the titular Goddess of Everything Else and her nemesis the Goddess of Cancer.

When you read Scott’s essays and stories in a Word doc, as opposed to on his blog, you see a surprisingly large number of red squigglies. These were bugging me, so I started combing through and correcting all the egregious typos, e.g. “memembers” instead of “members”, “guines” instead of “genius”, and “somthing” instead of “somethin...