trentmkelly/LessWrong-43k huggingface dataset:
Explanations as Hard to Vary Assertions
by Alexander — 2021-09-24T11:33:13.735Z

Update: After some investigation, I found out that The Beginning of Infinity by David Deutsch contains a few minor misquotes of Popper, Turing and others. Nevertheless, it is an excellent book.

Background

As I read through Rationality: A-Z, I kept seeing similarities to David Deutsch's worldview. Deutsch pioneered quantum computation in the 1970s, motivated by the possibility of gaining a deeper grasp of quantum physics and as a potential way to test many-worlds.

This post is adapted from my review of The Beginning of Infinity. I read it a couple of years ago, and it is among the most formative books I have read.

Overview

We have a great deal of knowledge about the vast and unfamiliar reality that causes our observations and the elegant, universal laws governing that reality. This knowledge consists of explanations: assertions about what is out there beyond appearances and how it works. Where do explanations come from? The source of our knowledge is a process of conjectures alternating with criticism. Humans possess the capacities for creativity and rationality, enabling them to actively pursue error correction through creating, combining, altering and criticising ideas in the quest for good explanations.

Good Explanations

The role of experiment and observation is to choose among the ideas we come up with. We interpret experiences through explanatory theories, but good explanations are not obvious. Fallibilism entails not looking to authorities but acknowledging that we may always be mistaken and that no belief can ever be rationally supported or justified conclusively. Always, there remains a possible doubt as to the truth of the belief. What distinguishes science from other belief systems is that scientific beliefs are always defeasible and never final. We ought to continuously be correcting errors and updating beliefs in the quest for knowledge. We correct errors by seeking good explanations.

Good explanations in science are hard-to-vary assertions about reality. They are hard-to-vary because they provide specific details that fit together so tightly that changing them ruins the explanation. This criterion helps to eliminate bad explanations that keep adding justifications in light of refutations and counterevidence to avoid falsification. An explanation that is hard-to-vary but does not survive a critical test can be considered falsified.

We can expla...
How's it going with the Universal Cultural Takeover? Part II
by David Hugh-Jones — 2021-09-24T09:37:41.287Z
Link: https://wyclif.substack.com/p/hows-it-going-with-the-universal-601

This is the second half of a response to an old blog post by Scott Alexander. Part I is here.

Preferences and desires

Universal culture as a summoner-eating demon is a fabulous metaphor, but not an entirely new idea. Hmm:

The bourgeoisie cannot exist without constantly revolutionising the instruments of production, and thereby the relations of production, and with them the whole relations of society…. All fixed, fast-frozen relations, with their train of ancient and venerable prejudices and opinions, are swept away, all new-formed ones become antiquated before they can ossify. All that is solid melts into air….

As the Coke example suggests, a lot of “universal culture” is really just “capitalist processes and products”, which indeed are selected to outcompete others by the rigour of the market. Alexander again:

Universal culture is the collection of the most competitive ideas and products. Coca-Cola spreads because it tastes better than whatever people were drinking before.

Economists have a way of thinking about this, which has become so influential that, like spectacles we’ve forgotten we are wearing, it is almost invisible. Competition means competition to satisfy preferences, and preference satisfaction is good because of “consumer sovereignty” — the basic liberal idea that only I can decide what is good for me. The modern idea of preferences is basic to a large amount of powerful microeconomic theory which can be very good at explaining the world. But you can think that, and still think that the idea itself is wrong.

An alternative view is that capitalist products exist, directly or indirectly, to satisfy desires, not preferences. Preferences are choices made by the sovereign consumer. Desires are the basic human drives that we are born with.

A picture may help.

Two views of capitalism and history

The picture shows human society as a complex multidimensional landscape. The contour lines are human happiness. The higher up we are, the better our life is. We start off as cavepeople in the paleolithic, not very high up because we are hungry and may be eaten by lions. Then, skipping some trivial details like the Bronze age, Iron age, medieval Europe and early modernity, capitalism gets going.

The view on the left panel is the Preferences view. Capitalism satisfies our preferences, which are always to move up the happiness gradient, with the red arrows. We can’t go dire...
Common knowledge about Leverage Research 1.0
by BayAreaHuman — 2021-09-24T06:56:14.729Z

I've spoken to people recently who were unaware of some basic facts about Leverage Research 1.0; facts that are more-or-less "common knowledge" among people who spent time socially adjacent to Leverage, and are not particularly secret or surprising in Leverage-adjacent circles, but aren't attested publicly in one place anywhere.

Today, Geoff Anders and Leverage 2.0 are moving into the "Progress Studies" space, and seeking funding in this area (see: Geoff recently got a small grant from Emergent Ventures). This seems like an important time to contribute to common knowledge about Leverage 1.0.

You might conclude that I'm trying to discredit people who were involved, but that's not my aim here. My friends who were involved in Leverage 1.0 are people who I respect greatly. Rather, I just keep being surprised that people haven't heard certain specific, more-or-less legible facts about the past, that seem well-known or obvious to me, and that I feel should be taken into account when evaluating Leverage as a player in the current landscape. I would like to create here a publicly-linkable document containing these statements.

Facts that are common knowledge among people I know:

Members of Leverage 1.0 lived and worked in the same Leverage-run building, an apartment complex near Lake Merritt. (Living there was not required, but perhaps half the members did, and new members were particularly encouraged to.)

Participation in the project involved secrecy / privacy / information-management agreements. People were asked to sign an agreement that prohibited publishing almost anything (for example, in one case someone I know starting a personal blog on unrelated topics without permission led to a stern reprimand).

Geoff developed a therapy technique, "charting". He says he developed it based on his novel and complete theory of psychology, called "Connection Theory". In my estimation, "charting" is in the same rough family of psychotherapy techniques as Internal Family Systems, Coherence Therapy, Core Transformation, and similar. Like those techniques, it leads to shifts in clients' beliefs and moods. I know people from outside Leverage who did charting sessions with a "coach" from Paradigm Academy, and reported it helped them greatly. I've also heard people who did lots of charting within Leverage report that it led to dissociation and fragmentation, that they have fou

...
Book Review: Who We Are and How We Got Here 
by Yair Halberstadt — 2021-09-24T05:05:46.609Z

It's rare to read a popular science book that's both factually accurate and engaging, but David Reich offers an excellent example of how to do that.

The book explores the newly emerging field of ancient DNA. Using techniques that are less than a decade old, we can now sequence the entire genome of humans up to hundreds of thousands of years old. The picture that's beginning to emerge as we analyse this DNA is shaking up classic views on history and prehistory, and offers some fascinating insights into our story as a species.

Overview

The book begins by introducing the techniques and history of the field. At the same time it applies these techniques to studying human population movements and mixtures. It starts off deep in the past and with humanity as a whole, then moves forwards in time to specific areas and peoples. Finally it ends up discussing a little bit about the politics of DNA, the future of the field, and a small amount of philosophy.

Techniques of the trade

DNA might be relatively stable over a human lifetime, but over thousands of years it degrades, such that by the time you're dealing with a neanderthal bone the vast majority of the DNA you'll extract from it is from microbes and other contaminants. The ability to separate any actual neanderthal DNA from it relies on a number of advances:

Improved techniques: for example there's a bone in the ear that preserves DNA at least 10 times better than the rest of the body.
Careful quality control: extracting ancient DNA is impossible if it becomes contaminated with modern human DNA. The bones are taken to a completely sterile room modeled after microchip fabrication plants. There most of the bone is milled away, leaving the center which is likely to be free of human contaminants. It is this remainder that is sequenced.
Powerful computers and advanced algorithms make DNA sequencing thousands of times cheaper than it was just 15 years ago, and make it possible to differentiate human DNA from microbial DNA.

The very first Neanderthal genome was sequenced over 3 years from 2007, and doing this at any kind of scale only became possible from about 2014. At the time of writing his book only a few hundred complete ancient genomes had been published, but thousands more had been recently analyzed and were awaiting publication from his laboratory alone. By now, 3 years after writing his book, over 3000 have been published...
Cartesian Frames and Factored Sets on ArXiv
by Scott Garrabrant — 2021-09-24T04:58:54.845Z

Papers on Cartesian frames and factored sets are now on arXiv.

Cartesian Frames: https://arxiv.org/abs/2109.10996

Factored Sets: https://arxiv.org/abs/2109.11513

The factored set paper is approximately identical to the sequence here, while the Cartesian frame paper is rewritten by Daniel Hermann and Josiah Lopez-Wild, optimized for an audience of philosophers.
Walkie-Talkies
by jefftk — 2021-09-24T01:10:08.107Z

In the discussion on

kids roaming a few
weeks ago, Cullyn

suggested
walkie-talkies as a way to keep in touch with kids who are out of
sight. This wasn't something I had thought about at all: my only
experience had been with short-range children's models in the early
'90s, but now you can get rechargeable ones that will reach about 1/4mi
through houses for

~$13/each
shipped.

I got a set of four a little over a week ago, and we've been trying
them out:

We started with a lot of playing with them in situations where
they're not really needed, with the kids running ahead or something.

Before using them each time, we test them to make sure we have
them set up right.

Other kids want to play with the walkie-talkies, but will be
rough and change the settings. We ended up needing to make a rule
that they weren't for sharing.

We've practiced setting the channel and volume several times,
in case they do get messed up.

While I can clip mine onto my belt, or put it in my pocket, the
kids rarely have appropriate clothes for that. Julia got a small
fanny pack for Lily, and tweaked it to fit better.

Initially, we're only using them for short times and distances,
at a range where I would be OK just walking over and checking on
them every few minutes.

Lily (7y) is much better at using them than Anna (5y). I don't
think I would let Anna go very far with one unless it was to meet
Lily.

These particular ones speak their settings out loud as you
change them, which the kids really like. On the other hand, the knobs
are quite hard to read, so Julia painted a white stripe on each so
you can see their setting at a glance.

We tested the range, and in our flat neighborhood of densely packed
three-story wooden buildings it was clear at 1,200ft, down to almost
unusably fuzzy at 1,500ft. This is farther than we are OK with the
kids going, and farther than their friends' houses, so I don't expect
it to be an issue for a while.

Before deciding to get these, we had also considered a watch with
cellphone + GPS like the Gizmo.
That seems like a nice idea, and I liked the idea of GPS, but it's
more fragile, more expensive up front, and has ongoing costs. Since the
walkie-talkie was so much cheaper, we decided to give that a try and
see how we liked it. So far so good!

Comment via: facebook
Forecasting Transformative AI, Part 1: What Kind of AI?
by HoldenKarnofsky — 2021-09-24T00:46:49.279Z

PASTA: Process for Automating Scientific and Technological Advancement.

Audio also available by searching Stitcher, Spotify, Google Podcasts, etc. for "Cold Takes Audio"

This is the first of four posts summarizing hundreds of pages of technical reports focused almost entirely on forecasting one number. It's the single number I'd probably most value having a good estimate for: the year by which transformative AI will be developed.

1

By "transformative AI," I mean "AI powerful enough to bring us into a new, qualitatively different future." The

Industrial Revolution

is the most recent example of a transformative event; others would include the Agricultural Revolution and the emergence of humans.

2

This piece is going to focus on exploring a particular kind of AI I believe could be transformative: AI systems that can essentially automate all of the human activities needed to speed up scientific and technological advancement. I will call this sort of technology Process for Automating Scientific and Technological Advancement, or PASTA.

3

(I mean PASTA to refer to either a single system or a collection of systems that can collectively do this sort of automation.)

PASTA could resolve the same sort of bottleneck discussed in

The Duplicator

and

This Can't Go On

- the scarcity of human minds (or something that plays the same role in innovation).

PASTA could therefore lead to

explosive science

, culminating in technologies as impactful as

digital people

.

And depending on the details, PASTA systems could have objectives of their own, which could be dangerous for humanity and could matter a great deal for

what sort of civilization ends up expanding through the galaxy

.

By talking about PASTA, I'm partly trying to get rid of some unnecessary baggage in the debate over "artificial general intelligence." I don't think we need artificial general intelligence in order for this century to be the most important in history. Something narrower - as PASTA might be - would be plenty for that.

To make this idea feel a bit more concrete, the rest of this post will discuss:

How PASTA could (hypothetically) be developed via roughly modern-day machine learning methods.
Why this could lead to explosive scientific and technological progress - and why it could be dangerous via PASTA systems having objectives of their own.

Future pieces will discuss how soon we might expect something like PASTA to be ...
Shared Frames Are Capital Investments in Coordination
by johnswentworth — 2021-09-23T23:24:51.263Z

Shared Frames Are Coordination Tools

When Claude Shannon invented information theory, he was working at Bell Labs. Naturally, the new theory was applied to Bell’s telephone system quite quickly. But information theory itself did not offer any immediately-usable methods to run more phone calls through the same wires;

practical

compression

algorithms

came later. So how did information theory create immediate practical value?

What information theory offered was a new way of framing communication infrastructure problems. Historically, the content transmitted (e.g. speech) was tightly coupled to the mode of transmission (e.g. voltage oscillations matching the audio oscillations). Information theory suggested that the content and the mode of transmission could be decoupled. Any kind of data (written words, speech, video) could in-principle all be transmitted over any kind of channel (copper wire, radio, microwave, messenger pigeon) with near-optimal efficiency and arbitrarily-low noise. We could even quantify “amount of information” in a message and “information capacity” of a channel, independently of each other, in terms of “bits” - a universal way to represent information.

So information theory naturally suggests a way to break up the problem of communication infrastructure design: separate content and encoding from transmission and channel design. More generally, information theory is a frame: it suggests questions, approximations, what to pay attention to vs ignore, and ways to break up the problem/system into subproblems/subsystems.

Questions:

What’s the information capacity of a given channel (e.g. a pair of microwave towers), and how can we increase it?
What’s the amount of information per second in a phone call, and how can we efficiently encode it?

Approximations:

Accurately transmitting an n-bit message through a channel requires a channel capacity of approximately n bits, regardless of the message details.
Conversely, an n-bit capacity channel can accurately transmit a message of approximately n bits, regardless of the channel details.

What to pay attention to vs ignore:

Ignore most interactions between channel-design and message content - e.g. we don’t need to worry about the idiosyncrasies of human speech when designing a satellite link.
Pay attention to how many bits per second we can send through the channel (i.e. throughput).
Pay attention to how long th

...
Review: Martyr Made Podcast
by Elizabeth — 2021-09-23T20:10:01.508Z

Introduction

Sometimes I consume media that makes factual claims. Sometimes I look up some of these claims to see how much trust I should place in said media, in a series I call epistemic spot checks. Over the years, I’ve gone back and forth on how useful this is. Focusing on evaluating particular works instead of developing a holistic opinion on an entire subject does feel perverse to me. OTOH, sometimes non-fiction is recreational, and I don’t think having some of my attention directed by people I find insightful and trustworthy is a bad thing, as long as I don’t swallow their views unquestioningly. Additionally, there’s a pleasant orderliness to doing ESCs, like the intellectual equivalent of cleaning my house. It’s not enough in and of itself, but it can free up RAM such that there’s room for deeper work.

I started listening to Darryl Cooper’s Martyr Made last year as part of a deep dive on cults, but kept going because I found him incredibly insightful. After listening to the 30+ hours of the God’s Socialist sequence, I Googled around and found a few accusations of racism against Cooper. I didn’t believe the accusations then, and I still don’t. People can go through the motions of saying what other people tell them to, but they can’t fake what Cooper does, which is to approach every human being as someone worthy of respect and compassion, whose actions are probably reasonable given their incentives. I value that a lot more than proper signaling.

Some time later I found an archive of Cooper’s deleted Twitter logs, and, uh, I get where people are coming from on the racism thing. I still absolutely believe in his respect and compassion for everyone except members of the USSR leadership (and even then, he’ll say very nice things about the intentions of early communists). However, the thing about doing that genuinely instead of choosing a side and signaling allegiance is that it doesn’t compress well to 140 characters, and he said a bunch of things that were extremely easy to round to terrible beliefs. I might also have mistaken him for racist, if all I had was his Twitter. But given the podcasts, I am very sure that he respects-and-has-compassion-for every human being.

[Between when I started listening and when I published this Cooper returned to Twitter, which I have mixed feelings about. Namely “I think this is bad for him intellectually and emot...
Matt Levine spots IRL Paperclip Maximizer in Reddit
by Nebu — 2021-09-23T19:10:16.546Z

Matt Levine is the author of regular column "Money Stuff" on Bloomberg, and he seems to have spotted a paperclip maximizer in the wild:

A key explanation of, like, modern life is that politics are so vicious and polarized because (1) everyone uses Facebook, (2) Facebook has an algorithm to decide what stuff to show people, (3) that algorithm is built to maximize engagement, (4) the algorithm is very effective, and (5) it turns out that what maximizes engagement is vicious polarizing stuff, oops. That probably exaggerates the role of Facebook’s algorithm and understates the agency of its users — “Everything you hate about the Internet is actually everything you hate about people,” says Balk’s First Law — but there’s something to it. Social media reflects and also shapes human behavior, in an emergent way. It targets a behavior it likes — spending more time on Facebook, basically — and tries to maximize it. To do that, it maximizes behaviors that correlate with the target behavior, like posting and reading vicious political content. It gives people what they want, as revealed by their actions, which is not what they actually want. (Or is it?) And so at some level you have behaviors in the real world that seem to be designed by an algorithm to maximize engagement on social media; politics are vicious and volatile because that’s what Facebook, deep down, wants.

Okay now let’s do stocks! At Margins, Ranjan Roy has a great post on “Memestocks and Reddit redesigns” arguing that some of the 2021 meme-stock craze is due to changes in Reddit’s algorithms to maximize engagement: The stock market is volatile and bizarre because that’s what Reddit, deep down, wants. You read about a stock, Reddit sends you “you might also like ...” notifications about that stock and other stocks, those notifications are designed to draw you in, they work, you spend more time looking at stocks on Reddit, your identity starts to become “person who cares about this stock,” you buy the stock, you start posting about it, you start buying weekly-expiry out-of-the-money call options on the stock and posting your daily profits and losses, the stock goes to $1,000, it becomes the biggest story in the national news, everyone in the country is obsessed with it, and they are all coming to Reddit because the story is not “stock goes up” but “stock goes up because people are having fun and getting rich

...