trentmkelly/LessWrong-43k huggingface dataset:
So You Want To Colonize The Universe Part 3: Dust
by Diffractor — 2019-02-27T10:20:14.780Z

(1, 2, 4, 5)

Part 3a: Dust and Explosions

To a first approximation, there's exactly one thing that sets the speed limit on going fast in space.

Dust.

In the future, dust will be a Very Big Deal, as it's the dominant constraint on the most-important instrumental goal of going fast.

Anders Sandberg's paper pointed out dust as a constraint on interstellar probe design, but I didn't realize exactly how huge of an obstacle dust was until I started playing around with a spreadsheet.

To start with, interstellar (and intergalactic) dust has a size distribution, which tells you, for a given diameter range, how many dust grains there are of that diameter in a given volume.

At least for the range of 35-120 nanometers, (which shows up especially strongly in astronomical observations) it follows a power-law distribution, with an exponent of -3.5.

However, this dust isn't what we're worried about. There's erosion from protons and small dust hitting your dust shield at relativistic speeds, but doubling the dust diameter means it's 8x as massive and hits with 8x the energy.

At 0.9 c, 180 nm dust hits with 1 joule of energy. So all the normal dust isn't that much of an issue.

Going up to dust that's 1 micrometer wide, it hits with about 100 joules of energy, the energy of a firecracker. For all the following explosion comparisions, note that it's going to take the form of a super-narrow pinprick of kinetic energy directed on a single point, which is more destructive than a simple explosion, which radiates in all directions and has much of its energy dissipated as heat.

Destructive power keeps scaling rapidly, with about a factor-of-ten increase for every doubling in dust diameter, until we reach 20 micrometer dust, which hits with the energy of a grenade.

40 micrometer dust hits with the energy of 1.5 kg of dynamite. 86 micrometer dust hits with the energy of 30 bricks of C4. 0.18 mm dust hits with the energy of half a cruise missile. 0.4 mm dust hits with the energy of the Oklahoma city bombing. 0.86 mm dust hits with the energy of the largest non-nuclear weapon, the Russian FOAB. 8.6 mm dust hits with the energy of the Fat Man nuclear weapon, and 1.8 cm dust (a ball bearing) hits with the energy of a W87 fission warhead.

So, from about 1 to 20 micrometers, we get a pretty decent amount of boom that's shieldable. Whipple shields are the current standard for micrometeor impact. They hav...
So You Want to Colonize the Universe Part 2: Deep Time Engineering
by Diffractor — 2019-02-27T10:18:18.209Z

Part 2: Deep Time Engineering

(1, 3, 4, 5)

So, with "Gotta go Fast" as the highest goal, and aware of the fact that the amount of computational resources and thinking time devoted to building fast starships will exceed by many orders of magnitude all human thought conducted so far, due to the importance of it...

I set myself to designing a starship to get to the Virgo supercluster (about 200 million light years away) in minimum time, as a lower-bound on how much of the universe could be colonized. I expect the future to beat whatever bar I set, whether humanity survives or not (it turned out to be about 0.9 c)

Now, most people focus on interstellar travel, but the intergalactic travel part is comparatively underexplored (see comments). We have one big advantage here, which is that we don't need to keep mammals around, and this lets us have a much smaller payload. Instead of delivering a vessel that can support earth-based life for hundreds of millions of years, we just have to deliver about 100 kg of Von Neumann probes and stored people, which build more of themselves. (The true number is probably a lot less than this, but as it turns out, it isn't harder to design for the 100 kg case than the 1 mg case because there's a minimum viable mass for dust shielding, and we'll be cheating the rocket equation.)

Before we get into intergalactic starship design (part 5), I want to take a minute to point out the field of Deep Time engineering, which is something that I just crystallized as a concept while working on this.

Note that whatever starship design you're building, it has to last for 200 million years, getting bombarded by relativistic protons and dust the whole way, and even with relativity speeding things up, you're still talking about building machinery that last for tens of millions of years and works with extremely high reliability the whole way. This is incredibly far beyond what engineering normally does, it takes god-like levels of redundancy and reliability, and if you've got something with moving parts, there's erosion by friction to consider, and also 200 million years worth of cosmic rays... I didn't focus on actual solutions that much, but just the awareness of the existence of tasks which require building machinery that works for hundreds of millions of years sparked something.

Engineers have shorter time horizons than you might expect. In environmental...
So You Want to Colonize The Universe
by Diffractor — 2019-02-27T10:17:50.427Z

Epistemic Status: Mix of facts, far-future-speculation with the inevitable biases from only considering techniques we know are physically possible, fermi calculations, and an actual spacecraft design made during a one-week research bender.

(this is a sequence. 2, 3, 4, 5)

Part 1a: Gotta Go Fast, Astronomy is a-Wasting

Once a civilization or agent grows up enough to set it sights on colonizing realms beyond its host planet as its first priority (instead of averting existential risks), there is a very strong convergent instrumental goal which kicks in. Namely, going as close to lightspeed (abbreviated as c) as possible.

This is because the universe is expanding, so there is a finite sphere of reachable galaxies, and more pass outside the horizon every year. IIRC (60% probability), about half of the galaxies in the Hubble Ultra-Deep Field are unreachable even if we traveled at lightspeed.

Arriving at a galaxy even one year faster nets you a marginal gain of (one galaxy of stars)*(average stellar luminosity)*(1 year) of energy, which for our Milky way comes out to about 1.6×1044 Joules. Assuming energy production on earth stays constant, that's enough energy for a billion years of earth civilization, 130 trillion times over. And I'd expect a transhuman civilization to be quite a few orders of magnitude better at getting value from a joule of energy than our current civilization. And that's just for a single galaxy. There are a lot of galaxies, a one-year speedup in reaching them has tremendous value.

This is basically Bostrom's astronomical waste argument, except Bostrom's version then goes on to apply this loss in total form (which is far larger) instead of marginal form, to argue for the value of reducing existential risk.

Now, there are a few corrections to this to take into account. The first and most important is that, by the Landauer limit, the amount of (irreversible) computations that can be done is inversely proportional to temperature, so waiting until the universe cools down from its current 2.7 K temperature nets you several orders of magnitude more computational power from a given unit of energy than spending it now. Also, if you do reversible computation, this limit doesn't apply except to bit erasures, which nets you a whole lot more orders of magnitude of computation.

Another correction is that if there are aliens, they'll be rushing to colonize the univ...
Rule Thinkers In, Not Out
by Scott Alexander — 2019-02-27T02:40:05.133Z

Imagine a black box which, when you pressed a button, would generate a scientific hypothesis. 50% of its hypotheses are false; 50% are true hypotheses as game-changing and elegant as relativity. Even despite the error rate, it’s easy to see this box would quickly surpass space capsules, da Vinci paintings, and printer ink cartridges to become the most valuable object in the world. Scientific progress on demand, and all you have to do is test some stuff to see if it’s true? I don’t want to devalue experimentalists. They do great work. But it’s appropriate that Einstein is more famous than Eddington. If you took away Eddington, someone else would have tested relativity; the bottleneck is in Einsteins. Einstein-in-a-box at the cost of requiring two Eddingtons per insight is a heck of a deal.

What if the box had only a 10% success rate? A 1% success rate? My guess is: still most valuable object in the world. Even an 0.1% success rate seems pretty good, considering (what if we ask the box for cancer cures, then test them all on lab rats and volunteers?) You have to go pretty low before the box stops being great.

I thought about this after reading this list of geniuses with terrible ideas. Linus Pauling thought Vitamin C cured everything. Isaac Newton spent half his time working on weird Bible codes. Nikola Tesla pursued mad energy beams that couldn’t work. Lynn Margulis revolutionized cell biology by discovering mitochondrial endosymbiosis, but was also a 9-11 truther and doubted HIV caused AIDS. Et cetera. Obviously this should happen. Genius often involves coming up with an outrageous idea contrary to conventional wisdom and pursuing it obsessively despite naysayers. But nobody can have a 100% success rate. People who do this successfully sometimes should also fail at it sometimes, just because they’re the kind of person who attempts it at all. Not everyone fails. Einstein seems to have batted a perfect 1000 (unless you count his support for socialism). But failure shouldn’t surprise us.

Yet aren’t some of these examples unforgiveably bad? Like, seriously Isaac – Bible codes? Well, granted, Newton’s chemical experiments may have exposed him to a little more mercury than can be entirely healthy. But remember: gravity was considered creepy occult pseudoscience by its early enemies. It subjected the earth an...
Is LessWrong a "classic style intellectual world"?
by Gordon Seidoh Worley — 2019-02-26T21:33:37.736Z

Robin Hanson just published a post titled "Classic Style Intellectual World". It's short, so I recommend reading it, as it sets up this question, but in short it suggests that many intellectual activities are organized in a way similar to the way paintings on the walls of Renaissance churches are. That is, small spaces are allocated to lower-status artists where they are rewarded mainly for blending in well with architecture and surrounding context and only already high-status artists are asked to be more creative by giving them large spaces where they might do something less conforming (but not too non-conforming!). The parallel he draws is that lower-status intellectuals are expected to talk within the frame society and other intellectuals have already set up for them within a frame sized to match their status, while higher-status intellectuals are allowed a larger frame in which they may be more creative, the point being that status drives permitted creativity (that is, creativity that will not be ignored in obscurity) rather than creativity and innovation (or whatever traits are ostensibly sought in intellectuals) being what confers high status.

So the question I pose here is, how much does LW look like this picture Robin has painted? Specifically, is it that high readership, karma, post scores, etc. accumulates to authors who are the best at whatever it is you think the values of LW are or do those things go to those who best work "within the system" with LW value satisfaction as secondary concern?
How to get value learning and reference wrong
by Charlie Steiner — 2019-02-26T20:22:43.155Z

Six months ago, I thought I was on to something interesting involving value learning and philosophy of reference. Then I had a series of breakthroughs - or do you still call it a breakthrough when it reveals that you were on the wrong track? Reverse breakthrough? How about "repair-around." Anyhow, this is my attempt to turn the unpublished drafts into a story about their own failure.

One post in this vein did get published, Can few-shot learning teach AI right from wrong? It was about the appeal and shortcomings of what Abram Demski titled model-utility learning:

This AI is made of three parts:

● An unsupervised learning algorithm that learns a model of the world and rules for predicting the future state of the model.

● A supervised algorithm that takes some labeled sensory examples of good behavior, plus the model of the world learned by the unsupervised algorithm, and tries to classify which sequences of states of the model are good.

● To take actions, the AI just follows strategies that result in strongly classified-good states of its predictive model.

We get a picture of an AI that has a complicated predictive model of the world, then tries to find the commonalities of the training examples and push the world in that direction. What could go wrong?

[...]

The thing it learns to classify is simply not going to be what we wanted it to learn. We're going to show it examples that, from our perspective, are an extensional definition of satisfying human values. But the concept we're trying to communicate is a very small target to hit, and there are many other hypotheses that match the data about as well.

[...]

Here's the missing ability to do reference - to go from referring speech-acts to the thing being referred to.

This post had its good points. But under the surface, there was a false assumption that the model of value learning and the skill of reference were separable - that the problem was understanding reference so you could bolt it onto a model-utility learner that lacked it.

That's what the unpublished second post, Reference as a game, was based on. It's not like I was blind to all problems - it's that I didn't see anything better. And so I pushed forward despite feeling stuck.

In this post I want to talk about doing a little bit better. Naively, "doing better" here means lear

...
Knowing I’m Being Tricked is Barely Enough
by Elizabeth — 2019-02-26T17:50:07.608Z

I think it was in Terry Pratchett’s Going Postal that a con man mused that the easiest people to rip off were other con men, or at least those who aspired to be so, because all you had to do was make it look like they were taking advantage of you. Honest men wouldn’t fall for it because they weren’t willing to rip you off.* It’s interesting watching myself fall for that.

A year ago Audible (owned by Amazon) offered me a year’s membership (=12 credits for free books, plus discounts on other books and some miscellaneous perks) for $100. That’s almost half off the per credit price (which itself can be can be half the price of buying a book with money, although those prices are clearly set to encourage subscribing rather than to be paid). I was about to start reading a bunch of dense history tomes, so this seemed like a pretty good deal. Then between my plan changing and the library gods being generous, I ended up not using a single credit. Suddenly 11 months had passed and Audible e-mailed me telling me they would renew my membership in a month.

“NBD, I’ll just cancel my account and keep the credits for when I need them” I thought quietly to myself. Only it turns out unused credits disappear if you cancel your membership. To keep them, I’d have to renew my subscription. But Audible only lets you keep 18 credits at a time. If I renewed for another year, 6 credits will immediately expire, meaning I’m paying for 12 credits but only getting 6. So I went on a spree buying books off my Goodreads queue and gave a few as gifts. Actually I gave one too many gifts, because I realized later there was another good candidate I didn’t have the credits for.

Audible to the rescue. No sooner had I spent my last credit than they offered me three for $30 each- more per credit than I’d paid for my first membership, but less than the annual membership. I went as far as picking out two more books before realizing this was stupid, I had gone an entire year without buying a single audio book, I did not need three more credits.

Then my roommate told me $10/credit isn’t even that good a price, they were sure to offer me a better one when I actually cancelled. And I wanted it. A product I empirically had to be forced to use, and I felt compelled to buy more because it was cheap.

I put this down to two things-...
RAISE is launching their MVP
by  — 2019-02-26T11:45:53.647Z

After over a year of building the organisation and working and reworking the product, I’m glad to announce that RAISE will begin publishing lessons on a regular basis starting next Monday.

These lessons will include:

An Inverse Reinforcement Learning (IRL) track, sourced from a set of papers that we have distilled by our own efforts.
A track on Iterated Distillation and Amplification (IDA), largely adapted from Paul Christiano’s LessWrong sequence on the matter.

A lesson on the former will be published on our platform every Monday, and a batch of lessons on the latter will be published every Thursday. The former will be finished in approximately 10 weeks, the latter in 4. If you plan to take these lessons, we recommend scheduling one full day per week for each track you’re following. Once published, the lessons will be available indefinitely for self-paced study.

In the meantime, we continue to semi-regularly publish textbook guides for our prerequisites track. These guides should eventually cover all the prerequisite material one should understand in order to properly follow work in AI Safety.

In the coming months, our focus will be on testing and refining our MVP. As long as we operate, we hope to make learning AI Safety less and less costly, creating common knowledge and paying our research debt.

To do this testing properly, we are seeking individuals that would like to join our testing panel. This means that, as you go through our lessons, we will guide you through the lessons and interview you in-depth about your experience, most importantly about where you’re most confused. If you would like to join us, please fill in this form.
This work has been sponsored, among others, by the EA Hotel. If you like our work, consider donating there.
Where to find Base Rates?
by adam demirel — 2019-02-26T10:44:59.226Z

As we are all aware from Thinking, Fast and Slow, one of the largest decision-making errors we make is the Base Rate Fallacy. Just to remind you - the error is that we don't correctly apply probability reference classes to ourselves.

Two examples:

The likelihood of winning the lottery is x, but I am more likely to win than x.
1 in x people will get lung cancer from smoking, but I won't.

This is all well and good, but my question is where you go about actually finding x. In other words - where do you find your base rates? Any particular repositories of knowledge that we should know about? Any processes that we should follow?

Let's say you've decided you want to reduce your risk of getting Coronary Heart Disease. Is your first stop to go to WHO and gather their list of recommendations, or something else?
New versions of posts in "Map and Territory" and "How To Actually Change Your Mind" are up (also, new revision system)
by habryka — 2019-02-26T03:17:28.065Z

A few weeks ago MIRI released new editions of the "Map and Territory" and "How To Actually Change Your Mind" books, and those changes are now finally also reflected in the posts on LessWrong.

To make that happen smoothly, I've also implemented a revision and versioning system, allowing you to view the past revisions of all the posts in those books, and marking comments made against older versions of the post. I did this because a significant fraction of the discussion in the comments was referencing problems in the posts that are addressed in the latest versions, as well as wording that has been changed.

A post having past revisions is indicated by a small history-icon next to its date, which when clicked will allow you to select from all past versions of the post after it has been published. This is currently only available on desktop, the mobile version will follow soon.

We are planning to make the revision system available to everyone soon, which will allow you to uniquely link to any article in its current version, without having to worry that it will ever change. I hope to also make it so that you can get notifications to new major versions of posts that you are subscribed to. This is part of a broader effort to make posts on LessWrong longer-lived and have more of a wiki-feeling to them, and also to make it harder to accidentally delete your whole post with a wrong submission.
Starting a rationality dojo; any tips?
by Bae's Theorem — 2019-02-25T23:56:24.923Z

I have a small (4 core members) existing group of SSC/LW readers who meet up for lunch once a week, which gives us the raw materials to start such a group.
We plan to meet at least once a month, if not once a week. We'll probably start with the Hammertime sequence for content.

I want to optimize for introducing non-Rationalists to Rationality; I have several friends who might get involved just because they trust me.

I most desire feedback from people who have experience with such a project; what are the common failure modes, what can I do to make this as easy as possible, what are the best meeting environments, what can I do to make this helpful for people not already in our community?
Native mental representations that give huge speedups on problems?
by two-ox-heads — 2019-02-25T23:42:44.634Z

[x-post from r/slatestarcodex]

Is there a list/textbook of immediately usable mental representations like memory palaces?

I'm not talking about theoretical models of cognitive processes here (although they might suggest entire classes of the sort of thing I'm looking for). I'm talking specific techniques and 'ways of thinking' about certain things that immediately give huge speedups.

In case I'm still not making sense, lemme provide another example. Around five years ago, Brienne Yudkowsky, then Brienne Strohl, made a video about how to remember things.

TL;DW: If you want to remember duck -> apple -> Overton window -> ischemia, encapsulate each concept in some sensually vivid scenario in your head and link these scenes like a story.

After learning what she calls 'binding' [1], I've never had problems remembering lists again. I've never lost a string of thoughts and ideas in the shower, nor forgotten what I needed to once I'm out of the house. This is an especially drastic improvement for me, because I have severe combined-type ADHD.

I'm not talking about chunked models per se (e.g., Meditations on Moloch gives you a lens through which to view the vast majority of social structures, but it's not a model that I'd call native to our wetware) but more of stuff a couple of levels down. Data structures that we as humans can immediately use, algorithms that by being "close to the metal" dramatically obviates lots of brain cycles that would have been wasted [2].

Some things in the cluster of what I'm getting at in conceptspace: heuristics, knowledge representation, perceptual differences, Tufte's body of work, neurolinguistics and related fields, Math vs Verbal distinction, Mazur's four kinds of mathematical intuitions

Why I think this is a real thing rather than some made-up attempt to crystallise nonsense: everyone knows about optical illusions. But why do we have them? Well, from Wikipedia:

The hypothesis claims that visual illusions occur because the neural circuitry in our visual system evolves, by neural learning, to a system that makes very efficient interpretations of usual 3D scenes based in the emergence of simplified models in our brain that speed up the interpretation process but give rise to optical illusions in unusual situations. In this sense, the cognitive processes hypothesis can be considered a framework for an understanding of optical illusions

...