trentmkelly/LessWrong-43k huggingface dataset:
[Hammertime Final Exam] Quantum Walk, Oracles and Sunk Meaning
by silentbob — 2019-09-03T11:59:45.696Z

One and a half years ago, alkjash published his Hammertime sequence. Admittedly a bit late to the party I recently went through the 30 days together with a few other aspiring rationalists, who too may post their "final exams" in the coming weeks.

The task was the following:

Design a instrumental rationality technique.
Introduce a rationality principle or framework.
Describe a cognitive defect, bias, or blindspot.

For each of these, we're supposed to spend 5 minutes brainstorming plus 5 minutes writing the text. This worked out for the brainstorming in my case, but the writing took somewhat longer, especially turning the whole thing into a semi-readable post (I couldn't figure out how to properly format stuff in here; took me 15 minutes to create proper headlines as they were never limited to my selection but turned my whole text into a headline, plus creating these fancy separators always removed big chunks of text; so, sorry about that).

Technique: Quantum Walk

Murphyjitsu is a CFAR technique that helps you bulletproof plans that might feel secure but in reality probably aren't. It makes you focus on all these things that could go wrong which you haven't taken into account yet.

I propose a similar yet inverse method: picking goals that you're hesitant to set because they seem unreachable for whatever reason. Given you have such a goal, imagine that x months from now you will have reached it, and now think what course of actions may have brought you from where you are now to that point.
So here we are not looking for unforeseen failure modes, but instead for a hidden path to success, one that may make the difference between shying away from that goal, and making tangible progress toward it.
The reason I'm calling it "quantum walk" is twofold: Firstly, I couldn't think of any better name, as "anti murphyjitsu" sounds suboptimal. Secondly, I recently read Life on the Edge and am now primed to see quantum mechanics everywhere. One thing the book explained was how photosynthesis contains a step that only works due to quantum mechanics: some particle has to find a certain very specific spot to settle in. If that particle behaved in a classical way, finding that spot would be incredibly unlikely and the whole process of photosynthesis wouldn't actually work. Yet utilizing quantum laws the particle finds its way to its destination.
The technique's approach is similar in th...
Book Review: Ages Of Discord
by Scott Alexander — 2019-09-03T06:30:01.543Z

I.

I recently reviewed Secular Cycles, which presents a demographic-structural theory of the growth and decline of pre-industrial civilizations. When land is plentiful, population grows and the economy prospers. When land reaches its carrying capacity and income declines to subsistence, the area is at risk of famines, diseases, and wars – which kill enough people that land becomes plentiful again. During good times, elites prosper and act in unity; during bad times, elites turn on each other in an age of backstabbing and civil strife. It seemed pretty reasonable, and authors Peter Turchin and Sergey Nefedov had lots of data to support it.

Ages of Discord is Turchin’s attempt to apply the same theory to modern America. There are many reasons to think this shouldn’t work, and the book does a bad job addressing them. So I want to start by presenting Turchin’s data showing such cycles exist, so we can at least see why the hypothesis might be tempting. Once we’ve seen the data, we can decide how turned off we want to be by the theoretical problems.

The first of Turchin’s two cyclic patterns is a long cycle of national growth and decline. In Secular Cycles‘ pre-industrial societies, this pattern lasted about 300 years; in Ages of Discord‘s picture of the modern US, it lasts about 150:

This summary figure combines many more specific datasets. For example, archaeologists frequently assess the prosperity of a period by the heights of its skeletons. Well-nourished, happy children tend to grow taller; a layer with tall skeletons probably represents good times during the relevant archaeological period; one with stunted skeletons probably represents famine and stress. What if we applied this to the modern US?

Average US height and life expectancy over time. As far as I can tell, the height graph is raw data. The life expectancy graph is the raw data minus an assumed constant positive trend – that is, given that technological advance is increasing life expectancy at a linear rate, what are the other factors you see when you subtract that out? The exact statistical logic be buried in Turchin’s source (Historical Statistics of the United States, Carter et al 2004), which I don’t have and can’t judge.

This next graph is the median wage divided by GDP, a crude measure of income equality:

Lower values represent more inequality.

This next graph is median female age at first marriage....
Open & Welcome Thread - September 2019
by Ruby — 2019-09-03T02:53:21.771Z

If it’s worth saying, but not worth its own post, here's a place to put it.

You can also make a shortform post.

And, if you are new to LessWrong, here's the place to introduce yourself.

Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ.

The Open Thread sequence is here.
The Transparent Society: A radical transformation that we should probably undergo
by mako yass — 2019-09-03T02:27:21.498Z

Edit (May, 2020): The purpose of this document is to examine the end, and ask whether it would be good or stable, the purpose is not to examine the means, to lay a path, or to tally the risks of trying. I don't know how clear that was. I am sorry to disappoint anyone who was looking for that. It's a sensible thing to be concerned about. I hope I'll study it later.

In 1998, David Brin published a vision of a potentially inevitable societal shift towards all-encompassing democratised surveillance. It could not be called a panopticon, because those who patrol the observation house would be as visible as anyone else.

We would be able to watch our watchers.

Privacy would disappear, but many kinds of evil would go along with it.

The book was okay. It's definitely worth reading the first few chapters and some of Brin's reflections on the development of the internet are really interseting, but I found the rest very meandering and a little bit unsatisfying. I'm going to summarise my own understanding of radical societal transparency and why I'm convinced that it would be extremely good, actually so that I don't have to keep recommending the entire book, which didn't capture my stance very well.

I'm also going to go over some of the potential problems a transparent society would have, and explain why I'm not yet deterred by them. Some of them could turn out to be lethal to the idea, but that seems unlikely to me so far. I'm eager to explore those doubts until we're sure.

Advantages

It should not surprise anyone that a radically open society would have many advantages. Information is useful. If we know more about each other, we can arrange more trades, and we can trust each other more easily.

In order of importance:

It would likely prevent most "easy nuke" technological x-risks discussed in Bostrom's black ball paper (Vulnerable Universe)

That is to say, if a very harmful technology rapidly emerges, for instance, a method for manufacturing a species-ending virus requiring only very common lab equipment, the transparent society would be able to police against it without any special laws. Every government could fail utterly to recognise the threat, and we might still be able to survive it; In this example, biotech workers would simply be able to watch each others' behaviour, make a note of it if anyone reads about this apocalyptic method, keeps copies of the method around, seems t

...
One Way to Think About ML Transparency
by Matthew Barnett — 2019-09-02T23:27:44.088Z

What makes a neural network interpretable?

One response is that a neural network is interpretable if it is human simulatable. That is, it is interpretable if and only if a human could step through the procedure that the neural network went through when given an input, and arrive at the same decision (in a reasonable amount of time). This is one definition of interpretable provided by Zachary Lipton.

This definition is not ideal, however. It misses a core element of what alignment researchers consider important in understanding machine learning models. In particular, in order for a model to be simulatable, it must also be at a human-level or lower. Otherwise, a human would not be able to go step by step through the decision procedure.

Under this definition, a powerful Monte Carlo Tree Search would not be interpretable since that would imply that a human could beat an MCTS algorithm by simply simulating its decision procedure. So this definition appears to exclude things that we humans would consider to be interpretable, and labels them uninterpretable.

A slight modification of this definition yields something more useful for AI alignment. We could distinguish decision simulatability with theory simulatability. In decision simulatability, a human could step through the procedure of what an algorithm is doing, and arrive at the same output for any input.

In theory simulatability, the human would not necessarily be able to simulate the algorithm perfectly in their head, but they would still say that they algorithm is simulatable in their head, "given enough empty scratch paper and time." Therefore, MCTS is interpretable because a human could in theory sit down and work through an entire example on a piece of paper. It may take ages, but the human would eventually get it done; at least, that's the idea. However, we would not say that some black box ANN is interpretable, because even if the human had several hours to stare at the weight matrices, once they were no longer acquainted with the exact parameters of the model, they would have no clue as to why the ANN was making decisions.

I define theory simulatability as something like, the ability for a human to operate the algorithm given a pen and a blank sheet of paper after being allowed to study the algorithm for a few hours ahead of time. After the initial few hours, the human would be "taken off" the source of inf...
Joy in Discovery: Galois theory
by Eigil Rischel — 2019-09-02T19:16:46.542Z

Inspired by: Joy in Discovery, Class Project

Your task: Prove either of the following statements:

There exists a polynomial p(x) of degree five over the rational numbers, with a root x, so that x cannot be obtained from the rationals by field operations and iterated extraction of roots.

There is no set formulas, given in terms of field operations and iterated extraction of roots, expressing the roots of a fifth-degree polynomial over the rationals in terms of the coefficients.

Note that the former clearly implies the latter, but you are free to prove either.

By "field operations", i mean just +,−,×,/.
By "extraction of roots", i mean operations of the form "^n\sqrt{(-)}$.
You are allowed to take even roots of negative numbers, i.e to work inside the complex numbers.

To clarify:

In the case of the first problem, I am looking for a specific equation of the form ax5+bx4+cx3+dx2+ex+f, with a,b,c,d,e,f rational numbers, (a nonzero) which has at least one solution x, so that x cannot be written in terms of rational numbers, addition, subtraction, multiplication, division, and taking roots.

When taking roots, you can also take a non-principal root. For instance, you can take both 5√10 (the positive real fifth root of ten), but also the four other roots, 5√10⋅ei2πk5, for k=1,2,3,4.

The solution x may be a complex number.

In the case of the second problem, we just want to prove that there is no single formula which works for all fifth-degree polynomials, expressing the roots in terms of the coefficients.

Reading further will give you more information, which may not be in the spirit of things. I will take care not to give too many spoilers, but read carefully.

(You have already been spoiled that the solution involves something called "Galois theory", but if you know what that is, you almost certainly already know it applies to this question).

I just started taking a course on Galois theory for my master's degree in mathematics.
The course is pretty far below my level; I am taking it for annoying bureaucratic reasons. Despite that fact, although I have been exposed to the some of the ideas before(which probably made it significantly easier for me), I've never really scratched the surface of Galois theory.
But during the first lecture, as the professor started sketching out the direction of the course, how we would [do Galois theory] and use this to show the impossibilit...
Stories About Academia
by Ben Pace — 2019-09-02T18:40:00.106Z

This is the 2nd post of 5 containing the transcript of a podcast hosted by Eric Weinstein interviewing Peter Thiel.

Interview

Nature and Culture

Eric Weinstein: Actually I'm scanning my memory and I don't know that we've had this conversation, so I'm curious to hear your answer. One of the things that I found surprising is that I think I can tell a reasonably decent story about how this is a result of a scientific problem rather than the mismanagement of our future. Do you believe that if we assume that there was this early 1970s structural change in the economy, that it was largely a sort of manmade problem? Which is what we seemingly implicitly always assume. Or, might it be a scientific one?

Eric Weinstein: And let me give you the one iconic example that really kind of drives it home for me. I think quarks were discovered in 1968. And to find out that the proton and neutron are comprised of up and down quarks is an incredible change in our picture of the world. Yet it has no seeming implications for industry. And I started thinking about this question: Are we somehow fenced out of whatever technologies are to come - that we sort of exhausted one orchard of low hanging fruit and haven't gotten to the next?

Peter Thiel: I think one way to parse this question of scientific, technological stagnation is sort of nature versus culture. Did the ideas in nature run out? Or, at least the useful ideas. Maybe we make some more discoveries, but they're not useful.

Eric Weinstein: Or the easily useful.

Peter Thiel: Easily useful. So it's a problem with nature. And then the cultural problem is that there was actually a lot to be discovered or a lot that could be made useful, but somehow the culture had gotten deranged. And I sort of go back and forth on those two explanations. I think it's very complicated. Yeah, I think in physics you'd say ... I mean, probably even the fundamental discoveries stopped after the mid 1970s, but certainly the translation didn't happen. Quarks don't matter for chemistry, and chemistry's what matters on a human level.

Peter Thiel: I would say there was a lot that happened in biochemistry. You know, not chemistry down, but sort of chemistry up; the interface between chemistry and biology. And that's where I would be inclined to say there's a lot more that could happen and has not quite happened, because maybe the problems are hard. But maybe also th...
Best utility normalisation method to date?
by Stuart_Armstrong — 2019-09-02T18:24:29.318Z

For some time, me and others have been looking at ways of normalising utility functions, so that we can answer questions like:

Suppose that you are uncertain between maximising U1 and U2, what do you do?

...without having to worry about normalising U1 or U2 (since utility functions are only defined up to positive affine transformations).

I've long liked the mean-max normalisation; in this view, what matters is the difference between a utility's optimal policy, and a random policy. So, in a sense, each utility function has a equal shot of moving the outcome away from an expected random policy, and towards themselves.

The intuition still seems good to me, but the "random policy" is a bit of a problem. First of all, it's not all that well defined - are we talking about a policy that just spits out random outputs, or one that picks randomly among outcomes? Suppose there are three options, option A (if A is output), option B' (if B' is output), or do nothing (any other output), should we really say that A happens twice as often as B' (since typing out A randomly is twice as likely that typing out B'?).

Relatedly, if we add another option C, which is completely equivalent to A for all possible utilities, then this redefines the random policy. There's also a problem with branching - what if option A now leads to twenty choices later, while B leads to no further choices, are we talking about twenty-one equivalent choices, or twenty equivalent choices and one other one as likely as all of them put together? Also, the concept has some problem with infinite option sets.

A more fundamental problem is that the random policy includes options that neither U1 nor U2 would ever consider sensible.

Random dictator policy

These problems can be solved by switching instead to the random dictator policy as the default, rather than a random policy.

Assume we are hesitating between utility functions U1, U2, ... Un, with π∗i the optimal policy for utility Ui. Then the random dictator policy is just πrd which picks a π∗i at random and then follows that. So

πrd=1nπ∗1+1nπ∗2+…+1nπ∗n.

Normalising to the random dictator policy

This πrd is an excellent candidate for replacing the random policy in the normalisation. It is well defined, it would never choose options that all utilities object to, and it doesn't care about how options are labelled or about how to count them.

Therefore we can pres...
The Power to Demolish Bad Arguments
by Liron — 2019-09-02T12:57:23.341Z

This is Part I of the Specificity Sequence

Imagine you've played ordinary chess your whole life, until one day the game becomes 3D. That's what unlocking the power of specificity feels like: a new dimension you suddenly perceive all concepts to have. By learning to navigate the specificity dimension, you'll be training a unique mental superpower. With it, you'll be able to jump outside the ordinary course of arguments and fly through the conceptual landscape. Fly, I say!

"Acme exploits its workers!"

Want to see what a 3D argument looks like? Consider a conversation I had the other day when my friend “Steve” put forward a claim that seemed counter to my own worldview:

Steve: Acme exploits its workers by paying them too little!

We were only one sentence into the conversation and my understanding of Steve’s point was high-level, devoid of specific detail. But I assumed that whatever his exact point was, I could refute it using my understanding of basic economics. So I shot back with a counterpoint:

Liron: No, job creation is a force for good at any wage. Acme increases the demand for labor, which drives wages up in the economy as a whole.

I injected principles of Econ 101 into the discussion because I figured they could help me expose that Steve misunderstood Acme's impact on its workers.

My smart-sounding response might let me pass for an intelligent conversationalist in non-rationalist circles. But my rationalist friends wouldn't have been impressed at my 2D tactics, parrying Steve's point with my own counterpoint. They'd have sensed that I'm not progressing toward clarity and mutual understanding, that I'm ignorant of the way of Double Crux.

If I were playing 3D chess, my opening move wouldn't be to slide a defensive piece (the Econ 101 Rook) across the board. It would be to... shove my face at the board and stare at the pieces from an inch away.

Here's what an attempt to do this might look like:

Steve: Acme exploits its workers by paying them too little!

Liron: What do you mean by “exploits its workers”?

Steve: Come on, you know what “exploit” means… Dictionary.com says it means “to use selfishly for one’s own ends”

Liron: You’re saying you have a beef with any company that acts “selfish”? Doesn’t every company under capitalism aim to maximize returns for its shareholders?

Steve: Capitalism can be good sometimes, but Acme has gone beyond the pale with their exploitation o

...
Specificity: Your Brain's Superpower
by Liron — 2019-09-02T12:53:55.022Z

Introduction

What if there was a magic superpower that simultaneously enabled you to:

Demolish bad arguments
Judge startup ideas
Make scientific breakthroughs
Solve climate change
Understand “God”
Be emotionally mature
Teach concepts better
Draw better
Be creative
Make small talk
Improve discourse
Be invisible

Amazingly, your brain does have a superpower that gives you all these powers. It’s called…

To activate the power of specificity, all you have to do is ask yourself the question, “What’s an example of that?” Or more bluntly, “Can I be more specific?” And then you unleash a ton of power in a surprisingly broad variety of domains.

It's an open secret in the rationality community how powerful this skill is of being specific. Eliezer captured the essence of it in his 2012 post, Be Specific. In it, he comments on the difficulty of teaching people specificity skills for the first time:

When I’m talking to anyone outside the local LessWrong community, I find that a very large amount of my conversation involves repeatedly asking them to be more specific.

He also describes how CFAR's Applied Rationality Workshops teach the power of specificity by osmosis. I can vouch that my personal experience as a workshop attendee fits this description:

Attendees picked [specificity] up from all the instructors having to repeatedly ask the attendees to be more specific, and then having to ask them again, while being specific themselves, until the attendees picked up the rhythm by example and feedback.

I hope you're curious to unpack my list of claims about what specificity lets you do, because I’m dead serious about all of it! Except being invisible; I was kidding about that one.

So without further ado, let’s get into the specifics.

Next post: The Power to Demolish Bad Arguments