trentmkelly/LessWrong-43k huggingface dataset:
Less Wrong Lacks Representatives and Paths Forward
by curi — 2017-11-08T19:00:20.866Z

In my understanding, there’s no one who speaks for LW, as its representative, and is *responsible* for addressing questions and criticisms. LW, as a school of thought, has no agents, no representatives – or at least none who are open to discussion.

The people I’ve found interested in discussion on the website and slack have diverse views which disagree with LW on various points. None claim LW is true. They all admit it has some weaknesses, some unanswered criticisms. They have their own personal views which aren’t written down, and which they don’t claim to be correct anyway.

This is problematic. Suppose I wrote some criticisms of the sequences, or some Bayesian book. Who will answer me? Who will fix the mistakes I point out, or canonically address my criticisms with counter-arguments? No one. This makes it hard to learn LW’s ideas in addition to making it hard to improve them.

My school of thought (Fallible Ideas – FI – https://fallibleideas.com) has representatives and claims to be correct as far as is known (like LW, it’s fallibilist, so of course we may discover flaws and improve it in the future). It claims to be the best current knowledge, which is currently non-refuted, and has refutations of its rivals. There are other schools of thought which say the same thing – they actually think they’re right and have people who will address challenges. But LW just has individuals who individually chat about whatever interests them without there being any organized school of thought to engage with. No one is responsible for defining an LW school of thought and dealing with intellectual challenges.

So how is progress to be made? Suppose LW, vaguely defined as it may be, is mistaken on some major points. E.g. Karl Popper refuted induction. How will LW find out about its mistake and change? FI has a forum where its representatives take responsibility for seeing challenges addressed, and have done so continuously for over 20 years (as some representatives stopped being available, others stepped up).

Which challenges are addressed? *All of them*. You can’t just ignore a challenge because it could be correct. If you misjudge something and then ignore it, you will stay wrong. Silence doesn’t facilitate error correction. For information on this methodology, which I cal...
Attending to Now
by ialdabaoth — 2017-11-08T16:53:07.419Z

I live among a community of humans who believe that they need to save the world.

They aren't good enough. They may still save the world completely on accident, but ultimately, on the measure, the odds are pretty grim.

And they know this. All around me I see people facing their task with grim determination, or resolute despair, or cheerful-but-insistent obliviousness.

The problem is that saving the world is hard. The Gods keep handing us monkeys new kinds of fire to play with, hoping that this time maybe we'll burn ourselves up and rid them of the pesky usurpers. So far we're surviving on sheer dumb luck.

When I was a child, the first truly holocaustic device had been created: the atomic bomb. For forty years, mankind stood on a palpable knife-edge overlooking oblivion - if not of the species, then at the very least of the hard-earned way of life that thousands of years of civilization had granted us.

Then in the late 1980s and early 1990s, seemingly by miracle, the Cold War ended and we managed to step back from the precipice of annihilation. It didn't have to happen that way; we dodged a bullet. And Mankind's collective reaction to this was, for the most part, "whew, glad THAT'S over!" instead of "oh fuck, BULLETS."

We're busy building more bullets, and noticing the bullets that we've already been building for along time, that were a little slower to reach the firing chamber. Global ecological upheaval. Nanotechnology. Molecular biotechnology. Full economic automation. Artificial general intelligence. And, should we get bored of the past 25 years of relative peace, there's always a global thermonuclear arsenal to dust off and point at ourselves again - because hey, why not?

So there's plenty of bullets, and we're not good enough to dodge all of them. Most of the people I know are only focusing on one of them - around here the fad seems to be AGI - and even within that single domain most everybody seems terrified that the genie will come out of the bottle before anyone knows how to leash it.

This isn't a post about becoming good enough. Around here, EVERYONE'S talking about becoming better, stronger, faster, smarter. This is a post about facing the grim prospect of your cultural annihilation and fighting on anyways. About, if not defeating your foes and seeing them driven before yo...
Does Age Bring Wisdom?
by Scott Alexander — 2017-11-08T07:20:00.376Z

I.

I turn 33 today. I can only hope that age brings wisdom.

We’ve been talking recently about the high-level frames and heuristics that organize other concepts. They’re hard to transmit, and you have to rediscover them on your own, sometimes with the help of lots of different explanations and viewpoints (or one very good one). They’re not obviously apparent when you’re missing them; if you’re not ready for them, they just sounds like platitudes and boring things you’ve already internalized.

Wisdom seems like the accumulation of those, or changes in higher-level heuristics you get once you’ve had enough of those. I look back on myself now vs. ten years ago and notice I’ve become more cynical, more mellow, and more prone to believing things are complicated. For example:

1. Less excitement about radical utopian plans to fix everything in society at once

2. Less belief that I’m special and can change the world

3. Less trust in any specific system, more resignation to the idea that anything useful requires a grab bag of intuitions, heuristics, and almost-unteachable skills.

4. More willingness to assume that other people are competent in aggregate in certain ways, eg that academic fields aren’t making incredibly stupid mistakes or pointlessly circlejerking in ways I can easily detect.

5. More willingness to believe that power (as in “power structures” or “speak truth to power”) matters and infects everything.

6. More belief in Chesterton’s Fence.

7. More concern that I’m wrong about everything, even the things I’m right about, on the grounds that I’m missing important other paradigms that think about things completely differently.

8. Less hope that everyone would just get along if they understood each other a little better.

9. Less hope that anybody cares about truth (even though ten years ago I would have admitted that nobody cares about truth).

All these seem like convincing insights. But most of them are in the direction of elite opinion. There’s an innocent explanation for this: intellectual elites are pretty wise, so as I grow wiser I converge to their position. But the non-innocent explanation is that I’m not getting wiser, I’m just getting better socialized. Maybe in medieval Europe, the older I grew, the more I would realize that the P...
Linking to Other Resources
by Chris_Leong — 2017-11-08T06:33:50.114Z

Lesser Wrong is De Facto the Main Site. Given this situation, it should take the role of being the central hub from which you can access all of the other resources within the rationality community. This will be incredibly easy as someone could just write up a post on Resources and it could be added in the sidebar, just like how the about page currently works.

Examples of links we could include on the Resources Page:

Meetups
LW Study Hall
Wiki
Reddit
Effective Altruism Forum

Ect...

Anyway, what do people think of this idea?
How Popper killed Particle Physics
by J Thomas Moros — 2017-11-07T16:55:21.727Z
Link: http://backreaction.blogspot.co.uk/2017/11/how-popper-killed-particle-physics.html

Discusses some failures in the process of modern particle physicis in the vien of Nobody does the thing that they are supposedly doing and The Failures of Eld Science.
Confessions of a Slacker
by Jacob Falkovich — 2017-11-07T14:20:00.518Z

Crossposting the entire thing from Putanumonit in honor of the Slack sequence.

Go read Slack, it’s short and important.

If you’re not a slacker, that post might change your life and explain why you feel like you have no control over your own life despite doing well on almost all counts. If you are a slacker, like me, this post gives our philosophy a name and provides a definition: slack is the absence of binding constraints on behavior.

Zvi’s post is abstract on purpose. I’ll continue his mission by getting more specific and, of course, by putting a num on it. To the latter purpose, I’ll modify the definition of slack to make it quantifiable:

Slack is the distance from binding constraints on your behavior.

Keep your Distance

Slack is a function of many resources. Running out of any single vital resource is enough to constrain your behavior: make you do something you didn’t want to, or prevent you from doing something you want. Freedom requires having spare time, spare money, spare energy, spare weirdness points, available friends etc. The “slack as distance” formula looks a little something like this:

Slack disappears when the spare capacity of any single resource goes to zero, regardless of how much of everything else you have. Maintaining slack requires balancing all the important resources, making sure to shore up the scarcest resources first.

My grandma just paid to replace a pipe on her floor that was flooding the entire apartment building. The other 20 tenants were supposed to participate in the cost, but due to diffusion of responsibility and greed, they decided collectively to weasel out of contributing. A lawyer suggested that my grandma should go to court but she refused, for reasons of slack. The question isn’t whether the time in court will be worth the money gained, but whether the lack of this particular sum of money will force my grandma into something as undesirable as spending weeks litigating against her neighbors. It almost certainly wouldn’t.

At her age, my grandma’s scarcest resource is stress-free time. She’s not trading it for money.

It’s remarkable how many people fuck up by doing the opposite: concentrating on the resources that are easiest for them to obtain, and neglecting their most pressing needs.

One half of Americans have less than $400 to spare, including millions of middle-cla...
11/07/2017 Development Update: LaTeX!
by habryka — 2017-11-07T11:06:58.012Z

LATEX!press $ for inserting LaTeX and CTRL+M for block-level LaTeXP(i,j)=min{|Vj|,|N(Vi)|∑x=|N(Vi)∩Vj|(|Vj|x)(|V−Vj||N(Vi)|−x)(|V||N(Vi)|)
Clarify Your No's
by Chris_Leong — 2017-11-07T07:25:46.881Z

As humans, we are often too quick to say that something is impossible and move on. Often it pays to double-check - what exactly have we shown to be impossible and does this really matter? Sometimes we'll find it is actually possible, sometimes it'll just give us another idea, but in both these cases it is useful.

Example 1: Can a computer work without power?

Suppose you've been teleported into the past, but you would to be able to solve a calculation. You have no way of generating electricity. Just because an electronic computer like the one you buy at the store needs power, it doesn't mean that you can't build a mechanical computer that doesn't need power. (clarifying computer)
Suppose you're making a computer. Surely, it can't be expected to do anything when the power is off? But what about if it is e-ink, then it can display something static. (clarifying work)
Suppose you want to give children laptop with educational programs, but the town has no sources of electricity. You invent the OLPC laptops, which are powered using handcranks. (clarifying without power)

In each of these cases, it would have been very easy to think, "Computers need power" there is nothing that I can do, when this wasn't the case. Not that we ended up clarifying each phrase - "computer", "work", "without power"

Example 2: Barber Paradox (wiki)

The barber says that they shave every man in the town, but no man shaves themselves.

If you haven't heard this before, this brainteaser might be hard or it might be obvious. Regardless, we can make it even easer by clarifying each phrase.

What does "in the town?" mean? Maybe they mean everyone who lives in the town and doesn't include people who work in the town.
What if the barber is not a man? What if they are a women?

The second is the proper solution, but it comes out very easily once we start clarifying exactly what is impossible?

Example 3: Being attractive

Suppose a person wants to be attractive. Suppose they find out that being attractive requires them to be tall and have lots of muscles because they live in Shallow Land. Unfortunately, they are short and can't gain muscles no matter how much they train. Perhaps, they even become depressed because of this.

On the other hand, what is meant by "attractive"? What do they actually want...
Moloch's Toolbox (2/2)
by Eliezer Yudkowsky — 2017-11-07T01:58:37.315Z

Follow-up to: Moloch's Toolbox (1/2)

vii. Sticky traditions in belief-dependent Nash equilibria without common knowledge

cecie: I could talk next about a tax system that makes it cheaper for corporations to pay for care instead of patients, and how that sets up a host of “decisionmaker is not the beneficiary” problems.

But I suspect a lot of people reading this conversation understand that part already, so instead I’ll turn my attention to venture capital.

visitor: It sounds like the “politicians” and the “voters” might be a more key issue, if the cultural translator is right about what those correspond to.

cecie: Ah! But it turns out that venture capitalists and startups can be seen as a simpler version of voters and politicians, so it’s better to consider entrepreneurs first.

Besides, at this point I imagine the Visitor is wondering, “Why can’t anyone make any money by saving those babies? Doesn’t your society have a profit incentive that fixes this?”

visitor: Actually, I don’t think that was high on my list of questions. It’s understood among my people that not every problem is one you can make a profit by fixing—persistent societal problems tend to be ones that don’t have easily capturable profits corresponding to their solution.

I mean, yes, if this was all happening on our world and it wasn’t already being addressed by the Serious People, then somebody would just mix the bleeping nutrients and sell it to the bleeping parents for bleeping money. But at this point I’ve already guessed that’s going to be illegal, or saving babies using money is going to be associated with the wrong Tower and therefore unprestigious, or your parents are using a particular kind of statistical analysis that requires baby sacrifices, or whatever.

cecie: Hey, details matter!

visitor: (in sad reflection) Do they? Do they really? Isn’t there some point where you just admit you can’t stop killing babies and it doesn’t really matter why?

cecie: No. You can never say that if you want to go on being a cynical economist.

Now, there are several different kinds of molasses covering the world of startups and venture capital. It’s the tradition-bound aspects of that e...
Kolmogorov complexity makes reward learning worse
by Stuart_Armstrong — 2017-11-06T20:08:02.000Z

A putative new idea for AI control; index here.

In a previous post, I argued that Kolmogorov complexity/simplicity priors do not help when learning human values - that some extreme versions of the reward or planners were of roughly equal complexity.

Here I'll demonstrate that it's even worse than that: the extreme versions are likely simpler than a "reasonable" one would be.

Of course, as with any statement about Kolmogorov complexity, this is dependent on the computer language used. But I'll aim to show that for a "reasonable" language, the result holds.

So let (p,R) be a reasonable pair that encodes what we want to encode in human rationality and reward. It is compatible with the human policy πH, in that p(R)=πH.

Let (pr,Rr) be the compatible pair where pr is the rational Bayesian expected reward maximiser, with Rr the corresponding reward so that pr(Rr)=πH.

Let (pi,0) be the indifferent planner (indifferent to the choice of reward), chosen so that pi(R′)=πH for all R′. The reward 0 is the trivial reward.

Information content present in each pair

The planer pi is simply a map to πH, so the only information in pi (and (pi,0)) is the definition of πH.

The policy πH and the brief definition of an expected reward maximiser pr are the only information content in (pr,Rr).

On the other hand, (p,R) defines not only πH, but, at every action, it defines the bias or inefficiency of πH, as the difference between the value of πH and the ideal R-maximising policy πR. This is a large amount of information, including, for instance, every single human bias and example of bounded rationality.

None of the other pairs have this information (there's no such thing as bias for the flat reward 0, nor for the expected reward maximiser pr), so (p,R) contains a lot more information than the other pairs, so we expect it to have higher Kolmogorov complexity.
Meditation and mental space
by Kaj_Sotala — 2017-11-06T13:10:03.612Z

One effect that I often notice after my meditation practice has been interrupted and I then manage to resume it again, is an increase in a kind of mental resilience.

That is, when I have a lower resilience, feeling bad for any reason feels much more like an emergency. It’s something that forces itself into my consciousness, takes over, and refuses to go away. I would like to ignore it, but I can’t; as long as it’s there, it’s hard to think of anything else.

When my resilience is higher, it’s like my mind has more room for thoughts and emotions. Something might be making me feel bad, but something else might also be making me feel good, and there’s space for those two to intermingle. It becomes much easier to accept that I’m feeling a little bad, but I don’t need to do anything about it. I can just go on and do something else, and the nasty feeling might go away on its own – or if it doesn’t, that’s fine too.

Interestingly, being on antidepressants can also give me a similar effect.

Of course, in itself this kind of an effect isn’t too surprising, given that it’s one of the explicit goals of the practice. Culadasa’s The Mind Illuminated notes that two of the goals of mindfulness practice are an increase in the amount of “conscious power” (roughly, the amount of things that can be consciously processed at a time), as well as learning to more intentionally shift the focus of attention, so that it won’t just automatically go to the most painful or pleasant thing and become preoccupied with that, but can rather be controlled in a more useful manner. Still, it’s nice to see that the practice is bearing fruit.
Bias in rationality is much worse than noise
by Stuart_Armstrong — 2017-11-06T11:08:38.000Z

A putative new idea for AI control; index here.

I've found quite a few people dubious of my "radical skepticism" post on human preferences. Most "radical skepticism" arguments - Descartes's Demon, various no-free-lunch theorems, Boltzmann Brains - generally turn out to be irrelevant, in practice, one way or another.

But the human preferences are in a different category. For a start, it's clear that getting them correct is important for AI alignement - we can't just ignore errors. But most importantly, simplicity priors/Kolmogorov complexity/Occam's razor don't help with learning human preferences, as illustrated most compactly with the the substitution of (-p,-R) for (p,R).

But this still feels like a bit of a trick. Maybe we can just assume rationality plus a bit of noise, or rationality most of the time, and get something a lot more reasonable.

Structured noise has to be explained

And, indeed, if humans were rational plus a bit of noise, things would be simple. A noisy signal has high Kolmogorov complexity, but there are ways to treat noise as being of low complexity.

The problem with that approach is that explaining noise is completely different from explaining the highly structured noise that we know as bias.

Take the anchoring bias, for example. In one of the classical experiments, American students were asked if they would buy a product for the price that was the last two digits of their social security number, and were then asked what price they would buy the product for. The irrelevant last two digits had a strong distorting effect on their willingness to pay.

Modelling anchoring behaviour

Let's model a simplified human, if not asked about their social security number, as valuing a cordless keyboard at $25, plus noise η drawn from a normal distribution of mean $0 and standard deviation $5.

If they are first asked about their social secuity number s, their valuation shifts to 3/4($25) + 1/4(s) + η.

To explain human values, we have two theories about their rewards:

R(0)=$25.

R(1)=3/4($25) + 1/4(s).

We have three theories about their planning rationality:

p(0): the human is rational.

p(1): p(0) + noise η.

p(2): the human is rational, but has an anchoring bias of 1/4 of the value of the number they hear.

p(3): p(2) + noise η.

If we count noise as being of low complexity, then R(0), p(0), and p(1) are low complexity rewards/planners. R(1), p(2), and p(3) are

...
Normative assumptions: regret
by Stuart_Armstrong — 2017-11-06T10:59:48.000Z

A putative new idea for AI control; index here.

In a previous post, I presented a model of human rationality and reward as pair (p, R), with p our (ir)rational planning algorithm (called a planner), and R our reward, with p(R) giving our actions/policy. I also showed that human behaviour is insufficient to establish R (or p) in any meaningful way whatsoever.

Yet humans seem to make judgements about each other's rationality and true rewards all the time. And not only do these judgements often agree with each other (almost everyone agrees that the anchoring bias is a bias - or a out-of-context heuristic, ie a bias - and almost nobody argues that it's actually a human value), but they often seem to have predictive ability. What's going on?

Adding normative assumptions

To tackle this, we need to add normative assumptions. Normative assumptions are simply assumptions that distinguish between two pairs (p, R) (p', R') that lead to the same policy: p(R)=p'(R').

Since those pairs predict the same policy, they cannot be distinguished via observations. So a normative assumption is an extra piece, that cannot itself be deduced from observation, and distinguishes between planner-reward pairs.

Because normative assumptions cannot be deduced from observations, they are part of the definition of the human reward function. They are not abstract means of converging on this true reward; they are part of the procedure that defines this reward.

What are they?

There are two way of seeing such normative assumptions. The first is as an extra update rule: upon seeing observation o, the probabilities of (p, R) and (p', R') would normally go to α and α', but, with the normative assumption, there is an extra update to the relative probabilities of the two.

Equivalently, a normative assumption can be seen as an adjustment to the priors of these pairs. The two approaches are equivalent, but sometimes an extra update is computationally tractable whereas the equivalent prior would be intractable (and vice-versa).

Example: regret

So, if normative assumptions are definitions of rationality/reward, what are good candidates to pick? How do we, as humans, define our own rationality and reward?

Well, one strong way seems to be through our feeling of regret (human regret, not regret in the machine learning sense). We often feel and express regret when something turns out the wrong way from what we wan

...