trentmkelly/LessWrong-43k huggingface dataset:
Expert Iteration From the Inside
by tristanm — 2017-11-02T21:11:02.825Z

Epistemic Status: Possibly generalizing from one example, but seems simple enough to apply more broadly.

Related To: &quot;Thinking Fast and Slow&quot;,

Fake Frameworks

,

Dive In.

I think it’s a good idea to try out bad ideas. No, I don't mean really bad ideas, such as ideas where both your System 1 and System 2 judgements warn against them, I mean bad as in your System 1 makes you feel like it might be worth a try, but your System 2 can’t verify that it’s truly the optimal strategy. In the majority of situations it’s just not possible to calculate fully all the possible consequences of an action during the window of opportunity where the action is feasible. And yet, society moves on. Despite a lot of people relying mainly on their System 1 intuitions, this does not result in disaster often enough to warrant a big shift in the weighting of these two systems.

System 1 often gets a bad rap. It’s viewed as something that the majority of people rely on to a fault over their System 2 thinking, which results in a lot of negative biases and other failure modes. It’s something that seems very difficult to tune and no one seems to know exactly how to change it, whereas System 2 is something that we can deliberately control by learning about the world, taking empirical data, creating new models, watching out for biases and so on.

Unfortunately, lots of real world problems need to be solved quickly, a lot of human endeavors require cooperation between people with lots of different beliefs and incentives and that are hard to accurately model, and we still have a need to take actions in this world. This can’t always be accomplished by sitting and thinking until we’re convinced that we have theoretical guarantees that our actions are optimal.

It seems that within the rationality community, there may be a tendency to mistrust intuition and to be very risk averse, requiring a lot of theoretical guarantees before progress can be made. I argue that this is a bad thing. And while there's obviously room for risk aversion, and reliance on System 2, I also argue that it’s not strictly necessary to lean fully on 2 in order to be a good rationalist. Yes, intuition can be the source of many of our biases and flaws in judgement, but intuition can be tuned with training, and as I want to stress, should be. In fact, as I will argue, having a well developed System 1 will...
Optimising Scientific Research
by DragonGod — 2017-11-02T19:52:39.291Z

Traditional science is too slow Yudkowsky says. Traditional science finds it acceptable for you to waste 30 years on a false theory as long as you disconfirm it at the end. Traditional science allows you to privilege hypothesis as long as you duly test them. Traditional science does not prevent you from 'discovering' magic explanations. Traditional science is insufficient. Traditional science has been tested and found wanting.

And I agree, research as it is currently practiced is (to the best of my knowledge) very unoptimised. There are easily correctable inefficiencies in research that could be eliminated, to increase the speed of converging on the correct theories. Merely practicing Bayesian updating is not enough to eliminate all the inefficiences in research. I get the sense that more is possible. but as best as I can tell, the methods I propose in this post is an optimum one. ,

Table of Contents

A Mathematical Model of Scientific Research
Efficient Experimental Selection
Optimised Experiment Design
Optimised Hypothesis Selection

Brief Summary

A Mathematical Model of Scientific Research

Introduces the model I use to analyse scientific research, and lays the foundations for the entire project. Without this paper, none of the rest would be intelligible.

Efficient Experimental Selection

When confronted with a research problem, what atrategy should we adopt when selecting experiments in order to converge on the correct hypothesis as soon as possible? How do we optimise our experiments in order to make the experimental process as efficient as possible?

Optimised Experiment Design

How do we design our experiments so that they are optimal? So that we only consider the best experiments at any point in time. How do we further optimise our research to make it as efficient as possible?

Optimised Hypothesis Selection

How do we select our hypotheses so that we only consider the best hypotheses? How do we prevent ourselves from wasting time on hypothesis that are not true (and not likely to be true). How do we optimise our research in the very hypotheses we investigate?

I ~expect to publish the first post within a (week p = (0.33)/fortnight p = (0.5), month p = (0.9)) of writing this~ have vastly underestimated the project, and have no idea when I'll finish....
Being Likened to a Trolley Problem
by Portodiovilla — 2017-11-02T17:49:04.341Z

(Epistemic status: Contains use of the trolley problem, eventually / the whole time, so what do you expect?)

(I'm aware that there are a lot of good, practical reasons to try and figure out whether someone likes you, and that many of these inform the weird extra-importance we place on knowing for sure that we're liked. Try to ignore those strictly practical reasons, if you can, in favour of thinking about why and how we should worry what our friends think on principle)

You are travelling through time when you notice that one of your relationships is heading towards...a hangout? Maybe? Looking back, you had foreseen a hangout at around this time every week, and steered towards it, and it seems again like there is a potential hangout ahead. You like hangouts, so you begin to type up a request to your relationship co-pilot to steer your mutual relationship towards the hangout, but something compels you to stop just before sending it.

It occurs to you that maybe you don't actually have to make a request? Maybe the hangout will occur without you having to do anything at all this time? After all, your hangout requests have been mostly approved so far, and, having arrived at the requested hangouts, and engaged in the associated hangings out, they all seemed pleasant and genial for both you and your co-pilot. Perhaps your co-pilot, to whom you have been making these hangout requests, does also foresee this hangout, and does also wants to steer towards it. Perhaps they will make a request to you this time to steer towards the hangout.

That, you think, would actually be kind of nice. There's always a bit of tension when you make a request to your relationship's co-pilot; you always worry that they'll say no. True, they've agreed many times before (and when they haven't their justifications have seemed unrelated to your mutual relationship), but still you wonder: what if this time they say no to your request?

That would suck! You quite like when your relationship leads to these hangouts, and you don't think it's just you; it seems like your co-pilot has liked them as well, like they genuinely enjoy piloting this relationship with you. That said, you're not their only co-pilot, nor they yours. Both of you pilot many other relationships, relationships which you co-pilot with other people, relationships which occasionally...
Our values are underdefined, changeable, and manipulable
by Stuart_Armstrong — 2017-11-02T11:09:15.669Z

Crossposted at Intelligent Agent Forum.

When asked whether "communist" journalists could report freely from the USA, only 36% of 1950 Americans agreed. A follow up question about Amerian journalists reporting freely from the USSR got 66% agreement. When the order of the questions was reversed, 90% were in favour of American journalists - and an astounding 73% in favour of the communist ones.

There are many examples of survey responses depending on question order, or subtle issues of phrasing.

So there are people whose answers depended on question order. What then are the "true" values of these individuals?

Underdetermined values

I think the best way of characterising their values is to call them "underdetermined". There were/are presumably some people for which universal freedom of the press or strict national security were firm and established values. But for most, there were presumably some soft versions of freedom of the press and nationalism, and the first question triggered one narrative more strongly than the other. What then, are their "real" values? That's the wrong question - akin to asking if Argentina really won the 1986 world cup.

Politicians can change the opinions of a large sector of the voting public with a single pronouncement - were the people's real opinions the ones before, or the ones after? Again, this seems to be the wrong question. But don't people fret about this inconsistency? I'd wager that they aren't really aware of this, because people are the most changeable on issues they've given the least thought to.

And rationalists and EAs are not immune to this - we presumably don't shift much on what we identify as our core values, but on less important values, we're probably as changeable as anyone. But such contingent values can become very strong if attacked, thus becoming a core part of our identity - even if it's very plausible we could have held the opposite position in a world slightly different.

Frameworks and moral updating

People often rely on a small number of moral frameworks and principles to guide them. When a new moral issue arises, we generally try and fit it into a moral framework - and when there are multiple ones that could fit, we can go in multiple directions, driven by mood, bias, tribalism, and many other contingent factors.

The moral framew...
Nightmare of the Perfectly Principled
by Benquo — 2017-11-02T09:10:00.979Z

My actual literal nightmares about civilizational collapse somehow manage to be insanely optimistic about human nature.

I dreamt that in response to the news of the Trumps’ probable successful intimidation or bribery of their New York prosecutors, the US devolved into a lawless hellscape, since the last shreds of pretense of “we’re punishing you because it’s what the law says” were gone. In my dream, I successively wished I’d transferred more of my assets to paper, then money, then gold, then firearms, as I realized how far things had gone.

If I’d been thinking sanely, the thing I should have wished I’d accumulated is the only real source of safety in a state of war: a bigger, better gang. But fundamentally, I should have known better than to imagine that things would collapse quickly.

What was I getting wrong? I was tacitly assuming that the majority of people were perfectly principled.

The rule of law and the structure of power

The crazy thing about my nightmare was that it assumed that we had rule of law in the first place, that most of the sorts of people who vote for “law and order” politicians meant the same sort of thing I do by law. In practice, at least for decades, they’ve meant whatever “we” can get the police to enforce, to preserve a civil order that keeps the right sort of people on top.

The normal attitude towards police officers, soldiers, and other authority figures entitled to use coercive force, is to feel safe if they seem like they’re on your side, and unsafe if they seem like they’re the enemy. Police are presumed to follow the laws insofar as there is a custom to do so, and they feel that the power structure they’re embedded in wants them to; exceptions are judged on the basis of whether they locally cause harm.

But my intuitions about whether I am protected by police are quite different. The motive I imagine for following legal procedure is a transcendent commitment to following the law in full formality, because it is the law laid down in the civil code.

This explains how other people - especially other people in groups comparatively unlikely to be the targets of state violence - can read stories about cops and courts colluding to screw over some group or other and maybe feel vaguely guilty, but not personally unsafe. My intuitions are basically always that if we have a good shared set of protocols / laws, then it doesn’t matter whether we’r...
Satoshi Nakamoto?
by MrRobot — 2017-11-02T05:24:45.144Z

1) How scary would it be to realize that Satoshi Nakamoto is actually Unfriendly AI slowly creating an insane amount of wealth? (For a broader, but more positive context read Max Tegmark's intro chapter "The Tale of the Omega Team" from his book "Life 3.0", pg 3-21)

2) Proof that "Satoshi Nakamoto is either: 1) dead or in prison, OR 2) is not a human": Any human who is alive and free would have withdrawn from their initial Bitcoin wallet by now.

3) Nobody will ever shutdown Bitcoin as long as its making them money! That plus distributed blockchain. How is that not a recipe for CPU immortality?

4) And if I'm wrong: LessWrong, will you make it your mission to figure out who Satoshi Nakamoto is? To make sure they are... human.

5) If I was a superintelligent AGI with lots of money, I'd pay Elon Musk whatever money he wants to create physical startups that will be able to create Terminators and go to Space.
The Craft & The Community - A Post-Mortem & Resurrection
by Bendini — 2017-11-02T03:45:46.384Z

Epistemic status: Broad, well-developed speculation.

Preface

To my knowledge, this essay contains the most comprehensive list of criticisms of the rationality community to date. Understandably, some people may take this as a rejection of the community as a whole. It is not. In order to fix problems affecting a non-hierarchical group, individuals within the group need to have a shared understanding of them. In order to do this, someone has to look under the hood and report back with their findings.

Most people are aware there is something wrong in a general sense. There is, to some extent, an awareness that things aren’t quite right but little consensus at to whether it's analogous to a vitamin deficiency or more like a malignant neoplasm.

Solutions vary, depending on the type and extent of the problems. As such, if the consensus is that they are relatively trivial, some solutions are going to look like amputating a leg to deal with a discoloured toenail. If however, the problems are more serious, the majority of solutions proposed up to now look a lot like putting a bandaid on a bullet wound. This is the crux of it, and I expect most negative responses to this will boil down to disagreement over the size, scope and urgency of addressing the problems.

That also includes disagreements over tone; if you spot an axe murderer prowling the library, you are well within your rights to warn people by screaming at the top of your lungs. Unless you are following an absolute version of Kantian ethics, the value tradeoffs you make should vary based on what you are trying to accomplish. In the above example, alerting potential victims of an axe murderer is of far higher importance than maintaining the decorum of the library. This applies to less extreme problems too, tailored to their size, scope and urgency, while remaining mindful of the long-term consequences of bending the rules "just this once" and the pitfalls of operating in a perpetual state of emergency.

In essence, I’m taking a nuanced view of the maxim "Kind, True, Necessary". I believe every issue not omitted from the finished essay is at least relevant to talk about, every factual claim I make is to my knowledge and best efforts true, but I am not being as kind as I could possibly be. I could be kinder by writing in the abstract, but in practice that often ends up obscuring ...
November 2017 Media Thread
by ArisKatsaris — 2017-11-02T00:35:26.761Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.
Yet another failed utopia
by Kazi Siddiqui — 2017-11-02T00:04:55.900Z

A DeepMind-like AI is trained and executed on a decentralized supercomputer like the Golem network, perhaps one that also permits microtransactions like the IOTA Tangle.

The algorithm collects texts posted by citizens on public fora and processes them by topic using sentiment analysis algorithms. From actions A1 to An, it takes action Ax which satisfies the following condition:

Find maximum discontent of the relevant kind voiced by citizens under each plan max(A1), ..., max(An). Across all plans, find the minimum value among the maximum discontent min(max(A1), ..., max(An)). Definition: max(Ax) = min(max(A1), ..., max(An)).

Even if such an AI is hard coded to forbid it from preventing free speech on those fora, how could we possibly trust it not to surreptitiously silence citizens in some other way to prevent them from voicing their discontent?
Research Syndicates
by  — 2017-11-01T18:54:11.199Z

Let us assume we want to encourage people following high risk long term research paths. How might you do this without also just getting people sucking up the energy and not producing good research.

One possibility I've been musing about is research syndicates. This idea is that it might be a good idea for groups of disparate researchers to band together and form a syndicate. The syndicate main purpose would be a form of income sharing, any income going to one member (from a grant or other income) a portion would be shared with the rest. Lets say 75% to the group that got the income and 25% being split between the rest.

Why might this work

1) It helps young researchers get some income to get going with a promise that they will pay back to the syndicate if their research pans out.

2) There would be incentives to mentor the other researchers

3) Being part of a research syndicate would be a form of social proof, so they are to be looked on more seriously.

4) People being added to the syndicate is a costly signal on behalf of the syndicate members.

5) Spreads risk between research teams

6) Technical specialists are more likely to be able to recognise interesting researchers.

There would be a need to have exit clauses worked out. Probably being pretty costly to the people that want to exit or make someone else exit.

Anyone come across something like this being tried out? I think the activation energy for this is probably the legal work necessary.
Competitive Truth-Seeking
by SatvikBeri — 2017-11-01T12:06:59.314Z

In many domains, you can get better primarily by being correct more frequently. If you’re managing a team, or trying to improve your personal relationships, it’s very effective to improve your median decision. The more often you’re right, the better you’ll do, so people often implicitly optimize their success rate.

But what about competitive domains, like prediction markets, investing, or hiring?

One of the most common mistakes I see people make is trying to apply non-competitive intuitions to competitive dynamics. But the winning strategies are very different! It is not enough to be right – others must be wrong. Rather than having a high success rate across all situations, you want to find an edge in some situations, and bet heavily when you find a mistake in the consensus.

For example, say you’re a startup trying to hire software engineers, and your process finds two excellent candidates – Alex and Bob. You make both of them offers at market rate for excellent software engineers. Alex knows how to interview well, so every company believes (correctly) that he’s excellent, while Bob is a bad interviewer, and most of your competitors think he’s merely good. As a result, there might be a 10% chance Alex accepts your offer in this situation, and a 50% chance Bob does.

Assuming a similar rate of Alexes and Bobs in the world, this means your process should be optimized almost entirely for finding Bobs – if you miss a few Alexes to find another Bob, that’s totally worth it! But from the outside, this will look crazy – you’re passing up obviously great candidates in order to specifically find people who aren’t obviously great.

Furthermore, this means that copying mainstream interviewing strategies is one of the worst things you can do – if you only get points for beating consensus predictions, then matching them will get you a 0.

The next time you find yourself thinking about one of these situations, try to figure out whether it’s competitive or non-competitive, and whether you should be optimizing for median predictions or edge.
The Copernican Revolution from the Inside
by Bird Concept — 2017-11-01T10:51:50.127Z

The Copernican revolution was a pivotal event in the history of science. Yet I believe that the lessons most often taught from from this period are largely historically inaccurate and that the most important lessons are basically not taught at all [1]. As it turns out, the history of the Copernican revolution carries important and surprising lessons about rationality -- about what it is and is not like to figure out how the world actually works. Also, it’s relevant to deep learning, but it’ll take me about 5000 words on renaissance astronomy to make that point.

I used to view the Copernican revolution as an epic triumph of reason over superstition, of open science over closed dogma. Basically, things went as follows: Copernicus figured out that the sun rather than the earth is at the center of our planetary system. This theory immediately made sense of the available data, undermining its contorted predecessors with dazzling elegance. Yet its adoption was delayed by the Catholic Church fighting tooth and claw to keep the truth at bay. Eventually, with the emergence of Newton’s work and the dawn of the Enlightenment, heliocentrism became undeniable and its adoption inevitable [2].

This view is inaccurate. Copernicus system was not immediately superior. It was rejected by many people who were not puppets of the Church. And among those who did accept it, better fit to the data was not a main reason. What did in fact happen will become clear in a moment. But in reading that, I’d like to prompt you to consider the events from a very particular vantage point: namely what they would be like from the inside. Ask yourself not what these events seem like for a millennial with the overpowered benefit of historical hindsight, but for a Prussian astronomer, an English nobleman or a Dominican priest.

More precisely, there are two key questions here.

First, if you lived in the time of the Copernican revolution, would you have accepted heliocentrism? I don’t mean this as a social question, regarding whether you would have had the courage and resources to stand up to the immensely powerful Catholic Church. Rather, this is an epistemic question: based on the evidence and arguments available to you, would you have accepted heliocentrism? For most of us, I think the answer is unfortunately, emphatically, and surprisingly, no. The more I’ve read about the Copernican revolution, the l...