trentmkelly/LessWrong-43k huggingface dataset:
Against easy superintelligence: the unforeseen friction argument
by Stuart_Armstrong — 2013-07-10T13:47:33.114Z

In 1932, Stanley Baldwin, prime minister of the largest empire the world had ever seen, proclaimed that "The bomber will always get through". Backed up by most of the professional military opinion of the time, by the experience of the first world war, and by reasonable extrapolations and arguments, he laid out a vision of the future where the unstoppable heavy bomber would utterly devastate countries if a war started. Deterrence - building more bombers yourself to threaten complete retaliation - seemed the only counter.

And yet, things didn't turn out that way. Against all past trends, the light fighter plane surpassed the heavily armed bomber in aerial combat, the development of radar changed the strategic balance, and cities and industry proved much more resilient to bombing than anyone had a right to suspect.

Could anyone have predicted these changes ahead of time? Most probably, no. All of these ran counter to what was known and understood, (and radar was a completely new and unexpected development). What could and should have been predicted, though, was that something would happen to weaken the impact of the all-conquering bomber. The extreme predictions would be unrealistic; frictions, technological changes, changes in military doctrine and hidden, unknown factors, would undermine them.

This is what I call the "generalised friction" argument. Simple predictive models, based on strong models or current understanding, will likely not succeed as well as expected: there will likely be delays, obstacles, and unexpected difficulties along the way.

I am, of course, thinking of AI predictions here, specifically of the Omohundro-Yudkowsky model of AI recursive self-improvements that rapidly reach great power, with convergent instrumental goals that make the AI into a power-hungry expected utility maximiser. This model I see as the "supply and demand curve" of AI prediction: too simple to be true in the form described.

But the supply and demand curves are generally approximately true, especially over the long term. So this isn't an argument that the Omohundro-Yudkowsky model is wrong, but that it will likely not happen as flawlessly as described. Ultimately, the "bomber will always get through" turned out to be true: but only in the form of the ICBM. If you take the old arguments and replace "bomber" with "ICBM", you end with strong and accurate predictions. So "the ...
The failure of counter-arguments argument
by Stuart_Armstrong — 2013-07-10T13:38:57.443Z

Suppose you read a convincing-seeming argument by Karl Marx, and get swept up in the beauty of the rhetoric and clarity of the exposition. Or maybe a creationist argument carries you away with its elegance and power. Or maybe you've read Eliezer's take on AI risk, and, again, it seems pretty convincing.

How could you know if these arguments are sound? Ok, you could whack the creationist argument with the scientific method, and Karl Marx with the verdict of history, but what would you do if neither was available (as they aren't available when currently assessing the AI risk argument)? Even if you're pretty smart, there's no guarantee that you haven't missed a subtle logical flaw, a dubious premise or two, or haven't got caught up in the rhetoric.

One thing should make you believe the argument more strongly: and that's if the argument has been repeatedly criticised, and the criticisms have failed to puncture it. Unless you have the time to become an expert yourself, this is the best way to evaluate arguments where evidence isn't available or conclusive. After all, opposite experts presumably know the subject intimately, and are motivated to identify and illuminate the argument's weaknesses.

If counter-arguments seem incisive, pointing out serious flaws, or if the main argument is being continually patched to defend it against criticisms - well, this is strong evidence that main argument is flawed. Conversely, if the counter-arguments continually fail, then this is good evidence that the main argument is sound. Not logical evidence - a failure to find a disproof doesn't establish a proposition - but good Bayesian evidence.

In fact, the failure of counter-arguments is much stronger evidence than whatever is in the argument itself. If you can't find a flaw, that just means you can't find a flaw. If counter-arguments fail, that means many smart and knowledgeable people have thought deeply about the argument - and haven't found a flaw.

And as far as I can tell, critics have constantly failed to counter the AI risk argument. To pick just one example, Holden recently provided a cogent critique of the value of MIRI's focus on AI risk reduction. Eliezer wrote a response to it (I wrote one as well). The core of Eliezer's and my response wasn't anything new; they were mainly a rehash of what had been said before, with a different emphasis.

And most responses to critics of...
Help please! Making a good choice between two jobs
by cjb230 — 2013-07-10T11:23:31.177Z

After about three months of unemployment, today I have been told I will receive two different job offers. Obviously I want to make a decision that maximises my utility, but there are some difficult-to-quantify factors involved. Can anyone suggest a useful or clarifying perspective on the choice? What questions should I be asking myself?

As background, I'm a 36-year old male techie based in the UK. What I would really like to do is build my own startup, or join a startup that I think has good prospects. However, having done that twice, and nearly bankrupted myself twice, I need to get a financial cushion under myself before trying again. For the sake of my CV, I think I need to stay in my next role for at least 18 months. After that, I hope to be able to try something entrepreneurial again, but I want to stay employable in technology as a financial safeguard.

One job is in London, and will certainly pay more. I have many friends there, and a social network I can get back in to straight away. The pool of women I am interested in dating is much bigger there. The job itself will allow me to improve my skills, but probably not broaden them; there are very skilled people there that I can learn from. With this job, I fear being bored, and getting more and more specialised in a skill that is getting less popular.

The other job is in Glasgow. It will pay less, probably by about £15k. The cost of living difference, after tax, will probably be bigger than this - I expect I will have more money in my pocket with this job than with the London job. I only know two people living in Glasgow, so I will need to make new friends and get a new social life. Dating prospects are probably less good, but I don't know by how much. The job itself looks more interesting, and I can broaden my skills. The job market I am in will probably be more open to me after I take this job. With this job, I fear getting depressed due to isolation, and also the long-term effect of accepting a pay cut - if I worked in London again later, would I be able to negotiate my salary back up?

Cost aside, London is a better location. It has more of everything I am interested in (including a LW meetup group!). Location and pay aside, the job in Glasgow is better. I expect it will be more interesting, and will make it less likely that I'll be unemployed in the future.

I don't know how to weight these fac...
Meetup : Buffalo LW - Biased Boardgaming
by StonesOnCanvas — 2013-07-10T02:34:15.521Z

Discussion article for the meetup : Buffalo LW - Biased Boardgaming

WHEN:
18 July 2013 06:00:00PM (-0400)

WHERE:
121 Wanda Ave. Cheektowaga, NY 14211

Buffalo-area Less Wrong meetups on the first Sunday and Third Thursday of every month. Join us on Meetup: http://www.meetup.com/Less-Wrong-Buffalo/
Hey Everyone,
So, instead of doing a doodle poll to figure out a good day for a game night, I figured we could have a game night for our next meetup. We'll order some pizza for everyone. This time we will try playing with biases (each player draws a bias at random from a deck, and plays their turn with that bias. If someone is unsure how to play the bias he/she can ask the group for future reference and draw another one instead. If we think we know what bias someone drew we can try guessing it). See you all there!
P.S. This meetup starts an hour earlier than our normal Thursday meetups and will be held at our house.

Discussion article for the meetup : Buffalo LW - Biased Boardgaming
Can we make Drake-like Fermi estimates of expected distance to the next planet with primitive, sentient or self-improving life?
by chaosmage — 2013-07-10T01:34:36.397Z

I expect everyone here has an opinion on the Drake Equation. (Comment if I'm wrong.) And that's because it is an easy story to remember and spread. Never mind its glaring inadequacy or the symbols it uses: it gives you a number of alien civilizations and somehow that sticks. I'd like to see if a science meme with similar properties could be created to carry a transhumanist payload. So. Could you convince a random person of the following three points if you wanted to?

We're getting increasingly confident estimates on the number and distribution of planets in our galaxy.

The other factors in the Drake equation have been discussed a lot - they remain guesses till we find something, but at least they aren't going to change a lot until we do.

So we should be able to estimate, very roughly and while mumbling about priors, an expected distance to the next planetary body with primitive life, with sentient life or with self-improving life (i.e. something like AIs that can exponentially grow that biosphere's cognitive capacity).

I think you could. And if you do, and if you can give a number of light-years, regardless of how much you emphasize the low confidence, aliens will suddenly seem more real to that random person. And so will, if not full transhumanism, at least some vague notion that intelligence must grow much like life does. I think that could reach a lot of people.

(If anybody complains that the expectation of some Singularity-like development is ideological: no, it is a reasonable guess based on the current evidence, much like Drake's expectation of every technological civilization's eventual self-destruction was reasonable in his Cold War era.)

The brain I'm typing this from knows too little math or astronomy to do this locally, so I'm throwing out the idea. Anyone care to play with this?
Effectiveness or altruism?
by KatjaGrace — 2013-07-09T23:19:59.000Z

Effective altruism is all the rage. And as we all know, a highly effectively altruistic thing to do is bring about more effective altruism. However, the optimal mixture of activities for promoting effectiveness is not identical to that for promoting altruism. They are upheld by different norms, and require different efforts to expand and arguments to proliferate. Which brings us to a need for prioritization. So, which is better: effectiveness or altruism?

By this I mean, which is the better recipient of marginal promotion effort?

For background, I’m visiting the Oxford Effective Altruists at the moment. Conversations here made it salient to me that my implicit views on this question have changed a lot since I was a teenager, and also differ from others’ in the EA movement. So I thought I’d lay them out.

When I was younger, I thought altruism was about the most promising way to make the world better. There were extremely cheap figures around for the cost to save a human life, and people seemed to not care. So prima facie it seemed that the highly effective giving opportunities were well worked out, and the main problem was that people tended to give $2 to such causes occasionally, rather than giving every spare cent they had, that wasn’t already earmarked for something more valuable than human lives.

These days I am much more optimistic about improving effectiveness than altruism, and not just because I’m less naive about cost-effectiveness estimates. Here are some considerations:

Past success

So far, effectiveness has bought more than altruism for EAs, arguably. That is, giving an ordinary amount highly effectively will buy more good than giving a highly altruistic amount in an ordinary way. The difference between a normal amount of giving and a highly altruistic one is around 20x by my reckoning. I’m not sure what the mean charity is, but my impression is that it is less than 1/20th as useful as the best things we know of. Better data on this welcome.

Furthermore, perhaps the quest for producing altruistic value can learn from the more popular quest to produce selfish value. There have been efforts both to encourage people to be more selfish (see Ayn Rand) and to find cost-effective ways to direct selfish efforts. As far as I can tell, the former might even produce negative selfish value for people, whereas mankind has become ...
Prisoner's dilemma tournament results
by AlexMennen — 2013-07-09T20:50:43.753Z

The prisoner's dilemma tournament is over. There were a total of 21 entries. The winner is Margaret Sy, with a total of 39 points. 2nd and 3rd place go to rpglover64 and THE BLACK KNIGHT, with scores of 38 and 36 points respectively. There were some fairly intricate strategies in the tournament, but all three of these top scorers submitted programs that completely ignored the source code of the other player and acted randomly, with the winner having a bias towards defecting.

You can download a chart describing the outcomes here, and the source codes for the entries can be downloaded here.

I represented each submission with a single letter while running the tournament. Here is a directory of the entries, along with their scores: (some people gave me a term to refer to the player by, while others gave me a term to refer to the program. I went with whatever they gave me, and if they gave me both, I put the player first and then the program)

A: rpglover64 (38)
B: Watson Ladd (27)
c: THE BLACK KNIGHT (36)
D: skepsci (24)
E: Devin Bayer (30)
F: Billy, Mimic-- (27)
G: itaibn (34)
H: CooperateBot (24)
I: Sean Nolan (28)
J: oaz (26)
K: selbram (34)
L: Alexei (25)
M: LEmma (25)
N: BloodyShrimp (34)
O: caa (32)
P: nshepperd (25)
Q: Margaret Sy (39)
R: So8res, NateBot (33)
S: Quinn (33)
T: HonoreDB (23)
U: SlappedTogetherAtTheLastMinuteBot (20)
"Can we know what to do about AI?": An Introduction
by JonahS — 2013-07-09T18:22:49.318Z

I'm currently working on a research project for MIRI, and I would welcome feedback on my research as I proceed. In this post, I describe the project.

As a part of an effort to steel-man objections to MIRI's mission, MIRI Executive Director Luke Muehlhauser has asked me to develop the following objection:

"Even if AI is somewhat likely to arrive during the latter half of this century, how on earth can we know what to do about it now, so far in advance?"

In Luke's initial email to me, he wrote:

I think there are plausibly many weak arguments and historical examples suggesting that P: "it's very hard to nudge specific distant events in a positive direction through highly targeted actions or policies undertaken today." Targeted actions might have no lasting effect, or they might completely miss their mark, or they might backfire.

If P is true, this would weigh against the view that a highly targeted intervention today (e.g. Yudkowsky's Friendly AI math research) is likely to positively affect the future creation of AI, and might instead weigh in favor of the view that all we can do about AGI from this distance is to engage in broad interventions likely to improve our odds of wisely handling future crises in general — e.g. improving decision-making institutions, spreading rationality, etc.

I'm interested in abstract arguments for P, but I'm even more interested in historical data. What can we learn from seemingly analogous cases, and are those cases analogous in the relevant ways? What sorts of counterfactual history can we do to clarify our picture?

Luke and I brainstormed a list of potential historical examples of people predicting the future 10+ years out, and using the predictions to inform their actions. We came up with the following potential examples, which I've listed in chronological order by approximate year:

1896: Svante Arrhenius's prediction of anthropogenic climate change.

1935: Leo Szilard's ~1935 attempts to keep his patent of the atomic bomb secret from Germany.

1950-1980: Efforts to win the Cold War decades later, such as increasing education for gifted children.

1960: Norbert Weiner highlighting the dangers of artificial intelligence.

1972: The circle of ideas and actions around the The Limits to Growth, a book about the consequences of unchecked populatio

...
Where Are We the Weakest?
by DiscyD3rp — 2013-07-09T18:07:25.823Z

As rationalists, we should be able to consistently and accurately make predictions that enable us to act effectively.

As humans, we don't. At least not perfectly.

We need to improve. Many of us have, or at least believe we have. However, it's a notably hacked improvement. PredictionBook is an excellent source of feedback on how well we're doing, but there's more detailed information that isn't easily available that I think could be incredibly useful. Questions I would like to see answered are:

What kinds of predictions are we the least successful at predicting? (weakest calibration, smallest accuracy)

What kinds of predictions have the most low-hanging fruit? What's the easiest to improve on right now?

What kinds of predictions are the most useful to us? (accurately predicting a close friend's behavior>predicting obscure political decision)

Where aren't we making quantitative predictions? Where does our behavior involve predictions that are underrepresented on PredictionBook?

Before we are able to improve as a community, we need to know where to improve. I'd love to hear suggestions on how to answer these questions in the comments.
Gauging interest for a Zurich, Switzerland meetup group
by roland — 2013-07-09T18:00:12.460Z

Who would be interested in meeting? Please reply to this thread or msg me in private.

Thanks!

Edit: There doesnt seem to be enough people willing to meet so Im not planning anything at the moment. But if you happen to be in Zurich and want to meet a fellow LWer just msg me in private and we can go for a talk/drink.
[LINK] Analysis of why excluding hostile people is worth it
by NancyLebovitz — 2013-07-09T16:01:03.363Z

http://blip.tv/tech-love-live/osb09-donnie-berkholz-assholes-are-killing-your-project-2464449

This is specifically about why it's important to get assholes out of open source projects, but it applies in general. It includes an analysis of the social cost of keeping people around who frequently make other people unhappy, and in particular a way to balance the social costs (distraction, people doing much less work or leaving, useful volunteers not joining, assholes recruiting other assholes, etc.) of assholes against the useful work some of them do.
HPMoR the Youtube Series! But in need of advice
by wobster109 — 2013-07-09T14:50:34.654Z

Hi Less Wrong! I was wondering if you have experience with video editing? I want to record footage and a soundtrack, and then overlay them on each other, and I'll also need to be able to do special effects, such as to float someone in the air. Is there a video editing program that you'd recommend?

Edit - Please let me know if you'd like to act in it and are able to get to Madison, WI on weekends :)
My Take on a Decision Theory
by ygert — 2013-07-09T10:46:25.548Z

Finding a good decision theory is hard. Previous attempts, such as Timeless Decision Theory, work, it seems, in providing a stable, effective decision theory, but are mathematically complicated. Simpler theories, like CDT or EDT, are much more intuitive, but have deep flaws. They fail at certain problems, and thus violate the maxim that rational agents should win. This makes them imperfect.

But it seems to me that there is a relatively simple fix one could make to them, in the style of TDT, to extend their power considerably. Here I will show an implementation of such an extension of CDT, that wins on the problems that classic CDT fails on. It quite possibly could turn out that this is not as powerful as TDT, but it is a significant step in that direction, starting only from the naivest of decision theories. It also could turn out that this is nothing more than a reformulation of TDT or a lesser version thereof. In that case, this still has some value as a simpler formulation, easier to understand. Because as it stands, TDT seems like a far cry from a trivial extension of the basic, intuitive decision theories, as this hopes to be.

We will start by remarking that when CDT (or EDT) tries to figure out the expected value or a action or outcome, the naive way which it does so drops crucial information, which is what TDT manages to preserve. As such, I will try to calculate a CDT with this information not dropped. This information is, for CDT, the fact that Omega has simulated you and figured out what you are going to do. Why does a CDT agent automatically assume that it is the "real" one, so to speak? This trivial tweak seems powerful. I will, for the purpose of this post, call this tweaked version of CDT "Simulationist Causal Decision Theory", or SCDT for short.

Let's run this tweaked version though Newcomb's problem. Let Alice be a SCDT agent. Before the problem begins, as is standard in Newcomb's problem, Omega looks at Alice and calculates what choice Alice will make in the game. Without to much loss of generality, we can assume that Omega directly simulates Alice, and runs the simulation through the a simulation of the game, in order make the determination of what choice Alice will make. In other formulations of Newcomb's problem, Omega figures in out some other way what Alice will do, say by doing a formal analysis of her source code, but that seems intuitiv...