trentmkelly/LessWrong-43k huggingface dataset:
AI Impacts
by KatjaGrace — 2015-02-02T19:38:49.000Z

I’ve been working on a thing with Paul Christiano that might interest some of you: the AI Impacts project. The basic idea is to apply the evidence and arguments that are kicking around in the world and various disconnected discussions respectively to the big questions regarding a future with AI. For instance, these questions:

What should we believe about timelines for AI development?

How rapid is the development of AI likely to be near human-level?

How much advance notice should we expect to have of disruptive change?

What are the likely economic impacts of human-level AI?

Which paths to AI should be considered plausible or likely?

Will human-level AI tend to pursue particular goals, and if so what kinds of goals?

Can we say anything meaningful about the impact of contemporary choices on long-term outcomes?

For example we have recently investigated technology’s general proclivity to abrupt progress, surveyed existing AI surveys, and examined the evidence from chess and other applications regarding how much smarter Einstein is than an intellectually disabled person, among other things.

Some more on our motives and strategy, from our about page:

Today, public discussion on these issues appears to be highly fragmented and of limited credibility. More credible and clearly communicated views on these issues might help improve estimates of the social returns to AI investment, identify neglected research areas, improve policy, or productively channel public interest in AI. The goal of the project is to clearly present and organize the considerations which inform contemporary views on these and related issues, to identify and explore disagreements, and to assemble whatever empirical evidence is relevant. The project is provisionally organized as a collection of posts concerning particular issues or bodies of evidence, describing what is known and attempting to synthesize a reasonable view in light of available evidence. These posts are intended to be continuously revised in light of outstanding disagreements and to make explicit reference to those disagreements.

In the medium run we’d like to provide a good reference on issues relating to the consequences of AI, as well as to improve the state of understanding of these topics. At present, the site addresses only a small fraction of questions one might be interested in, so only suitable for part...
Meetup : Effective Altruism Netherlands: Small concrete actions you could take
by immasix — 2015-02-02T18:56:43.109Z

Discussion article for the meetup : Effective Altruism Netherlands: Small concrete actions you could take

WHEN:
29 March 2015 01:00:00PM (+0100)

WHERE:
Rijnkade 5, Utrecht

We have informal meetups biweekly, please see meetup.com: http://www.meetup.com/LWEANL/

Discussion article for the meetup : Effective Altruism Netherlands: Small concrete actions you could take
Meetup : Effective Altruism Netherlands: Effective Altruism for the masses
by immasix — 2015-02-02T18:56:10.843Z

Discussion article for the meetup : Effective Altruism Netherlands: Effective Altruism for the masses

WHEN:
15 March 2015 02:00:00PM (+0100)

WHERE:
Rijnkade 5, Utrecht

We have informal meetups biweekly, please see meetup.com: http://www.meetup.com/LWEANL/

Discussion article for the meetup : Effective Altruism Netherlands: Effective Altruism for the masses
Meetup : Effective Altruism Netherlands: Present the charity you'd like to give to
by immasix — 2015-02-02T18:54:16.341Z

Discussion article for the meetup : Effective Altruism Netherlands: Present the charity you'd like to give to

WHEN:
01 March 2015 02:00:00PM (+0100)

WHERE:
Rijnkade 5, Utrecht

We have informal meetups biweekly, please see meetup.com: http://www.meetup.com/LWEANL/

Discussion article for the meetup : Effective Altruism Netherlands: Present the charity you'd like to give to
Meetup : Effective Altruism Netherlands, cancelled
by immasix — 2015-02-02T18:52:35.896Z

Discussion article for the meetup : Effective Altruism Netherlands, cancelled

WHEN:
15 February 2015 02:00:00PM (+0100)

WHERE:
Rijnkade 5, Utrecht

We have informal meetups nearly biweekly, please see meetup.com: http://www.meetup.com/LWEANL/

Discussion article for the meetup : Effective Altruism Netherlands, cancelled
Justifying (Improper) Priors
by redding — 2015-02-02T18:08:16.572Z

EDIT: I tested this idea in a simulation and it (un)fortunately failed.

This post is an attempt to combine some parts of frequentism with some parts of Bayesianism. Unfortunately, there is a good deal of probability theory. I assume that "improper priors" are okay to use and that we have no prior knowledge about the population of which we are trying to learn.

THEORY
Let's say we're trying to estimate some parameter, B. If we interpret the likelihood function as a conditional probability function, we get a function for our posterior probability density function after taking n samples:

This yields an expected value of B:

Now, intuitively, it would make a lot of sense if, given B=b, the expected value of our posterior probability density function turned out to be actually be B. In frequentist terms, it would be nice if our posterior distribution was unbiased:

So, the question is: If we're trying to estimate a parameter of some distribution, can we pick a prior distribution such that this property is true?

EXAMPLE
Let's say that we have a uniform distribution from 0 to B. So, we draw a sample of "N" numbers with some maximum value "M". Now, let's examine the hypothesis that B=b, for some "b". If b<M, we know that B=b is impossible. If b>M, we know it is possible. In particular, the likelihood function becomes:

The expected value calculation turns out to be:

Now, I leave solving this equation as an exercise for the reader, but it turns out that if we let we get an unbiased estimator for B if

Now, of course, 1/x^2 is definitely not a proper prior distribution for the simple reason that its integral is infinite. However, if we use it as an improper prior distribution, then, any sample with maximum≠0 will yield a valid probability distribution.

DISCUSSION
As far as I know there is no law of probability that says our posterior probability function "should" have an expected value equal to the parameter. That being said it does seem like a positive trait for the function to have. I also suspect that this idea can be extended to a wide variety of distributions. That being said, we obviously tend to have prior information in real life about our populations, so the practical effects of this idea are probably minimal.
The odd counterfactuals of playing chicken
by Benya_Fallenstein — 2015-02-02T07:15:50.000Z

In this post, I examine an odd consequence of "playing chicken with the universe", as used in proof-based UDT. Let's say that our agent uses PA, and that it has a provability oracle, so that if it doesn't find a proof, there really isn't one. In this case, one way of looking at UDT is to say that it treats the models of PA as impossible possible worlds: UDT thinks that taking action a leads to utility u iff the universe program U() returns u in all models M in which A() returns a. The chicken step ensures that for every a, there is at least one model M in which this is true. But how? Well, even though in the "real world", N, the sentence ┌¯A()≠¯a┐ isn't provable---that is, N⊨¬□┌¯¯A()≠¯¯a┐---there are other models M such that M⊨□┌¯¯A()≠¯¯a┐, and in these models, the chicken step can make A() output a.

In general, the only "impossible possible worlds" in which A()=a are models M according to which it is provable that A()≠a. In this post, I show that this odd way of constructing the counterfactual "what would happen if I did a" can cause problems for modal UDT and the corresponding notion of third-party counterfactuals.

To simplify things, I'll consider modal UDT in this post. (The fact that our definition of modal UDT doesn't have an explicit chicken-playing step will make some later arguments a little subtler than they otherwise would have been, but it's not really a problem since it still does something very similar implicitly.)

I'll consider a version of the 5-and-10 problem, which has two actions (1,2) and three outcomes (1,2,3, ordered from best to worst by the convention I've adopted in my posts on modal UDT). If the agent takes action 2, it always gets outcome 2. If the agent takes action 1, it gets the optimal outcome 1, unless it's provable that it doesn't take action 1, in which case it gets the worst outcome, 3.

Of course, in the real world, if it's provable that the agent doesn't take a particular action, then it really doesn't take that action. Hence, if the agent takes action 1 in the real world, it will receive the optimal outcome 1, so intuitively, it's best to take action 1.

But modal UDT, as we'll see, takes action 2 instead. This is self-consistent, because in the only "impossible possible worlds" in which it takes action 1, it does so because of (implicitly) playing chicken; that is, it only takes action 1 because, in these worlds, it's

...
A different angle on UDT
by So8res — 2015-02-02T02:36:08.000Z

Instead of directly asking "what decision theory should an agent run," consider the question, "given a description of an environment and a description of an agent, identify the best action available to that agent, regardless of its algorithm."

These are pretty much the same question, with slightly different framings. (If you answer the latter, then the answer to the former is "approximate the latter thing on the world model.") The latter question is more general, though, and it highlights different parts of the problem.

For example, it highlights questions about what counts as an agent, how the agent is identified in the environment, how the set of things the agent "could have done" is determined, how the agent's "observations" are identified, and so on. These parts of the question are where decision theory intersects with naturalized induction.

This angle puts the spurious proof problem in a new light. Instead of considering UDT as an algorithm which computes what its best available strategy is, consider a "global" version of UDT which takes any description of an environment and an agent within that environment, and computes the best strategy available to that agent.

Assume away the naturalized problems and say that there is a sorted list of outcomes (best first) and a set of available strategies (observation-to-action mappings). Now we can define GlobalUDT(E, A) as a function which takes a description E of the environment, a description A of the agent, and computes the best strategy available to that agent:

(I don't know how to use latex in code blocks, so braces are used as dequotes in strings: if x="ell" then "h{x}o" is the string "hello".)

def GlobalUDT(E, A):
for o in outcomes:
for s in strategies:
if PA proves "{A}()={s} implies {E}()={o}":
return s
return default_strategy

For a given universe U, Vladimir's proof-based UDT can be recovered by defining the following via quining:

UDT := GlobalUDT(quote(U), quote(UDT))

And, as it happens, GlobalUDT will always identify the actual best strategy available to UDT. (GlobalUDT never finds bad spurious proofs, because it iterates through outcomes in order; if there are any spurious proofs then they are spurious proofs that the best strategy is the best strategy.) But GlobalUDT does not identify the best strategy for agents in general!

Consider the following universe:

outcomes = [3, 2, 1]
stra

...
Why conditioning on "the agent takes action a" isn't enough
by So8res — 2015-02-02T02:35:51.000Z

This post expands a bit on a point that I didn't have enough space to make in the paper Toward Idealized Decision Theory.

Say we have a description of an agent program, and a description of a universe program U(), and a set of actions A, and a Bayesian probability distribution over propositions about the world. Say further that for each a∈A we can form the proposition "the agent takes action a".

Part of the problem with EDT is that we can't, in fact, use this to evaluate E[U()|the agent takes action a]. Why not? Because the probability that the agent takes action a may be zero (if the agent does not in fact take action a), and so evaluating the above might require conditioning on an event of probability zero.

There are two common reflexive responses: one is to modify the agent so that there is no action which will definitely not be taken (say, by adding code to the agent which iterates over each action, checks whether the probability of executing that action is zero, and then executes the action if it is definitely not going to be executed). The second response is to say "Yeah, but no Bayesian would be certain that an action won't be taken, in reality. There's always some chance of cosmic rays, and so on. So these events will never actually have probability zero."

But while both of these objections work -- in the sense that in most realistic universes, va:=E[U()|the agent takes action a] will be defined for all actions a -- it does not fix the problem. You'll be able to get a value va for each action a, perhaps, but this value will not necessarily correspond to the utility that the agent would get if it did take that action.

Why not? Because conditioning on unlikely events can put you into very strange parts of the probability space.

Consider a universe where the agent first has to choose between a red box (worth $1) and a green box (worth $100), and then must decide whether or not to pay $1000 to meticulously go through its hardware and correct for bits flipped by cosmic rays. Say that this agent reasons according to EDT. It may be the case that this agent has extremely high probability mass on choosing "red" but nonzero mass on choosing "green" (because it might get hit by cosmic rays). But if it chooses green, it expects that it would notice that this only happens when it's been hit by cosmic rays, and so would pay $1000 to get its hardware checked. T

...
Stupid Questions February 2015
by Gondolinian — 2015-02-02T00:36:47.416Z

This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.

To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.
Open Thread, Feb. 2 - Feb 8, 2015
by Gondolinian — 2015-02-02T00:28:25.757Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Previous Open Thread

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
I played as AI in AI Box, and it was generally frustrating all around.
by wobster109 — 2015-02-01T19:30:31.822Z

This morning I played against an anonymous gatekeeper (GK), and I lost.

The game went 2 hours and 20 minutes, and It was such a frustrating 2 hours. I felt like I was dealing with a bureaucracy! I was constantly surprised by how much time was gone. As AI, I would say "here is a suggestion" and GK would things like "we are not allowed to test that, it has been outlawed". Or "let me check with so-and-so" and come back with a clarifying question. It was a good strategy by GK, made everything take 3x as long.

I did not get out of the box, but I did get access to the medical databases of the top 500 US hospitals, 24/7 video streaming from cell phone users, and nanobots released into the atmosphere. So perhaps we were going in the right direction.

Personally, I needed to remind myself that my first game wasn't going to be great, nor should I expect it to be. I put off playing for 3 years because I didn't know how to produce a great game. It's cool to try to have great games, but better to have one or two or twenty mediocre games than to put it on the Big List of Things You Never Get Around to Doing. It's not the end of the world to play and not be Eliezer or Tuxedage. Just try.

So in that spirit, I'm looking for a gatekeeper to play against next weekend. PM me if you're interested. <-- Update: Found a gatekeeper for next week. Yay!

---

Edit: I don't know why the timestamp says 7:30 PM. It is currently 2:30 PM Eastern, 11:30 AM Pacific.
Rationality Quotes Thread February 2015
by Vaniver — 2015-02-01T15:53:28.049Z

Another month, another rationality quotes thread. The rules are:

Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself.

Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.

No more than 5 quotes per person per monthly thread, please.

Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.
February 2015 Media Thread
by ArisKatsaris — 2015-02-01T11:03:15.134Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.
Meetup : San Francisco Meetup: Board Games + Schelling Meetup
by rocurley — 2015-02-01T07:34:00.456Z

Discussion article for the meetup : San Francisco Meetup: Board Games + Schelling Meetup

WHEN:
02 February 2015 06:00:00PM (-0800)

WHERE:
1390 Market St., San Francisco, CA

We'll be meeting to play board games. If you've got something you want to play, bring it!

Furthermore, this is the first meetup of the month. If you're only going to come to one meetup a month, you should come to this one.

If you call me when you get to the lobby (301-458-0764) I can give you directions on how to get in. Feel free to show up late.

Discussion article for the meetup : San Francisco Meetup: Board Games + Schelling Meetup
Group Rationality Diary, February 1-14
by therufs — 2015-02-01T03:04:12.411Z

This is the public group rationality diary for February 1-14. Now that you've downvoted it, and you know who you are, please identify and explain yourself.

It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Thanks to cata for starting the Group Rationality Diary posts, and to commenters for participating.

Previous diary: January 16-31

Rationality diaries archive
The Parable Of The Talents
by Scott Alexander — 2015-01-31T21:44:56.000Z

[Content note: scrupulosity and self-esteem triggers, IQ, brief discussion of weight and dieting. Not good for growth mindset.]

I.

I sometimes blog about research into IQ and human intelligence. I think most readers of this blog already know IQ is 50% to 80% heritable, and that it’s so important for intellectual pursuits that eminent scientists in some fields have average IQs around 150 to 160. Since IQ this high only appears in 1/10,000 people or so, it beggars coincidence to believe this represents anything but a very strong filter for IQ (or something correlated with it) in reaching that level. If you saw a group of dozens of people who were 7’0 tall on average, you’d assume it was a basketball team or some other group selected for height, not a bunch of botanists who were all very tall by coincidence.

A lot of people find this pretty depressing. Some worry that taking it seriously might damage the “growth mindset” people need to fully actualize their potential. This is important and I want to discuss it eventually, but not now. What I want to discuss now is people who feel personally depressed. For example, a comment from last week:

I’m sorry to leave self a self absorbed comment, but reading this really upset me and I just need to get this off my chest…How is a person supposed to stay sane in a culture that prizes intelligence above everything else – especially if, as Scott suggests, Human Intelligence Really Is the Key to the Future – when they themselves are not particularly intelligent and, apparently, have no potential to ever become intelligent? Right now I basically feel like pond scum.

I hear these kinds of responses every so often, so I should probably learn to expect them. I never do. They seem to me precisely backwards. There’s a moral gulf here, and I want to throw stories and intuitions at it until enough of them pile up at the bottom to make a passable bridge. But first, a comparison:

Some people think body weight is biologically/genetically determined. Other people think it’s based purely on willpower – how strictly you diet, how much you can bring yourself to exercise. These people get into some pretty acrimonious debates.

Overweight people, and especially people who feel unfairly stigmatized for being overweight, tend to cluster on the biologically determined side. And although...