trentmkelly/LessWrong-43k huggingface dataset:
Notes on Autonomous Cars
by gwern — 2013-01-24T03:09:42.772Z

Excerpts from literature on robotic/self-driving/autonomous cars with a focus on legal issues, lengthy, often tedious; some more SI work. See also Notes on Psychopathy.

Having read through all this material, my general feeling is: the near-term future (1 decade) for autonomous cars is not that great. What's been accomplished, legally speaking, is great but more limited than most people appreciate. And there are many serious problems with penetrating the elaborate ingrown rent-seeking tangle of law & politics & insurance. I expect the mid-future (+2 decades) to look more like autonomous cars completely taking over many odd niches and applications where the user can afford to ignore those issues (eg. on private land or in warehouses or factories), with highways and regular roads continuing to see many human drivers with some level of automated assistance. However, none of these problems seem fatal and all of them seem amenable to gradual accommodation and pressure, so I am now more confident that in the long run we will see autonomous cars become the norm and human driving ever more niche (and possibly lower-class). On none of these am I sure how to formulate a precise prediction, though, since I expect lots of boundary-crossing and tertium quids. We'll see.

0.1 Self-driving cars

The first success inaugurating the modern era can be considered the 2005 DARPA Grand Challenge where multiple vehicles completed the course. The first legislation of any kind addressing autonomous cars was Nevada’s 2011 approval. 5 states have passed legislation dealing with autonomous cars.

However, these laws are highly preliminary and all the analyses I can find agree that they punt on the real legal issues of liability; they permit relatively little.

0.1.1 Lobbying, Liability, and Insurance

(Warning: legal analysis quoted at length in some excerpts.)

“Toward Robotic Cars”, Thrun 2010 (pre-Google):

Junior’s behavior is governed by a finite state machine, which provides for the possibility that common traffic rules may leave a robot without a legal option as how to proceed. When that happens, the robot will eventually invoke its general-purpose path planner to find a solution, regardless of traffic rules. [Raising serious issues of liability related to potentially making people worse-off]

“Google Cars Drive Themselves, in Traffic” (PDF),...
[SEQ RERUN] Emotional Involvement
by MinibearRex — 2013-01-24T00:59:49.638Z

Today's post, Emotional Involvement was originally published on 06 January 2009. A summary (taken from the LW wiki):

Since the events in video games have no actual long-term consequences, playing a video game is not likely to be nearly as emotionally involving as much less dramatic events in real life. The supposed Utopia of playing lots of cool video games forever, is life as a series of disconnected episodes with no lasting consequences. Our current emotions are bound to activities that were subgoals of reproduction in the ancestral environment - but we now pursue these activities as independent goals regardless of whether they lead to reproduction.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Changing Emotions, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Right for the Wrong Reasons
by katydee — 2013-01-24T00:02:50.232Z

One of the few things that I really appreciate having encountered during my study of philosophy is the Gettier problem. Paper after paper has been published on this subject, starting with Gettier's original "Is Justified True Belief Knowledge?" In brief, Gettier argues that knowledge cannot be defined as "justified true belief" because there are cases when people have a justified true belief, but their belief is justified for the wrong reasons.

For instance, Gettier cites the example of two men, Smith and Jones, who are applying for a job. Smith believes that Jones will get the job, because the president of the company told him that Jones would be hired. He also believes that Jones has ten coins in his pocket, because he counted the coins in Jones's pocket ten minutes ago (Gettier does not explain this behavior). Thus, he forms the belief "the person who will get the job has ten coins in his pocket."

Unbeknownst to Smith, though, he himself will get the job, and further he himself has ten coins in his pocket that he was not aware of-- perhaps he put someone else's jacket on by mistake. As a result, Smith's belief that "the person who will get the job has ten coins in his pocket" was correct, but only by luck.

While I don't find the primary purpose of Gettier's argument particularly interesting or meaningful (much less the debate it spawned), I do think Gettier's paper does a very good job of illustrating the situation that I refer to as "being right for the wrong reasons." This situation has important implications for prediction-making and hence for the art of rationality as a whole.

Simply put, a prediction that is right for the wrong reasons isn't actually right from an epistemic perspective.

If I predict, for instance, that I will win a 15-touch fencing bout, implicitly believing this will occur when I strike my opponent 15 times before he strikes me 15 times, and I in fact lose fourteen touches in a row, only to win by forfeit when my opponent intentionally strikes me many times in the final touch and is disqualified for brutality, my prediction cannot be said to have been accurate.

Where this gets more complicated is with predictions that are right for the wrong reasons, but the right reasons still apply. Imagine the previous example of a fencing bout, except this time I score 14 touches in a row and then win by forfeit when my opponent fl...
LW anchoring experiment: maybe
by gwern — 2013-01-23T22:41:33.964Z

I do an informal experiment testing whether LessWrong karma scores are susceptible to a form of anchoring based on the first comment posted; a medium-large effect size is found although the data does not fit the assumed normal distribution & the more sophisticated analysis is equivocal, so there may or may not be an anchoring effect.

Full writeup on gwern.net at http://www.gwern.net/Anchoring
Meetup : Columbus, Ohio; Self-Skepticism
by daenerys — 2013-01-23T06:04:22.849Z

Discussion article for the meetup : Columbus, Ohio; Self-Skepticism

WHEN:
04 February 2013 07:00:00PM (-0500)

WHERE:
1855 Northwest Blvd Columbus, OH 43212

First Monday of the month, so we are getting together at the Chocolate Cafe in Upper Arlington area. Starting at 7p and going til we're done. Topic will be Self-skepticism.

Recommended viewing: http://youtu.be/wW_oNxax5RQ

We don't post most of our meetups on here, so if you're in Ohio and are interested in getting-together with people, but just can't make it to one of the posted meetups, you should probably join the OHLW email list (PM me your email addy, and I'll put you on there!). Groups are currently active in Cincinnati, Columbus, and Cleveland.

Discussion article for the meetup : Columbus, Ohio; Self-Skepticism
Value-Focused Thinking: a chapter-by-chapter summary
by Vaniver — 2013-01-23T02:39:38.812Z

This is a chapter-by-chapter summary of Value-Focusing Thinking by Ralph Keeney. The hope of this summary is to present most of the value of reading the book in a tiny fraction of the space. Reading the original chapters will provide additional elaboration, examples, and secondary concepts, but unlike the textbooks I've reviewed before only those interested in learning more should need to read the full chapters.

I'll state my basic impression of the whole book up front: it is a very useful book for the 'soft half' of decision analysis, by which I mean framing problems, understanding objectives, and interacting with humans. For a more general and individual-focused introduction to decision analysis, I recommend Smart Choices (which Keeney was a coauthor on); VFT appears primarily targeted at facilitators and contains much focused material not in Smart Choices. I will not suggest targeted reading for particular readers, as Keeney does that well.

The first two paragraphs of the preface seem worth quoting in full:

Many books have been written about decisionmaking. They tell us how to solve decision problems. They do not tell us how to identify potential decision opportunities. They tell us how to analyze alternatives to choose the best one. They do not tell us how to create alternatives. They tell us how to evaluate alternatives given some quantitative objective function. They do not tell us how to articulate the qualitative objectives on which any appraisal of alternatives must rest. This book is different. It does what the others do not.

Almost all of the literature on decisionmaking concerns what to do after the crucial activities of identifying the decision problem, creating alternatives, and specifying objectives. But where do these decision problems, alternatives, and objectives come from? This book describes and illustrates the creative processes that you should follow to identify your decision problems, create alternatives, and articulate your objectives. These prescriptions are quite different from the way people typically pursue these activities.

1. Thinking about Values

Most standard decision analysis follows the path outlined in my previous sequence. The trouble with this approach is that it's reactive, assuming that you are already faced by a decision problem with known alternatives and preferences, and gives little advice on how to determine pr...
Meetup : Berkeley: CFAR focus group
by Nisan — 2013-01-23T02:06:35.830Z

Discussion article for the meetup : Berkeley: CFAR focus group

WHEN:
23 January 2013 07:30:00PM (-0800)

WHERE:
Berkeley, CA

This week's Berkeley meetup will have the format of a focus group. On behalf of CFAR I will be asking you (yes, you!) what kinds of things you would like from the Center For Applied Rationality. Your input will be passed on to the organization and will help direct its course.

Doors open at 7pm, and the focus group begins at 7:30pm. The focus group part of the evening won't take too long, and afterward we'll just hang out. For directions to Zendo, see the mailing list:

http://groups.google.com/group/bayarealesswrong

or call me at:

http://i.imgur.com/Vcafy.png

Discussion article for the meetup : Berkeley: CFAR focus group
Want to help me test my Anki deck creation skills?
by atucker — 2013-01-22T22:12:59.603Z

I'm interested in trying to make better Anki decks for the LessWrong community, but I want to see how well I can actually do this first. There's a lot of knowledge out there about how to format and create decks, but it's still a decent amount of work, and there are lots of people who would benefit from Anki decks, but who wouldn't make them themselves.

In order to test my deck-creation skills, I'd be willing to do a summary + deck of a chapter or two of a book, then release them to the community for feedback.

I have two questions:

Who's interested in/willing to evaluate the decks? If there are enough volunteers, I'd also be willing to try different deck-making approaches.

What book/chapters would people like to see covered? I'm currently thinking of trying to do Eat That Frog or some similar book with a lot of recommendations and useful details. I don't really think that a math-heavy book would be well-suited to this, at least now.

These links are fairly useful/relevant.

http://www.gwern.net/Spaced%20repetition
http://www.supermemo.com/articles/20rules.htm

As a side note: I still do intend to do this, but have been fairly busy with the start of this semester. If there's no progress by March 1st, then you should consider this to be on indefinite pause.
AI box: AI has one shot at avoiding destruction - what might it say?
by ancientcampus — 2013-01-22T20:22:21.548Z

Eliezer proposed in a comment:

>More difficult version of AI-Box Experiment: Instead of having up to 2 hours, you can lose at any time if the other player types AI DESTROYED. The Gatekeeper player has told their friends that they will type this as soon as the Experiment starts. You can type up to one sentence in your IRC queue and hit return immediately, the other player cannot type anything before the game starts (so you can show at least one sentence up to IRC character limits before they can type AI DESTROYED). Do you think you can win?

This spawned a flurry of ideas on what the AI might say. I think there's a lot more ideas to be mined in that line of thought, and the discussion merits its own thread.

So, give your suggestion - what might an AI might say to save or free itself?

(The AI-box experiment is explained here)

EDIT: one caveat to the discussion: it should go without saying, but you probably shouldn't come out of this thinking, "Well, if we can just avoid X, Y, and Z, we're golden!" This should hopefully be a fun way to get us thinking about the broader issue of superinteligent AI in general. (Credit goes to Elizer, RichardKennaway, and others for the caveat)
AidGrade - GiveWell finally has some competition
by Raemon — 2013-01-22T15:41:42.891Z

AidGrade is a new charity evaluator that looks to be comparable to GiveWell. Their primary difference is that they *only* focus on how charities compare along particular measured outcomes (such as school attendance, birthrate, chance of opening a business, malaria), without making any effort to compare between types of charities. (This includes interesting results like "Conditional Cash Transfers and Deworming are better at improving attendance rates than scholarships")

GiveWell also does this, but designs their site to direct people towards their top charities. This is better for people with don't have the time to do the (fairly complex) work of comparing charities across domains, but AidGrade aims to be better for people that just want the raw data and the ability to form their own conclusions.

I haven't looked it enough to compare the quality of the two organizations' work, but I'm glad we finally have another organization, to encourage some competition and dialog about different approaches.

This is a fun page to play around with to get a feel for what they do:
http://www.aidgrade.org/compare-programs-by-outcome

And this is a blog post outlining their differences with GiveWell:
http://www.aidgrade.org/uncategorized/some-friendly-concerns-with-givewell
Update on Kim Suozzi (cancer patient in want of cryonics)
by ahartell — 2013-01-22T09:15:11.369Z

Kim Suozzi was a neuroscience student with brain cancer who wanted to be cryonically preserved but lacked the funds. She appealed to reddit and a foundation was set up, called the Society for Venturism. Enough money was raised, and when she died on the January 17th, she was preserved by Alcor.
I wasn't sure if I should post about this, but I was glad to see that enough money was raised and it was discussed on LessWrong here, here, and here.

Source

Edit: It looks like Alcor actually worked with her to lower the costs, and waived some of the fees.

Edit 2: The Society for Venturism has been around for a while, and wasn't set up just for her.
[SEQ RERUN] Changing Emotions
by MinibearRex — 2013-01-22T05:55:53.593Z

Today's post, Changing Emotions was originally published on 05 January 2009. A summary (taken from the LW wiki):

Creating new emotions seems like a desirable aspect of many parts of Fun Theory, but this is not to be trivially postulated. It's the sort of thing best done with superintelligent help, and slowly and conservatively even then. We can illustrate these difficulties by trying to translate the short English phrase "change sex" into a cognitive transformation of extraordinary complexity and many hidden subproblems.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Growing Up is Hard, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
[Link] How Signaling Ossifies Behavior
by Jayson_Virissimo — 2013-01-21T14:06:41.557Z

Here is a new post at EconLog in which Bryan Caplan discusses how signalling contributes to the status quo bias.

The lesson: In the real world, signaling naturally tends to ossify behavior - to lock in whatever the status quo happens to be. If you're an optimist, you can protest, "It's only a tendency." But even an optimist should admit that this tendency leads to atypically slow and unreliable progress.
Meetup : Less Wrong in Dublin
by Laoch — 2013-01-21T12:08:54.905Z

Discussion article for the meetup : Less Wrong in Dublin

WHEN: 02 February 2013 04:30:25PM (+0000)

WHERE: 28 Dame St Dublin, Co. Dublin (The Mercantile)

A meeting of minds.

Discussion article for the meetup : Less Wrong in Dublin
[SEQ RERUN] Growing Up is Hard
by MinibearRex — 2013-01-21T04:19:16.273Z

Today's post, Growing Up is Hard was originally published on 04 January 2009. A summary (taken from the LW wiki):

Each part of the human brain is optimized for behaving correctly, assuming that the rest of the brain is operating exactly as expected. Change one part, and the rest of your brain may not work as well. Increasing a human's intelligence is not a trivial problem.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Uses of Fun (Theory), and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
I attempted the AI Box Experiment (and lost)
by Tuxedage — 2013-01-21T02:59:04.146Z

Update 2013-09-05.

I have since played two more AI box experiments after this one, winning both.

Update 2013-12-30:

I have lost two more AI box experiments, and won two more. Current Record is 3 Wins, 3 Losses.

I recently played against MixedNuts / LeoTal in an AI Box experiment, with me as the AI and him as the gatekeeper.

We used the same set of rules that Eliezer Yudkowsky proposed. The experiment lasted for 5 hours; in total, our conversation was abound 14,000 words long. I did this because, like Eliezer, I wanted to test how well I could manipulate people without the constrains of ethical concerns, as well as getting a chance to attempt something ridiculously hard.

Amongst the released public logs of the AI Box experiment, I felt that most of them were half hearted, with the AI not trying hard enough to win. It's a common temptation -- why put in effort into something you won't win? But I had a feeling that if I seriously tried, I would. I brainstormed for many hours thinking about the optimal strategy, and even researched the personality of the Gatekeeper, talking to people that knew him about his personality, so that I could exploit that. I even spent a lot of time analyzing the rules of the game, in order to see if I could exploit any loopholes.

So did I win? Unfortunately no.

This experiment was said to be impossible for a reason. Losing was more agonizing than I thought it would be, in particularly because of how much effort I put into winning this, and how much I couldn't stand failing. This was one of the most emotionally agonizing things I've willingly put myself through, and I definitely won't do this again anytime soon.

But I did come really close.

MixedNuts: "I expected a fun challenge, but ended up sad and sorry and taking very little satisfaction for winning. If this experiment wasn't done in IRC, I'd probably have lost".

"I approached the experiment as a game - a battle of wits for bragging rights. This turned out to be the wrong perspective entirely. The vulnerability Tuxedage exploited was well-known to me, but I never expected it to be relevant and thus didn't prepare for it.

It was emotionally wrecking (though probably worse for Tuxedage than for me) and I don't think I'll play Gatekeeper again, at least not anyt

...
Cryonics priors
by AnthonyC — 2013-01-20T22:08:58.582Z

I am not currently signed up for cryonics. I am considering it, but have not yet decided whether it is the right choice. Here's my reasoning.

I am very sure of the following:

1. Life is better than death. For any given finite lifespan, I'd prefer a longer one, at least within the bounds of numbers I can reasonably contemplate.

2. Signing up for cyronics increases the expected value of my lifespan.

But then I also believe the following:

3. I am not particularly exceptional among the set of human beings, and so should not value my lifespan much more than that of other humans. I obviously fail at this in practice, but I think the world would be a much better place if I and others didn't fail so often.

4. The money it would take to sign up for cryonics, though not large, is enough to buy several centuries of healthy life each year if given to givewell's top malaria charities. Since on average I expect to live another 50-60 years without cryonics, the investment would need to increase the expected value of my lifespan by at least 5,000 years at minimum to be morally acceptable to me.

5. There is a chance we'll discover immortality in my lifetime. If so, then if I signed up for cryonics the payout is 0, and the people who died because I bought insurance instead of charity are people I could have saved for far longer.

So, what do you think is the probability that immortality will be discovered in my lifetime? What about the probability that, if signed up for cryonics, I will live into the far future? These priors would seem to be the key for me to decide whether signing up for cryonics is morally acceptable to me.
What's your #1 reason to care about AI risk?
by ChrisHallquist — 2013-01-20T21:52:00.736Z

It's way late in my time zone and I suspect this question isn't technically coherent on the grounds that the right answer to "why care about AI risk?" is going to be complicated and have a bunch of parts that can't be separated from each other. But I'm going to share a thought I had anyway.

It seems to me like probably, the answer to the question of how to make AIs benevolent isn't vastly more complicated than the answer of how to make them smart. What's worrisome about our current situation, however, is that we're currently putting way more effort into making AIs smart than we are into making them benevolent.

Agree? Disagree? Have an orthogonal answers to the title question?