trentmkelly/LessWrong-43k huggingface dataset:
Rationality Quotes December 2014
by Salemicus — 2014-12-03T22:33:20.202Z

Another month, another rationality quotes thread. The rules are:

Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself.

Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.

No more than 5 quotes per person per monthly thread, please.

Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.
The Bay Area Solstice
by Alex_Altair — 2014-12-03T22:33:17.760Z

As the holiday season approaches, we continue our tradition of celebrating the winter solstice.

This event is the offspring of Raemon's New York Solstice. The core of the event is a collection of songs old and new, silly and profound, led by the well-calibrated Bayesian choir. There will be bean bag chairs and candles. There will be campfire and chocolates (in case of dementors).

When: The Bay Area Solstice will be held on 13 December at 7:00 PM.

Where: We've rented the Humanist Hall, at 390 27th St, Oakland, CA 94612.

All humanists or transhumanists are welcome. We'll be diving our minds into the nature of the universe, both good and bad. We'll stare into the abyss of death, and into the radiance of our ability to remove it. We will recognize each other as allies and agents.

We're glad to provide aspiring rationalists with an alternative or addition to any holiday celebrations. There is an expected attendance of around 80 people.

Get your tickets here! And if you'd like to help us put it together, PM me.
Good things to have learned....
by NancyLebovitz — 2014-12-03T19:29:34.580Z

I was looking at a discussion of what should be in a college curriculum, and as such discussions seem to go, there was a big list of things everyone should study, and some political claims about what's being offered but shouldn't be.

Instead, what do you wish you'd studied in college? What do you wish other people had studied in college? On the latter, do you think everyone should have studied it, or do you just wish more people knew about it? Approximately what percentage of people?

Of course, this doesn't have to be limited to college. People could learn the same things earlier or later.
Autonomy, utility, and desire; against consequentialism in AI design
by sbenthall — 2014-12-03T17:34:57.989Z

For the sake of argument, let's consider an agent to be autonomous if:

It has sensors and actuators (important for an agent)

It has an internal representation of its goals. I will call this internal representation its desires.

It has some kind of internal planning function that given sensations and desires, chooses actions to maximize the desirability of expected outcomes

I want to point to the desires of the agent specifically to distinguish them from the goals we might infer for it if we were to observe its actions over a long period of time. Let's call these an agent's empirical goals. (I would argue that it is in many cases impossible to infer an agent's empirical goals from its behavior, but that's a distraction from my main point so I'm just going to note it here for now.)

I also want to distinguish them from the goals it might arrive on stably if it were to execute some internal goal-modification process optimizing for certain conditions. Let's call these an agent's reflective goals.

The term utility function, so loved by the consequentialists, frequently obscures these important distinctions. An argument that I have heard for the expansive use of the term "utility function" in describing agents is is: The behavior of all agents can be characterized using a utility function, therefore all agents have utility functions. This argument depends on a fallacious conflation of desire, empirical goals, and reflective goals.

An important problem that I gather this community thinks about deeply is how to think about agents whose reflective goals are different from its present desires--say, the desires I have and have transferred over to it. For example, if I want to program an agent with desires and the capacity to reflect, then can I guarantee that it will be faithful to the desires I intended for it?

A different problem that I am interested in is what other classes of agents there may be besides autonomous agents. Specifically, what if an agent does not have an internal representation of its desires. Is that possible? Part of my motivation for this is my interest in Buddhism. If an enlightened agent is one with no desires, how could one program a bodhisattva AI whose only motivation was the enlightenment of all beings?

An objection to this line of thinking is that even an agent that desires enlightenment for all beings has desir

...
Link: LessWrong and AI risk mentioned in a Business Insider Article
by James_Miller — 2014-12-03T17:13:59.505Z

Google Has An Internal Committee To Discuss Its Fears About The Power Of Artificial

"Worryingly, cofounder Shane Legg thinks the team's advances could be what finishes off the human race. He told the LessWrong blog in an interview 'Eventually, I think human extinction will probably occur, and technology will likely play a part in this'. He adds he thinks AI is the 'no.1 risk for this century'. It's ominous stuff. (Read about Elon Musk discussing his concerns about AI here.)"
A hypothetical question for investors 
by bokov — 2014-12-03T16:39:24.576Z

Let's suppose you start with $1000 to invest, and the only thing you can invest it in is stock ABC. You are only permitted to occupy two states:

* All assets in cash

* All assets in stock ABC

You incur a $2 transaction fee every time you buy or sell.

Kind of annoying limitations to operate under. But you have a powerful advantage as well. You have a perfect crystal ball that each day gives you the [probability density function](http://en.wikipedia.org/wiki/Probability_density_function) of ABC's closing price for the following day (but no further ahead in time).

What would be an optimal decision rule for when to buy and sell?
Meetup : December Rationality Dojo - Online Communication: Conveying Ideas Effectively
by MelbourneLW — 2014-12-03T11:49:35.338Z

Discussion article for the meetup : December Rationality Dojo - Online Communication: Conveying Ideas Effectively

WHEN:
07 December 2014 03:30:00PM (+0800)

WHERE:
Ross House Association, 247-251 Flinders Lane, Melbourne

[ATTN: Please remember the new location for the dojos: the Jenny Florence Room, Level 3, Ross House at 247 Flinders Lane,Melbourne. 3:30pm start / arrival - formal dojo activities will commence at 4:00pm.]

The Less Wrong Sunday Rationality Dojos are crafted to be serious self-improvement sessions for those committed to the Art ofRationality and personal growth. Each month a community member will run a session involving a presentation of content, discussion, and exercises.

Continuing the succession of immensely successful dojos, Chris will run a session on conveying ideas effectively in online communication.

As always, we will review the personal goals we committed to at the previous Dojo (I will have done X by the next Dojo). Our goals are now being recorded via Google Forms here - https://docs.google.com/forms/d/1MCHH4MpbW0SI_2JyMSDlKnnGP4A0qxojQEZoMZIdopk/viewform, and Melbourne Less Wrong organisers have access to the form results if you wish to review the goals you set last month.

This month, we are also seeking 2-3 lightning talks from members. Speakers will be limited to 5 minutes with room for questions. We will be asking for talks from attendees present, but if you already have a talk topic in mind, please contact Louise at lvalmoria@gmail.com

The Dojo is likely to run for 2-3 hours, after which some people will get dinner together.

If you have any trouble finding the venue or getting in, call Louise on 0419 192 367.

If you would like to present at a future Dojo or suggest a topic, please fill it in on the Rationality Dojo Roster: http://is.gd/dojoroster

Discussion article for the meetup : December Rationality Dojo - Online Communication: Conveying Ideas Effectively
Meetup : Canberra: End of year party
by DanielFilan — 2014-12-03T11:49:07.022Z

Discussion article for the meetup : Canberra: End of year party

WHEN:
13 December 2014 06:00:00PM (+1100)

WHERE:
10 Thynne St, Bruce, ACT

It's a party! The year is ending! Come to my place and we can chill, eat party foods, and chat about LW stuff. I'm in room 70, call me on my mobile number (0468 312 414) and you will get let in.

Note the change in location!

General meetup info:

If you use Facebook, please join our group.

Structured meetups are (usually) held on the second Saturday and fourth Friday of each month from 6 pm until late in the CSIT building, room N101.

Discussion article for the meetup : Canberra: End of year party
Meetup : Moscow Meetup: CBT is back
by Alexander230 — 2014-12-02T19:21:15.309Z

Discussion article for the meetup : Moscow Meetup: CBT is back

WHEN:
07 December 2014 02:00:00PM (+0300)

WHERE:
Russia, Moscow, ulitsa L'va Tolstogo 16

This meetup will be in semi-closed format! If you hadn't come to our meetups before, please wait for the next open meetup: it's planned to December, 21.

Regular visitors of our meetups, please wait for announce in our group:

https://groups.google.com/forum/#!forum/rationality-in-moscow

You will feel more comfortable in our meetups if you read the needed materials and became familiar with some base ideas:

You understand the Bayes theorem (what is bayesianism - http://lesswrong.com/lw/1to/what_is_bayesianism or http://schegl2g.bget.ru/bayes/YudkowskyBayes.html ).

You understand what is "System 1" and "System 2" (Kahneman and Yudkowsky).

You know what is "rational agent".

You've read the base sequences: "Map and Territory" - http://wiki.lesswrong.com/wiki/Map_and_Territory_(sequence) and "Mysterious Answers to Mysterious Questions" - http://wiki.lesswrong.com/wiki/Mysterious_Answers_to_Mysterious_Questions .

It will allow you to understand better, what we talk about, how do we think, and will give you the spirit of our activities.

Discussion article for the meetup : Moscow Meetup: CBT is back
Model-free decisions
by paulfchristiano — 2014-12-02T17:39:04.000Z

Much concern about AI comes down to the scariness of goal-oriented behavior. A common response to such concerns is "why would we give an AI goals anyway?" I think there are good reasons to expect goal-oriented behavior, and I've been on that side of a lot of arguments. But I don't think the issue is settled, and it might be possible to get better outcomes by directly specifying what actions are good. I flesh out one possible alternative here.

(As an experiment I wrote the post on medium, so that it is easier to provide sentence-level feedback, especially feedback on writing or low-level comments. Big-picture discussion should probably stay here.)
[LINK] Steven Hawking warns of the dangers of AI
by Salemicus — 2014-12-02T15:22:58.849Z

From the BBC:

[Hawking] told the BBC:"The development of full artificial intelligence could spell the end of the human race."

...

"It would take off on its own, and re-design itself at an ever increasing rate," he said. "Humans, who are limited by slow biological evolution, couldn't compete, and would be superseded."

There is, however, no mention of Friendly AI or similar principles.

In my opinion, this is particularly notable for the coverage this story is getting within the mainstream media. At the current time, this is the most-read and most-shared news story on the BBC website.
Link: Biotech Corporate Email Hacked
by ilzolende — 2014-12-02T06:12:42.880Z

Here's the NYTimes story: Hackers With Apparent Investment Banking Background Target Biotech

You should be able to download a copy of the report from the FireEye website here. Alternatively, you can request a free copy of the FireEye report here by pretending to be a company (for example, entering "no company" in the "company" field). There may be a time delay in between requesting and receiving the report.

Luckily for all of us, just because the hackers, referred to as FIN4, had financial motivations (getting "inside information about impending market catalysts") did not mean that they attempted to maximize their financial gain. If they had, this could would have been on the front page instead of in the technology section, and the headline could have been "Terrorists Hired Hackers to Manufacture Synthetic Disease," or alternately "Hacker Group Threatens to Release Synthetic Plague if Demands Not Met."

I sincerely hope that, if artificial gene synthesis devices were not kept air-gapped before, that they will be now. If hackers were able to compromise the email accounts of researchers and scientists (listed separately in the report for some reason), and artificial gene synthesis devices took requests from authorized users by internet, then these hackers could have ordered genes synthesized.
[link] The Philosophy of Intelligence Explosions and Advanced Robotics
by Kaj_Sotala — 2014-12-02T03:44:12.745Z

The philosopher John Danaher has posted a list of all the posts that he's written on the topic of robotics and AI. Below is the current version of the list: he says that he will keep updating the page as he writes more.

The Singularity: Overview and Framework: This was my first attempt to provide a general overview and framework for understanding the debate about the technological singularity. I suggested that the debate could be organised around three main theses: (i) the explosion thesis -- which claims that there will be an intelligence explosion; (ii) the unfriendliness thesis -- which claims that an advanced artificial intelligence is likely to be "unfriendly"; and (iii) the inevitability thesis -- which claims that the creation of an unfriendly AI will be difficult to avoid, if not inevitable.

The Singularity: Overview and Framework Redux: This was my second attempt to provide a general overview and framework for understanding the debate about the technological singularity. I tried to reduce the framework down to two main theses: (i) the explosion thesis and (ii) the unfriendliness thesis.

The Golem Genie and Unfriendly AI (Part One, Part Two): This two-parter summarises what I think is the best argument for the unfriendliness thesis. The argument was originally presented by Muehlhauser and Helm, but I try to simplify its main components.

AIs and the Decisive Advantage Thesis: Many people claim that an advanced artificial intelligence would have decisive advantages over human intelligences. Is this right? In this post, I look at Kaj Sotala's argument to that effect.

Is there a case for robot slaves? - If robots can be persons -- in the morally thick sense of "person" -- then surely it would be wrong to make them cater to our every whim? Or would it? Steve Petersen argues that the creation of robot slaves might be morally permissible. In this post, I look at what he has to say.

The Ethics of Robot Sex: A reasonably self-explanatory title. This post looks at the ethical issues that might arise from the creation of sex robots.

Will sex workers be replaced by robots? A Precis: A short summary of a longer article examining the possibility of sex workers being replaced by robots. Contrary to the work of others, I suggest that sex work might be resilient to the phenomenon of technological unemployment.

Bostrom on Superintelligence (1) The Orth

...
Superintelligence 12: Malignant failure modes
by KatjaGrace — 2014-12-02T02:02:24.576Z

This is part of a weekly reading group on Nick Bostrom's book, Superintelligence. For more information about the group, and an index of posts so far see the announcement post. For the schedule of future topics, see MIRI's reading guide.

Welcome. This week we discuss the twelfth section in the reading guide: Malignant failure modes.

This post summarizes the section, and offers a few relevant notes, and ideas for further investigation. Some of my own thoughts and questions for discussion are in the comments.

There is no need to proceed in order through this post, or to look at everything. Feel free to jump straight to the discussion. Where applicable and I remember, page numbers indicate the rough part of the chapter that is most related (not necessarily that the chapter is being cited for the specific claim).

Reading: 'Malignant failure modes' from Chapter 8

Summary

Malignant failure mode: a failure that involves human extinction; in contrast with many failure modes where the AI doesn't do much.

Features of malignant failures

We don't get a second try

It supposes we have a great deal of success, i.e. enough to make an unprecedentedly competent agent

Some malignant failures:

Perverse instantiation: the AI does what you ask, but what you ask turns out to be most satisfiable in unforeseen and destructive ways.

Example: you ask the AI to make people smile, and it intervenes on their facial muscles or neurochemicals, instead of via their happiness, and in particular via the bits of the world that usually make them happy.

Possible counterargument: if it's so smart, won't it know what we meant? Answer: Yes, it knows, but it's goal is to make you smile, not to do what you meant when you programmed that goal.

AI which can manipulate its own mind easily is at risk of 'wireheading' - that is, a goal of maximizing a reward signal might be perversely instantiated by just manipulating the signal directly. In general, animals can be motivated to do outside things to achieve internal states, however AI with sufficient access to internal state can do this more easily by manipulating internal state.

Even if we think a goal looks good, we should fear it has perverse instantiations that we haven't appreciated.

Infrastructure profusion: in pursuit of some goal, an AI redirects most resources to infrastructure

...
December 2014 Monthly Bragging Thread
by elharo — 2014-12-01T23:32:31.543Z

Your job, should you choose to accept it, is to comment on this thread explaining the most awesome thing you've done this month. You may be as blatantly proud of yourself as you feel. You may unabashedly consider yourself the coolest freaking person ever because of that awesome thing you're dying to tell everyone about. This is the place to do just that.

Remember, however, that this isn't any kind of progress thread. Nor is it any kind of proposal thread. This thread is solely for people to talk about the awesome things they have done. Not "will do". Not "are working on". Have already done. This is to cultivate an environment of object level productivity rather than meta-productivity methods.

So, what's the coolest thing you've done this month?
The new GiveWell recommendations are out: here's a summary of the charities
by tog — 2014-12-01T21:20:23.819Z

GiveWell have just announced their latest charity recommendations! What are everyone’s thoughts on them?

A summary: all of the old charities (GiveDirectly, SCI and Deworm the World) remain on the list. They're rejoined by AMF, as the room for more funding issues that led to it being delisted have been resolved to GiveWell's satisfaction. Together these organisations form GiveWell's list of 'top charities', which is now joined by a list of other charities which they see as excellent but not quite in the top tier. The charities on this list are Development Media International, Living Goods, and two salt fortification programs (run by GAIN and ICCIDD).

As normal, GiveWell's site contains extremely detailed writeups on these organisations. Here are some shorter descriptions which I wrote for Charity Science's donations page and my tool for donating tax-efficiently, starting with the new entries:

GiveWell's newly-added charities

Boost health and cognitive development with salt fortification

The charities GAIN and ICCIDD run programs that fortify the salt that millions of poor people eat with iodine. There is strong evidence that this boosts their health and cognitive development; iodine deficiency causes pervasive mental impairment, as well as stillbirth and congenital abnormalities such as severe retardation. It can be done very cheaply on a mass scale, so is highly cost-effective. GAIN is registered in the US and ICCIDD in Canada (although Canadians can give to either via Charity Science, which for complex reasons helps others who donate tax-deductibly to other charities), allowing for especially efficient donations from these countries, and taxpayers from other countries can also often give to them tax-deductibly. For more information, read GiveWell's detailed reviews of GAIN and ICCIDD.

Educate millions in life-saving practices with Development Media International

Development Media International (DMI) produces radio and television broadcasts in developing countries that tell people about improved health practices that can save lives, especially those of young children. Examples of such practices include exclusive breastfeeding. DMI are conducting a randomized controlled trial of their program which has found promising indications of a large decrease in children's deaths. With more funds they would be able to reach millions of people, due to the unparallel...
Group Rationality Diary, December 1-15
by therufs — 2014-12-01T19:44:37.494Z

This is the public group rationality diary for December 1-15.

It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Thanks to cata for starting the Group Rationality Diary posts, and to commenters for participating.

Previous diary: November 16-30

Next diary: December 16-31

Rationality diaries archive