trentmkelly/LessWrong-43k huggingface dataset:
Be careful with thought experiments
by lukeprog — 2012-05-18T09:54:54.341Z

Thagard (2012) contains a nicely compact passage on thought experiments:

Grisdale’s (2010) discussion of modern conceptions of water refutes a highly influential thought experiment that the meaning of water is largely a matter of reference to the world rather than mental representation. Putnam (1975) invited people to consider a planet, Twin Earth, that is a near duplicate of our own. The only difference is that on Twin Earth water is a more complicated substance XYZ rather than H2O. Water on Twin Earth is imagined to be indistinguishable from H2O, so people have the same mental representation of it. Nevertheless, according to Putnam, the meaning of the concept water on Twin Earth is different because it refers to XYZ rather than H2O. Putnam’s famous conclusion is that “meaning just ain’t in the head.”

The apparent conceivability of Twin Earth as identical to Earth except for the different constitution of water depends on ignorance of chemistry. As Grisdale (2010) documents, even a slight change in the chemical constitution of water produces dramatic changes in its effects. If normal hydrogen is replaced by different isotopes, deuterium or tritium, the water molecule markedly changes its chemical properties. Life would be impossible if H2O were replaced by heavy water, D2O or T2O; and compounds made of elements different from hydrogen and oxygen would be even more different in their properties. Hence Putnam’s thought experiment is scientifically incoherent: If water were not H2O, Twin Earth would not be at all like Earth. [See also Universal Fire. --Luke]

This incoherence should serve as a warning to philosophers who try to base theories on thought experiments, a practice I have criticized in relation to concepts of mind (Thagard, 2010a, ch. 2). Some philosophers have thought that the nonmaterial nature of consciousness is shown by their ability to imagine beings (zombies) who are physically just like people but who lack consciousness. It is entirely likely, however, that once the brain mechanisms that produce consciousness are better understood, it will become clear that zombies are as fanciful as Putnam’s XYZ. Just as imagining that water is XYZ is a sign only of ignorance of chemistry, imagining that consciousness is nonbiological may well turn out to reveal ignorance rather than some profound conceptual trut

...
[SEQ RERUN] Timeless Physics
by MinibearRex — 2012-05-18T03:48:21.204Z

Today's post, Timeless Physics was originally published on 27 May 2008. A summary (taken from the LW wiki):

What time is it? How do you know? The question "What time is it right now?" may make around as much sense as asking "Where is the universe?" Not only that, our physics equations may not need a t in them!

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Relative Configuration Space, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : DC Meetup
by rocurley — 2012-05-18T01:28:31.060Z

Discussion article for the meetup : DC Meetup

WHEN:
20 May 2012 03:00:00PM (-0400)

WHERE:
National Portrait Gallery, Washington, DC 20001, USA

If the SIAI response to the Givewell critique has been posted, that will be the meetup topic.

Discussion article for the meetup : DC Meetup
Holden's Objection 1: Friendliness is dangerous
by PhilGoetz — 2012-05-18T00:48:14.898Z

Nick_Beckstead asked me to link to posts I referred to in this comment. I should put up or shut up, so here's an attempt to give an organized overview of them.
Since I wrote these, LukeProg has begun tackling some related issues. He has accomplished the seemingly-impossible task of writing many long, substantive posts none of which I recall disagreeing with. And I have, irrationally, not read most of his posts. So he may have dealt with more of these same issues.
I think that I only raised Holden's "objection 2" in comments, which I couldn't easily dig up; and in a critique of a book chapter, which I emailed to LukeProg and did not post to LessWrong. So I'm only going to talk about "Objection 1: It seems to me that any AGI that was set to maximize a "Friendly" utility function would be extraordinarily dangerous." I've arranged my previous posts and comments on this point into categories. (Much of what I've said on the topic has been in comments on LessWrong and Overcoming Bias, and in email lists including SL4, and isn't here.)

The concept of "human values" cannot be defined in the way that FAI presupposes

Human errors, human values: Suppose all humans shared an identical set of values, preferences, and biases. We cannot retain human values without retaining human errors, because there is no principled distinction between them.
A comment on this post: There are at least three distinct levels of human values: The values an evolutionary agent holds that maximize their reproductive fitness, the values a society holds that maximizes its fitness, and the values a rational optimizer holds who has chosen to maximize social utility. They often conflict. Which of them are the real human values?
Values vs. parameters: Eliezer has suggested using human values, but without time discounting (= changing the time-discounting parameter). CEV presupposes that we can abstract human values and apply them in a different situation that has different parameters. But the parameters are values. There is no distinction between parameters and values.
A comment on "Incremental progress and the valley": The "values" that our brains try to maximize in the short run are designed to maximize different values for our bodies in the long run. Which are human values: ...
Imposing FAI
by asparisi — 2012-05-17T21:24:01.165Z

All the posts on FAI theory as of late have given me cause to think. There's something in the conversations about it that has always bugged me, but it is something that I haven't found the words for before now.

It is something like this:

Say that you manage to construct an algorithm for FAI...

Say that you can show that it isn't going to be a dangerous mistake...

And say you do all of this, and popularize it, before AGI is created (or at least, before an AGI goes *FOOM*)...

...

How in the name of Sagan are you actually going to ENFORCE the idea that all AGIs are FAIs?

I mean, if it required some rare material (like nuclear weapons) or large laboratories (like biological wmds) or some other resource that you could at least make artificially scarce, you could set up a body that ensures that any AGI created is an FAI.

But if all it is, is the right algorithms, the right code, and enough computing power... even if you design a theory for FAI, how would you keep someone from making UFAI anyway? Between people experimenting with the principles (once known), making mistakes, and the prospect of actively malicious *humans*... it just seems like unless you somehow come up with an internal mechanism that makes FAI better and stronger than any UFAI could be, and the solution turns out to be such that any idiot could see that it was a better solution... that UFAI is going to exist at some point no matter what.

At that point, it seems like the question becomes not "How do we make FAI?" (although that might be a secondary question) but rather "How do we prevent the creation of, eliminate, or reduce potential damage from UFAI?" Now, it seems like FAI might be one thing that you do toward that goal, but if UFAI is a highly likely consequence of AGI even *with* an FAI theory, shouldn't the focus be on how to contain a UFAI event?
[LINK] Relational models of Exchange
by Salemicus — 2012-05-17T19:19:30.118Z

The title may make it seem like it's mind-killing stuff (politics, religion, etc) but in fact it's about the different way people relate to one another. A lot of the things discussed in the article get mentioned here as "cognitive biases" but in fact they are a fundamental part of how people interact.

http://www.theatlantic.com/business/archive/2012/05/mitt-romney-one-night-stands-and-the-economics-of-relationships/257239/

For me, the highlight is when he talks about people approaching the same exchange with different relational models.
Short article on AI in UK online Wired
by Stuart_Armstrong — 2012-05-17T18:26:27.731Z

A short article, quoting me and Luke:

http://www.wired.co.uk/news/archive/2012-05/17/the-dangers-of-an-ai-smarter-than-us

It makes the point that it's not the shambling robots that are the risks here, but the other powers of intelligence.
Meetup : Moscow meetup: general rationality
by Yuu — 2012-05-17T17:08:43.367Z

Discussion article for the meetup : Moscow meetup: general rationality

WHEN:
20 May 2012 05:00:00PM (+0400)

WHERE:
Rossiya, Moscow, ulitsa Ostozhenka 14

We will meet at “Subway” restaurant, entrance from Lopukhinskiy pereulok. Look for a table with “LW” banner, I will be there from 17:00 MSK.

Topics will be:

General applications of rationality (in life)

Words meaning and usage

Fun part: discussion about examples from book and motion pictures, how can characters be more rational

Topics, proposed by you

Please let me know, can you come this Sunday, or not.

I also propose to organize next meetup on 9th of June, Saturday, at the same time. Feel free to comment you thoughts about this proposal.

Discussion article for the meetup : Moscow meetup: general rationality
Meetup : Small Berkeley Meetup
by Thausler — 2012-05-17T15:55:15.334Z

Discussion article for the meetup : Small Berkeley Meetup

WHEN:
23 May 2012 07:00:00PM (-0700)

WHERE:
2128 Oxford St, Berkeley, CA

This will be a small Berkeley meetup. We will be meeting at Oxford St Starbucks and then proceeding to a local restaurant for dinner.

Discussion article for the meetup : Small Berkeley Meetup
Being a Realist (even if you believe in God)
by scav — 2012-05-17T14:07:44.053Z

I have a small prediction. Some time before the end of the next paragraph you are going to form a strong opinion as to whether this article has any value to you. I'm going to be using the G word, appealing to emotion, and generally flinging around rhetoric not backed by any mathematics. If it helps, you can think of this as epistemic wheelchair access for those of us unable to leap tall equations at a single bound and enter the temple of pure reason by flying in through the upper windows.

According to a casual Google search I did, St Francis was said to have asked:

"My God, what art thou, and what am I?"

I am not going to be offering any answers to the first question. But for the purposes of this article I will be describing myself as a christian realist. What I mean by a realist is that I identify myself as someone who seeks to engage with reality - the universe as it really is. And by christian, I mean that I came to this desire by the rather circuitous route of becoming a Christian, and then getting deconverted from most of the associated dogma, while retaining many of the ethical heuristics and respect for the teachings attributed to Jesus.

Most readers can now sigh sadly, roll their eyes and go read something more interesting. But if you are still here, either having a belief in God that you are not willing to abandon, or you want to try to engage in rational discussions with those who do, what this article is about is that without requiring self-identification as a "rationalist", theists and untheists can and should meet half way and at least agree on the logical necessity of being a realist.

First of all, if God is real, then rejecting reality runs the risk of rejecting the real God in favour of a personal and therefore flawed delusion about Him. This is generally held to be a bad thing from a theological point of view, (BTW, I am using the conventional spelling of Him with the capital letter as a gender-irrelevant variable name rather than a male personal pronoun).

Secondly, I don't think it is honest or sane to insist on the reality of God but exempt any opinions from testing against reality just because they relate to God in some way. It's almost redundant to point out the atrocities that have been carried out on the pretext ...
[SEQ RERUN] Relative Configuration Space
by MinibearRex — 2012-05-17T05:43:57.161Z

Today's post, Relative Configuration Space was originally published on 26 May 2008. A summary (taken from the LW wiki):

Maybe the reason why we can't observe absolute speeds, absolute positions, absolute accelerations, or absolute rotations, is that particles don't have absolute positions - only positions relative to each other. That is, maybe quantum physics takes place in a relative configuration space.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Mach's Principle: Anti-Epiphenomenal Physics, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
How can we ensure that a Friendly AI team will be sane enough?
by Wei Dai — 2012-05-16T21:24:58.681Z

One possible answer to the argument "attempting to build FAI based on Eliezer's ideas seems infeasible and increases the risk of UFAI without helping much to increase the probability of a good outcome, and therefore we should try to achieve a positive Singularity by other means" is that it's too early to decide this. Even if our best current estimate is that trying to build such an FAI increases risk, there is still a reasonable chance that this estimate will turn out to be wrong after further investigation. Therefore, the counter-argument goes, we ought to mount a serious investigation into the feasibility and safety of Eliezer's design (as well as other possible FAI approaches), before deciding to either move forward or give up.

(I've been given to understand that this is a standard belief within SI, except possibly for Eliezer, which makes me wonder why nobody gave this counter-argument in response to my post linked above. ETA: Carl Shulman did subsequently give me a version of this argument here.)

This answer makes sense to me, except for the concern that even seriously investigating the feasibility of FAI is risky, if the team doing so isn't fully rational. For example they may be overconfident about their abilities and thereby overestimate the feasibility and safety, or commit sunken cost fallacy once they have developed lots of FAI-relevant theory in the attempt to study feasibility, or become too attached to their status and identity as FAI researchers, or some team members may disagree with a consensus of "give up" and leave to form their own AGI teams and take the dangerous knowledge developed with them.

So the question comes down to, how rational is such an FAI feasibility team likely to be, and is that enough for the benefits to exceed the costs? I don't have a lot of good ideas about how to answer this, but the question seems really important to bring up. I'm hoping this post this will trigger SI people to tell us their thoughts, and maybe other LWers have ideas they can share.
Off to Alice Springs
by Psy-Kosh — 2012-05-16T19:32:29.872Z

Am about to pack up computer then go to the airport to start a sequence of flights to give this a try.

I already have a room in a hostel booked for a few nights for when I get there, and will see how stuff goes.

Anyways, since there's been on and off discussion on this, just thought I'd post that I'm actually giving this a try.

(Will likely be a day or two before I can reply/comment/etc, given length of flights, etc.)

EDIT: Ugh. You take care of one aspect of the planning fallacy, and fail elsewhere. Long story short, I missed my flight and had to reschedule it to friday.

EDIT2: Packing up computer and going off to airport. Again. This time will be early.

EDIT3: And am here. and am exhausted. :) Will start looking for work stuff tomorrow. There's a job board at this hostel, but apparently there's not much currently. But right now am rather sleep deprived.

EDIT4: So today (Monday, May 21st) went to the visitor information center. I must have misunderstood the original article, was under the impression that the visitor center had job boards. Didn't, but pointed me to a nearby recruiting/contracting agency which they said might have appropriate stuff for visitors on a work&holiday visa. Went there. said that at least as of today there's nothing, but also needed a resume (which I didn't have with me, and my work experience is limited anyways.) Anyways, got a copy of the form, will dig out/fix up what resume I do have, and also keep looking. The board at this hostel didn't have much of anything in the way of work that I saw. Will look again, though, and see if I can find others.

EDIT5 (May 29th): Still looking for work, been asking/applying to various places, including that recruiting/contracting agency, and am right now waiting (well, and still looking.) Over the weekend, though, MileyCyrus and I went on an organized 3day Uluru/Kata Tjuta/Kings Canyon trip/hikes, which was awesome. But again, as far as work, tossing out inquiries and stuff all over, trying to find out who's hiring at the moment.
[SEQ RERUN] Mach's Principle: Anti-Epiphenomenal Physics
by MinibearRex — 2012-05-16T18:59:40.734Z

Today's post, Mach's Principle: Anti-Epiphenomenal Physics was originally published on 24 May 2008. A summary (taken from the LW wiki):

Could you tell if the whole universe were shifted an inch to the left? Could you tell if the whole universe was traveling left at ten miles per hour? Could you tell if the whole universe was accelerating left at ten miles per hour? Could you tell if the whole universe was rotating?

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was My Childhood Role Model, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : Weekly Chicago Meetups Resume 5/26
by Unnamed — 2012-05-16T17:53:54.836Z

Discussion article for the meetup : Weekly Chicago Meetups Resume 5/26

WHEN:
26 May 2012 01:00:00PM (-0500)

WHERE:
360 N Michigan Ave, Chicago IL

Because of all of the craziness happening in downtown Chicago, with the NATO meeting and associated protests, the 5/19 Chicago meetup is cancelled.

We will continue with our regularly scheduled weekly meetups the following Saturday, 5/26, 1pm at the Corner Bakery (Michigan & Wacker).

Join the mailing list to stay up-to-date on Chicago meetup plans.

Discussion article for the meetup : Weekly Chicago Meetups Resume 5/26
Meetup : Vancouver Minicamp Report-Back
by  — 2012-05-16T16:47:23.801Z

Discussion article for the meetup : Vancouver Minicamp Report-Back

WHEN:
18 May 2012 05:00:00PM (-0700)

WHERE:
Vancouver

Minicamp was totally awesome and I learned so much. I think I can put it best like this: I no longer have "a sense that more is possible" it is now "More is demonstratably possible. Period."

Rationalist communities can be so much better than I previously knew, and at this meetup I want to discuss how we can get from here to there. The bar has been raised and I have become correspondingly more ambitious.

So if you are interested in what minicamp was like, come out. If you want to be part of an awesome rationalist community, come out. If 2+2=4, come out.

(This technically isn't our big public meetup, so if you have for some odd reason commited to only coming to big public meetups, this isn't it.)

Also we are going to re-run Politics is the Mind Killer, as we didn't have attendance while I was away. You don't have to read it, some of us will know it well enough to present the ideas.

This week the meetup is on Friday at 17:00 (SCHEDULE CHANGE!!!). This is unconfirmed an may be changed. Details and address are on the list See you all there!

Discussion article for the meetup : Vancouver Minicamp Report-Back
Tools versus agents
by Stuart_Armstrong — 2012-05-16T13:00:04.336Z

In his critique of the Singularity Institute, Holden Karnofsky presented a distinction between an AI functioning as a tool versus one functioning as an agent. In his words, a tool AI would

(1) Calculate which action A would maximize parameter P, based on existing data set D. (2) Summarize this calculation in a user-friendly manner, including what Action A is, what likely intermediate outcomes it would cause, what other actions would result in high values of P, etc.

In contrast, an agent AI would:

(1) Calculate which action, A, would maximize parameter P, based on existing data set D. (2) Execute Action A.

The idea being that an AI, asked to "prevent human suffering", would come up with two plans:

Kill all human.

Cure all diseases, make everyone young and immortal.

Then the agent AI would go out and kill everyone, while the tool AI would give us the list and we would pick the second one. In the following, I'll assume the AI is superintelligent, and has no other objectives than what we give it.

Long lists

Of course, we're unlikely to get a clear two element list. More likely we'd get something like:

Kill all humans with engineered plagues.

Kill all humans with nukes.

Kill all humans with nanobots.

Kill all humans with...

...

...

Lobotomise all humans with engineered plagues.

Lobotomise all humans with surgery.

Lobotomise all humans with...

...

...

Kill some humans, lobotomise others, cure still others.

...

The nice solutions might not even appear on the list. Of course, this is still very worthwhile information! This allows us to go into the tool AI, and rewire it again, so that it gets our meanings more accurately. Maybe after a few iterations, we'll have refined the AIs understanding of what we want, and we'll get a nice implementable solution near the top. Of course, this presupposes that we understand the options, and that it's safe for us to read the list.

Understanding the options

The key, and difficult requirement is that the AI "summarize this calculation in a user-friendly manner". The most efficient action won't be "kill all humans"; it will instead be "implement this algorithm, fund that research lab, send this email to this politician..." In fact, it'll probably be "type this sequence of commands..."

So if we're to judge the relative merit of the plans, we really are dependent on the tool AI's summary skills. So the AI needs to ha...