trentmkelly/LessWrong-43k huggingface dataset:
Evaluating the feasibility of SI's plan
by JoshuaFox — 2013-01-10T08:17:29.959Z

(With Kaj Sotala)

SI's current R&D plan seems to go as follows:

1. Develop the perfect theory.
2. Implement this as a safe, working, Artificial General Intelligence -- and do so before anyone else builds an AGI.

The Singularity Institute is almost the only group working on friendliness theory (although with very few researchers). So, they have the lead on Friendliness. But there is no reason to think that they will be ahead of anyone else on the implementation.

The few AGI designs we can look at today, like OpenCog, are big, messy systems which intentionally attempt to exploit various cognitive dynamics that might combine in unexpected and unanticipated ways, and which have various human-like drives rather than the sort of supergoal-driven, utility-maximizing goal hierarchies that Eliezer talks about, or which a mathematical abstraction like AIXI employs.

A team which is ready to adopt a variety of imperfect heuristic techniques will have a decisive lead on approaches based on pure theory. Without the constraint of safety, one of them will beat SI in the race to AGI. SI cannot ignore this. Real-world, imperfect, safety measures for real-world, imperfect AGIs are needed. These may involve mechanisms for ensuring that we can avoid undesirable dynamics in heuristic systems, or AI-boxing toolkits usable in the pre-explosion stage, or something else entirely.

SI’s hoped-for theory will include a reflexively consistent decision theory, something like a greatly refined Timeless Decision Theory. It will also describe human value as formally as possible, or at least describe a way to pin it down precisely, something like an improved Coherent Extrapolated Volition.

The hoped-for theory is intended to provide not only safety features, but also a description of the implementation, as some sort of ideal Bayesian mechanism, a theoretically perfect intelligence.

SIers have said to me that SI's design will have a decisive implementation advantage. The idea is that because strap-on safety can’t work, Friendliness research necessarily involves more fundamental architectural design decisions, which also happen to be general AGI design decisions that some other AGI builder could grab and save themselves a lot of effort. The assumption seems to be that all other designs are based on hopelessly misguided desi...
[SEQ RERUN] Imaginary Positions
by MinibearRex — 2013-01-10T04:58:55.375Z

Today's post, Imaginary Positions was originally published on 23 December 2008. A summary (taken from the LW wiki):

People who are not members of a minority group may somehow come to believe that members of this group possess certain traits which seem to "fit". These traits are not required to have any connection to the real traits of that group.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Living By Your Own Strength, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Group rationality diary, 1/9/13
by cata — 2013-01-10T02:22:31.232Z

This is the public group instrumental rationality diary for the week of January 7th. It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Thanks to everyone who contributes! Happy New Year to folks; my resolution is to always post these on Monday evenings instead of letting them slip to Tuesday or Wednesday : >

Previous diary; archive of prior diaries.
Post-college: changing nature of friend interactions
by calcsam — 2013-01-09T22:26:00.032Z

As a working professional a couple of years out college, I’ve been noticing how interactions with my friends has changed since the beginning of college – and especially since graduation.

In college, my social groups typically formed around groups with common meeting places -- freshman dorm, newspaper, church, “draw group” (essentially a group of friends that ‘draw’ into the same dorm).

Because there was a common space where everyone could hang out, everyone else felt comfortable just showing up (at least at designated times), and so there were always people to talk to. No-permission-required-meeting was a self-sustaining norm.

With jobs and schedules, we shift to a permission-required-meeting-situation – you don’t just show up at your friend’s house, we say “Hey, what’s a good time to meet up?”

This adds an additional barrier to meeting, and so that happen less often.

People usually realize this at some level, and employ a variety of ad-hoc strategies to counteract this. These are usually well-deployed in our professional lives, but in our personal lives, there are some complications, and usually room for improvement.

Group meetings. There are 10 connections between five people, as opposed to one connection between two people. But generally – assuming people share fairly common schedules – it will take less than 10x initiative to get five people together as two

Disadvantage: Often most of our close friends don’t form groups. Only a small subset of mine does.

Non-face-to-face communication. Christmas cards are a time-honored way of doing this. E-mail, like mail, is a no-permission-required system. Every year, I send out a general Life Update email to my old and current friends and family. My friends and I more frequently email each other interesting links. When I read something cool online, I often think “who could I send this to?”

Disadvantage: for most people, compared to face-to-face interaction, it’s not the same.

Scheduling regular meetings: I live in CA and my girlfriend lives in NY, so for the last five months we have set aside 10am PST / 1pm EST to talk every weekday. For the last 8 months, I have met my friend Caleb* have weekly 1-to-2 hour meetings on Sunday mornings where we discuss how the last week went and make go

...
Meetup : Washington DC measurement meetup
by rocurley — 2013-01-09T19:06:18.699Z

Discussion article for the meetup : Washington DC measurement meetup

WHEN:
13 January 2013 03:00:00PM (-0500)

WHERE:
Washington DC

Instead of the courtyard, we will be meeting at someone's house: please PM me, maia or PhilipL, and we'll send you the address. If you're subscribed to the mailing list (of course you are!) then you could also just look at it.
It's very close to the Wheaton metro station.
We will be doing measurement-based calibration games. That is to say, we'll be guessing the physical (or as Robin Hanson suggested, economic) properties of objects with confidence intervals, and then testing how well calibrated we are. If you have objects with properties (you probably do) and can bring them, please do so so we can guess at them. In particular, dense or (sparse?) objects would be interesting, as well as items that you know the price of.

Discussion article for the meetup : Washington DC measurement meetup
False vacuum: the universe playing quantum suicide
by Stuart_Armstrong — 2013-01-09T17:04:23.741Z

Imagine that the universe is approximately as it appears to be (I know, this is a controversial proposition, but bear with me!). Further imagine that the many worlds interpretation of Quantum mechanics is true (I'm really moving out of Less Wrong's comfort zone here, aren't I?).

Now assume that our universe is in a situation of false vacuum - the universe is not in its lowest energy configuration. Somewhere, at some point, our universe may tunnel into true vacuum, resulting in a expanding bubble of destruction that will eat the entire universe at high speed, destroying all matter and life. In many worlds, such a collapse need not be terminal: life could go one on a branch of lower measure. In fact, anthropically, life will go on somewhere, no matter how unstable the false vacuum is.

So now assume that the false vacuum we're in is highly unstable - the measure of the branch in which our universe survives goes down by a factor of a trillion every second. We only exist because we're in the branch of measure a trillionth of a trillionth of a trillionth of... all the way back to the Big Bang.

None of these assumptions make any difference to what we'd expect to see observationally: only a good enough theory can say that they're right or wrong. You may notice that this setup transforms the whole universe into a quantum suicide situation.

The question is, how do you go about maximising expected utility in this situation? I can think of a few different approaches:

Gnaw on the bullet: take the quantum measure as a probability. This means that you now have a discount factor of a trillion every second. You have to rush out and get/do all the good stuff as fast as possible: a delay of a second costs you a reduction in utility of a trillion. If you are a negative utilitarian, you also have to rush to minimise the bad stuff, but you can also take comfort in the fact that the potential for negative utility across the universe is going down fast.

Use relative measures: care about the relative proportion of good worlds versus bad worlds, while assigning zero to those worlds where the vacuum has collapsed. This requires a natural zero to make sense, and can be seen as quite arbitrary: what would you do about entangled worlds, or about the non-zero probability that the vacuum-collapsed worlds may have worthwhile life in them? Would the relative measure user also put

...
Course recommendations for Friendliness researchers
by Louie — 2013-01-09T14:33:50.300Z

When I first learned about Friendly AI, I assumed it was mostly a programming problem. As it turns out, it's actually mostly a math problem. That's because most of the theory behind self-reference, decision theory, and general AI techniques haven't been formalized and solved yet. Thus, when people ask me what they should study in order to work on Friendliness theory, I say "Go study math and theoretical computer science."

But that's not specific enough. Should aspiring Friendliness researchers study continuous or discrete math? Imperative or functional programming? Topology? Linear algebra? Ring theory?

I do, in fact, have specific recommendations for which subjects Friendliness researchers should study. And so I worked with a few of my best interns at MIRI to provide recommendations below:

University courses. We carefully hand-picked courses on these subjects from four leading universities — but we aren't omniscient! If you're at one of these schools and can give us feedback on the exact courses we've recommended, please do so.

Online courses. We also linked to online courses, for the majority of you who aren't able to attend one of the four universities whose course catalogs we dug into. Feedback on these online courses is also welcome; we've only taken a few of them.

Textbooks. We have read nearly all the textbooks recommended below, along with many of their competitors. If you're a strongly motivated autodidact, you could learn these subjects by diving into the books on your own and doing the exercises.

Have you already taken most of the subjects below? If so, and you're interested in Friendliness research, then you should definitely contact me or our project manager Malo Bourgon (malo@intelligence.org). You might not feel all that special when you're in a top-notch math program surrounded by people who are as smart or smarter than you are, but here's the deal: we rarely get contacted by aspiring Friendliness researchers who are familiar with most of the material below. If you are, then you are special and we want to talk to you.

Not everyone cares about Friendly AI, and not everyone who cares about Friendly AI should be a researcher. But if you do care and you might want to help with Friendliness research one day, we recommend you consume the subjects below. Please contact m...
DRAFT:Ethical Zombies - A Post On Reality-Fluid
by MugaSofer — 2013-01-09T13:38:02.754Z

I came up with this after watching a science fiction film, which shall remain nameless due to spoilers, where the protagonist is briefly in a similar situation to the scenario at the end. I'm not sure how original it is, but I certainly don't recall seeing anything like it before.

Imagine, for simplicity, a purely selfish agent. Call it Alice. Alice is an expected utility maximizer, and she gains utility from eating cakes. Omega appears and offers her a deal - they will flip a fair coin, and give Alice three cakes if it comes up heads. If it comes up tails, they will take one cake away her stockpile. Alice runs the numbers, determines that the expected utility is positive, and accepts the deal. Just another day in the life of a perfectly truthful superintelligence offering inexplicable choices.

The next day, Omega returns. This time, they offer a slightly different deal - instead of flipping a coin, they will perfectly simulate Alice once. This copy will live out her life just as she would have done in reality - except that she will be given three cakes. The original Alice, however, receives nothing. She reasons that this is equivalent to the last deal, and accepts.

(If you disagree, consider the time between Omega starting the simulation and providing the cake. What subjective odds should she give for receiving cake?)

Imagine a second agent, Bob, who gets utility from Alice getting utility. One day, Omega show up and offers to flip a fair coin. If it comes up heads, they will give Alice - who knows nothing of this - three cakes. If it comes up tails, they will take one cake from her stockpile. He reasons as Alice did an accepts.

Guess what? The next day, Omega returns, offering to simulate Alice and give her you-know-what (hint: it's cakes.) Bob reasons just as Alice did in the second paragraph there and accepts the bargain.

Humans value each other's utility. Most notably, we value our lives, and we value each other not being tortured. If we simulate someone a billion times, and switch off one simulation, this is equivalent to risking their life at odds of 1:1,000,000,000. If we simulate someone and torture one of the simulations, this is equivalent to risking a one-in-a-billion chance of them being tortured. Such risks are often acceptable, if enough utility is gained by success. We...
Meetup : Melbourne Social Meetup
by Maelin — 2013-01-09T12:27:13.277Z

Discussion article for the meetup : Melbourne Social Meetup

WHEN:
18 January 2013 07:00:00PM (+1100)

WHERE:
See mailing list, Carlton VIC 3053

Melbourne's next regular social meetup will be held on Friday 18th January at the usual venue in Carlton. All are welcome from 6:30pm for a 7:00pm official start, but don't stress about being on time.

Our social meetups are informal events held on the third Friday of each month, where we lounge about playing boardgames and chatting, with occasional group parlour games such as Mafia/Werewolf or Resistance if people are interested. If you haven't been to a Melbourne meetup before/recently, the social meetup can be less intimidating way to meet us as it's very informal.

Some snacks will be provided and we'll probably arrange some form of delivered food for dinner. BYO drinks and games.

For the location/any questions, please see the Melbourne Less Wrong google group, or feel free to SMS me on 0421 231 789.

Discussion article for the meetup : Melbourne Social Meetup
Meetup : Moscow: Applied Rationality
by Yuu — 2013-01-09T07:42:17.345Z

Discussion article for the meetup : Moscow: Applied Rationality

WHEN:
19 January 2013 04:00:00PM (+0400)

WHERE:
Russia, Moscow, ulitsa L'va Tolstogo 16

Our venue is changed:

We will meet in the Yandex Money office, this is the second door at the address I mentioned. Please use the following guide to get there: link. I will be there at 16:00 MSK. And I will also check the entrance at 16:15 and 16:30, so please do not be late.

Main topics:

Applied rationality: practice. The next one will be calibration session.

Cognitive biases analysis. Here is the link to the list of biases we will work on (in Russian).

Solving cases. Please prepare some problems for discussion. You can contact me to propose your problems and cases in advance.

If you are going for the first time, you can fill this one minute form (in Russian), to share your contact information. You can also use personal messages here, or drop a message at lw@lesswrong.ru to contact me for any reason. Please also contact me, if you are not sure you can be in time and I provide you with an emergency phone number.

Discussion article for the meetup : Moscow: Applied Rationality
[SEQ RERUN] Living By Your Own Strength
by MinibearRex — 2013-01-09T06:42:35.003Z

Today's post, Living By Your Own Strength was originally published on 22 December 2008. A summary (taken from the LW wiki):

Our hunter-gatherer ancestors strung their own bows, wove their own baskets and whittled their own flutes. Part of our alienation from our design environment is the number of tools we use that we don't understand and couldn't make for ourselves. It's much less fun to read something in a book than to discover it for yourself. Specialization is critical to our current civilization. But the future does not have to be a continuation of this trend in which we rely more and more on things outside ourselves which become less and less comprehensible. With a surplus of power, you could begin to rethink the life experience as a road to internalizing new strengths, not just staying alive efficiently through extreme specialization.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Sensual Experience, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : Berkeley meetup: Board games!
by Nisan — 2013-01-08T20:40:42.392Z

Discussion article for the meetup : Berkeley meetup: Board games!

WHEN:
09 January 2013 07:00:00PM (-0800)

WHERE:
Berkeley, CA

Hello, all! Zendo will host its first Less Wrong meetup of the year on Wednesday at 7pm. Come and play a game from Zendo's library of board games, or bring a board game that you particularly want to play. I look forward to seeing everyone again. If anyone is interested I will share stories about the Solstice Eve ritual that Raemon from Less Wrong put together.

For directions to Zendo, see the mailing list:

http://groups.google.com/group/bayarealesswrong

or call me at:

http://i.imgur.com/Vcafy.png

Discussion article for the meetup : Berkeley meetup: Board games!
[Link] Your Elusive Future Self
by Richard_Kennaway — 2013-01-08T12:27:03.046Z

A current article in Science reports on this study about how good people are at predicting what their future selves will be like. Not very good, apparently. Daniel Gilbert, a psychologist at Harvard, with other colleagues conducted several experiments online, in which 19,000 people were asked about such things as personality traits, preferences in music, etc., answering about the present, about themselves 10 years earlier, and about what they expected 10 years hence. More precisely, this not being a longitudinal study, people of any age X predicted less difference with their X+10 selves than people of age X+10 recollected of themselves at age X. The effect did not go away with increasing age: 58-year-olds still expected less change in the next 10 years than 68-year-olds reported in the last ten.

Gilbert and colleagues call this effect "the end of history illusion," because it suggests that people believe, consciously or not, that the present marks the point at which they've finally stopped changing.

"What these data suggest, and what scads of other data from our lab and others suggest, is that people really aren't very good at knowing who they're going to be and hence what they're going to want a decade from now," Gilbert says.

Someone suggests an alternative explanation:

Another possibility is that people "might well anticipate substantial change, yet not know how they would change, and thus, just predict the status quo"

An actionable moral:

"The single best way to make predictions about what you're going to want in the future isn't to imagine yourself in the future, … it's to look at other people who are in the very future you're imagining," [Gilbert] says.
[SEQ RERUN] Sensual Experience
by MinibearRex — 2013-01-08T06:30:44.942Z

Today's post, Sensual Experience was originally published on 21 December 2008. A summary (taken from the LW wiki):

Much of the anomie and disconnect in modern society can be attributed to our spending all day on tasks (like office work) that we didn't evolve to perform (unlike hunting and gathering on the savanna). Thus, many of the tasks we perform all day do not engage our senses - even the most realistic modern video game is not the same level of sensual experience as outrunning a real tiger on the real savanna. Even the best modern video game is low-bandwidth fun - a low-bandwidth connection to a relatively simple challenge, which doesn't fill our brains well as a result. But future entities could have different senses and higher-bandwidth connections to more complicated challenges, even if those challenges didn't exist on the savanna.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Complex Novelty, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.