trentmkelly/LessWrong-43k huggingface dataset:
Rational subjects and rational practitioners
by Richard_Kennaway — 2012-12-11T14:40:17.527Z

Half-closing my eyes and looking at the recent topic of morality from a distance, I am struck by the following trend.

In mathematics, there are no substantial controversies. (I am speaking of the present era in mathematics, since around the early 20th century. There were some before then, before it had been clearly worked out what was a proof and what was not.) There are few in physics, chemistry, molecular biology, astronomy. There are some but they are not the bulk of any of these subjects. Look at biology more generally, history, psychology, sociology, and controversy is a larger and larger part of the practice, in proportion to the distance of the subject from the possibility of reasonably conclusive experiments. Finally, politics and morality consist of nothing but controversy and always have done.

Curiously, participants in discussions of all of these subjects seem equally confident, regardless of the field's distance from experimental acquisition of reliable knowledge. What correlates with distance from objective knowledge is not uncertainty, but controversy. Across these fields (not necessarily within them), opinions are firmly held, independently of how well they can be supported. They are firmly defended and attacked in inverse proportion to that support. The less information there is about actual facts, the more scope there is for continuing the fight instead of changing one's mind. (So much for the Aumann agreement of Bayesian rationalists.)

Perhaps mathematicians and hard scientists are not more rational than others, but work in fields where it is easier to be rational. When they turn into crackpots outside their discipline, they were actually that irrational already, but have wandered into an area without safety rails.
Group rationality diary, 12/10/12
by cata — 2012-12-11T11:50:26.990Z

This is the public group instrumental rationality diary for the week of December 10th. It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Thanks to everyone who contributes!

Previous diary; archive of prior diaries.
Meetup : Moscow: Applied Rationality
by Yuu — 2012-12-11T09:14:00.685Z

Discussion article for the meetup : Moscow: Applied Rationality

WHEN:
22 December 2012 04:00:00PM (+0400)

WHERE:
Rossiya, Moscow, ulitsa Ostozhenka 14/2

We will meet at “Subway” restaurant, entrance from Lopukhinskiy pereulok. Look for a table with “LW” banner, I will be there from 16:00 MSK.

Main topics:

Applied rationality: practice. We will improve our rationality skills.

Solving cases. You can propose any problem and together we will propose a solution using our rationality skills.

Cognitive biases analysis. Here is the link to the list of biases we will work on (in Russian).

If you are going for the first time, please fill this one minute form (in Russian), to share your contact information. You can also use personal messages here, or drop a message at lw@lesswrong.ru to contact me for any reason. Please use the same from to propose the date for the next meetup, if you can't come this time.

N. B. Google may show incorrect location, please use Yandex maps.

Discussion article for the meetup : Moscow: Applied Rationality
[SEQ RERUN] Evolved Desires
by MinibearRex — 2012-12-11T02:21:42.006Z

Today's post, Evolved Desires was originally published on December 5, 2008. A summary:

How much can evolutionary biology calculations tell us about how humans will consider competitive scenarios, and therefore help us predict whether or not a singleton will occur.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Beware Hockey Stick Plans, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Call for a Friendly AI channel on freenode
by Mitchell_Porter — 2012-12-10T23:27:08.618Z

I visited #lesswrong on freenode yesterday and was able to get in some discussion of FAI-related matters. But that channel also exists to allow discussion of rationalist fanfiction, political opinions, and whatever else people want to talk about.
I would like for there to be a place on that network where the topic actually is Friendly AI - where you can go to brainstorm, and maybe you'll have to wait because they're already talking about cognitive neuroscience or automated theorem provers, but not because they're talking about ponies or politics.
Surely there are enough people with a serious, technical interest in FAI and related topics (and I don't just mean among LW regulars) to make such a channel sustainable. I'll bet that there are other people holding back from participation precisely because existing forums are so full of uninformed noise and conversational tangents. It's inevitable that entropy would set in after a while, but if the default baseline was still that even the chatter was technically informed and focused on what's coming - that would be mission accomplished.
I explored the freenode namespace a little. #FAI redirects to #unavailable, so it may be an abandoned project. #AGI exists but is invite-only. #AI exists but I'm told it's dull, and besides, the agenda here is meant to be, not just AI, but singularity-relevant AI. So there seems to be an opening. Or am I reinventing the wheel?
The challenges of bringing up AIs
by Stuart_Armstrong — 2012-12-10T12:43:24.333Z

At the current AGI-12 conference, some designers have been proponents of keeping AGI's safe by bringing them up in human environments, providing them with interactions and feedback in a similar way to how we bring up human children. Obviously that approach would fail for a fully smart AGI with its own values - it would pretend to follow our values for as long as it needed, and then defect. However, some people have confidence if we started with a limited, dumb AGI, then we could successfully inculcate our values in this way (a more sophisticated position would be that though this method would likely fail, it's more likely to succeed than a top-down friendliness project!).

The major criticism of this approach is that it anthropomorphises the AGI - we have a theory of children's minds, constructed by evolution, culture, and our own child-rearing experience. And then we project this on the alien mind of the AGI, assuming that if the AGI presents behaviours similar to a well-behaved child, then it will become a moral AGI. The problem is that we don't know how alien the AGI's mind will be, and if our reinforcement is actually reinforcing the right thing. Specifically, we need to be able to find some way of distinguishing between:

An AGI being trained to be friendly.

An AGI being trained to lie and conceal.

An AGI that will behave completely differently once out of the training/testing/trust-building environment.

An AGI that forms the wrong categories and generalisations (what counts as "human" or "suffering", for instance), because it lacks human-shared implicit knowledge that was "too obvious" for us to even think of training it on.
[LINK] Two Modes of Discourse: Taking everything personally v. debate as sport
by Vaniver — 2012-12-10T07:46:24.438Z

A blog post by Alistair Roberts, as curated by Steve Sailer. (Steve's version is shorter and more targeted; the original blog post is the fourth in a series on triggering and suffers for its reliance on the particular issue.)
It seems like a very useful dichotomy, and strongly reminds me of Ask and Guess.
Do I really not believe in God? Do you?
by Shmi — 2012-12-10T07:02:14.394Z

Well, I used to think that I do not believe in anything supernatural that affects what happens to me, but I'm wondering if maybe I actually do alieve in it. For example, a few days ago I had a close call in traffic, and when a collision I fully expected to happen just a second prior did not transpire, I mentally thanked... whom? I definitely had a clear feeling of gratitude for escaping, and I don't normally mean it literally when I say "Thank God!". So, who or what did I feel thankful to? I've never been religious, and I got rid of most of my superstitions over the years, but apparently there is still something there, and I do not know how to react to this knowledge.

What would be the proper reaction after a close call? Shrug and say "got lucky this time, should be more cautious next time"? What about when waiting for a diagnosis, what does sort-of-praying "please, please, let everything be OK" say about one's true beliefs? I know that I am much better at not blaming the world when something bad happens to me by chance than at not thanking the world when something good happens. Should it not be symmetric? Which part of a normally non-religious person wakes up and asserts itself in a crisis situation out of their control? Should it be embraced, suppressed, worked on?
Taking into account another's preferences
by SarahNibs — 2012-12-10T05:06:00.819Z

Question regarding: If you don't know the name of the game, just tell me what I mean to you

I've been thinking about what it means to prefer that someone else achieve their preferences. In particular, what happens if you and I both prefer to adopt and chase after each other's preferences to some extent. This has clear good points, like cooperating and making more resources more fungible and thus probably being more efficient and achieving more preferences overall, and clear failure modes, like "What do you want to do? -- I don't know, I want to do whatever you want to do. Repeat."

My first thought: okay, simple, I'll just define my utility function U' to be U + aV where U was my previous utility function and V is your utility function and a is an appropriate scaling factor as per Stuart's post and then I can follow U'!1

This has a couple problems. First, if you're also trying to change your actions based on what I want, there's a circular reference issue. Second, U already contains part of V by definition, or something2.

My second thought: Fine, first we'll both factor our preferences into U = U1 + aV where U1 is my preference without regards to what you want. (Yours is V = V1 + bU) Basically what I want to say is "What do you want to do, 'cause ignoring you I want burgers a bit more than Italian, which I want significantly more than sandwiches from home" and then you could say "well ignoring you I want sandwiches more than Italian more than burgers but it's not a big thing, so since you mean b to me, let's do Italian". It's that "ignoring you" bit that I don't know how to correctly intuit. And by intuit I mean put into math.

Assuming it means something coherent to factor U into U1 + aV, there's still a problem. Watch what happens when we remove the self-reference. First scale U and V to something you and I can agree is approximately fungible. Maybe marginal hours, maybe marginal dollars, whatever. Now U = U1 + a(V1 + bU), so U - abU = U1 + aV1 and as long as ab<1, you can maximize U by maximizing U1 + aV1. Which sounds great, except that my intuition screams that maximizing U should depend on b. So what's up there? My guess is that somewhere I snuck a dependence on b into a...

(I like that the ab<1 constraint appears... intuitively I think it should mean that if we both try to care too much about what the other person wants, neither...
New Sequences and General Update From Castify
by scott_from_castify — 2012-12-10T04:45:12.063Z

You may have noticed the announcement last week of Less Wrong's partnership with Castify to create Less Wrong podcasts. We're happy to announce that we now offer three new sequences. For those who want to try us out, we are also offering Eliezer's seminal essay, The Simple Truth, as a standalone 99¢ purchase (it's also part of the Map and Territory sequence).

Our channels:

The Simple Truth

Map and Territory

Ethical Injunctions

and our pilot sequence, Mysterious Answers to Mysterious Questions

We have audio samples on our site if you're curious.

In other news, we're still working to make the promoted posts available as a monthly subscription. We'll keep you up to date with the progress on that. It's still a few weeks away.

Side note: Because of user feedback, we have removed the one year download restriction for sequences purchased. However, we have added a clause in our terms of service that gives you a minimum of two years to download the audio, if only because living forever is hard, even for digital goods.

See you in the comments.
By Which It May Be Judged
by Eliezer Yudkowsky — 2012-12-10T04:26:23.835Z

Followup to: Mixed Reference: The Great Reductionist Project

Humans need fantasy to be human.

"Tooth fairies? Hogfathers? Little—"

Yes. As practice. You have to start out learning to believe the little lies.

"So we can believe the big ones?"

Yes. Justice. Mercy. Duty. That sort of thing.

"They're not the same at all!"

You think so? Then take the universe and grind it down to the finest powder and sieve it through the finest sieve and then show me one atom of justice, one molecule of mercy.

- Susan and Death, in Hogfather by Terry Pratchett

Suppose three people find a pie - that is, three people exactly simultaneously spot a pie which has been exogenously generated in unclaimed territory. Zaire wants the entire pie; Yancy thinks that 1/3 each is fair; and Xannon thinks that fair would be taking into equal account everyone's ideas about what is "fair".

I myself would say unhesitatingly that a third of the pie each, is fair. "Fairness", as an ethical concept, can get a lot more complicated in more elaborate contexts. But in this simple context, a lot of other things that "fairness" could depend on, like work inputs, have been eliminated or made constant. Assuming no relevant conditions other than those already stated, "fairness" simplifies to the mathematical procedure of splitting the pie into equal parts; and when this logical function is run over physical reality, it outputs "1/3 for Zaire, 1/3 for Yancy, 1/3 for Xannon".

Or to put it another way - just like we get "If Oswald hadn't shot Kennedy, nobody else would've" by running a logical function over a true causal model - similarly, we can get the hypothetical 'fair' situation, whether or not it actually happens, by running the physical starting scenario through a logical function that describes what a 'fair' outcome would look like:

So am I (as Zaire would claim) just assuming-by-authority that I get to have everything my way, since I'm not defining 'fairness' the way Zaire wants to define it?

No more than mathematicians are flatly ordering everyone to assume-without-proof that two different numbers can't have the same successor. For fairness to be what everyone thinks is "fair" would be entirely circular, structurally isomorphic to "Fzeem is what everyone thinks is fzeem"... or like trying to define the counting numbers a...
[SEQ RERUN] Beware Hockey Stick Plans
by MinibearRex — 2012-12-10T00:45:36.358Z

Today's post, Beware Hockey Stick Plans was originally published on December 4, 2008. A summary:

Most hockey stick style growth models have a problem, which is that they have almost never come true. It seems likely, therefore, that some part of the reasoning is long.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Underconstrained Abstractions, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Claiming Connotations
by Error — 2012-12-09T23:40:37.485Z

I was chasing links a few days ago and ran into Disputing Definitions from the sequences (again), and was thinking about the idea of having definition-argument participants expand their definition of a word to dissolve the dispute. And I thought, I've tried it a couple of times, and I don't think it has ever actually worked.

I think something else is going on here: People arguing about the "true" definition of a word are trying to lay social claim to its connotations by dictating its denotation.
Consider an atheist gay-rights supporter and a Christian gay-rights opponent. The former says "homosexuality is not immoral" the latter says "homosexuality is immoral."
Expand on "immoral." Perhaps the atheist considers an act immoral if it harms someone else without their consent, while the Christian considers an act immoral if it goes against the teachings of the Bible.
The expanded forms look like:
A: "Homosexuality does not harm anyone without their consent.
B: "Homosexuality goes against the teachings of the Bible.
The atheist is unlikely to disagree with B; Leviticus 18:22 is pretty straightforward. He won't *care* that homosexuality is against biblical teachings, but he won't disagree that it does. The Christian may disagree with A, but assume for the sake of argument that he doesn't.
It would seem the factual disagreement has been dissolved; they aren't actually contradicting each other, they are making *entirely orthogonal statements.*
Present the above analysis to both sides, though, and I suspect that instead of acknowledging the dissolution, they would fall to arguing over the *correct* definition of the *label* "immoral." Even after the expansion is presented. This has happened to me a couple of times.
Question: If they're not arguing about the biblical status or harm status of homosexuality, and they acknowledge that they mean entirely different things by the label "immoral," what are they actually contesting when they argue the proper denotation of that label?
This seems to me to be the reversed version of sneaking in connotations. For that someone applies a word to a case for which it is denotationally true but connotation-ally questionable, as when referring to Martin Luther King, Jr as a criminal.
Arging over definitions, though, strikes me as trying to *lay claim* to connotations rather than sneak them in. If you can dictate the denotation of a commo...
My experience as an Australian work-holiday maker
by MileyCyrus — 2012-12-09T22:09:21.984Z

I read Optimal Employment and decided to try a work-holiday in Australia. Several people have asked me how it's going, so here's my first take. I haven’t finished my work-holiday yet, but I want to provide this information now so that potential travelers can make a decision before the hiring season peaks next summer.

Alice Springs

I came to Alice Springs in late April, and secured a job at a fast-food immediately. They offered me $26/hr after super and penalties. I didn’t get many hours, but it was enough to cover expenses. I took the job so that I wouldn’t lose money while I was looking for a better job.

Finding better job proved tough. I quickly applied to all of the sit-down restaurants and bars in town, and they all said they preferred locals. I signed up for a couple of employment agencies, but they were concerned that I had didn’t have any experience serving alcohol. I borrowed a phonebook and called random business on the Plenty Highway and that didn’t work.

This continued for five weeks. I found this period stressful because I get anxious asking businesses if they’re hiring, and then I feel humiliated when they say they’re not. There were fun times: I toured Uluru and met cool people in the hostel. But in the back of my mind I was always stressed about finding a better job. After five weeks I was about to head south in search of fruit picking work. But then I saw a flyer.

Remote Australia

A gas-station/convenience store was looking for staff. It was a full-time job, with meals and accommodation costing $40/week. This was exactly what I was looking for! I called immediately and got the job because I was American and nobody else had called before me.

This job was pretty much everything Louie described in Optimal Employment. I had practically no expenses, no responsibilities, and free entertainment after hours. At any given time there were three backpackers (including myself) working there, and one would be switched out about every month. We played a lot of video games and ate a lot of barbeques. We also attended a concert and a rodeo. I saved a lot of money and had a lot of fun, but I was anxious to leave after six months.

Adelaide

Australia prohibits backpackers from working the same job for more than six months, so I had t

...