trentmkelly/LessWrong-43k huggingface dataset:
Building rationalist communities: lessons from the Latter-day Saints
by calcsam — 2011-05-09T15:14:53.295Z

Or: How I Learned Everything I Know About Group Organization By Spending Two Years on a Mormon Mission in India.

The official name is the Church of Jesus Christ of Latter-day Saints. You may know us as ‘Mormons.’ We like to call ourselves ‘Latter-day Saints.’

If you’re a Less Wrongian and trying to organize a rationalist community, you should be interested in the Latter-day Saint organizational model for four reasons:

- it’s a nonprofit, but franchise-based and designed to propagate itself,

- everyone has a responsibility,

- no one is paid, and

- it works.

This is an introductory post. I'm not trying to persuade you to join, but rather that there’s something to learn here.

Here, I will give you some basic details about what the LDS Church is. In later posts, I will explain more how it works. A series overview is here.

A franchise model

The Church has about 55,000 missionaries worldwide, all of whom follow the same basic dress code and go about in pairs, basically recruiting people to join the organization. For men, white shirt and ‘conservative’ tie, suitjacket if it’s cold. Clean-shaven. No chewing gum in public. Short hair. And so forth.

Church buildings are selected from a basic set of designs. Each congregation unit has about 150 people each week at Sunday services. The internal organization is the same for each congregation, albeit with procedures for simplification in smaller units. Everyone has a responsibility, from the congregation head down to the teenage boys who prepare and serve the ‘sacrament.’[1]

And nobody is paid.

Everyone has a responsibility

The Church is an organization, but members also comprise a distinct culture. Within the culture, there is an expectation that church members accept a ‘calling’ or specific unpaid organizational responsibility.

Callings are assigned by the head of the local congregation. You can privately decline, but there is an expectation is to accept the responsibility.

Examples include:

- visiting specific church member families monthly (“home teacher”)

- helping local unemployed church members f

...
Intelligence vs Friendliness
by snarles — 2011-05-09T14:28:36.609Z

An agent is composed of two components: a predictive model of the world, and a utility function for evaluating world states

I would say that the 'intelligence' of an agent corresponds to the sophistication and accuracy of their world model. The 'friendliness' of an agent depends on how closely their utility function aligns with human values.

What is baffling to me is the vague idea that developing the theory of friendliness has any significant synergy with developing the theory of intelligence. (Such an idea has surfaced in discussions of on the plausibility of SIAI developing friendly AI before unfriendly AI is developed by others.)

One argument for the latter (and the only one I can think of) is that:

a) A perfect world model is not possible: one must make the best tradeoff given limited resources

b) The more relevant a certain feature of the world is to the utility function, the more accurately we want to model it in the world model

However, for the large part, the world model would differ, in totality, very little between a paperclip maximizer and a friendly AI. While the Friendly AI certainly has to keep track of more things which are irrelevant to the paperclip maximizer, both AIs would have to have world models which have to be able to model human behavior in order for the AIs to be effective, which one would expect would account for the bulk of the complexity of the world model in the first place.
Shanghai Less Wrong Meetup #2 Wednesday May 11th, 7pm
by Patrick — 2011-05-09T09:09:47.801Z

John Teddy has asked me to post the following meetup on Less Wrong:

Shanghai Lesswrong Meeting #2

Wednesday May 11th, 7pm, XuHui District, Shanghai, China (private residence not listed, please contact John Teddy)

The first meeting was a success. There are plans to have regular meetings, goal setting, and various topics. Some of the planned topics are porting the mind to software, bayes theorem, cognitive biases, paleo diet, etc. The first meeting was at a public venue. Our next meeting will be at a private residence. Please text or call John Teddy ( http://lesswrong.com/user/johntheodore ) at 18621732925 to receive the address.
But Butter Goes Rancid In The Freezer
by JenniferRM — 2011-05-09T06:01:34.941Z

I broached the subject of cryonics with a family member today. He offered almost none of the normal objections and I've been happy all day about the way the conversation went. One interesting issue that he raised that I'd like to find an answer for is the question in the title.

Butter goes rancid after a while at room temperature. It also goes rancid in the fridge and can absorb the other flavors if things aren't well contained inside the refrigerator. Butter also goes rancid if left in a normal freezer, which mostly is designed to bring things very close to the melting point of water around 273 kelvin.

This suggests that lipid chemistry responds to temperatures in a different way than intuitions mostly educated by other examples of freezing, which is relevant because the brain is mostly made out of fat, with some complicating proteins mixed in. My guess is that developing a "rancid brain" isn't likely to be a serious issue when you get down to the 77 kelvin of liquid nitrogen, but its still something I'd like to be able to answer directly and honestly, after really thinking about it in terms of "safety engineering".

One way to answer the direct question about butter might be to just perform the basic experiment with some butter samples at different temperatures (room, fridge, freezer, -80C freezer in a bio lab) and figure out how long butter stored each way takes to go rancid and then do some curve fitting, but that seems like it would take months or maybe even years, and butter doesn't even necessarily answer neurological questions directly. Even if I learned about butter chemistry, there could be open questions about brain chemistry. I've tentatively googled around for 30 minutes but organic chemistry isn't a primary area of expertise and I wasn't sure out to dig up the specialist scientific literature that might answer my question.

This community seemed like a good place to get help on the subject!

Here are some specific questions I'd love to know the answers to...

1. What are the precise chemical reactions are that are collectively referred to as rancidity in english, and how to they change at cryogenic temperatures? Does butter stop going rancid in liquid nitrogen?

2. Are these or similar reactions possible in the brain, given all the cell membranes and mylenation and so on that are primarily made out of fat?

3. ...
Toronto Meetup, May 10th
by Spencer_Sleep — 2011-05-09T02:50:26.592Z

When: Tuesday, May 10th, 20:00

Where: The Bedford Academy, 36 Prince Arthur Avenue

Hi everyone,

The Toronto meetup group is having one of our bi-weekly meetings this Tuesday at the Bedford Academy. The reservation is under the name Spencer Sleep. I have requested a table upstairs, as it tends to be much quieter up there.

Regulars take note: This meeting is on Tuesday, not Thursday. This change is because Thursday night is a busy night for bars, so it tends be much louder than Tuesday nights.

Newcomers are, as always, extremely welcome.

The discussion topic for this meeting is the feasibility of nuclear power. If that is too abstract for you, then it is the feasibility of nuclear power as an energy source when compared to alternatives as they stand today, judged on the basis of economic and environmental effects and logistics (availability of fuel, creation of waste, etc...). This discussion topic comes from a meeting a few weeks ago where we wanted to try applying rationality to "solve" (or at least gain a better understanding of) controversial problems, such as that of nuclear power. Since these are such hot topics, discussions are usually very charged with emotions, and biases run rampant. We wanted to see if we could make headway by approaching it from a rational standpoint. Five minutes into the discussion, we determined that we did not have enough facts and that all we were doing was quoting contradictory numbers at each other. We determined to put it off for another day when we had had time to prepare. That day is today (well, Tuesday).

Note: This discussion topic will probably not take up more than an hour of the meeting, so even if this topic does not particularly interest you please show up anyway, as there will be many other discussions throughout the night.

If you want to hear about upcoming LessWrong events in Toronto, or have ideas about how or when those events should be run, join the Toronto LessWrong Google Group!

See you Tuesday
[Reference request] Article by scientist giving lower and upper bounds on the probability of superintelligence
by multifoliaterose — 2011-05-08T22:16:23.190Z

A few months back somebody posted an article by a scientist giving lower and upper bounds on the probability of superintelligence. He broke up the calculation as a Fermi calculation with three parts (EDIT: See LocustBeanGum's answer). Does anybody remember this article and if so can you provide a link?
Shane Legg's Thesis: Machine Superintelligence, Opinions?
by Zetetic — 2011-05-08T20:04:34.183Z

I searched the posts but didn't find a great deal of relevant information. Has anyone taken a serious crack at it, preferably someone who would like to share their thoughts? Is the material worthwhile? Are there any dubious portions or any sections one might want to avoid reading (either due to bad ideas or for time saving reasons)? I'm considering investing a chunk of time into investigating Legg's work so any feedback would be much appreciated, and it seems likely that there might be others who would like some perspective on it as well.
[SEQ RERUN] The Scales of Justice, the Notebook of Rationality
by Tyrrell_McAllister — 2011-05-08T17:43:27.892Z

Today's post, The Scales of Justice, the Notebook of Rationality, was originally published on 13 March 2007. Two summaries (taken from the LW wiki):

People have an irrational tendency to simplify their assessment of things into how good or bad they are without considering that the things in question may have many distinct and unrelated attributes.

(alternate summary:)

In non-binary answer spaces, you can't add up pro and con arguments along one dimension without risk of getting important factual questions wrong.

Discuss the post here (rather than in the comments to the original post).

This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Burch's Law, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.

Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Death Note, Anonymity, and Information Theory
by gwern — 2011-05-08T15:44:00.291Z

I don't know if this is a little too afar field for even a Discussion post, but people seemed to enjoy my previous articles (Girl Scouts financial filings, video game console insurance, philosophy of identity/abortion, & prediction market fees), so...

I recently wrote up an idea that has been bouncing around my head ever since I watched Death Note years ago - can we quantify Light Yagami's mistakes? Which mistake was the greatest? How could one do better? We can shed some light on the matter by examining DN with... basic information theory.

Presented for LessWrong's consideration: Death Note & Anonymity.
Book Review: Predictably Irrational by Dan Ariely
by erratio — 2011-05-08T08:13:31.325Z

I'm ambivalent about recommending this book to LW readers. On the one hand, it's well-written and has lots of interesting anecdotes about Ariely's experiments. He also includes a full reference list at the end of all the papers published based on the studies he describes in the book, which is nice. On the other hand, almost none of the material should be really new to most people here, and a lot of the chapters turned out to be summarisable in a single sentence, indicating a large amount of fluff.

Anyway, I spent a couple of hours today writing up a quick summary of each chapter, and have included my notes below.

Chapter 1 - Everything is relative

When presented with multiple options, some of which are similar to each other and some of which are in separate reference classes, people will consistently pick the best option from the set of similar options because they are directly comparable. Similarly, the use of decoy options when offering products will influence their choices - including a ludicrously expensive option makes the others look good by comparison, and when offering three choices in the same reference class the majority of consumers will pick the middle one.

Chapter 2 - Supply and demand:

Examines the anchoring effect. People asked to list the last two digits of their Social Security number unconsciously used that number as an anchor when bidding for various items later. Black pearls went from worthless to priceless after being displayed next to other high-priced gemstones. When sequentially given 3 different anchoring points, people stuck to the first one presented even if the later ones would have been more beneficial for them. It is also possible to self-anchor, ie. if I buy something expensive despite a previous habit of being cheap, I am now more likely to buy something expensive again because I use my own novel behaviour as an anchor.

Chapter 3 - The power of FREE!

People are irrationally attached to getting free stuff. In one experiment, he set up a table where they had high-quality chocolate for 15c and low-quality for 1c, and most people chose the high-quality ones. But when they decreased each price by 1c, so that it was 14c versus free, most people chose the free ones (and in another experiment they controlled for the inconvenience of getting money out for one but not the other and got the same result). People will also buy more books in o...
Religious Behaviorism
by PhilGoetz — 2011-05-08T00:11:36.687Z

Willard Quine described, in his article "Ontological Relativity" (Journal of Philosophy 65(7):185-212), his doctrine of the indeterminability of translation. Roughly, this says that words are meaningful (a collection of words emitted by an agent can help predict that agent's actions), but don't have meanings (any word taken by itself corresponds to nothing at all; there is no correspondence between the word "rabbit" and the Leporidae).

In Quine's words,

Seen according to the museum myth, the words and sentences of a language have their determinate meanings. To discover the meanings of the native's words we may have to observe his behavior, but still the meanings of the words are supposed to be determinate in the native's mind, his mental museum, even in cases where behavioral criteria are powerless to discover them for us. When on the other hand we recognize with Dewey that "meaning. . . is primarily a property of behavior," we recognize that there are no meanings, nor likenesses nor distinctions of meaning, beyond what are implicit in people's dispositions to overt behavior. For naturalism the question whether two expressions are alike or unlike in meaning has no determinate answer, known or unknown, except insofar as the answer is settled in principle by people's speech dispositions, known or unknown.

Quine got my hackles up by using the word "naturalism" when he meant "behaviorism", implicitly claiming that naturalistic science was synonymous (or would be, if he believed in synonyms) with behaviorism. But I'll try to remain impartial. (Quine's timing was curious; Chomsky had demolished behaviorist linguistics in 1959, nine years before Quine's article.)

Quine's basic idea is insightful. To phrase it in non-behaviorist terms: If all words are defined in terms of other words, how does meaning get into that web of words? Can we unambiguously determine the correct mapping between words and meanings?

Quine's response was to deny that that is an empirical question. He said you should not even talk about meaning; you can only observe behavior. You must remain agnostic about anything inside the head.

But it is an empirical question. With math, plus with some reasonable assumptions, you can prove that you can unambiguously determine the correct mapping even from the outside. In a world where you can tel...
[Altruist Support] Fix the SIAI's website (EDIT: or not. I'll do it)
by Giles — 2011-05-07T18:36:13.570Z

EDIT: This post no longer reflects my current attitude. I'm now signed up as a volunteer for SIAI and will help them with the website and/or whatever else needs doing. Add a comment or contact me if you're curious as to what my attitude was or why it changed.

What I've learnt: People want something more specific

What I've also learnt: Not to commit to donating money to an organization without carefully reading their website first

Imagine you are a prospective SIAI donor. You've learnt about AI and its risks, about how hardly anyone takes these risks seriously, about how people are fundamentally not mentally equipped to handle issues of existential risk in a sane way. You've looked around and seen that the SIAI is the only (or one of just a few) organizations that appear to realise this and want to do anything about it.

So you go to their website. What are you looking for? You're looking for a reason not to give money to them.

Here's one:

The Singularity Institute exists to carry out the mission of the Singularity-aware – to accelerate the arrival of the Singularity in order to hasten its human benefits; ...

This seems a somewhat gung-ho attitude which is not consistent with the message on the rest of the site. And this isn't just my misreading or quoting out of context - apparently that page is very out of date and no longer represents the worldview of the more mature, grown up SIAI.

But people reading the site don't know that. And remember, they're looking for reasons not to give - for reasons to retreat back to their comfort zone where everything's basically OK and the SIAI are just a bunch of weirdos.

The fact that an organization dedicated to shaping the future of humanity can't keep their website up to date would seem to be one of those reasons.

So, if you really believe the SIAI to be the most effective charity right now, you should help them by offering to fix their website for them - in order to help attract more donors.

Some possible objections and counter-objections:

1. If Giles thinks fixing the SIAI's website is so important, he'd already be doing it himself.

Essentially this boils down to the fact that you probably trust the SIAI a lot more than I do. So for me the community-building effort is the higher priority.

2. If the website was so important, the SIAI would already have fixed it. Better just to give them money and they'll spend it on fixing t...
Shifting Load to Explicit Reasoning
by Vladimir_Nesov — 2011-05-07T18:00:22.319Z

Related to: Which Parts Are "Me"?, Making your explicit reasoning trustworthy, The 5-Second Level.

What's damaging about moralizing that we wish to avoid, what useful purpose does moralizing usually serve, and what allows to avoid the damage while retaining the usefulness? It engages psychological adaptations that promote conflict (by playing on social status), which are unpleasant to experience and can lead to undesirable consequences in the long run (such as feeling systematically uncomfortable interacting with a person, and so not being able to live or work or be friends with them). It serves the purpose of imprinting your values, which you feel to be right, on the people you interact with. Consequentialist elucidation of reasons for approving or disapproving of a given policy (virtue) is an effective persuasion technique if your values are actually right (for the people you try to confer them on), and it doesn't engage the same parts of your brain that make moralizing undesirable.

What happens here is transfer of responsibility for important tasks from the imperfect machinery that historically used to manage them (with systematic problems in any given context that humans but not evolution can notice), to explicit reasoning.

Taking advantage of this requires including those tasks in the scope of things that can be reasoned about (instead of ignoring them as not falling into your area of expertise; for example flinching from reasoning about normative questions or intuition as "not scientific", or "not objective"), and developing enough understanding to actually do better than the original heuristics (in some cases by not ignoring what they say), making your explicit reasoning worth trusting.

This calls for identifying other examples of problematic modes of reasoning that engage crude psychological adaptations, and developing techniques for doing better (and making sure they are actually better before trusting them). These examples come to mind: rational argument (don't use as arguments things that you expect other person disagrees with, seek a path where every step will be accepted), allocation of responsibility (don't leave it to unvoiced tendencies to do things, discuss effort and motivation explicitly), development of emotional associations with a given situation/person/thought (take it in your own hands, explicitly train your emotion to be what you prefer ...