trentmkelly/LessWrong-43k huggingface dataset:
The Belief Signaling Trilemma
by Scott Garrabrant — 2013-09-20T00:50:32.810Z

One major issue for group rationality is signaling with dishonest beliefs. Beliefs are used as a signal to preserve your reputation or to show membership in a group. It happens subconsciously, and I believe it is the cause of most of the issues of both religions and political parties. It also happens on Less Wrong, most commonly though not sharing beliefs that you think people will disagree with.

First, let's identify the problem. This is mostly from the viewpoint of ideal agents as opposed to actual humans. You are part of a community of rationalists. You discuss lots of issues, and you become familiar with the other members of your community. As a result, you start to learn which members of the community are the smartest. Of course, your measure of the intelligence of the members is biased towards people who say things that you agree with. The members who say things that you agree with build up more reputation in your mind. This reputation makes you more likely to trust other things that this person says. I am also a member of this community. I have opinions on many things, but there is one issue that I think really does not matter at all. On this issue, most of the community believes X, but I believe not X. By signaling belief in X, I increase my reputation in the community, and will cause other people to take more seriously my views on other issues I think are more important. Therefore, I choose to signal belief in X.

What is happening here is that:

(A) People are assigning reputation based on claims, and judging claims based partially on other beliefs signaled by the same person.

(B) People want their claims are taken seriously, and so take actions which will preserve and improve their reputation.

Therefore,

(C) People take signal beliefs that they believe are false because they are shared by the community.

Signaling honest beliefs is kind of like cooperating in a prisoners dilemma. It helps the community push towards reaching what you believe are valid conclusions, at the cost of your own reputation. It is possible for us to decide as a community that we want to cooperate, especially with tools such as anti-kibitzer. However, there is more than one way to do that. I think there are three options. I think they are all theoretically possible, but I think they are all bad.

(1) We can agree to stop assigning reputation based on beliefs.

This option is bad because t...
Meetup : Intro to AI Existential Risk
by chronophasiac — 2013-09-20T00:33:46.347Z

Discussion article for the meetup : Intro to AI Existential Risk

WHEN:
22 September 2013 08:33:39PM (-0400)

WHERE:
25 Ames St, Cambridge, MA 02139

Organizations like Oxford's Future of Humanity Institute and the Machine Intelligence Research Institute consider artificial intelligence to be a possible extinction risk to humanity. In this presentation, I'll be introducing the concept of AI as an 'existential risk'.

The presentation will take about 30 minutes, followed by Q&A and discussion.

Cambridge/Boston-area Less Wrong meetups are every Sunday at 2pm in MIT's building 66 at 25 Ames St, room 156. Room number subject to change based on availability; signs will be posted with the actual room number.

Our default schedule is as follows:

—Phase 1: Arrival, greetings, unstructured conversation.
—Phase 2: The headline event. This starts promptly at 2:30, and lasts 30-60 minutes.
—Phase 3: Further discussion. We'll explore the ideas raised in phase 2, often in smaller groups.
—Phase 4: Dinner. It's about a ten minute walk to the usual restaurant.

Discussion article for the meetup : Intro to AI Existential Risk
Meetup : London meetup: thought experiments
by Paul Crowley — 2013-09-19T20:29:33.168Z

Discussion article for the meetup : London meetup: thought experiments

WHEN:
29 September 2013 02:00:00PM (+0100)

WHERE:
LShift, Hoxton Point, 6 Rufus St, London, N1 6PE

NB note the change of location!

We are so good at providing reasons for our decisions after the fact that the reasons before the fact that cause our decisions are often not clear to us. At this meeting we'll discuss and practice techniques for getting past our rationalisations and learning more about the real reasons we find ourselves making the choices we do.

Discussion article for the meetup : London meetup: thought experiments
Help us Optimize the Contents of the Sequences eBook
by lukeprog — 2013-09-19T04:31:20.391Z

MIRI's ongoing effort to publish the sequences as an eBook has given us the opportunity to update their contents and organization.

We're looking for suggested posts to reorder, add, or remove.

To help with this, here is a breakdown of the current planned contents of the eBook and any currently planned modifications. Following that is a list of the most popular links within the sequences to posts that are not included therein.

Now's a good time to suggested changes or improvements!

———

Map and Territory

Added …What's a Bias Again? because it's meant to immediately follow Why Truth, And….

Mysterious Answers to Mysterious Questions

No changes.

A Human's Guide to Words

No changes.

How to Actually Change Your Mind

Politics is the Mind-Killer

Removed The Robbers Cave Experiment because it already appears in Death Spirals and the Cult Attractor, and there in the original chronological order which flows better.

Death Spirals and the Cult Attractor

Removed The Litany Against Gurus because it already appears in Politics is the Mind-killer.

Seeing with Fresh Eyes

Removed Asch's Conformity Experiment and Lonely Dissent because they both appear at the end of Death Spirals. Removed The Genetic Fallacy because it's in the Metaethics sequence: that's where it falls chronologically and it fits better there with the surrounding posts.

Noticing Confusion

Removed this entire subsequence because it is entirely contained within Mysterious Answers to Mysterious Questions.

Against Rationalization

Added Pascal's Mugging (before Torture vs Dust Specks) because it explains the 3^^^3 notation. Added Torture vs Dust Specks before A Case Study of Motivated Continuation because A Case Study refers to it frequently.

Against Doublethink

No changes.

Overly Convenient Excuses

Removed How to Convince Me that 2+2=3 because it's already in Map & Territory.

Letting Go

No change.

The Simple Math of Evolution

Added Evolutionary Psychology because it fits nicely at the end and it's referred to by other posts many times.

Challenging the Difficult

No change.

Yudkowsky's Coming of Age

No change.

Reductionism

No change. (Includes the Zombies subsequence.)

Quantum Physics

No change. Doesn't include any "Preliminaries" posts, since they'd all be duplicates

Metaethics

No change.

Fun Theory

No change.

The Craft and the Community

No change.

Appendix

Includes:

The Simple

...
Book Review: Basic Category Theory for Computer Scientists (MIRI course list)
by So8res — 2013-09-19T03:06:02.195Z

I'm reviewing the books on the MIRI course list. After finishing Cognitive Science I picked up Basic Category Theory for Computer Scientists, by Benjamin C. Pierce.

Basic Category Theory for Computer Scientists

This book is tiny, clocking in at around 80 pages. Don't be fooled, it packs a punch.

A word of warning: when the title says "for Computer Scientists", it is not abusing the term. I went in expecting Category Theory for Computer Programmers and a tone like "Welcome, Java Programmer, to the crazy world of math!". What I got was a lean, no-bullshit introduction to category that assumes mathematical competence.

"Computer Scientist" and "Programmer" get mixed up in common parlance, so casual programmers are cautioned that this book targets the former group. Category Theory for Computer Scientists assumes you're familiar with proof-writing, set theory, functional programming, and denotational semantics.

In return, Category Theory wastes none of your time. I found this refreshing, but unexpected.

I'll give a brief overview of the contents of the book before discussing them.

Chapter Summaries

Basic Constructions

Functors, Natural Transformations, and Adjoints

Applications

Further Reading

1. Basic Constructions

Introduces Categories.

Introduces monomorphisms (f∘g = f∘h → g=h) and epimorphisms (g∘f = h∘f → g=h), then uses these to introduce categorical duals (similar constructs with the direction of the arrows swapped).

Introduces isomorphisms and the concept of equality up to isomorphism.

Introduces initial and terminal objects (objects with exactly one arrow from/to each object).

Introduces binary products and coproducts. Generalizes to arbitrary products.

Introduces equalizers and coequalizers.

Introduces pullbacks, briefly mentions pushforwards.

Generalizes equalizers, products, and pullbacks to terminal cones (which are limits).

Introduces exponentiation.

Mentions Cartesian Closed Categories (categories with products, exponents, and a terminal object).

2. Functors, Natural Transformations, and Adjoints

Introduces Functors (arrows between categories).

Introduces F-Algebras (an extreme generalization of algebra).

Introduces Natural Transformations (structure-preserving mappings between functors).

Introduces Adjoints (functors related in way that generalizes efficiency/optimality).

3. Applications

Discusses four applications of category theory:

Closed Car

...
Proxy Donating as Spam Filter
by beth — 2013-09-19T01:55:53.878Z

One thing that sometimes makes me hesitate to donate to a cause is that, unless you're donating in person and using cash, you're inevitably signing up for a gigantic stream of junk mail, not just from the organization you gave money to, but other, often totally unrelated charities as well. I haven't noticed a lot of these charities offering a privacy policy that lets you avoid this, but I haven't paid close attention because frankly, I don't think I'd have a lot of confidence in such a privacy policy even if I saw one in some literature.

I wonder if there are donations to be gained in guaranteeing this sort of privacy by going through a third party. Charities could include the usual pre-addressed envelope in their mailings, only instead of their own address it would go to an organization called Givepal. The envelope would include the charity's id, and donors would be instructed to make their checks out to Givepal, who would then distribute the money to the specified charity, keeping the transaction anonymous. Givepal could survive by taking a cut of the donations if necessary, or could itself operate as a non-profit.
Meetup : The Anthropic Principle and the Great Filter
by Danny_Hintze — 2013-09-19T01:46:15.194Z

Discussion article for the meetup : The Anthropic Principle and the Great Filter

WHEN:
21 September 2013 02:00:00PM (-0700)

WHERE:
300 E Orange Mall Tempe, AZ 85281 (480) 965-6164

We will be meeting up at Hayden for a few hours to discuss why we don't see other life in the cosmos and the implications of that.

Discussion article for the meetup : The Anthropic Principle and the Great Filter
Meetup : Vancouver "Monthly" Super Meetup
by  — 2013-09-19T00:30:08.868Z

Discussion article for the meetup : Vancouver "Monthly" Super Meetup

WHEN:
22 September 2013 03:00:00PM (-0700)

WHERE:
3598 Main Street, Vancouver, BC

It's that time! The Vancouver meetup would like to invite you all to our "monthly" super public social hangout meetup. We'll hang out at a cafe on main street and probably have some good discussion about something or other that we have not decided yet. This is supposed to be casual and newperson friendly, so please come.

Meetup is Sunday the 22nd at Bean Around the World at main and 20th at 15:00.

Our mailing list is vancouver-rationalists on google groups.

Discussion article for the meetup : Vancouver "Monthly" Super Meetup
Help FHI crowdsource a directory of researchers with high consequentialist significance and win a prize 
by  — 2013-09-18T20:11:58.583Z

Let me introduce myself. My name is Kristian Rönn and I work for the Future of Humanity Institute (FHI) as a project manager. This is my first LessWrong post.

tl;tr:

Wouldn't it be useful to have a directory of researchers (dependent or independent) that do research with high consequentialist significance?

Solution: sign up at mirivolunteers.org and help us crowdsource such a list!

The best contributor will win a 30 minute skype conversation with one of the FHI researchers!

The long version:

We all know that some scientific research has extremely high consequentialist significance (good or bad). But much research with high consequentialist significance also falls squarely within the scope of philosophy (see William MacAskill's blog post on high-impact philosophy). Such important areas of philosophical research include ethics, epistemology and philosophy of mind (especially applied to existential risk research). Yet professional philosophy is a diseased discipline.

First, professional philosophers very often work on trivial questions (e.g., “how is knowledge best analyzed?”).

Secondly, when they do address questions of importance, they are seldom motivated by a concern to find the truth, and answers are instead assessed on poor criteria, such as originality, elegance, “cleverness”, and tendency to vindicate cherished beliefs.

Fortunately, there are a few philosophers addressing many of the relevant questions, and from whose work we can learn a great deal. It also seems like the problem of building a friendly AI forces us to think about the philosophical questions mentioned in a more formal/rigorous way. Because in order to design a friendly AI I need to figure out:

What utility function it ought to have (ethics). E.g. Coherent Extrapolated Volition.

How to make correct inferences in an optimal way (epistemology). E.g. Gödel machines and AIXI.

What set of things/concept exists in the agents ontology (metaphysics); and more specifically.

How to distinguish between those things that are a part of the set of conscious things and those who are not (philosophy of mind).

How to track the identity of things in time and space (personal identity over time).

(Author's note: I believe some of the problems listed above are not solvable, since they are just a quibble over semantics)

This means that a lot of the most important philosophy ...
Understanding Simpson's Paradox
by Vaniver — 2013-09-18T19:07:56.653Z

An article by Judea Pearl, available here. It's quick at 8 pages, and worth reading if you enjoy statistics (though I think people who already are familiar with the math of causality1 will get more out of it than others2). I'll talk here about the part that I think is generally interesting:

Any claim to a resolution of a paradox, especially one that has resisted a century of attempted resolution must meet certain criteria. First and foremost, the solution must explain why people consider the phenomenon surprising or unbelievable. Second, the solution must identify the class of scenarios in which the paradox may surface, and distinguish it from scenarios where it will surely not surface. Finally, in those scenarios where the paradox leads to indecision, we must identify the correct answer, explain the features of the scenario that lead to that choice, and prove mathematically that the answer chosen is indeed correct. The next three subsections will describe how these three requirements are met in the case of Simpson's paradox and, naturally, will proceed to convince readers that the paradox deserves the title "resolved."

I've never really liked the name "paradox," because what it seems to mean is "unintuitive phenomenon." (Wikipedia puts it as "something which seems false and yet might be true.") The trouble is that "unintuitive" is a two-place word, and it makes sense to think like reality, so that true things seem true to you, instead of still seeming false. (For example, when I first learned about Zeno's Paradox, I already knew calculus, and so Zeno's position was the one that seemed confusing and false.)

What I like most about Pearl's article is that it explicitly recognizes the importance of fully dissolving the paradox,3 and seems to do so. Simpson's Paradox isn't an unsolvable problem in statistics, it's a straightforward reversal effect--only if you use the language of causality.

1. My review of Causality gives a taste of what it would look like to be familiar with the math, but you'd need to actually read the book to pick it up. The Highly Advanced Epistemology 101 for Beginners sequence is relevant, and contains Eliezer's attempt to explain the basics of causality in Causal Diagrams and Causal Models.

2. Pearl discusses how you would go about using simulations to show that do calculus gives you the right result, but leaves it as an exercise f...
Help CFAR Take Manhattan
by palladias — 2013-09-18T18:40:16.882Z

CFAR is coming to New York City at the beginning of November for our first full-scale workshop outside the Bay Area. And we need your help to run it smoothly.

We're looking for volunteers in the tri-state area who can help us carry out all the behind the scenes logistics that keep our workshops running smoothly. And we'll need a bit more help this time, since we'll be in this location for the first time.

Our volunteers help us with tasks like:

Set up/Break down for meals

Organization for the end of workshop parties

Running errands to replenish supplies

Filming some presentations

etc

Usually, volunteers get the chance to sit in on a handful of the classes we run during the workshop. For our Berkeley workshops, our volunteer ranks have been largely filled out by alumni, who want to make sure other people get to experience the workshop experience they enjoyed.

If you think you might be interested and available to help us out this November in New York, please fill out this five question survey, and we'll be in touch. And if you have questions, I'll meet you in the comments.
[LINK] Larry = Harry sans magic? Google vs. Death
by Shmi — 2013-09-18T16:49:17.876Z

Google's announcement, Time magazine rather sensationalist headline.

In any case, it's nice to know that Google set its sights to "challenge ... aging and associated diseases". Apple's Tim Cook:

For too many of our friends and family, life has been cut short or the quality of their life is too often lacking. Art is one of the crazy ones who thinks it doesn’t have to be this way.

One more step towards "world optimization".
What did governments get right? Gotta list them all!
by Stuart_Armstrong — 2013-09-18T12:59:00.625Z

When predicting future threats, we also need to predict future policy responses. If mass pandemics are inevitable, it matters whether governments and international organisations can rise to the challenge or not. But its very hard to get a valid intuitive picture of government competence. Consider the following two scenarios:

Governments are morasses of incompetence, saturated by turf wars, perverse incentives, inefficiencies, regulatory capture, and excessive risk aversion. The media reports a lot of the bad stuff, but doesn't have nearly enough space for it all, as it has to find some room for sport and naked celebrities. The average person will hear 1 story of government incompetence a day, anyone following the news will hear 10, a dedicated obsessive will hear 100 - but this is just the tip of the iceberg. The media sometimes reports good news to counterbalance the bad, at about a rate of 1-to-10 of good news to bad. This rate is wildly over-optimistic.

Governments are filled mainly by politicians desperate to make a positive mark on the world. Civil servants are professional and certainly not stupid, working to clear criteria with a good internal culture, in systems that have learnt the lessons of the past and have improved. There is a certain amount of error, inefficiency, and corruption, but these are more exceptions than rules. Highly politicised issues tend to be badly handled, but less contentious issues are dealt with well. The media, knowing that bad news sells, fills their pages mainly with bad stuff (though they often have to exaggerate issues). The average person will hear 1 story of government incompetence a day, anyone following the news will hear 10, a dedicated obsessive will hear 100 - but some of those are quite distorted. The media sometimes reports good news to counterbalance the bad, at about a rate of 1-to-10 of good news to bad. This rate is wildly over-pessimistic.

These two situations are, of course, completely indistinguishable for the public. The smartest and most dedicated of outside observers can't form an accurate picture of the situation. Which means that, unless you have spent your entire life inside various levels of government (which brings its own distortions!), you don't really have a clue at general government competence. There's some very faint clues that governments may be working better than we generally thi...
Meetup : Philadelphia - The Sword of Good
by mare-of-night — 2013-09-18T03:09:34.800Z

Discussion article for the meetup : Philadelphia - The Sword of Good

WHEN:
22 September 2013 12:30:00PM (-0400)

WHERE:
1515 Chestnut St, Philadelphia, PA

Location: Givovani's Pizza

Discussion topic: The Sword of Good (A short story - I highly recommend reading it before the meetup if you are able, because it's meant to have some element of surprise.)

Fiction doesn't seem to be used as a discussion topic for meetups very often, but I noticed that we tend to end up talking about stories together anyway, and it usually leads to interesting conversations.

If you have any simple games you'd like to play as a group, please bring them - it will give us something to do in the beginning while people are still arriving.

Discussion article for the meetup : Philadelphia - The Sword of Good
Thought experiment: The transhuman pedophile
by PhilGoetz — 2013-09-17T22:38:06.160Z

There's a recent science fiction story that I can't recall the name of, in which the narrator is traveling somewhere via plane, and the security check includes a brain scan for deviance. The narrator is a pedophile. Everyone who sees the results of the scan is horrified--not that he's a pedophile, but that his particular brain abnormality is easily fixed, so that means he's chosen to remain a pedophile. He's closely monitored, so he'll never be able to act on those desires, but he keeps them anyway, because that's part of who he is.

What would you do in his place?

In the language of good old-fashioned AI, his pedophilia is a goal or a terminal value. "Fixing" him means changing or erasing that value. People here sometimes say that a rational agent should never change its terminal values. (If one goal is unobtainable, the agent will simply not pursue that goal.) Why, then, can we imagine the man being tempted to do so? Would it be a failure of rationality?

If the answer is that one terminal value can rationally set a goal to change another terminal value, then either

any terminal value of a rational agent can change, or

we need another word for the really terminal values that can't be changed rationally, and a way of identifying them, and a proof that they exist.