trentmkelly/LessWrong-43k huggingface dataset:
Political topics attract participants inclined to use the norms of mainstream political debate, risking a tipping point to lower quality discussion
by emr — 2015-03-26T00:14:18.806Z

(I hope that is the least click-baity title ever.)

Political topics elicit lower quality participation, holding the set of participants fixed. This is the thesis of "politics is the mind-killer".

Here's a separate effect: Political topics attract mind-killed participants. This can happen even when the initial participants are not mind-killed by the topic.

Since outreach is important, this could be a good thing. Raise the sanity water line! But the sea of people eager to enter political discussions is vast, and the epistemic problems can run deep. Of course not everyone needs to come perfectly prealigned with community norms, but any community will be limited in how robustly it can handle an influx of participants expecting a different set of norms. If you look at other forums, it seems to take very little overt contemporary political discussion before the whole place is swamped, and politics becomes endemic. As appealing as "LW, but with slightly more contemporary politics" sounds, it's probably not even an option. You have "LW, with politics in every thread", and "LW, with as little politics as we can manage".

That said, most of the problems are avoided by just not saying anything that patterns matches too easily to current political issues. From what I can tell, LW has always had tons of meta-political content, which doesn't seem to cause problems, as well as standard political points presented in unusual ways, and contrarian political opinions that are too marginal to raise concern. Frankly, if you have a "no politics" norm, people will still talk about politics, but to a limited degree. But if you don't even half-heartedly (or even hypocritically) discourage politics, then a open-entry site that accepts general topics will risk spiraling too far in a political direction.

As an aside, I'm not apolitical. Although some people advance a more sweeping dismissal of the importance or utility of political debate, this isn't required to justify restricting politics in certain contexts. The sort of the argument I've sketched (I don't want LW to be swamped by the worse sorts of people who can be attracted to political debate) is enough. There's no hypocrisy in not wanting politics on LW, but accepting political talk (and the warts it entails) elsewhere. Of the top of my head, Yvain is one LW affiliate who now largely writes about more politically cha...
Meetup : Rationality Meetup Vienna
by AnnaLeptikon — 2015-03-25T20:14:25.322Z

Discussion article for the meetup : Rationality Meetup Vienna

WHEN:
09 May 2015 02:00:00PM (+0100)

WHERE:
Kaisermühlenstraße 24/2

directions: http://web.student.tuwien.ac.at/~e0326238/rationality_meetup/directions.html

FB-Event for more details and discussions:
https://www.facebook.com/events/811775762240996/
(You need to become member of "Rationality Vienna" to see it)

Discussion article for the meetup : Rationality Meetup Vienna
Models as definitions
by Stuart_Armstrong — 2015-03-25T17:46:03.300Z

A putative new idea for AI control; index here.

The insight this post comes from is a simple one: defining concepts such as “human” and “happy” is hard. A superintelligent AI will probably create good definitions of these, while attempting to achieve its goals: a good definition of “human” because it needs to control them, and of “happy” because it needs to converse convincingly with us. It is annoying that these definitions exist, but that we won’t have access to them.

Modelling and defining

Imagine a game of football (or, as you Americans should call it, football). And now imagine a computer game version of it. How would you say that the computer game version (which is nothing more than an algorithm) is also a game of football?

Well, you can start listing features that they have in common. They both involve two “teams” fielding eleven “players” each, that “kick” a “ball” that obeys certain equations, aiming to stay within the “field”, which has different “zones” with different properties, etc...

As you list more and more properties, you refine your model of football. There are some properties that distinguish real from simulated football (fine details about the human body, for instance), but most of the properties that people care about are the same in both games.

My idea is that once you have a sufficiently complex model of football that applies to both the real game and a (good) simulated version, you can use that as the definition of football. And compare it with other putative examples of football: maybe in some places people play on the street rather than on fields, or maybe there are more players, or maybe some other games simulate different aspects to different degrees. You could try and analyse this with information theoretic considerations (ie given two model of two different examples, how much information is needed to turn one into the other).

Now, this resembles the “suggestively labelled lisp tokens” approach to AI, or the Cyc approach of just listing lots of syntax stuff and their relationships. Certainly you can’t keep an AI safe by using such a model of football: if you try an contain the AI by saying “make sure that there is a ‘Football World Cup’ played every four years&rdqu...
[POLITICS] Jihadism and a new kind of existential threat
by MrMind — 2015-03-25T09:37:27.036Z

Politics is the mind-killer. Politics IS really the mind-killer. Please meditate on this until politics flows over you like butter on hot teflon, and your neurons stops fibrillating and resume their normal operations.

Preface

I've always found silly that LW, one of the best and most focused group of rationalists on the web isn't able to talk evenly about politics. It's true that we are still human, but can't we just make an effort at being calm and level-headed? I think we can. Does gradual exposure works on group, too? Maybe a little bit of effort combined with a little bit of exposure will work as a vaccine.
And maybe tomorrow a beautiful naked valkyrie will bring me to utopia on her flying unicorn...
Anyway, I want to try. Let's see what happens.

Intro

Two recent events has prompted me to make this post: I'm reading "The rise of the Islamic State" by Patrick Coburn, which I think does a good job in presenting fairly the very recent history surrounding ISIS, and the terrorist attack in Tunis by the same group, which resulted in 18 foreigners killed.
I believe that their presence in the region is now definitive: they control an area that is wider than Great Britain, with a population tallying over six millions, not counting the territories controlled by affiliate group like Boko Haram. Their influence is also expanding, and the attack in Tunis shows that this entity is not going to stay confined between the borders of Syria and Iraq.
It may well be the case that in the next ten years or so, this will be an international entity which will bring ideas and mores predating the Middle Age back on the Mediterranean Sea.

A new kind of existential threat

To a mildly rational person, the conflict fueling the rise of the Islamic State, namely the doctrinal differences between Sunni and Shia Islam, is the worst kind of Blue/Green division. A separation that causes hundreds of billions of dollars (read that again) to be wasted trying kill each other. But here it is, and the world must deal with it.
In comparison, Democrats and Republicans are so close that they could be mistaken for Aumann agreeing.
I fear that ISIS is bringing a new kind of existential threat: one where is not the existence of humankind at risks, but the existence of the idea of rationality.
The funny thing is that while people can be extremely irrational, they can still work on technology to disco...
Personal Notes On Productivity (A categorization of various resources)
by  — 2015-03-25T01:35:34.912Z

For each topic, I’ve curated a few links that I’ve found to be pretty high quality.

Meta:(Epiphany Addiction, Reversing Advice, Excellence Porn)

@Learning:

Success People: (Mastery),(ChoosingTopics: Osci,PG)

Thinking: (Ikigai, Stoicism, Rationality)

HabitChange: (!ShootDog)

Productivity.Principles/Energy/Relaxation:(FullEngagement, ArtOfLearning)

Productivity.Systems/Hacks: (Autofocus, GTD/ZTD, EatFrog),(Scott Young)

Depression/Anxiety:

Social:

Meditation

Full List: https://workflowy.com/s/zUTEaY0ZcJ

I'd like feedback on:

What other categories/links would you include (I'm sure there's lots of interesting stuff I'm missing.)? What do you think of the categorization ("Thinking" is a pretty large category.)?

Whether you think I should make cross-posts about sub-topics here. The main benefit of making more cross posts is that the discussion/comments would be more focused on those topics. In particular, I think that looking at SuccessfulPeople.Startups, SuccessfulPeople.Science, and the Meditation document are the most original parts of this post.

SuccessfulPeople.Startups contains a categorization of some of Paul Graham's essays (e.g. Having ideas, fund-raising, executing, etc).

The SuccessfulPeople.Science link contains a separate categorization of advice specifically for scientists (e.g. Picking ideas, the importance of being persistent, the importance of reading widely, etc).

The meditation document lists a few high quality meditation resources that I've found (and I've read ~10 books on meditation. Most of it is crap. Some of the stuff I list is orders of magnitude better than the median meditation book I've read.).

Whatever seems salient to you.
Forum Digest: Corrigibility, utility indifference, & related control ideas
by Benya_Fallenstein — 2015-03-24T17:39:09.000Z

This is a quick recap of the posts of this forum that deal with corrigibility (making sure that if you get an agent's goal system wrong, it doesn't try to prevent you from changing it), utility indifference (the idea to remove incentives to manipulate you so that you change or not change the agent's goal system, by adding rewards to its utility function that make it get the same utility in both cases), and related AI control ideas. It's current as of 3/21/15.

Papers

As background to the posts listed below, the following two papers may be helpful.

Corrigibility, by Nate Soares, Benja Fallenstein, Eliezer Yudkowsky, and Stuart Armstrong (2015). This paper introduces the problem of corrigibility and analyzes some simple models, including a version of Stuart Armstrong's utility indifference. Abstract:

As artificially intelligent systems grow in intelligence and capability, some of their available options may allow them to resist intervention by their programmers. We call an AI system “corrigible” if it cooperates with what its creators regard as a corrective intervention, despite default incentives for rational agents to resist attempts to shut them down or modify their preferences. We introduce the notion of corrigibility and analyze utility functions that attempt to make an agent shut down safely if a shutdown button is pressed, while avoiding incentives to prevent the button from being pressed or cause the button to be pressed, and while ensuring propagation of the shutdown behavior as it creates new subsystems or self-modifies. While some proposals are interesting, none have yet been demonstrated to satisfy all of our intuitive desiderata, leaving this simple problem in corrigibility wide-open.

Utility Indifference, by Stuart Armstrong (2010). An older paper by Stuart explaining the utility indifference approach.

Corrigibility

Generalizing the Corrigibility paper's impossibility result?, Benja Fallenstein. The Corrigibility paper looks at a particular linear way to combine two utility functions, UN and US, which incentivize normal operation and shutdown, respectively. It shows that most such linear combinations lead to unintended behavior. Is it possible to avoid this problem by considering non-linear combinations? It turns out that this question isn't quite well-formed.

Utility indifference

Utility indifference and infinite improbability drives,

...
Social prerequisites of rationality
by  — 2015-03-24T12:33:46.170Z

Summary: it is a prerequisite that you think you are entitled to your own beliefs, your beliefs matter, you think your actions follow your own beliefs and not from commands issued by others, and your actions can make a difference, at the very least in your own life. This may correlate with what one may call either equality or liberty.

Religion as not even attire, just obedience

I know people, mainly old rural folks from CEE, who do not think they are entitled to have a vote in whether there is a God or not. They simply obey. This does NOT mean they base their beliefs on Authority: rather they think their beliefs do not matter, because nobody asks them about their beliefs. They base their behavior on Authority, because this is what is expected of them. The Big Man in suit tells you to pay taxes, you do. The Big Man in white lab coat tells you to take this medicine, you do. The Big Man in priestly robes tells you to kneel and cross yourself, you do. They think forming their own beliefs is above their "pay grade". One old guy, when asked any question outside this expertise, used to tell me "The Paternoster is the priests's business." Meaning: I am not entitled to form any beliefs regarding these matters, I lack the expertise, and lack the power. I think what we have here is not admirable epistemic humility, rather a heavy case of disempoweredness, inequality, oppression, lack of equality or liberty and of course all that internalized.

Empoweredness, liberty, equality

Sure, on very high levels liberty and equality may be enemies: equality beyond a certain level can only be enforced by reducing liberties, and liberty leads to inequality. But only beyond a certain level: low and mid-levels they go hand in hand. My impression is that Americans who fight online for one and against the other simple take the level where they go hand in hand for granted, having had this for generations. But it is fairly obvious that on lower levels, some amount of liberty presumes some about of equality and vice versa. Equality also means an equality of power, and with that it is hard to tyrannize over others and reduce their liberties. You can only succesfully make others un-free if you wield much higher power than theirs and then equality goes out the window. The other way around: liberty means the rich person cannot simply decide to bulldoze the poor persons mud hut and buil...
Indifferent vs false-friendly AIs
by Stuart_Armstrong — 2015-03-24T12:13:19.825Z

A putative new idea for AI control; index here.

For anyone but an extreme total utilitarian, there is a great difference between AIs that would eliminate everyone as a side effect of focusing on their own goals (indifferent AIs) and AIs that would effectively eliminate everyone through a bad instantiation of human-friendly values (false-friendly AIs). Examples of indifferent AIs are things like paperclip maximisers, examples of false-friendly AIs are "keep humans safe" AIs who entomb everyone in bunkers, lobotomised and on medical drips.

The difference is apparent when you consider multiple AIs and negotiations between them. Imagine you have a large class of AIs, and that they are all indifferent (IAIs), except for one (which you can't identify) which is friendly (FAI). And you now let them negotiate a compromise between themselves. Then, for many possible compromises, we will end up with most of the universe getting optimised for whatever goals the AIs set themselves, while a small portion (maybe just a single galaxy's resources) would get dedicated to making human lives incredibly happy and meaningful.

But if there is a false-friendly AI (FFAI) in the mix, things can go very wrong. That is because those happy and meaningful lives are a net negative to the FFAI. These humans are running dangers - possibly physical, possibly psychological - that lobotomisation and bunkers (or their digital equivalents) could protect against. Unlike the IAIs, which would only complain about the loss of resources to the FAI, the FFAI finds the FAI's actions positively harmful (and possibly vice versa), making compromises much harder to reach.

And the compromises reached might be bad ones. For instance, what if the FAI and FFAI agree on "half-lobotomised humans" or something like that? You might ask why the FAI would agree to that, but there's a great difference to an AI that would be friendly on its own, and one that would choose only friendly compromises with a powerful other AI with human-relevant preferences.

Some designs of FFAIs might not lead to these bad outcomes - just like IAIs, they might be content to rule over a galaxy of lobotomised humans, while the FAI has its own galaxy off on its own, where its humans take all these dangers. But generally, FFAIs would not come about by someone designing a FFAI, let alone someone designing a FFAI that can safely trade with a FA...
Why I Reject the Correspondence Theory of Truth
by pragmatist — 2015-03-24T11:00:28.828Z

This post began life as a comment responding to Peer Gynt's request for a steelman of non-correspondence views of truth. It ended up being far too long for a comment, so I've decided to make it a separate post. However, it might have the rambly quality of a long comment rather than a fully planned out post.

Evaluating Models

Let's say I'm presented with a model and I'm wondering whether I should incorporate it into my belief-set. There are several different ways I could go about evaluating the model, but for now let's focus on two. The first is pragmatic. I could ask how useful the model would be for achieving my goals. Of course, this criterion of evaluation depends crucially on what my goals actually are. It must also take into account several other factors, including my cognitive abilities (perhaps I am better at working with visual rather than verbal models) and the effectiveness of alternative models available to me. So if my job is designing cannons, perhaps Newtonian mechanics is a better model than relativity, since the calculations are easier and there is no significant difference in the efficacy of the technology I would create using either model correctly. On the other hand, if my job is designing GPS systems, relativity might be a better model, with the increased difficulty of calculations being compensated by a significant improvement in effectiveness. If I design both cannons and GPS systems, then which model is better will vary with context.

Another mode of evaluation is correspondence with reality, the extent to which the model accurately represents its domain. In this case, you don't have much of the context-sensitivity that's associated with pragmatic evaluation. Newtonian mechanics may be more effective than the theory of relativity at achieving certain goals, but (conventional wisdom says) relativity is nonetheless a more accurate representation of the world. If the cannon maker believes in Newtonian mechanics, his beliefs don't correspond with the world as well as they should. According to correspondence theorists, it is this mode of evaluation that is relevant when we're interested in truth. We want to know how well a model mimics reality, not how useful it is.

I'm sure most correspondence theorists would say that the usefulness of a model is linked to its truth. One major reason why certain models work better than others is that they are b...
Michael Oakeshott's critique of something-he-called-rationalism
by  — 2015-03-24T09:20:41.196Z

Ideally participants in this discussion would have read his relevant essays (collected in the Rationalism in Politics book) but as an introduction this will do and this one is also good.

Clealy Oakeshott means something different under Rationalism than LW. I will call it SOCR (Something Oakeshott Calls Rationalism) now.

SOCR is the the idea that you can learn to cook from a recipe book, following the algorithms. He argues it used to be a popular idea in early 20th century Britain and it is false. Recipe books are written for people who can already cook, and this knowledge only comes from experience, not books. Either self-discovery, or apprenticing. Try to learn to cook from a recipe book and the book will not teach you, but you own failed experiments will, the hard way, you end up rediscovering cooking by a trial and error basis. Apprenticing is easier. The recipe book writer assumes the recipe works on an empty mind, while it only works on a mind already filled with experience. And what is worse, often minds are filled with the wrong kind of experience.

While Oakeshott can be accusing of endorsing "life experience as conversation stopper" his main argument is basically how is knowledge communicable. You have knowledge in your head, much of it gathered through experience, you may not be able to communicate all aspect of it through training an apprentice and even less through writing a book. Doing things is often more of an art than science. Worse, you would expect having the students cup pre-filled by the right kind of stuff, but often it is empty or filled with the wrong kind of stuff, which makes your book misunderstood.

Oakeshott focused on politics because his main point was that following a recipe book like Marxism-Leninism or Maoism is not simply a bad idea, but literally impossible, the doctrine you learn will be colored by your pre-existing experience and you will do whatever your experience dictates anyway. The issue is misleading yourself and others into thinking you are implementing a recipe, an algorithm, when it is not the case.

Oakeshott is basically saying e.g. you can never predict what the Soviet Union will do by looking at the Marxist books they read. However, if you add up the experience of the Tzarist imperialism and the experience of being a very reasonably paranoid revolutionary on the run from the Ohrana and fearing betra...