trentmkelly/LessWrong-43k huggingface dataset:
Reduced impact in practice: randomly sampling the future
by Stuart_Armstrong — 2014-03-17T16:30:55.370Z

This was an idea developed along with Eliezer when he was at the FHI in November.

In my post on "reduced impact AI" I talked about how you could estimate the impact of an AI by (hypothetically) observing a slice of the future. When Eliezer was here last November, we talked about how this could be cached out, and came up with a model.

Ignore the "output channel" from the original reduced impact post, and consider this situation:

We have two computers running in two similarly shielded rooms. One of them is a simple algorithm running through the digits of e (call it run-e), that was soon after deleted. Another is our dear old friend clippy, the AI motivated to maximise paperclips. Without removing clippy's paperclip preferences, we'd like to motivate it to have next to no impact on the world. How should we define "next to no impact"?

The obvious think is to compare it with run-e, and require it to have no more expected impact than that machine. How could this be measured? Well, the original idea was to put a stochastic process on clippy as it was turned on (inside the shielded room) and say that clippy has reduced impact if we can't find any extra evidence, from a vantage point 50 years in the future, that clippy was successfully turned on. Now put the stochastic same process on run-e and define:

Clippy has reduced impact if, from a vantage of 50 years into the future, we have no more evidence that clippy was turned on than we have of run-e being turned on.

Since run-e was simply thinking about e for a while, within a shielded room, then getting deleted, we should have very little posterior evidence about it. But how do we determine what counts as evidence in 50 years time - after all, run-e emitted some radiation (if it was on), so there should be some evidence in 50 years that it was on. So how to define "no more evidence"?

By using a random sampling. If clippy went wild and filled the universe with paperclips, it would be rapidly obvious: there would be paperclips all over the place, lack of unprocessed iron in planetary crusts, light beams showing images of mass paperclip factories, etc... If some super-intelligence started sampling random data about particles in 50 years time, it wouldn't take long for it to deduce whether there'd been a unconstrained clippy or not - in any case, it would (almost certainly) deduce this information long before it deduced whether...
My book: Simulating Dennett - This Wednesday in Sao Paulo
by diegocaleiro — 2014-03-17T08:15:43.933Z

There's been somewhat frequent coverage of Daniel Dennett on Lesswrong:

How not to be a Naïve Computationalist

Dennett's "Consciousness Explained": Prelude

"Where Am I?", by Daniel Dennett

Dennett's heterophenomenology

My personal favorite: Zombies: The Movie

I've written a book called Simulating Dennett nearly five years ago now (if you are considering an academic career, keep that slow paced speed in mind, for good or ill). It summarizes Dennett's philosophy while trying to make the reader able to think like Dennett. It seemed to me at the time, and still does now, that Dennett's kind of mind is very interesting and we should have more of those, so I tried my best to create a Dennett installer in book form.

Simulating Dennett: Tools and Constructions of a Naturalist

Is the 244 pages that ensued. Portuguese or Spanish reading skills advised. Or use it to learn Portuguese prior to your trip to Rio, Pantanal, Iguaçu Falls and the Amazon Forest. (for legal reasons I've chopped out the second half of the file, but there are instructions on how to get it when you get to the end of the first half)

Abstract

This dissertation intends to provide the reader with an inner simulation of Daniel Dennett’s form of reasoning, spreading over his whole philosophy, emphasizing his treatment of patterns, the evolutionary algorithm, consciousness, and his use of illata, abstracta, semantic, and syntax, to carve nature at its joints, especially biology and the human mind. It recasts, in a new light, great part of his most important ideas, and reverse engineers what made him think in particular ways, walking the reader through similar pathways, fostering an active learning of a thinking style, above and beyond a mere exposition of the results obtained by this thinking style over the years.

Keywords: Daniel Dennett, Consciousness, Memetics, Intentional stance, Evolution,

Algorithm.

This Wednesday 2013-03-19 at 14:00 I’ll be presenting it as thesis in the University of São Paulo. Lesswrongers passing by Brazil, or the 20 of us who actually live here are welcome to join.

Here is the Facebook event.
Meetup : Moscow, Chaotic Meet up
by Yuu — 2014-03-17T07:59:55.192Z

Discussion article for the meetup : Moscow, Chaotic Meet up

WHEN:
23 March 2014 04:00:00PM (+0400)

WHERE:
Russia, Moscow, ulitsa L'va Tolstogo 16

We will have:

Board games

Non-board games

You can see proposed program in more details here (in Russian).

We gather in the Yandex office, you need the second revolving door with the sign “Яндекс”, here is the photo of the entrance you need. You need to pass the first entrance and through the archway. Here is additional guide how to get there: link. We will gather at the same second entrance, but we will go to a room inside the building at 16:00. So please do not be late.

You can fill this one minute form (in Russian), to share your contact information.
We start at 16:00 and sometimes finish at night. Please pay attention that we only gather near the second entrance and then come inside.

Discussion article for the meetup : Moscow, Chaotic Meet up
Group Rationality Diary, March 16-31
by therufs — 2014-03-16T22:47:18.311Z

This is the public group instrumental rationality diary for March 16-31.

It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Thanks to cata for starting the Group Rationality Diary posts, and to commenters for participating.

Immediate past diary: March 1-15

Next diary: April 1-15

Rationality diaries archive
Meetup : Utrecht: Famine, Affluence and Morality
by SoerenMind — 2014-03-16T19:56:44.267Z

Discussion article for the meetup : Utrecht: Famine, Affluence and Morality

WHEN: 22 March 2014 07:00:00PM (+0100)

WHERE: Oudegracht 158, Utrecht

At the last meetup we decided to have biweekly meetups on Saturday evenings. We will see if this turns out to be convenient. This time we want to discuss Peter Singer's essay "Famine, Affluence and Morality". Link: http://commonsenseatheism.com/wp-content/uploads/2010/04/Singer-Famine-Affluence-and-Morality.pdf Wiki-link: en.wikipedia.org/wiki/Famine,Affluence,and_Morality The first half of the essay contains the most important information. Other effective altruism and rationality topics are welcome too of course. Especially if you have any specific questions to discuss don't hesitate to bring them up. We have a meetup.com presence of Lesswrong/Effective Altruism Nederland now. Please click attend if you want to join. http://www.meetup.com/Less-Wrong-Nederland/events/170182532/

We will meet in a café called De Winkel van Sinkel, which is 400m walking distance from Utrecht Centraal. The meetup will be held in English. I will be holding a sign that says "LW" on it.

Discussion article for the meetup : Utrecht: Famine, Affluence and Morality
Reference Frames for Expected Value
by ozziegooen — 2014-03-16T19:22:39.976Z

Puzzle 1: George mortgages his house to invest in lottery tickets. He wins and becomes a millionaire. Did he make a good choice?

Puzzle 2: The U.S. president questions if he should bluff a nuclear war or concede to the USSR. He bluffs and it just barely works. Although there were several close calls for nuclear catastrophe, everything works out ok. Was this ethical?

One interpretation of consequentialism is that decisions that produce good outcomes are good decisions, rather than decisions that produce good expected outcomes.12 One would be ethical if their actions end up with positive outcomes, disregarding the intentions of those actions. For instance, a terrorist who accidentally foils an otherwise catastrophic terrorist plan would have done a very ‘morally good’ action.3 This general view seems to be surprisingly common.4

This seems intuitively strange to many, it definitely is to me. Instead, ‘expected value’ seems to be a better way of both making decisions and judging the decisions made by others. However, while ‘expected value’ can be useful for individual decision making, I make the case that it is very difficult to use to judge other people’s decisions in a meaningful way.5 This is because ‘expected value’ is typically defined in reference to a specific set of information and intelligence rather than an objective truth about the world.

Two questions to help guide this:

Should we judge previous actions based on ‘expected’ or ‘actual’ value?

Should we make future decisions to optimize ‘expected’ or ‘actual’ value?

I believe these are in a sense quite simple, but require some consideration to definitions.6

Optimizing Future Decisions: Actual vs. Expected Value

The second question is the easiest of the two, so I’ll begin with that one. The simple answer is that this is a question of defining ‘expected value’. Once we do so the question kind of goes away.

There is nothing fundamentally different between expected value and actual value. A more fair comparison may be ‘expected value from the perspective of the decision maker’ with ‘expected value from a later, more accurate prospective’.

Expected value converges on actual value with lots of information. Said differently, actual value is expect...
Meta-error: I like therefore I am
by KatjaGrace — 2014-03-16T19:10:12.000Z

I like Scott’s post on what LessWrong has learned in its lifetime. In general I approve of looking back at your past misunderstandings and errors, and trying to figure out what you did wrong. This is often very hard, because it’s hard to remember or imagine what absurd thoughts (or absences of thought) could have produced your past misunderstandings. I think this is especially because nonsensical confusions and oversights tend to be less well-formed, and thus less organizable or memorable than e.g. coherent statements are.

In the spirit of understanding past errors, here is a list of errors which I think spring from a common meta-error. Some are mentioned in Scott’s post, some were mine, some are others’ (especially those who are a combination of smart and naive I think), a few are hypothetical:

Because I believe agent-like behavior is obviously better than randomish reactions, I assume I am an agent (debunked!).

Because I think it is good to be sad about the third world, and not good to be sad about not having enough vitamin B, I assume the former is why I am sad.

Because I explicitly feel that racism is bad, I am presumably not racist.

Because my mind contains a line of reasoning that suggests I should not update much against my own capabilities because I am female, presumably I do no such thing.

Because I have formulated this argument that it is optimal for me to think about X-risks, I assume I am motivated to (also debunked on LW).

Because I follow and endorse arguments against moral realism, and infer that on reflection I prefer to be a consequentialist, I assume I don’t have any strong moral feelings about incest.

Because I have received sufficient evidence that I should believe Y, I presumably believe Y now.

I don’t believe Y, and the only reason I endorse to not believe things is that you haven’t got enough evidence for them, therefore I must not have enough evidence to believe Y.

Because I don’t understand the social role of Christmas, I presume I don’t enjoy it (note that this is a terrible failing of the outside view: none of those people merrily opening their presents understands the social role either).

Because I don’t endorse the social role of drinking, I assume I don’t enjoy it.

Because signaling sounds bad to me, I assume I don’t do it, or at least not as much as others.

...
Intelligence-disadvantage
by casebash — 2014-03-16T07:14:57.212Z

While LessWrong contains a large amount of high-quality material, most of the rationality advice isn't actually targeted at our core audience. The focus seems to be more on irrational things that people do, rather than irrational things that smart people do. (Sidenote: If we wanted to create a site focused on spreading general rationality, then we'd need to simplify the discussion, remove a lot of the maths/controversial ideas and add in some friendly images. Does such a site exist?).

This has led to a number of comments questioning the real world value of having read the sequences. If your average person had the patience to read through the core sequences and understand them, they'd find them extremely valuable. It'd provide them with a glimpse into a new way of thinking and even though they would likely hardly appear to be very logical to most Less Wrongers, they'd be much better than they were at the start.

On the other hand, most Less Wrongers already know the basics of logic. That's not to say that we don't act extremely irrational much of the time, but just that going over the basics of logic again probably provides minimal benefit. What is needed is something specifically targeted at the kind of irrational mistakes and beliefs that intelligent people make. I would argue that if this were a sequence, it would be the most important sequence in the entire site. But, since I lack that level of writing ability, I'm not even going to attempt such a project. So I just created a post where we can list articles or ideas that should be part of such a sequence in the hope that someone else might pick it up

Here are some examples of mistakes that intelligent people make:

Taking a fixed instead of a growth mindset - shying away from challenges, convincing oneself that we are just naturally bad at non-intellectual things and that we shouldn't focus on them

Directly pointing out people's flaws

Overthinking issues that are really very simple

Counter-signalling by ignoring the value of fashion, money, being liked

Valuing intelligence above all other qualities

Rigidly adhering to rules

Expecting other people to always be rational

Not considering popularity as a signal that is worthwhile understanding

Overvaluing being right
I'm sure there are plenty more. Any other suggestions or relevant articles?
Why are Harvard's alumni so wealthy?
by JonahS — 2014-03-15T18:47:24.880Z

[Added: Solipsist made good comments that partially account for the phenomenon, and Douglas_Knight pointed out that the figure of $200m+ below should be $30m+]

This article reports that Harvard has 2,964 alumni worth $200+ million, with a total wealth of $622 billion. These figures are staggering:

The university with the next highest figures (University of Pennsylvania) are ~2x less, and the figures for Yale (for example) are ~4x less.

If you assume that Harvard has ~60 living classes of alumni, each of which has ~2,000 students, then the wealth per graduate coming from the alumni worth $200+ million alone is ~$5 million. Furthermore, the fraction of Harvard alumni with $200+ million is on the order of 2%. [Added: from here: "Of Americans, some 30,000 have more than $100m and 2,800 have more than $500m", so < 0.01% of Americans have that much money.]

Note that Bill Gates "only" has ~$75 billion and that Mark Zuckerberg "only" has ~$30 billion, so that they don't account for Harvard's decisive advantage over other universities.

What is going on here? Why would Harvard come out ahead by such a large margin? Its acceptance rate is smaller than those of Stanford, Yale and Princeton, but by no more than 25%. Moreover, this has been true historically

One can ask a similar question of University of Pennsylvania, which is significantly less selective than Yale and Princeton, but has a decisive advantage over them.

Some possible explanations for the discrepancy:

Harvard selects students with higher expected earnings.

Harvard selects students from families who are already wealthy (more than other top schools: the wealthy alumni are not self-made).

Students who aspire to make a lot of money choose Harvard (over other top schools).

Going to Harvard increases students' earning power (more than other top schools), for example, through better networking opportunities.

Regardless of which of these explanations hold, there remains a question of why they would hold.

Is Harvard a better choice than other universities for students who aspire to be wealthy than other top schools?

Even if Harvard has historically been better for increasing expected earnings, this may have ceased to be the case because of the rise of the tech sector and Stanford and MIT's focus on tech.
High school students and epistemic rationality
by VipulNaik — 2014-03-15T17:40:58.192Z

In a recent post, I considered the feasibility and desirability of exposing high school students to the ideas of effective altruism. In this post, I consider the value of exposing them to the idea of epistemic rationality. Epistemic rationality refers to rationality in thinking about stuff. This is related to but distinct from instrumental rationality, which is rationality in one's actual decisions and actions in the pursuit of life goals. For more on the distinction, see here, here, and here.

Epistemic rationality is championed at LessWrong and by the organizations affiliated with LessWrong (including CFAR and MIRI). It's also potentially of broader interest than effective altruism, although in my mind, the two idea clusters are closely intertwined.

As with my effective altruism post, I consider two questions:

Are people in high school ready to understand the ideas of epistemic rationality?

Are there benefits from exposing people to epistemic rationality ideas when they are still in high school?

1. Are people in high school ready to understand the ideas of epistemic rationality?

The answer to this question largely depends on what people you're referring to, and what ideas you are referring to. The ideas involved range from the sort that anybody who plans to go to college should be able to understand, to ones that require a good grounding in probability theory, economics, calculus, or other subjects. An abstract understanding of basic cognitive biases, such as correlation versus causation, confirmation bias, the fundamental attribution error, or the illusion of transparency, is at the easy end. Something like the litany of Tarski is probably somewhere in the middle. A proper understanding of conditional probabilities and Bayes' theorem is at the hard end. It's possible to convey such understanding without the technical mathematics, but that arguably requires even more skill on the part of both the teacher and the learner. There's also a significant gap between just having an abstract understanding of a cognitive bias and actually applying it when thinking about specific problems. The factors that predict whether a person will actually apply their epistemic rationality to specific situations is unclear. In particular, it's not necessarily true that more intelligent people will apply their abstractly acquired rationality to thinking about problems, at least onc...
Democracy and individual liberty; decentralised prediction markets
by Chrysophylax — 2014-03-15T12:27:37.050Z

A pair of links I found recently (via Marginal Revolution) and haven't found on LW:

http://www.cato-unbound.org/2014/03/10/mark-s-weiner/paradox-modern-individualism

https://bitcointalk.org/index.php?topic=475054.0;all

The former discusses liberty in the context of clannish behaviour, arguing that it is the existence of the institutions of modern democracies that allows people individual liberty, as it precludes the need for clan structures (extended family groups, crime syndicates, patronage networks and such).

The latter is a author's summary of a white paper on the subject of decentralised Bitcoin prediction markets with a link to the paper.
Shoulds can be changed to Cans
by Stabilizer — 2014-03-15T09:05:18.916Z

Here is an interesting exercise. Whenever, you have a 'should' statement, see if you can change it to a 'can' statement, and notice being more empowered. Examples:

Not, I should be grateful. Instead, I can be grateful! [To clarify, I mean that being grateful is something that will make you feel good. As an analogy, if you have tasty cookies lying around the house, you will say to yourself: "well, I can certainly get cookie."]

Not, I should leave a generous tip. Instead, I can leave a generous tip! [As in, you can leave a generous tip and feel good about it.]

Not, I should donate to charity. Instead, I can donate to charity!

Not, I should loosen-up sometimes. Instead, I can loosen-up sometimes!

Not, I should keep in touch with old friends. Instead, I can keep in touch with old friends!

Not, I should learn to program. Instead, I can learn to program!

Not, I should eat healthy. Instead, I can eat healthy!

The general pattern here is that, instead of making the activity a moral duty, you can make it something fulfilling which you choose to do because of its benefits.

For 'should not' statements you can substitute: 'I choose not to.' One example (you can make up more):

Not, I should not eat refined sugar. Instead, I choose not to eat refined sugar.

I'm sure that this procedure doesn't always work and you can generate counterexamples. I have not done them here. Please share your examples (counterexamples).