trentmkelly/LessWrong-43k huggingface dataset:
Form Your Own Opinions
by Ben Pace — 2018-04-28T19:50:18.321Z

Follow-up to: A Sketch of Good Communication

Question:

Why should you integrate an expert's model with your model at all? Haven’t you heard that people weigh their own ideas too heavily - you should just defer to them.

Here’s a quick reminder of the three step process:

And this new proposal, I think, suggests changing what you do after step 2.

The people who have the best ideas, as it seems to me, often change their plans as a result of their debates in all the other fields. Here’s three examples[1] of people doing this with AI timelines.

[Person 1] Oh, AI timelines? Well, I recall reading that it took evolution 109years to go from eukaryotes to human brains. I’d guess that human developers are about 106 times more efficient than evolution, so I expect it to take 1000 years to get there from the point where we built computers. Which puts my date at 2956.

If I’m wrong I’ll likely have to learn something new about developers competence relative to evolution, or about how humans get to do a type of intelligence evolution wasn’t allowed to for some reason.

[Person 2] Oh, AI timelines? Well, given my experience working on coding projects, it seems to me that projects take 50% extra time to run than you'd expect once you've got the theory down, so I'll take the date by which we should have enough hardware to build a human brain, estimate the coding work required for the necessary project, and add 1.5x time to it.

And if you change my mind on this, it will help make my models of project time more accurate, and change how I do my job.

[Person 3] Oh, AI timelines? Well, given my basic knowledge of GDP growth rates I’d guess that being able to automate this percentage of the workforce would cause a doubling every X unit of time, which I expect for us (at current rates) only to be able to do after K years.

If you show me I’m wrong it’ll either be because GDP is not as reliable as I think, or because I’ve made a mistake extrapolating the trend as it stands.

Can you see how a conversation between either of these two people would lead them both to learn not only about AI, but also about models of evolution/large scale coding projects/macroeconomics?

Recently, Jacob Lagerros and I were organising a paper-reading session on a recent Distill.pub paper, and Jacob was arguing for a highly-structured and detailed read-through of the paper. I wanted to focus m...
Mere Addition Paradox Resolved
by David Cooper — 2018-04-27T21:13:22.857Z

Diagram:

See https://en.wikipedia.org/wiki/Mere_addition_paradox for the diagram, though in case the key parts of it are edited into a different form in the future, I'll provide a description here (adding appropriate numbers of my own invention, chosen to reflect the height of the bars in the diagram)

Description of diagram:

Population A might have 1000 people in it with a quality of life of 8, which we'll call Q8.

Population A+ is a combination of 1000 people at Q8 (population A) plus another 1000 people at Q4 (population A', though this population is not normally named).

Population B- is a combination of two lots of 1000 people which are both at Q7.

Population B has 2000 people at Q7.

The distinction between group B and B- is that B- keeps the two lots of 1000 people apart, which should reduce their happiness a bit as they have fewer options for friends, but we're supposed to imagine that they're equally happy whether they're kept apart (as in B-) or merged (in B).

Parfit's argument (to illustrate the paradox):

"Parfit observes that i) A+ seems no worse than A. This is because the people in A are no worse-off in A+, while the additional people who exist in A+ are better off in A+ compared to A [where they simply wouldn't exist] (if it is stipulated that their lives are good enough that living them is better than not existing)."

"Next, Parfit suggests that ii) B− seems better than A+. This is because B− has greater total and average happiness than A+."

"Then, he notes that iii) B seems equally as good as B−, as the only difference between B− and B is that the two groups in B− are merged to form one group in B."

"Together, these three comparisons entail that B is better than A. However, Parfit observes that when we directly compare A (a population with high average happiness) and B (a population with lower average happiness, but more total happiness because of its larger population), it may seem that B can be worse than A."

Paradox Lost:

First of all, we need to understand why there should be an optimal population size for a given amount of available resources, and if the population grows too high, total happiness goes down rather than up. This must be the case because the happiness of a population falls to zero long before the resources per person approach zero, and if you drag people out of ...
Noticing the Taste of Lotus
by Valentine — 2018-04-27T20:05:23.898Z

Recently I started picking up French again. I remembered getting something out of

Duolingo

a few years ago, so I logged in.

Since the last time I was there, they added an “achievements” mechanic:

I noticed this by earning one. I think it was “Sharpshooter”. They gave me the first of three stars for something like doing five lessons without mistakes. In the “achievements” section, it showed me that I could earn the second star by doing twenty lessons in a row flawlessly.

And my brain cared.

I watched myself hungering to get the achievements. These arbitrary things that someone had just stuck on there… in order to get me to want them. I noticed that I could get the second and maybe third star of “Sharpshooter” by doing earlier lessons and googling words and phrases I wasn’t quite sure about…

…which

really doesn’t help me learn French

.

Yes, we could quibble about that. Maybe perfect practice makes perfect, yada yada. But the point is: I disagree, I think my disagreement comes from knowing what I’m talking about when it comes to my learning, and someone’s arbitrary gold stars immediately overrode all that insight by grabbing my motivations directly.

I don’t have a problem with gamification per se. What bugs me here is that this specific gamification didn’t fit my goals, and that fact didn’t at all affect how well the system grabbed my wanting. I just… wanted those achievements. Because they were there.

If I hadn’t noticed this, and if I’m right about what I need to learn French, then I would have wasted a bunch of time pursuing a useless proxy goal. And I would have felt pleasure in achieving it. I might have even thought that was a meaningful sign that I was learning French — never mind that my goal of holding my own in conversations isn’t really helped by carefully avoiding typos.

Duncan Sabien sometimes talks about “lotus-eating”. He’s referring to

a part of the Odyssey

where they land on an island of “lotus-eaters”. It turns out that once you eat some of this kind of lotus, all you want to do is eat more. You stop caring about your other goals. The lotus just grabs your wants directly.

I claim you can notice when something grabs your wanting. Just… look. Just pay atten...
Double Cruxing the AI Foom debate
by agilecaveman — 2018-04-27T06:46:39.468Z

This post is generally in response to both Paul Christiano’s post and the original foom debate. I am trying to find the exact circumstances and conditions which would point to fast or slow AI takeoff.

Epistemic Status: I have changed my mind a few times about this. Currently thinking that both fast and slow takeoff scenarios are plausible, however I put more probability mass on slow takeoff under normal circumstances.

Just to be clear about the terminology. When we say fast take-off, we talk about an AGI that achieves a decisive unstoppable advantage from a “seed AI” to a “world optimizing AGI” in less than a week. That is, it can gather enough resources to improve itself to the point where it’s values come to dominate human ones for the direction of the future. That does not necessarily mean that the world could end in the week if everything went badly, rather than the process of human decline would become irreversible at that point.

From here

When we talk about slow takeoff, we are talking about a situation where the pace of economic doubling is being gradually reduced to less than 15 years, potentially to 1-2 years. Note, this “slow” is still a fundamental transition of the human society on the order of the agricultural or industrial revolution. Most people in the broader world think of significantly slower transitions that what the AI community considers “slow”. “Middle takeoff” is something in between the two scenarios.

There are several analogies that people use to favor slow take-off, such as:

a) Economic Model of gradually replacing humans out of the self-improvement loop

b) Corporate model of improvement or lack there of

c) Previous shifts in human society

There are several analogies that people use to favor fast take-off, such as:

a) Development of nuclear weapons

b) The actual model of how nuclear weapons work

c) Development of human brains compared to evolution

There are also several complicating factors, such as:

a) The exact geo-political situation, such the number, sophistication and adversarial nature of the players.

b) The number of separate insights required to produce general intelligence

c) Whether progress occurs in hardware or software or both

d) Improvement in human-computer interfaces compared to improvement in stand-alone algorithms.

The exact scenario we are considering is the following. Sometime in the lab, we get an insight that fina...
LW Update 4/26/2018 – Images, Events and Recent Discussion
by Raemon — 2018-04-27T04:55:55.310Z

Smaller update (trying to get into a shorter dev cycle)

Images – We have a quick-and-dirty solution to most of the missing images from the sequences. Longterm, we plan to replace them with the higher res images from Said Achmiz's ReadTheSequences.com. Some non-sequence image links are still broken, which will be a bit more annoying to get up and running.

Events in Recent Discussion – We're still thinking about how to properly incorporate meetups (both in having them appear on the frontpage, and in having the community page be a place that people have a reason to check regularly).

But, meanwhile, we added Event posts to the Recent Discussion section, to help give them a bit more visibility. (They'll show up when someone first posts an event, and again if they event is commented on)

Shorter window of comments appearing in Recent Discussion – Some people have complained that some comments in Recent Discussion are old, which makes it harder to quickly check if you've seen all the latest things in that section. Previously, I'd made it so that if a post was recently commented on, in appeared in Discussion with all comments from the past 2 days visible.

I just shortened that window to 18 hours, hoping that'd capture enough recent comments to see the latest chunk of conversation, but no so much that you'll see a bunch of old comments over and over.

Slight Tweak to Padding on mobile – Special thanks to James Lamine who did his first open source contribution to site, updating the padding on mobile-size screens to look a bit nicer.

Link to Git Log (commit ca50a9f)
The 3% Incline (theferrett.com)
by Raemon — 2018-04-27T00:56:30.869Z
Link: http://www.theferrett.com/ferrettworks/2018/04/the-3-improvement/

I liked this recent post by The Ferrett (a blogger who discusses a number of issues including gaming, polyamory, handling mental illness, politics and various kinds of relationships).

In particular, this seemed like a clear encapsulation of one of the concepts I touch on at Sunset at Noon, without all the surrounding baggage.

The rationality community has this vague undercurrent narrative of "Self Improvement! You can Radically Change! All problems are fixable! Growth Mindset!". And this isn't false per se, but the tone of it often sort of implies overnight change.

I think occasionally people can change radically, but this most often seems to have to do with a change in environment (new job, new house, new community, etc). Changing environments is pretty expensive and a bit of a crapshoot.

For personal, dedicated self improvement, a more realistic rate-of-return is something like 3% (not quite sure on what timescale this means. "Per month" feels a bit too short and "per year" feels a bit too long. Maybe per quarter of 6-month interval?)

In Sunset at Noon I had referred to this as "the 2% incline". I like Ferrett's post as a canonical Post Referring To This Concept, and had sort of picked 2% out of a hat, so I'm editing the original post to use 3%.
Computational Morality (Part 4) - Consequentialism
by David Cooper — 2018-04-26T23:58:40.689Z

I suggested in part 3 that all the best proposals may be converging in on the same destination, and that we might be able to use my method of calculating morality to help with the process of unification of the best ones (and rejection of the hopeless ones). So, in the absence of a league table of the best proposals, we'll just have to examine them as they turn up and hope that we encounter all the ones that matter. In the comments under Part 1, a link was provided to a page which provides a good starting point, and I'm going to use it to show where people have been making fundamental errors which have driven them in ill-advised directions: https://plato.stanford.edu/entries/consequentialism/ - it may be worth keeping it open in another tab as you read this.

What do we see on that page? Well, let's start at section 3. In the second paragraph we see an objection to the idea of an unsophisticated game being regarded as being as good as highly intellectual poetry. Now, such poetry is outside of my experience, but we can substitute a play by Shakespeare, such as King Lear, while the unsophisticated game can be darts. A common man who likes spending time in the pub might well turn down an invitation to go to see this Shakespeare play in favour of a few games of darts with his friend, while a more sophisticated individual might scoff at that and head for the theatre. If both derive exactly the same amount of pleasure from their chosen activity (which includes all the intellectual satisfaction of the appreciation of the high quality content of the play, and the intellectual satisfaction of the calculating the score for the darts player), who's to say that the Shakespeare play is really the superior option? Another intellectual with no tolerance for cheap manipulation may rate the Shakespeare play very differently from the average intellectual, finding the play deeply unsatisfying because of the extreme artificiality where two of the daughters express no love at all for anyone other than their father while the third is so cold that she doesn't make any attempt to explain that she loves her father just as much as her sisters do (or more). She is so unnaturally cold that she comes across as not caring at all. Right from the opening, the rest of the play can be predicted, all apart from the ending which will either be happy (like a lightweight fa...
Welcome to Ottawa Rational Dojo [Edit With Your Details]
by daniel zev — 2018-04-26T18:36:25.944Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
The Leading and Trailing Edges of Development
by Gordon Seidoh Worley — 2018-04-26T18:02:23.681Z

I've recently been working with the ox herding pictures in my zen practice. For those not familiar, the ox herding pictures depict a developmental model of practice and stages through which a person passes on their way to enlightenment. The pictures describe progress generally and capability within specific aspects of practice, like meditation, specifically, and in using them to guide my own practice I've been reminded of an idea from my study of developmental psychology that I live with but not written much about—the leading and trailing edges of development.

The name gives away most of the idea, but for the sake of clarity I'll describe it in some more detail. If we assume a developmental model of human psychology (like the ones of Kegan, Commons, me, Beck & Cowan, and others) or even just a positive psychology model (e.g. growth mindset, positive disintegration, etc.) and permit that development doesn't happen evenly (a person may be more developed in some ways than others), then we can talk about the leading and trailing edges of development as referring to those aspects of a person that are most and least developed in terms of any particular model. I'll make this concrete with some personal examples.

I had an awakening experience a few years ago that I interpret as tipping me between predominately systems-relations thinking (Kegan 4) and predominately holonic thinking (Kegan 5). That said this was only true of my leading edge in aspects of my life related to things like my work in philosophy and organizational development. My trailing edge, by comparison, remained possibly as far back as thing-level thinking (Kegan 2) when it came to things related to, for example, romantic relationships. Now my leading edge gave me tools that helped me pull forward my trailing edge, but that was work I had to do and did not happen automatically, so knowing about my trailing edge helped me know where to work to catch up certain aspects of myself to be closer to the leading edge.

In my zen practice I similarly notice that, in some ways I express things you might associate with the 9th or 10th ox herding picture, like my general equanimity, surrender to the perfection of existence, and acceptance of emptiness, but in other ways I lag behind because I have a sense of self of the sort you might associate more with the 6th or 7th picture and my meditation practice could be ...
The One-Channel Paradigm
by Noah Walton — 2018-04-25T21:49:56.176Z

Under the one-channel paradigm, one thing does one thing at a time.

The general thing that follows the one-channel paradigm is the game. Or maybe physics or a Markov process.

This seems to be a common paradigm. It also leads to bad mistakes.

“What shall I do? Well, I want to learn some economics. So I’ll make a tree (a DAG, actually) of concepts that I need to know, and learn from the bottom up. At the end, I’ll know the whole tree. Then I can get building a bigger tree off of that tree.”

This is better than thinking of learning as Level 0, Level 1, Level 2, etc., where each Level is one concept. Since each Level can have many members, and you can branch out in many different ways, a tree makes more sense than a tower. (What about a cyclic knowledge graph? We might want to account for iff statements or meta-stable principles.)

But maybe the one-channel paradigm is not a useful way to think about problems, a lot of the time. Even if it’s “true". Even if we live inside a Markov process, it might be useful to think of things as not being that way.

Because subscribing the one-channel paradigm destroys a lot of valuable things humans can do. It tends to overrate conscious thought, for example. Conscious thought is the obvious candidate for the channel to identify with. Notice the mistake here. We went from treating the whole world as a Markov process, to singling out a rather unnatural part of spacetime (conscious thought) and describing it as a coherent stream separate from the rest of the world.

For me, I think this error leads to wanting to do too many atomic activities, or at least thinking of my activities as atomic. Playing a game. Playing a piece of music. Writing a post. Eating a meal. Learning a topic.

There should be alternative ways of thinking about these things, under a paradigm other than the one-channel one. I don’t know how to express those views in language, though. But maybe stories are good examples.

It seems that reading a whole book can often give much greater returns than reading many isolated pieces of information. Would you rather read the dictionary or Anna Karenina?

Why? I could give some bullshit answer about maybe Anna Karenina giving us a better chance to accomplish interesting compression: a lot of the compression done while reading the dictionary seems much duller — a dictionary is just filling in a template over and over again, and ...
Effective Altruism's Ultimate Goal: Eradicate Human Suffering. 
by willfranks — 2018-04-25T12:12:15.660Z

The Highest Good We Can Serve

The Effective Altruism movement aims to answer the question "how can we use our resources to help others the most?". Well, let us ask: what is the most we could ever help someone? In the quest for human progress, what would it mean to go all the way? What’s the highest, most altruistic goal we could shoot for, both as individuals and as a society? I venture that it is precisely this: to end (or fully minimise) suffering for every person on Earth. (The suffering of other sentient beings I consider a slightly separate issue and will exclude from this post for the sake of brevity).

If there are states of mind and brain that correspond to minimal suffering and maximal psychological freedom, awareness, happiness and love, should we not be prioritising dissemination of these states to as many people as possible, in line with the goal of Effective Altruism to help as many people as possible, as much as possible? A strategy for reaching this goal becomes obvious when we consider that there exist reliable, systematic methods to attain these mind-states; methods of training the mind that can be easily and often freely transmitted from human to human through verbal instruction.

The task of alleviating psychological suffering, even partially, meets the framework of a high-impact problem as described in an Introduction to Effective Altruism:

Great in scale (just about every single human is subject to unnecessary psychological suffering)
Highly neglected (techniques to alleviate suffering are esoteric and practiced only by a tiny majority), and
Highly solvable (techniques and tools exist to alleviate suffering, and allocating funds towards their dissemination will increase their adoption - especially since they are mostly free or very cheap!).

Over and above meeting people's basic physiological and financial needs - which dominates the focus of modern philanthropy - we are faced with a 7-billion-instance problem of individual human psychology, and how to alter it to minimise suffering and maximise its positive aspects. Because once everybody on Earth is clothed, educated, working and fed, what then? We in the developed, materially abundant West know that there will still be widespread mental affliction (as evidenced by an epidemic of mental health issues such as anxiety and depression).

Eradicating human suffering is perhaps the high...