trentmkelly/LessWrong-43k huggingface dataset:
Why no total winner?
by Paul Crowley — 2017-10-15T22:01:37.920Z

Why doesn't a single power rule the world today?

[I'm taking advantage of the new "LW posts as blog posts" format to post something I'm pretty unsure about. I'm working from my memories of the blog posts, and from a discussion I had with Robin Hanson and Katja Grace in late 2012. Please let me know if any of this is inaccurate!]

One of the key differences of opinion in the Hanson-Yudkowsky AI-Foom Debate is about the idea of a "decisive advantage". If I'm not misrepresenting the parties horribly, the idea is that some point in a world with AGI, some AGI-enabled party uses their greater intelligence to increase their general power: money, resources, control over others, intelligence and suchlike. That greater power increases their ability to gain power, resulting in a snowball effect that ends with some party having control over the outcomes for all of Earth-originating life.

Robin asks the very reasonable question: if that's how things work, why hasn't it already happened? What stops the largest business using its decisive power over smaller ones to defeat and absorb them, growing ever larger and more powerful until all other businesses fall to its power? Why do we have multiple nations today, when this model would seem to predict that a single state should ultimately conquer and rule all? I don't remember Robin proposing an answer of his own: a mechanism or theoretical model that would lead us to expect multiple powers. But it seems like a good question, and it's bugged me ever since.

I think I'd need to be much more of a student of history than I am to have any confidence in an answer, so let me share some wild speculation that might at least start discussion:

Regulation: Such growth isn't an option for legal businesses at all, because states exist. So powerful a business would challenge the power of the state, and the state is in a position to disallow that. The explicit purpose of monopoly legislation is to stop a business which has become very powerful in one area from leveraging that to become powerful elsewhere.

Principal-agent problems: it sure would be easier to keep an empire together if you could reliably appoint generals and rulers who always did what you told them to. Especially if the round-trip time for getting them a message is on the order of weeks, and you have to entrust

...
Identities are [Subconscious] Strategies
by Ruby — 2017-10-15T18:10:46.042Z

We all have identities. Arguably, any statement of the form “I am a __” is an identity. Of course, we usually reserve the term for the statements which feel especially core to us in describing and predicting ourselves, and in expressing our values and aspirations. Such identities may have their benefits, but they also come with a number of perils. In particular, we are prone to a) become distressed by any perceived threat to an identity, and b) become utterly inflexible around shifting, modifying, or discarding identities. These effects can be detrimental to both personal wellbeing and goal attainment. Sanity, however, can be regained if we recognize that our identities do not exist unto themselves, and are instead [often subconscious] strategies towards the attainment of specific goals and values.

Identities are Strategies towards Goals

Consider a person who prides themselves on their identity as a writer: “I am a writer.” This identity is precious because there is an implicit statement of the form “I am a writer[, and therefore I will have a job, income, status, friends, lovers, and my life will be good].” The implicit statement is the goal to be obtained and the explicit identity is the strategy for achieving that goal. The value of the identity derives from the goal is supports.

I describe these plans as subconscious because more often than not they are not articulated. Many people have an identity around being intelligent, but I expect that if you ask them why this important, they will need a few moments to generate their answer. I also expect that in many cases the belief in the goodness of an identity is absorbed from society and it is social drives which motivate it for an individual. In that case, the full identity statement might go “I am a __ [and therefore society will approve of me]” whether or not an individual would admit it. In the most general case, it’s “I am a __ [and therefore goodness].”

Threats to the Identity are Threats to the Goal

Given that an identity is a strategy for achieving a goal, any threat to the identity is a threat to the goal. The degree of threat perceived is proportional to the importance of the goal and to the extent that the identity is sole strategy for achieving the goal. If someone believes that being a writer is their sole avenue for having a good and fulfilling life, they are going to get upset when that identity i

...
Building an civilisation  scale OODA loop for the problem of AGI
by  — 2017-10-15T17:56:55.582Z

You can break down our civilizations reaction to the problem of AGI into a massive decentralized OODA loop. Each part of the OODA loop is not one person but an aggregate of many people and organisations.

My current major worry is that we do not have a robust loop.

Observations: There are a few major observations we have made that inform our current work. AGI systems might be more powerful than humans and able to make themselves more powerful again via RSI. We can't control our current reinforcement learning systems

Orientation: This is the philosophy and AI strategy work of FHI and others

Decide: This is primarily done inside the big foundations and soon the governments

Act: This is OpenAIs work on AI safety for RL. Or instituting AI policy.

What I think we are not doing is investing much money into the observation phase. There is a fair amount of observation of the RL work, but we do not have much observation going into how much more powerful AGI systems will be and can be made via RSI.

One example of the observations we could make, would be to try and get an estimate of how much speeding up human cognitive work would speed up science. We could could look at science from a systemic perspective and see how long various steps take. The steps might be

Gathering data

Analysing data

Thinking about the analysis

Each of these will have a human and a non-human component (either collecting data or computational analysis of the data).
If we could get better observations of how much each component has we can get an estimate of how quickly things could be sped up.

Similar observations might be made for programming, especially programming of machine learning systems.
I will try and write a longer post at some point, fleshing things out more. But I would be interested in peoples other ideas on how we could improve the OODA loop for AGI.
The Fish-Head Monk
by Zvi — 2017-10-14T12:10:00.359Z

Epistemic Status: Parable

By way of the comments on SlateStarCodex, from Essential Sufism:

‘There was a poor fisherman who was a Sufi teacher. He went fishing every day, and each day he would distribute his catch to the poor of his village, except for a fish head or two that he used to make soup for himself His students dearly loved and admired their ‘fish-head sheikh.”

One of the students was a merchant. Before traveling to Cordoba, the teacher asked him to convey his greetings to his own teacher, the great sage Ibn ‘Arabi, and to ask the sage for some advice to help him in his own spiritual work, which he felt was going very slowly.

When the merchant arrived at Ibn ‘Arabi’s house, he found, much to his surprise, a veritable palace surrounded by elaborate gardens. He saw many servants going back and forth and was served a sumptuous meal on gold plates by beautiful young women and handsome young men. Finally he was brought to Ibn ‘Arabi, who was wearing clothing fit for a sultan. He conveyed his teacher’s greetings and repeated his teacher’s request for spiritual guidance. Ibn ‘Arabi said simply, “Tell my student that he is too worldly!” The merchant was shocked and offended by this advice coming from someone living in such worldly opulence.

When he returned, his teacher immediately asked about his meeting with Ibn ‘Arabi. The merchant repeated Ibn ‘Arabi’s words and added that this sounded totally absurd coming from such a wealthy, worldly man.

His teacher replied, “You should know that each of us can have as much material wealth as his soul can handle without losing sight of God. What you saw in him was not merely material wealth but great spiritual attainment.” Then the teacher added, with tears in his eye, “Besides, he is right. Often at night as I make my simple fish-head soup, I wish it were an entire fish!” ‘

The comment finishes, and others agree:

Plausible, but one might think “pretty convenient for those with temporal power, especially holders of hereditary positions, as a lot of Sufi leadership is.”

Responders were also not thrilled by Ibn ‘Arabi’s behavior:

Truly, The Donald is the greatest Sufi master.

and:

Yyyyeah… that sounds a lot like the kind of abuse that gets used as an example when they warn you about joining a cult.

The fish-head monk understands.

Often at night as he makes his simple fish-head soup, he wishes it were an entire fi...
Beginners’ Meditation
by Zvi — 2017-10-14T12:10:00.338Z

I was about to meditate. There was a beginners’ meditation class three blocks away at the 14th Street Y. Seemed wrong to not check it out. Low risk, potential high reward. Non-zero story value. Might even learn something.

It was held in Room 403. Not in the gym. In the preschool. Not great.

I looked inside. I saw a circle of about twenty folding chairs. Sitting in most of the chairs were very old women, chatting. Whatever filter was operating was not subtle.

It said, you are not the target.

Welcome to Meditators Anonymous. My name is Zvi. Hi, Zvi.

I asked if I was in the right room, hoping I wasn’t. I was. Despite lowered expectations, I sat down.

They passed around a petition to keep the class going after its one month engagement. This was class two. Impressive customer loyalty. Was a there there after all?

Several minutes late, the circle was complete. Me. One instructor. One Asian man. One Amy Poehler lookalike. Twenty-two very senior female citizens.

We were told meditation was us getting to know ourselves. Accepting our own friend request on Facebook. I tried not to make too much of a face. I tried not to make too little of a face. Would have been a lie. People deserve honest feedback.

We went around, said our names and mentioned something that warmed our hearts this past week. To get to know each other. I noticed I was confused. What did that have to do with meditation?

Some mentioned small niceties. Most mentioned babies. They especially loved fathers with babies, doing heroic things like pushing strollers or carrying the baby.

Here’s to you, baby carrying father. Here’s to you.

My son Gideon had been born the previous Thursday. So I won.

I like victory. Big fan. Passed the test just like all the rest. But never really understood the reasons why I took it in the first place.

We were told to sit comfortably in our folding chairs. Not a beginner task.

We were told to sit up fully straight, in a relaxed position. Some people can do that.

We were told to keep our eyes open, aiming six feet ahead on the floor, to take our practice into the world. We were told to relax our shoulders, and put our hands on our thighs. Can do.

Focus on the breath, either at the belly, chest or nose. If we have thoughts, label that thinking and come back to the breath. She mentioned things we might be distracted by. They were distracting. I got briefly hungry. I came bac...
Welcome to the World
by Zvi — 2017-10-14T12:10:00.282Z

Epistemic Status: Good news, everyone!

On October 5, 2017, my second son was born. Everyone say hello to the wonderful Gideon Mowshowitz:

As you can imagine, it’s been quite a week. We are all doing well, and slowly recovering aside from lack of sleep.

I encourage you to send your congratulations, but as an experiment I’m adding a twist: You need to say congratulations differently than all previous congratulations. No duplicates!

As you can also imagine, my available time will be limited in the coming weeks and months, so posting likely will be lighter. This is also a natural stopping point. I’ve said the community-related things I most needed to say. I have more thoughts, and will doubtless share some, but they seem less pressing. Sequence complete.

If you wish to join us for Friday night Sabbath dinner, please email me to let us know so we can prepare. If you can found this and can figure out my email, you’re probably invited. We’d love to meet some new people, and to see some old friends. We start at 6:30. Also let me know if you’d like to swing by on a Saturday to chat or watch some football.

My offer of strategic consultation for all causes I don’t actively oppose remains open. Only two people have taken me up on it, so I am reminding everyone that it exists. Response times might be not so fast right about now, and if I get too many requests I’ll have to limit this, but for now the line is open.

In conclusion, celebrate. Good times. Everyone around the world, come on!
You can never be universally inclusive
by Kaj_Sotala — 2017-10-14T11:30:04.250Z

A discussion about the article “We Don’t Do That Here” (h/t siderea) raised the question about the tension between having inclusive social norms on the one hand, and restricting some behaviors on the other hand.

At least, that was the way the discussion was initially framed. The thing is, inclusivity is a bit of a bad term, since you can never really be universally inclusive. Accepting some behaviors is going to attract people who like engaging in those behaviors while repelling people who don’t like those behaviors; and vice versa for disallowing them.

Of course, you can still create spaces that are more inclusive than others, in being comfortable to a broader spectrum of people. But the way you do that, is by disallowing behaviors that would, if allowed, repel more people that the act of disallowing them does.

If you use your social power to shut up people who would otherwise be loudly racist and homophobic and who then leave because they don’t want to be in a place where those kinds of behaviors aren’t allowed, then that would fit the common definition of “inclusive space” pretty well.

That said, the “excluding racists and homophobes” thing may make it sound like you’re only excluding “bad” people, which isn’t the case either. Every set of rules (including having no rules in the first place) is going to repel some completely decent people.

Like, maybe you decide to try to make a space more inclusive by having a rule like “no discussing religion or politics”. This may make the space more inclusive towards people of all kinds of religions and political backgrounds, since there is less of a risk of anyone feeling unwelcome when everyone else turns out to disagree with their beliefs.

But at the same time, you are making the space less inclusive towards people who are perfectly reasonable and respectful people, but who would like to discuss religion or politics. As well as to people who aren’t so good at self-regulation and will feel uncomfortable about having to keep a constant eye on themselves to avoid saying the wrong things.

And maybe these people would feel more comfortable at a different event with different rules, which was more inclusive towards them. Which is fine. Competing access needs:

Competing access needs is the idea that some people, in order to be able to participate in a community, need one thing, and other people need a conflicting thing, and inste...
"Focusing," for skeptics.
by Conor Moreton — 2017-10-14T07:00:00.000Z
Link: https://medium.com/@ThingMaker/focusing-for-skeptics-6b949ef33a4f

Gendlin’s Focusing technique is super rad. I know this because everybody keeps telling me so.

(Okay, not quite everybody, but a really tediously large percentage of the people in my online social circle.)

But I’ve tried it a bunch of times, in a bunch of variants, with a bunch of qualified mentors trying to help, and it’s just never clicked. I’ve listened to the audio book and gone through all the steps, and it just doesn’t do anything for me.

So here’s my variant—the thing that I do instead, which I claim is using the same hardware and software and providing me with the same kind of improved introspective access. If you’re one of those skeptics who thought it all sounded nuts, or one of those unlucky people who thought it sounded awesome but could never make it work, this post has your name on it.

The “big idea” of Focusing (according to me) is that parts of your subconscious System 1 are storing up massive amounts of accurate, useful information that your conscious System 2 isn’t really able to access. There are things that you’re aware of “on some level,” data that you perceived but didn’t consciously process (see

blindsight

as both concrete example and metaphor), competing goalsets that you’ve never explicitly articulated, and so on and so forth.

Focusing is a technique for bringing some of that data up into conscious awareness, where you can roll it around and evaluate it and do something about it. Half of the value comes from just discovering that the information exists at all (e.g. noticing feelings that were always there and strong enough to

Imperius

you but which were somewhat “under the radar” and subtle enough that they’d never actually caught your attention), and the other half comes from having new models to work with and new theories to test. If I manage to recognize that e.g. a significant chunk of my romantic problems stem from self-censorship pressure because of a strong aversion to seeming needy, I suddenly have threads to pull on rather than falling back to just “Yeah, things with Cameron are not great.”

The actionable claim of Focusing (again, according to me) is that this information expresses itself in “felt senses” in the body—think butterflies in the stomach, or your throat closing up, or the heat of embarrassment in your cheeks, or a heavy sense of doom that makes your arms feel leaden and numb, or whatever physiological sensation happens ...
Activated social login, temporarily deactivated normal signup
by habryka — 2017-10-14T03:59:11.873Z

I have just activated Facebook, Google and Github signup. It currently automatically gives you a username that is just your first name and last name concatenated (or your Github username), but it probably makes sense to allow you to change your username manually in the long-run. Let me know if you run into any problems.

We've also had some spam accounts being a bit annoying, so I've deactivated normal account signup. I will activate it again after I've built some smarter spam account detection systems. If you need an account not associated to a social login in the meantime, just ping me on Intercom and I will set you up with something.

Oh, and also comments at -5 karma or below are now by default collapsed.
Instrumental Rationality 5: Interlude II
by  — 2017-10-14T02:05:37.208Z

[Instrumental Rationality Sequence 5/7]

[This Interlude once again goes over two additional ideas that are separate from the well-researched stuff: There Is No Akrasia and Recovering from Failure. The first endorses a reductionist, specific view towards tackling akrasia, while the second is about having a policy of self-care when you inevitably fail at your endeavors.]

There Is No Akrasia:

[This essay is about how the term “akrasia” isn’t too useful. I argue against using any sort of general label for the feeling of “anti-wantiness”, i.e. when you don’t want to do something. Instead, I push for a reductionist approach to look at the problem.]

“Akrasia” is a term often used to mean “weakness of will”, aka the intention-action gap we covered in Scaling Up in Instrumental Rationality 4.2. It’s when you somehow “want” to do something, yet you still don’t actually do it.

I also think it's an idea that incurs potentially major costs when you hold it in your bag of mental models. I claim that:

Akrasia is often treated as a “thing” by people who learn about it, and this can lead to problems, even though akrasia a sorta-coherent concept.

If we want to move forward and solve the problems that fall under the akrasia-umbrella, it’s better to Taboo the term akrasia altogether and instead employ a more reductionist approach that favors specificity

First off, I do think that akrasia is a term that resonates with a lot of people. When I’ve described this concept to friends, they’ve all had varying degrees of reactions along the lines of “Aha! This term perfectly encapsulates something I feel!”

It does seem, then, that this concept of “want-want versus want” or “being unable to do what you ‘want’ to do” seems to point at a real group of things in the world, at least from a perception standpoint.

However, I think that this might have inadvertent problems.

Once people learn the term akrasia and what it represents, they can now pattern-match it to their own associated experiences. I think that, once you’ve reified akrasia, i.e. turned it into a “thing” inside your worldview, problems can occur.

First off, treating akrasia as a real thing gives it additional weight and power over you:

When you start to notice the patterns, you can start see akrasia everywhere, even in places where it might not be immediately useful. One example is when people may try less hard because they

...
Offloading Executive Functioning to Morality
by weft — 2017-10-14T01:43:39.507Z

tl;dr- "My executive functioning doesn't work, so I use morality instead."
From an outside perspective it's a little hard to tell that I don't have a lot of executive functioning. I keep my living space relatively clean, I get places relatively on time, I maintain a regular 9-to-5, and I have a lot of side projects in my free time that I generally manage to complete. But my life wasn't always that way, and most of the general rationality/CFAR methods of coping don't work at all for me. Getting diagnosed as an adult with ADD and trying a variety of medications also didn't work at all for me. Obviously, generalizing from one mind is a particularly Bad Idea in my case as I'm rather an outlier, but hopefully things that work for me might be useful for other people. (This may end up as a sequence of Things That Work for Me, but for today I'm just focusing on one.)
One way I manage to function is to have employment or other life aspects where I feel it is a Moral Imperative to Do The Thing. I call this "Offloading my executive functioning onto my morality," and it pretty much means setting up my life in a way where Doing Things comes naturally, because otherwise there are immediate negative effects on people I care about. I don't want to get into EvPsych stuff, but I feel it is a more natural motivation and How Humans Are Wired.
A bunch of examples:
Bad Idea: Have a job that lets you arrive whenever. Your arrival time has no effect on anyone else's work.
Good Idea: Have a job where a specific person is depending on you to arrive on time. Maybe your friend can't leave his shift until you arrive for yours. Maybe you are teaching a karate class and arriving late is wasting a bunch of people's time.
----------
Bad Idea: RSVP to events with "Maybe", so that no one's put out if you don't show.
Good Idea: RSVP to events with "Yes, AND I need a ride" or "Yes, AND I'll bring dessert" so that not showing up is causing a detriment to other people.
------------
Bad Idea: Have a job like academia where even if your research might be hugely useful, it is theoretic in application and far in the future.
Good Idea: Have a job like teaching or nursing, where people are relying on you, and not-working has immediate, visible, detrimental results for other people.

--------...