trentmkelly/LessWrong-43k huggingface dataset:
Bug: Inconsistent session status
by SquirrelInHell — 2018-01-05T13:32:57.354Z

The status of my user session on LW2 is halfway-preserved by cookies (with everything else reset). The top bar shows the username on the initial page load, but changes to "LOGIN" on any refresh.

I have not investigated, but I suspect using some mechanism for session permanence that was not meant for that purpose (LocalStorage etc.).
Choice begets regret
by bryjnar — 2018-01-04T20:28:21.574Z

Epistemic status: speculative

Choice is bad I want to focus on one aspect of this badness: regret. I'm going to argue that increased choice predictably increases the amount of regret that an agent feels, even if they are actually better off, and that this is bad for humans in particular.

Suppose that John is an uneducated child of subsistence farmers. Then John has a very limited set of options available to him (his "choice set") - work at his parents' farm, work at someone else's farm in the village, start his own farm... and that's about it. While John might prefer to be a lawyer, the fact that he was unable to become one does not reflect badly on him , since it was just not an available option. Given the limited set of alternatives, John ends up taking over the family farm (gaining 100 utils of satisfaction), and is quite satisfied with himself (starting his own farm being the best option at 110 utils).

Now, suppose that Kim is an educated child of farmers (with exactly the same preferences as John). Kim could still become a farmer, but also has some new, intriguing possibilities as a clerk, or clergyman, or a lawyer. Some of these options are much better than becoming a farmer, but it is increasingly unclear how to reach them. Kim ends up becoming a clerk (gaining 150 utils of satisfaction), but is mildly dissatisfied (becoming a lawyer being the best at 200 utils).

Now, suppose Liam is an educated child of professionals in the developed world (with the same preferences as John and Kim). Liam has many options far better than being a farmer or a clerk, including some that seem vastly better but are hard to acquire (the best is starting a spaceship company). Liam ends up being a well-paid software developer (gaining 300 utils of satisfaction), but is perennially dissatisfied with his choices (spaceship company CEO being the best at 1000 utils).

John, Kim, and Liam have the same preferences, and they each satisfy them progressively more. But Liam has the most regret, despite having done best. What's going on?

Well, first of all I've been casually using this word "regret" without really saying what I mean. Let's assume for the moment that "regret" works a bit like it works in decision theory: an individual's regret about a decision is the difference between the best option (post facto) and ...
[Paper]: Artificial Intelligence in Life Extension: from Deep Learning to Superintelligence
by turchin — 2018-01-04T14:28:57.965Z

There are two views on the best strategy among transhumanists and rationalists: The first involves the belief that one must invest in life extension technologies, and the latter, that it is necessary to create an aligned AI that will solve all problems, including giving us immortality or even something better. In our article, we showed that these two points of view do not contradict each other, because it is the development of AI that will be the main driver of increased life expectancy in the coming years, and as a result, even currently living people can benefit (and contribute) from the future superintelligence in several ways.

Firstly, because the use of machine learning, narrow AI will allow the study of aging biomarkers and combinations of geroprotectors, and this will produce an increase in life expectancy of several years, which means that tens of millions of people will live long enough to survive until the date of the creation of the superintelligence (whenever it happens) and will be saved from death. In other words, the current application of narrow AI to life extension provides us with a chance to jump on the “longevity escape velocity”, and the rapid growth of the AI will be the main factor that will, like the wind, help to increase this velocity.

Secondly, we can—here in the present—utilize some possibilities of the future superintelligence, by collecting data for “digital immortality”. Based on these data, the future AI can reconstruct the exact model of our personality, and also solve the identity problem. At the same time, the collection of medical data about the body will help both now—as it can train machine learning systems in predicting diseases—and in the future, when it becomes part of digital immortality. By subscribing to cryonics, we can also tap into the power of the future superintelligence, since without it, a successful reading of information from the frozen brain is impossible.

Thirdly, there are some grounds for assuming that medical AI will be safer. It is clear that fooming can occur with any AI. But the development of medical AI will accelerate the development of BCI interfaces, such as a Neuralink, and this will increase the chance of AI not appearing separately from humans, but as a product of integration with a person. As a result, a human mind will remain part of the AI, ...
Paper: Artificial Intelligence in Life Extension: from Deep Learning to Superintelligence

by avturchin — 2018-01-04T14:21:40.945Z

There are two views on the best strategy among transhumanists and rationalists: The first involves the belief that one must invest in life extension technologies, and the latter, that it is necessary to create an aligned AI that will solve all problems, including giving us immortality or even something better. In our article, we showed that these two points of view do not contradict each other, because it is the development of AI that will be the main driver of increased life expectancy in the coming years, and as a result, even currently living people can benefit (and contribute) from the future superintelligence in several ways.

Firstly, because the use of machine learning, narrow AI will allow the study of aging biomarkers and combinations of geroprotectors, and this will produce an increase in life expectancy of several years, which means that tens of millions of people will live long enough to survive until the date of the creation of the superintelligence (whenever it happens) and will be saved from death. In other words, the current application of narrow AI to life extension provides us with a chance to jump on the “longevity escape velocity”, and the rapid growth of the AI ​​will be the main factor that will, like the wind, help to increase this velocity.

Secondly, we can—here in the present—utilize some possibilities of the future superintelligence, by collecting data for “digital immortality”. Based on these data, the future AI can reconstruct the exact model of our personality, and also solve the identity problem. At the same time, the collection of medical data about the body will help both now—as it can train machine learning systems in predicting diseases—and in the future, when it becomes part of digital immortality. By subscribing to cryonics, we can also tap into the power of the future superintelligence, since without it, a successful reading of information from the frozen brain is impossible.

Thirdly, there are some grounds for assuming that medical AI will be safer. It is clear that fooming can occur with any AI. But the development of medical AI will accelerate the development of BCI interfaces, such as a Neuralink, and this will increase the chance of AI not appearing separately from humans, but as a product of integration with a person. As a result, a human mind will remain part of the AI, ​​and from within, the human will direct its go...
Papers for 2017
by Kaj_Sotala — 2018-01-04T13:30:01.406Z

I had three new papers either published or accepted into publication last year; all of them are now available online:

How Feasible is the Rapid Development of Artificial Superintelligence? Physica Scripta 92 (11), 113001.

Abstract: What kinds of fundamental limits are there in how capable artificial intelligence (AI) systems might become? Two questions in particular are of interest: 1) How much more capable could AI become relative to humans, and 2) how easily could superhuman capability be acquired? To answer these questions, we will consider the literature on human expertise and intelligence, discuss its relevance for AI, and consider how AI could improve on humans in two major aspects of thought and expertise, namely simulation and pattern recognition. We find that although there are very real limits to prediction, it seems like AI could still substantially improve on human intelligence.

Links: published version (paywalled), free preprint.

Disjunctive Scenarios of Catastrophic AI Risk. AI Safety and Security (Roman Yampolskiy, ed.), CRC Press. Forthcoming.

Abstract: ​ Artificial intelligence (AI) safety work requires an understanding of what could cause AI to become unsafe. This chapter seeks to provide a broad look at the various ways in which the development of AI sophisticated enough to have general intelligence could lead to it becoming powerful enough to cause a catastrophe. In particular, the present chapter seeks to focus on the way that various risks are disjunctive—on how there are multiple different ways by which things could go wrong, any one of which could lead to disaster. We cover different levels of a strategic advantage an AI might acquire, alternatives for the point where an AI might decide to turn against humanity, different routes by which an AI might become dangerously capable, ways by which the AI might acquire autonomy, and scenarios with varying number of AIs. Whereas previous work has focused on risks specifically only from superintelligent AI, this chapter also discusses crucial capabilities that could lead to catastrophic risk and which could emerge anywhere on the path from near-term “narrow AI” to full-blown superintelligence.

Links: free preprint.

Superintelligence as a Cause or Cure for Risks of Astronomical Suffering. Informatica 41 (4).

(with Lukas Gloor)

Abstract: Discussions about the possible conseque

...
Insights from 'The Strategy of Conflict'
by DanielFilan — 2018-01-04T05:05:43.091Z

Cross-posted from my blog.

I recently read Thomas Schelling's book 'The Strategy of Conflict'. Many of the ideas it contains are now pretty widely known, especially in the rationalist community, such as the value of Schelling points when coordination must be obtained without communication, or the value of being able to commit oneself to actions that seem irrational. However, there are a few ideas that I got from the book that I don't think are as embedded in the public consciousness.

Schelling points in bargaining

The first such idea is the value of Schelling points in bargaining situations where communication is possible, as opposed to coordination situations where it is not. For instance, if you and I were dividing up a homogeneous pie that we both wanted as much of as possible, it would be strange if I told you that I demanded at least 52.3% of the pie. If I did, you would probably expect me to give some argument for the number 52.3% that distinguishes it from 51% or 55%. Indeed, it would be more strange than asking for 66.67%, which itself would be more strange than asking for 50%, which would be the most likely outcome were we to really run the experiment. Schelling uses as an example

the remarkable frequency with which long negotiations over complicated quantitative formulas or ad hoc shares in some costs or benefits converge ultimately on something as crudely simple as equal shares, shares proportionate to some common magnitude (gross national product, population, foreign-exchange deficit, and so forth), or the shares agreed on in some previous but logically irrelevant negotiation.

The explanation is basically that in bargaining situations like these, any agreement could be made better for either side, but it can't be made better for both simultaneously, and any agreement is better than no agreement. Talk is cheap, so it's difficult for any side to credibly commit to only accept certain arbitrary outcomes. Therefore, as Schelling puts it,

Each party's strategy is guided mainly by what he expects the other to accept or insist on; yet each knows that the other is guided by reciprocal thoughts. The final outcome must be a point from which neither expects the other to retreat; yet the main ingredient of this expectation is what one thinks the other expects the first to expect, and so on. Somehow, out of this fluid and indeterminate situation that seemin

...
Video - Subject - Object Shifts and How to Have Them
by Matt Goldenberg — 2018-01-04T02:11:22.142Z

A frame is a lens through which we see the world. We're always framing things, but very rarely are we aware of those frames.

In this Mental Model Monday, I go over "Subject-Object Shifts", or the process by which we become aware of the frames through which we see things. I also give two practical ways to quickly make subject-object shifts in your own life.

View Video

Transcript

Hi, this is Matt, and this is Mental Model Monday number 26. Today I wanted to talk about subject-object shifts. These are incredibly important in the development of people and how they think. If you can learn to make subject-object shifts faster, then you can exponentially increase your rate of development and growth and understanding in a new field.

Defining a Subject-Object Shift

The idea behind a subject-object shift is that what was once something that you were subject to, something that was unconscious to you, something that you saw through, now becomes object to you, something that you can look at, something that you can play with, something that is not the way the world is but is just a way to look at the world.

One way to look at them is that they are a way to examine unconscious assumptions. That's sort of a fake and not quite true model of what subject-object shifts are, is they are a way examine assumptions which at one point were unexaminable to you.

Subject-Object Shifts and Kegan's Developmental Stages

An example of where subject-object shifts happen are in Kegan's developmental stages. This is one model of how adults develop. In Kegan's developmental stages, he's the one who coined the term subject-object shift, I believe. In Kegan's developmental stages, at each stage the thing that you originally saw as the world becomes a part of the world. At stage two, your impulses and perceptions, you can now see they're not me. They're part of me but my needs, interests, and desires are me. That's who is am, that's what I am. That's all there is. Whereas in stage three you see, oh, my needs, interests, and desires are not all of my. They're not all of the things that matter. I can see beyond that. Those are just individual things that I can play with. My whole world is now relationships and mutuality and how I interact with others, and the relations between people, and my friend group.

Then the relationships be...
TSR #8 Operational Consistency
by Hazard — 2018-01-03T02:11:32.274Z

This is part of a series of posts where I call out some ideas from the latest edition of The Strategic Review (written by Sebastian Marshall), and give some prompts and questions that I think people might find useful to answer. I include a summary of the most recent edition, but it's not a replacement for reading the actual article. Sebastian is an excellent writer, and your life will be full of sadness if you don't read his piece. The link is below.

Background Ops #8: Operational Consistency

SUMMARY

Carthage getting wiped out probably had a lot to do with them flubbing the Battle of Utica.
The Roman sneak attack could have been prevented by following existing known military best practices.
Military has a unique issue of being 90% boring and then 10% get-it-right-or-die.
The Problem

1. There’s lots of best practices that things will go slightly better if they’re followed.
2. If they’re neglected, most of the time nothing bad will happen and it won’t be noticed.
3. If they’re neglected, everything will be slightly worse all the time.
4. If they’re neglected, occasionally something extremely bad will happen.
5. Following those best practices can be very boring and they don’t have a naturally good feedback loop on learning and following them.

The General Orders for Sentries is a pretty solid example of a set of operations that have been very well designed. Read them for insight.
Design operations that respect human nature.
General steps to building consistent awesomeness

1. Codification
2. Accounting for human nature, biases, and errors
3. Stripping away inessentials
4. Training
5. Inspection and verification

It’s an iterative process.

Making a plan that involves more willpower than you can reasonably expect yourself to muster is a common example of “forgetting to account for being human”. Often, when I try to install a new productivity system, it fails because I assume that a lot of trivial inconveniences can just be willpowered. Or maybe I just haven’t thought about the system enough to realize how many trivial inconveniences there are.

Isolating and eliminating trivial inconveniences is a great way to increase your consistency. This winter, I’ve noticed that one of the reasons it’s easier to sleep in is because it’s a lot colder outside of my bed than in it (who could have seen that coming?). Now I go to sleep wearing a hoodie, which makes it a whole lot easier to g...
How I accidentally discovered the pill to enlightenment but I wouldn’t recommend it.
by Elo — 2018-01-03T00:37:53.005Z

Main post: http://bearlamp.com.au/how-i-accidentally-discovered-the-pill-to-enlightenment-but-i-wouldnt-recommend-it/

Brief teaser:

Eastern enlightenment is not what you think. I mean, maybe it is. But it’s probably not. There’s a reason it’s so elusive, and there’s a reason that it hasn’t joined western science and the western world the way that curiosity and discovery have as a driving force.

This is the story of my mistake accidentally discovering enlightenment.

February 2017

I was noticing some weird symptoms. I felt cold. Which was strange because I have never been cold. Nicknames include “fire” and “hot hands”, my history includes a lot of bad jokes about how I am definitely on fire. I am known for visiting the snow in shorts and a t-shirt. I hit 70kg, The least fat I have ever had in my life. And that was the only explanation I had. I asked a doctor about it, I did some reading – circulation problems. I don’t have circulation problems at the age of 25. I am more fit than I have ever been in my life. I look into hesperidin (orange peel) and eat myself a few whole oranges including peel. No change. I look into other blood pressure supplements, other capillary modifying supplements… Other ideas to investigate. I decided I couldn’t be missing something because there was nothing to be missing. I would have read it somewhere already. So I settled for the obvious answer. Being skinnier was making me colder.

Flashback to February 2016

This is where it all begins. I move out of my parents house into an apartment with a girl I have been seeing for under 6 months. I weigh around 80kg (that’s 12.5 stones or 176 pounds or 2822 ounces for our imperial friends). Life happens and by March I am on my own. I decide to start running. Make myself a more desirable human.

I taught myself a lot about routines and habits and actually getting myself to run. Running is hard. Actually, running is easy. Leaving the house is hard. But I work that out too.

For the rest of the post please visit: http://bearlamp.com.au/how-i-accidentally-discovered-the-pill-to-enlightenment-but-i-wouldnt-recomme...
How I accidentally discovered the pill to enlightenment but I wouldn’t recommend it.
by Elo — 2018-01-03T00:37:18.450Z

Main post: http://bearlamp.com.au/how-i-accidentally-discovered-the-pill-to-enlightenment-but-i-wouldnt-recommend-it/

Brief teaser...

Eastern enlightenment is not what you think. I mean, maybe it is. But it’s probably not. There’s a reason it’s so elusive, and there’s a reason that it hasn’t joined western science and the western world the way that curiosity and discovery have as a driving force.

This is the story of my mistake accidentally discovering enlightenment.

February 2017

I was noticing some weird symptoms. I felt cold. Which was strange because I have never been cold. Nicknames include “fire” and “hot hands”, my history includes a lot of bad jokes about how I am definitely on fire. I am known for visiting the snow in shorts and a t-shirt. I hit 70kg, The least fat I have ever had in my life. And that was the only explanation I had. I asked a doctor about it, I did some reading – circulation problems. I don’t have circulation problems at the age of 25. I am more fit than I have ever been in my life. I look into hesperidin (orange peel) and eat myself a few whole oranges including peel. No change. I look into other blood pressure supplements, other capillary modifying supplements… Other ideas to investigate. I decided I couldn’t be missing something because there was nothing to be missing. I would have read it somewhere already. So I settled for the obvious answer. Being skinnier was making me colder.

Flashback to February 2016

This is where it all begins. I move out of my parents house into an apartment with a girl I have been seeing for under 6 months. I weigh around 80kg (that’s 12.5 stones or 176 pounds or 2822 ounces for our imperial friends). Life happens and by March I am on my own. I decide to start running. Make myself a more desirable human.

I taught myself a lot about routines and habits and actually getting myself to run. Running is hard. Actually, running is easy. Leaving the house is hard. But I work that out too.

For the rest of the post please visit http://bearlamp.com.au/how-i-accidentally-discovered-the-pill-to-enlightenment-but-i-wouldnt-recommend-it/