trentmkelly/LessWrong-43k huggingface dataset:
Every Implementation of You is You: An Intuition Ladder
by lolbifrons — 2018-03-29T05:14:07.396Z

I was recently arguing in /r/transhumanism on reddit about the viability of uploading/forking consciousness, and I realized I didn't have any method of assessing where someone's beliefs actually lay - where I might need to move them from if I wanted to convince them of what I thought.

So I made an intuition ladder. Please correct me if I made any mistakes (that aren't by design), and let me know if you think there's anything past the final level.

Some instructions on how to use this: Read the first level. If you notice something definitely wrong with it, move to the next level. Repeat until you come to a level where your intuition about the entire level is either "This is true" or "I'm not sure." That is your level.

1. Clones and copies (the result of a medical procedure that physically reproduces you exactly, including internal brain state) are the same thing. Every intuition I have about a clone, or an identical twin, applies one-to-one to copies as well, and vice versa. Because identical twins are completely different people on every level except genetically, copies are exactly the same way.

2. Clones and copies aren't the same thing, as copies had a brain and memories in common with me in the past, but for one of us those memories are false and that copy is just a copy, while my consciousness would remain with the privileged original.

3. Copies had a common brain and memories, which make them indistinguishable from each other in principle, so they believe they're me, and they're not wrong in any meaningful sense, but I don't anticipate waking up from any copying procedure in any body but the one I started in. As such, I would never participate in a procedure that claims to "teleport" me by making a copy at a new location and killing the source copy, because I would die.

4. Copies are indistinguishable from each other in principle, even from the inside, and thus I actually become both, and anticipate waking up as either. But once I am one or the other, my copy doesn't share an identity with me. Furthermore, if a copy is destroyed before I wake up from the procedure, I might die, or I might wake up as the copy that is still alive. As such, the fork-and-die teleport is a gamble for my life, and I would only attempt it if I was for some reason comfortable with the chance that...
Open-Category Classification
by TurnTrout — 2018-03-28T14:49:23.665Z

Introduction

If I present you with five examples of burritos, I don’t want you to pursue the simplest way of classifying burritos versus non-burritos. I want you to come up with a way of classifying the five burritos and none of the non-burritos that covers as little area as possible in the positive examples, while still having enough space around the positive examples that the AI can make a new burrito that’s not molecularly identical to the previous ones.

- AI Alignment: Why It's Hard and Where to Start

Consider the problem of designing classifiers that are able to reliably say "I don't know" for inputs well outside of their training set. In the literature, this problem is known as open-category classification [7]; a closely-related problem in AI safety is inductive ambiguity detection [4].

Solution of generalized inductive ambiguity detection could allow for agents who robustly know when to ask for clarification; in this post, we discuss the problem in the context of classification. Although narrower in scope, the solution of the classification subproblem would herald the arrival of robust classifiers which extrapolate conservatively and intuitively from their training data.

Obviously, we can't just train classifiers to "recognize" the unknown class. One, this class isn't compact, and two, it doesn't make sense - it's a map/territory confusion (the label unknown is a feature of the world model, not a meaningful ontological class). Let's decompose the concept a bit further.

Weight regularization helps us find a mathematically-simple volume in the input space which encapsulates the data we have seen so far. In fact, a binary classifier enforces a hyperplane which cleaves the entire space into two volumes in a way which happens to nearly-optimally classify the training data. This hyperplane may be relatively simple to describe mathematically, but the problem is that the two volumes created are far too expansive.

In short, we'd like our machine learning algorithms to learn explanations which fit the facts seen to date, are simple, and don't generalize too far afield. This is an example of conservatism:

Conservatism and conservative planning seems like it might directly tackle some standard concerns [in alignment] head-on and in a sufficiently-basic way to avoid loopholes, and might also be subject to

... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
'Trivial Inconvenience Day' Retrospective
by namespace — 2018-03-28T05:14:51.524Z

Introduction

Shortly after reading Scott Alexander's LessWrong Crypto Autopsy I found myself agreeing with the point so strongly I was brainstorming ways that its dismal outcome could have been prevented. Peoples personal accounts of why they didn't buy bitcoin seemed to converge on a central theme: Buying Bitcoin was a trivial inconvenience. Pondering what might be done in light of this, I was reminded of Boston Rat's Bureaucracy Day. The Bureaucracy Day is essentially a designated day for people to beat the ugh field effect by getting together and going through the whole mess of annoying tasks, paperwork, and other trivially inconvenient things people have been putting off. Having been impressed by the concept the first time I read about it, Scott's dire analysis convinced me to try something like it in the hope that it would be a useful tool against this sort of thing happening again.

Design

Preliminary Steps & Research

The actual idea came to me when a friend linked Scott's post. From the first paragraphs I was taken by the premise, and lamented how I had the opportunity to buy Bitcoin when it was pennies but didn't. For me the sting is even greater because I'd been tempted to buy an entire thousand dollars worth, but eventually decided against it. (As Scott points out in his post, I was especially being an idiot since just buying 10 bucks worth would have made me a much wealthier man.) Part of why I didn't was I was under the impression my only option for buying was essentially meeting someone in person that mined it. I'm unsure exactly when this was, but it's quite possible I was thinking about this pre-exchange and that was the reality.

As I was sitting there thinking about this, I remembered the Boston Rat post and felt it was a good starting template for solving this issue. I said to my friend:

Okay, here's my idea. Are you aware of Boston rat's "Bureaucracy day?" We do "Trivial Inconvenience Day". Where we get people into a giant IRC room. Or Discord or whatever. (IRC is easier to use a web client for without sign up.) And have people make a list of s*** they know in the abstract they should do but are putting off. And then we all do it together and talk about having done it. Sort of like a baptism, but for buying cryptocurrency or fixing my website. >_>

The friend...
Welcome to Pittsburgh Rats [Edit With Your Details]
by zlrth — 2018-03-28T04:56:05.085Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Karnofsky on forecasting and what science does
by Rob Bensinger — 2018-03-28T01:55:26.495Z
Link: https://80000hours.org/2018/02/holden-karnofsky-open-philanthropy/

[...]

Holden Karnofsky: I think a decent counterpoint to this [project of predicting future AI progress] is like, "Look. You can do all the work you want and try and understand the brain. It is not possible to predict the future 20 years out. No one’s ever done it well." That’s an argument someone could make. I think it’s a decent argument. But I think there’s also a counterpoint to that.

One, I think people do somewhat overestimate how futile it is to predict the future. We have an ongoing project on this. We have a contractor working right now on looking back at a bunch of 10- or 20- or 30-year predictions, and scoring them according to whether they came true or false. These are predictions that were made decades ago. We’ll see how that goes.

The other thing I’d say is: you can’t know the future, but it seems possible that you can be well-calibrated about the future. If you look at, for example, Slate Star Codex, every year it’s putting up a whole bunch of probabilities about things that that blogger is not an expert on and doesn’t really necessarily know a whole bunch about. There will be a probability that things improve in the Middle East, a probability that someone wins a Presidential election... This person doesn’t necessarily know what’s going to happen, but they have some knowledge about their own state of knowledge. They have some knowledge about their own state of ignorance.

What they’re not able to do is accurately predict what’s going to happen. What they are able to do is make predictions such that when they say something is 90% likely, they’re wrong about 10% of the time. When they say something is 80% likely, they’re wrong about 20% of the time. When they say something is 50% likely, they’re wrong about half the time. They know how likely they are to be wrong, which is different from knowing what’s going to happen.

And so one of the things that I’d look for in this timelines person is a deep familiarity with the science of forecasting, which is something that we’re very interested in and we try to incorporate into our grantmaking. A deep familiarity with that, and an understanding of what is realistic for them to say 20 years out.

[...]

Robert Wiblin: What things do you think you’ve learned, over the last 11 years of doing this kind of research, about in what situations you can trust expert consensus and in what cases you should think ther...
The fundamental complementarity of consciousness and work
by KatjaGrace — 2018-03-28T01:20:00.563Z

Matter can experience things. For instance, when it is a person. Matter can also do work, and thereby provide value to the matter that can experience things. For instance, when it is a machine. Or also, when it is a person.

An important question for what the future looks like, is whether it is more efficient to carry out these functions separately or together.

If separately, then perhaps it is best that we end up with a huge pile of unconscious machinery, doing all the work to support and please a separate collection of matter specializing in being pleased.

If together, then we probably end up with the value being had by the entities doing the work.

I think we see people assuming that it is more efficient to separate the activities of producing and consuming value. For instance, that the entities whose experiences matter in the future will ideally live a life of leisure. And that lab grown meat is a better goal than humane farming.

Which seems plausible. It is at least in line with the general observation that more efficient systems seem to be specialized.

However I think this isn’t obvious. Some reasons we might expect working and benefiting from work to be done by overlapping systems:

We don’t know which systems are conscious. It might be that highly efficient work systems tend to be unavoidably conscious. In which case, making their experience good rather than bad could be a relatively cheap way to improve the overall value of the world.
For humans, doing purposeful activities is satisfying, so much so that there are concerns about how humans will cope when they are replaced by machines. It might be hard for humans to avoid being replaced, since they are probably much less efficient than other possible machines. But if doing useful things tends to be gratifying for creatures—or for the kinds of creatures we decide are good to have—then it is less obvious that highly efficient creatures won’t be better off doing work themselves, rather than being separate from it.
Consciousness is presumably cheap and useful for getting something done, since we evolved to have it.
Efficient production doesn’t seem to evolve to be entirely specialized, especially if we take an abstract view of ‘production’. For instance, it is helpful to produce the experience of being a sports star alongside the joy of going to sports games.
Specialization seems especially helpful if keeping track of

...
Defining the ways human values are messy
by Stuart_Armstrong — 2018-03-27T22:42:06.002Z

In many of my posts, I've been using phrases like "human values are contradictory, underdefined, changeable, and manipulable". I also tend to slide between calling things preferences, values, morals, rewards, and utilities. This post will clarify some of this terminology.

I say that human values are contradictory, when humans have firm and strong opinions that are in conflict. For instance, a respect for human rights versus desires to reduced harm, when those two come in conflict (more broadly, deontology versus utilitarian conflicts). Or enjoying food (or wanting to be someone who enjoys food) versus wanting to get thin (or wanting to be the someone who gets thin). Or family loyalty versus more universal values.

I say that human values are underdefined, when humans don't have a strong opinion on something, and where their opinion can be very different depending on how the something is phrased. This includes how the issue is framed (saving versus dying), or how people interpret moral choices (such as abortion or international press freedom) depending on what category they put that choice in. New technologies often open up new areas where old values don't apply, forcing people to define new values in the space (often by analogy to old values).

Notice that there is no clear distinction between contradictory and underdefined: as the values in conflict or potential conflict get firmer, this moves from underdefined to contradictory.

I say that human values are changeable, because of the way that values shift, often in predictable ways, depending on such things as social pressure, tribalism, changes in life-roles or positions, or new information (fictional as well as factual information). I suspect that most of these shifts are undetectable to the subject, just as most belief changes are.

I say that human values are manipulable, in that capable humans and potentially advanced AI, can use the vulnerabilities of human cognition to push values in a particular direction. This is a subset of changeable, but with a different emphasis.

Rewards/values/preferences...

At the object level, I see values, preferences, and morals as the same thing. All express the fact that a certain state of the world or a certain course of action, is better than another one.

At the meta level, humans tend to distinguish between them, seeing values and morals as fundamental, w...
Optimal level of hierarchy for effective altruism?
by Jan_Kulveit — 2018-03-27T22:38:27.967Z
Link: http://effective-altruism.com/ea/1mj/optimal_level_of_hierarchy_for_effective_altruism/

Epistemic status: a question. Way of looking.

Hierarchical modular network structure is an organizing principle found in many complex systems, ranging from ecosystems to human brain, and of course it is commonly seen in social networks. By "modularity" we mean, technically speaking, that there are parts of a network that are more densely connected than in random networks. This can have various causes, like spatial proximity or functional consideration. These densely connected parts often form hierarchies, where smaller units are parts of larger ones. Hierarchical structures is the way how things are done in real world.

We can view the effective altruism movement as a network. Then, we may ask what its network structure is and what processes generated it. We may also ask a question what network structure it should have: how the network should be optimized for a given purpose (like changing the world for better).

In general, this seems a hard question, but possibly just looking from this perspective leads to some easier and more practical questions. Like:

What is the optimal level of hierarchicality?
How much should geography correlate with function?
How much should things be localized

For the first question, we can easily imagine two possible answers:

1) Two level structure, where there are global EA organizations, and local groups.

2) Three level structure, where there are global EA organizations, country, region or language-specific organizations, and local groups.

It seems what effective altruism movement has at the moment is a mixture of both. There is CEA, which is at the same time the global organization and an anglosphere one, but also has some UK and Oxford flavour. On the other hand, we have also Effective Altruism Foundation, which is a mixture of country-wide organization for Germany, language-based organization, and a global one.

For the second question, at the moment we have a strong correlation of physical space and functional or meme-space. For example in Germany, people are much more than average focused on suffering reduction. People in Bay Area are more than average on long-term plans to influence AI, etc.

For the third question, it seems there are various forces pulling in different directions.

If we take an outside look, the whole does not seem to be a result of a design or some intentional optimization, but more likely a legacy of some “or...
Learn Bayes Nets!
by abramdemski — 2018-03-27T22:00:11.632Z

It recently occurred to me that there are a lot of people in the rationalist community who want to deeply absorb intuitions about how Bayes' theorem works and how to think with it in practice, who have not been specifically told that learning inference algorithms for Bayesian networks is one of the best ways forward.

Well, I'm telling you now.

Bayesian networks were the innovation which made probabilistic reasoning really practical and interesting for artificial intelligence -- and none of the reasons for that are special to trying to squeeze intelligence into a computer. They're also, more or less, describing the way people have to think in order to do probabilistic reasoning in practice. There have been many innovations in probabilistic reasoning since Bayes nets, but those are arguably more about how to get good results on a computer and less about fundamental conceptual issues that you'll get a lot from.

I would argue that the most important inference algorithm to learn about to get practical intuitions is belief propagation. There are others who would argue for monte carlo algorithms, like MCMC (monte carlo markov chain). You may want to learn both, to form your own opinion (and of course, there are many more algorithms beyond this which you may want to learn, in order to gain more connections so your knowledge of the field sticks, and gain more insights). Belief prop and MCMC are more or less the first two algorithms people thought of; there are a lot of newer developments, but they're largely elaborations.

Here is what I claim you can get out of it, through careful study:

Understanding how bayesian networks define probability distributions, and how probabilities spread through the network via belief propagation, makes your understanding of Bayes' theorem and probability theory in general much more "load bearing" -- so it'll break under the strain if it isn't solid (which is a good thing).
It'll give you a useful fake model of what you're doing when you're thinking. Global Bayesian updates don't just happen; they result from (something like) local updates which you have to spread across your web of beliefs, partly through conscious attention and cogitation.
It's also a useful analogy for aspects of group epistemics, like avoiding double counting as messages pass through the social netw

...
Welcome to Astral Codex Ten Miami Meetup
by eric135 — 2018-03-27T21:27:16.518Z

What kind of events does your group usually run? What does it usually do?

We have monthly meetups that typically involve an activity related to applied rationality. Throughout the month there are social meetups happening from Miami up to Palm Beach.

Who would be a good fit for you group?

Anyone who is interested in LessWrong, ACX, or Effective Altruism will probably fit in. Everyone is welcome, even if you aren't familiar with these things.

Should they have any particular skills or have done some specific background reading?

No skills are required. Background reading might be suggested for a specific meetup but will usually not be required.
Evaluating Existing Approaches to AGI Alignment
by Gordon Seidoh Worley — 2018-03-27T19:57:39.207Z
Link: https://mapandterritory.org/evaluating-existing-approaches-to-agi-alignment-70fe1037d999

My read of the AI safety space is that there are currently two major approaches to AGI alignment being researched: agent foundations and agent training. We can contrast them in part by saying the ultimate goal of the agent foundations program is to figure out how to make an AGI, possibly from scratch, that will align itself with human values even if humans don’t know what their values are or feed the AGI the “wrong” values, while the agent training program is to figure out how to teach an AGI, and really any sufficiently capable agent, human values. Having now formally stated the AI alignment problem, we can ask to what extent each program stands to satisfy the technical requirements for alignment.

To refresh your memory, I presented AI alignment as the problem of ensuring that an agent adopts the values of humanity and human operators know enough about the agent to believe it shares humanity’s values. For shorthand we can call these two properties value alignment and believable alignment, respectively. Of particular note is that alignment it not possible unless both values are aligned and that alignment is believable as one without the other allows failure cases like trivial alignment, as in the paperclip maximizer that is trivially aligned by having no model of human values at all, and the treacherous turn, where an agent appears to be aligned but will express unaligned behaviors after it is too powerful to stop. Specific instances of these properties are well known through thought experiments about how AI alignment might fail, but the formalization as two categorical properties is new, so it’s in light of these properties that we assess how well each approach to alignment addresses them.

Starting with the older of the two approaches, although it was only in 2014 that the agent foundations approach coalesced enough to have an agenda, its threads stretch back to the turn of the 21st century and the earliest attemptsto address alignment, then primarily known as the problem of building Friendly AI. MIRI is the clear champion of this approach, with support from researchers at or funded through FLI, FHI, CHAI, and other groups. We might summarize the agent foundations approach to alignment as starting from the assumption that we need to build a rational agent if we want to have any hope that it will be aligned, and then having built a rational agent instill in it...