trentmkelly/LessWrong-43k huggingface dataset:
OpenAI releases functional Dota 5v5 bot, aims to beat world champions by August
by habryka — 2018-06-26T22:40:34.825Z
Link: https://blog.openai.com/openai-five/

Our team of five neural networks, OpenAI Five, has started to defeat amateur human teams at Dota 2. While today we play with restrictions, we aim to beat a team of top professionals at The International in August subject only to a limited set of heroes. We may not succeed: Dota 2 is one of the most popular and complex esports games in the world, with creative and motivated professionals who train year-round to earn part of Dota’s annual $40M prize pool (the largest of any esports game).

Commentary by Sam Altman: http://blog.samaltman.com/reinforcement-learning-progress

This is the game that to me feels closest to the real world and complex decision making (combining strategy, tactics, coordinating, and real-time action) of any game AI had made real progress against so far.

The agents we train consistently outperform two-week old agents with a win rate of 90-95%. We did this without training on human-played games—we did design the reward functions, of course, but the algorithm figured out how to play by training against itself.

This is a big deal because it shows that deep reinforcement learning can solve extremely hard problems whenever you can throw enough computing scale and a really good simulated environment that captures the problem you’re solving. We hope to use this same approach to solve very different problems soon. It's easy to imagine this being applied to environments that look increasingly like the real world.
The Beauty and the Prince
by Chris_Leong — 2018-06-26T13:10:29.889Z

This post will address a problem proposed by Radford Neal in his paper Puzzles of Anthropic Reasoning Resolved Using Full Non-indexical conditioning. In particular, he defined this problem - The Beauty and the Prince - to argue against the halver solution to the Sleeping Beauty Problem. I don't think that this is ultimately a counter-example, but I decided to dedicate a post to it because I felt that it was quite persuasive when I first saw it. I'll limit the scope of this post to arguing that his analysis of the halver solution is incorrect and providing a correct analysis instead. I won't try to justify the halver solution as being philosophically correct as I plan to write another post on the Anthropic Principle later, just show how it applies here.

The Beauty and the Prince is just like the Sleeping Beauty Problem, but with a Prince who is also interviewed and memory-wiped. However, he is always interviewed on both Monday and Tuesday regardless of what the coin shows and he is told whether or not Sleeping Beauty is awake. If he is told that she is awake, what is the probability that the coin came up heads. The argument is that 3/4 times she will be awake and 1/4 times she is asleep so only 1/3 times when he is told she is awake will the coin be heads. Further, it seems that Sleeping Beauty should adopt the same odds as him. They both have the same information, so if he tells her the odds are 1/3, on what basis can she disagree? Further, she knows what he will say before he even asks her.

I want to propose that the Prince’s probability estimate as above is correct, but it is different from Sleeping Beauty’s. I think the key here is to realised that indexicals aren’t a part of standard probability, so we need to de-indexicalise the situation. However, we'll de-indexicalise the original problem first. We'll do this be ensuring that only one interview ever "counts", by which we mean that we will calculate the probability of events over the interviews that count. We'll do this by flipping a second coin if the first comes up tails. If it is heads, only the first interview counts, whilst for tails only the second interview counts. We then get the odds being: 1/2 heads + Monday counts; 1/4 tails + Monday counts; 1/4 tails + Tuesday counts.

We similarly de-indexalise the Prince, though we flip the second coin in the heads cas...
What could be done with RNA and DNA sequencing that's 1000x cheaper than it's now?
by ChristianKl — 2018-06-26T12:24:57.030Z

In How to Invent the Future Alan Kay proposes that a good way of inventing the feature is to take an exponential and project it into the feature. Afterwards it's useful to ask what can be done with the new capabilities.

On important exponential is the price of DNA and RNA sequencing that fell faster than Moore's law over the last 30 years.

What can be done in a future is radically cheaper sequencing?
Logical uncertainty and Mathematical uncertainty
by AlexMennen — 2018-06-26T01:08:20.559Z

There is a significant difference between uncertainty about mathematical truths in cases where there isn't a known procedure for checking whether a mathematical claim is true or false, versus when there is but you do not have the computational resources to carry it out. Examples of the former include the Collatz and twin prime conjectures, and examples of the latter include whether or not a given large number is a semi-prime, and what the first decimal digit of Graham's number is.

The former should not be called logical uncertainty, because it is about what is true, not about what can be proved; I'll call it mathematical uncertainty instead. The latter really is uncertainty about logic, since we would know that the claim is either proved or refuted by whatever theory we used to prove the algorithm correct, and we would just be uncertain as to which one.

It's well-known that standard probability theory is a poor fit for handling logical uncertainty because it assumes that the probabilities are logically coherent, and uncertainty about what the logical coherence constraints are is exactly what we want to model; there are no possible outcomes in which the truth-value of a decidable sentence is anything other than what it actually is. But this doesn't apply to mathematical uncertainty; we could study the probability distribution over complete theories that we converge to as time goes to infinity, and reason about this probability distribution using ordinary probability theory. Possible sources of evidence about math that could be treated with ordinary probability theory include physical experiments and black-boxed human intuitions. But another important source of evidence about mathematical truth is checking examples, and this cannot be reasoned about in ordinary probability theory because each of the examples is assigned probability 1 or 0 in the limit probability distribution, since we can check it. So just because you can reason about mathematical uncertainty using ordinary probability theory doesn't mean you should.

Logical induction, taken at face value, looks like an attempt at handling mathematical uncertainty, since logical inductors assign probabilities to every sentence, not just sentences known to be decided by the deductive process. But most of the desirable properties of logical inductors that have been proved refer to seq... ,,,,,
Book Review: Why Honor Matters
by ryan_b — 2018-06-25T20:53:48.671Z

People who live in honor cultures have a sense of purpose and meaning. They dwell in solidarity with their fellows, are courageous in the face of danger, set great store in hospitality, and put the welfare of the group above their own.

Mostly.

I expect when readers from this site think of honor, it brings to mind

Culture of Honor: the Psychology of Violence in the South

, the gist of which is that American Southerners kill each other over insults more often than the rest of the country because the biggest chunk of colonial immigrants there were cattle-herders from the border between Scotland and England. I also guess that through Scott Alexander's review of

Albion's Seed

, the usual view of these people (and how they think) is perhaps unflattering.

In

Why Honor Matters

, Tamler Sommers offers a defense of honor. It is only a defense, and not an apologia, so he makes no excuses for the evils associated with it (eternal feuds, subjugation of women, etc). He speaks for the general case of honor with a variety of examples, rather than any specific implementation of it. The writing is clear and untechnical.

I.

Why write the book?

Courage, integrity, solidarity, drama, hospitality, a sense of purpose and meaning - these are attractive values and characteristic, important for living a good and worthwhile life. But there was something else drawing me to honor too, something more fundamental and harder to describe. Though I subscribe to liberal values of toleration and respect for individual freedom, I've come to believe that the Western liberal approach to ethics is deeply misguided. The approach is too systematic, too idealized and abstract - incapable of reckoning with the messy complexity of the real world.

How does honor differ?

Idealized, systematic, abstract, and universalizable - honor has none of these attributes. Honor, unlike dignity, is not abstract; it's grounded in fact. Honor is real only when people recognize and acknowledge it. Honor codes are local, not universal, tailored to the particular needs of each community. Most important, honor codes are tailored to people as they are, not how we wish to them to be or how we imagine they would be if they were "rational." Honor is full of compromises; it deals in grays rather than black and white. It seeks better alternatives, not ideal alternatives. In short, honor is a thoroughly nonidea

...
The Alignment Newsletter #12: 06/25/18
by Rohin Shah — 2018-06-25T16:00:42.856Z

Highlights

Factored Cognition

(Andreas Stuhlmuller): This is a presentation that Andreas has given a few times on Factored Cognition, a project by

Ought

that is empirically testing one approach to amplification on humans. It is inspired by

HCH

and

meta-execution

. These approaches require us to break down complex tasks into small, bite-sized pieces that can be solved separately by copies of an agent. So far Ought has built a web app in which there are workspaces, nodes, pointers etc. that can allow humans to do local reasoning to answer a big global question.

My opinion: It is unclear whether most tasks can actually be decomposed as required for iterated distillation and amplification, so I'm excited to see experiments that can answer that question! The questions that Ought is trying seem quite hard, so it should be a good test of breaking down reasoning. There's a lot of detail in the presentation that I haven't covered, I encourage you to read it.

Summary: Inverse Reinforcement Learning

This is a special section this week summarizing some key ideas and papers behind inverse reinforcement learning, which seeks to learn the reward function an agent is optimizing given a policy or demonstrations from the agent.

Learning from humans: what is inverse reinforcement learning?

(Jordan Alexander): This article introduces and summarizes the first few influential papers on inverse reinforcement learning.

Algorithms for IRL

attacked the problem by formulating it as a linear program, assuming that the given policy or demonstrations is optimal. However, there are many possible solutions to this problem -- for example, the zero reward makes any policy or demonstration optimal.

Apprenticeship Learning via IRL

lets you learn from an expert policy that is near-optimal. It assumes that the reward function is a weighted linear combination of features of the state. In this case, given some demonstrations, we only need to match the feature expectations of the demonstrations in order to achieve the same performance as the demonstrations (since the reward is linear in the features). So, they do not need to infer the underlying reward function (which may be ambiguous).

Maximum Entropy Inverse Reinforcement Learning

(Brian D. Ziebart et al): While matching empirical feature counts helps to deal with the ambiguity of the reward functions, exactly matching featuer counts will...
UDT can learn anthropic probabilities
by cousin_it — 2018-06-24T18:04:37.262Z

Today I realized that UDT doesn't have to clash with "objective" views of anthropic probability, like SSA or SIA: instead it can, in some sense, learn which of these views is true!

The argument goes like this. First I'll describe a game where SSA and SIA lead to different decisions. Unlike Sleeping Beauty, my game doesn't involve memory loss, so someone can play it repeatedly and keep memories of previous plays. Then we'll figure out what UDT would do in that game, if it valued each copy's welfare using the average of SSA and SIA. It turns out that different instances of UDT existing at once will act differently: the instances whose memories are more likely under SSA will make decisions recommended by SSA, and likewise with SIA. So from the perspective of either view, it will look like UDT is learning that view, though from an agnostic perspective we can't tell what's being learned.

First let's describe the game. Imagine there's a small chance that tomorrow you will be copied many times - more than enough to outweigh the smallness of the chance. More precisely, let's say some event happens at 1:N odds and leads to N² copies of you existing. Otherwise (at N:1 odds) nothing happens and you stay as one copy. You're offered a choice: would you rather have each copy receive a dollar if the event happens, or receive a dollar if the event doesn't happen? The former corresponds to SIA, which says you have N:1 odds of ending up in the world with lots of copies. The latter corresponds to SSA, which says you have N:1 odds of ending up in a world with no copying.

We can repeat this game for many rounds, allowing the players to keep their memories. (Everyone existing at the end of a round gets to play the next one, leading to a whole lot of people in the end.) Consider for a moment what happens when you're in the middle of it. Imagine you started out very sure of SSA, but round after round, you kept finding that you were copied. It would be like seeing a coin come up heads again and again. At some point you'll be tempted to say "what the hell, let's do a Bayesian update in favor of SIA". This post is basically trying to give a UDT justification for that intuition.

Our anthropic repeated game is quite complicated, but if SSA and SIA were equally likely, it would be equivalent to thi...
What am I?
A bit of poetic prose.
by AureliusGray — 2018-06-23T17:19:18.421Z

What is my identity? What is the core that defines me? If I identify with all of humanity, then my core is as follows.

I am a sleeper, as I sleep and I dream. I am a thinker, as I have thoughts and I hear them in my mind. They differ and vacillate and sometimes take a sharp turn in content and direction, but I know that they exist. I am an eater, as I require cellular inputs to subsist; to not die. I am a breather, as I require air. Thus, at my core, I am a breathing, eating, sleeping thinker.

And I am no more than that. Every other aspect of myself is an add-on that can be removed at will depending on what is necessary in the world as it exists in that moment.

I am the slate that is to be written; the bootstrap of the most minimal kernel; the translucent box whose contents are only what the world requires me to be.

And if my breathing and my eating and my sleeping and my thinking can be digitized—then I am nothing at all.
Informational hazards and the cost-effectiveness of open discussion of catastrophic risks
by avturchin — 2018-06-23T13:31:13.641Z
Link: http://effective-altruism.com/ea/1q9/informational_hazards_and_the_costeffectiveness/

TL;DR: In order to prevent x-risks, our strategic vision should outperform technical capabilities of the potential malevolent agents, which means that strategic discussion should be public and open, but the publication of technical dangerous knowledge should be prevented.

Risks and benefits of the open discussion

Bostrom has created a typology of info-hazards, but any information could also have “x-risk prevention positive impact”, or info-benefits. Obviously, info-benefits must outweigh the info-hazards of the open public discussion of x-risks, or the research of the x-risks is useless. In other words, the “cost-effectiveness” of the open discussion of a risk A should be estimated, and the potential increase in catastrophic probability should be weighed against a possible decrease of the probability of a catastrophe.

The benefits of public discussion are rather obvious: If we publicly discuss a catastrophic risk, we can raise awareness about it, prepare for the risk, and increase funding for its prevention. Publicly discussed risks are also more likely to be viewed by a larger group of scientists than those that are discussed by some closed group. Interdisciplinary research and comparison of different risks is impossible if they are secrets of different groups; as a result of this phenomenon, for example, asteroid risks are overestimated (more publicly discussed) and biorisks are underestimated (less-discussed).

A blanket "information hazard counterargument" is too general, as any significant new research on x-risks changes the information landscape. For example, even good news about new prevention methods may be used by bad actors to overcome these methods.

The problem of informational hazards has already been explored in the field of computer security, and they have developed best practices in this area. The protocol calls for the discoverer of an informational hazard to first try to contact the firm which owns the vulnerable software, and later, if there is no reply, to publish it openly (or at least hint at it), to provide users with an advantage over bad actors who may use it secretly.

The relative power of info-hazards depends on the information that has already been published and on other circumstances

Consideration 1. If something is already public knowledge, then discussing it is not an informational hazard. Example: AI risk. The same is tru...
Extending the Private Language Argument
by Ashwin Kumar — 2018-06-23T09:07:52.272Z
Link: https://medium.com/@ashwinkumar.ap/what-is-experience-and-what-do-we-do-about-it-60a52396e85e

Two instances of our ways of using the notion of experience:

"Our experience is unavailable to others" (Eg: women's experience, minorities' experience not being available to non-women and non-minorities.)
"We have experienced oppression" (The problem of what is an experience and what is its explanation. While it is always the case that the two tend to collapse into each other, thinking is an activity of pricing the two apart.)

In both the above-mentioned cases, the idea of experience is that it is a "private episode". Private, because, it is unique to an individual and occurs in the inner theater of the mind, and episode, because, it has a duration and is some kind of recording of a public episode. So, the assumption is that while there are public episodes of specific duration, there are their mental imprints which are episodes of another kind, which are called experience.

Sensations, for example, are private episodes and are unique to the sufferer. No amount of physical pain one suffers in one's toe or stomach or head can be shared with or transferred to another. That makes sensation private. Similarly, sensations can and do have a duration. I could have had a headache the whole of yesterday. But since today morning, I could be feeling better. It hurts whenever someone stamps on my toe and not before or long after. That makes sensations episodic.

But experience is like neither of the two. To say that I have experience, it minimally requires that one specifies an object about which we can be said to have experience. So, for instance, we can be said to have experience in novel writing, driving and even in dealing with failure. That we indeed have had such experience is established both for ourselves and for others by looking at our behavior in the relevant circumstances. To be experienced in driving is to be able to maneuver an automobile, or speak about the distinctions involved in driving. It is not to have a mental record about driving in the privacy of our mind or a "muscle memory" in our limbs.

Experience is not a record of sensations, but a grasp of the world.

What we have is experience if what we have is publicly available as a disposition to be learnt, imparted, shared or talked about with some degree of intelligibility. Without a world of objects to respond to and without a sense of appropriateness of...
Two prescriptions for fixing a  procedural/declarative knowledge mismatch.
by Andrew Quinn — 2018-06-22T22:17:08.900Z

Prescriptions

If you think you have more procedural knowledge than declarative:

use the Feynman technique (ideally once per day for a few days) and
read other people's exposition when you get stuck.

If you think you have more declarative knowledge than procedural:

work problems (ideally with clear solutions) and
generate concrete examples for yourself.

Rationale

In whales' excellent Book Review: How Learning Works, each one of the 7 Principles gets expanded upon with a "Research consensus". In the first principle, discussing how prior knowledge affects a students' learning, they write:

Declarative knowledge (object-level concepts) and procedural knowledge (how and when to apply those concepts) do not always go hand in hand. One without the other is a knowledge gap that can be tricky to spot, especially in self-assessment.

(Emphasis whales', not mine.)

I've noticed this to be an especially useful asymmetry to keep in mind myself, since a lot of what I do is filling one, then using that to bootstrap the other. So I thought I would share my two "prescriptions" for it.

They're nothing groundbreaking, but they might help someone who's stuck in a state of analysis paralysis decide what to do next.

EDIT: The Feynman technique and a variant

My apologies, I thought this was a little better known than it was. To summarize very quickly, the Feynman technique is where you

Sit down with a piece of paper (or whatever "thought->thing" tool you prefer);
Write down the concept you want to build declarative knowledge on first;
Start to write out an explanation of the concept, as if you were explaining it to a generally intelligent layman, who doesn't know most of your fancy words;
and any time you have difficulty explaining in plain English what you're talking about, you pinpoint that as a gap you need to build declarative knowledge on further.

That was all taken from this helpful link from commentator habryka below. (Thanks!) Let me add a bit of my own value, by suggesting a variant for people with a lot of "know-how".

Pick a problem and write that at the top of the page.
Start to work through the problem, but at each stage, pause and write a plain-English explanation of what you're doing here.

It's a little more forgiveable to use jargon here, but you should try to explain whatever jargon you use i

...
Anthropics: Where does Less Wrong lie?
by Chris_Leong — 2018-06-22T10:27:16.592Z

I'm currently reading some papers various problems related to the anthropic problem and I'm planning to write up a post when I'm finished. To this end, it would be useful to know what position people hold so that I know what arguments I need to address. So what is your position on anthropics?

Various anthropic problems include the Sleeping Beauty problem, the Absent-minded driver, the Dr Evil problem, the Doomsday Argument, the Presumptuous Philosopher, the Sailor's Child, Fermi Paradox and the Argument from Fine-Tuning.