trentmkelly/LessWrong-43k huggingface dataset:
Eternal, and Hearthstone Economy versus Magic Economy
by Zvi — 2018-02-10T20:20:00.512Z

The game Eternal (that is my referral link), created by Magic professionals including lead designer Patrick Chapin, is modern Magic: The Gathering, with some simplifications and tweaks, on a phone, with a Hearthstone interface and economy.

That is super high praise.

Magic: The Gathering is the best game of all time. Eternal gives you the core Magic game play, things like mana bases and Magic-style attacking and blocking and even a stack. It even gives you genuine drafts. And it does all of that on a phone, with a good free play experience. Like Hearthstone, it looks crisp and good, plays light and fun. And in contrast to Magic: The Gathering Online, it works. Which is nice.

Do I have quibbles? Of course I have quibbles! I hate the move to 75 card constructed decks instead of 60. The changes to the color pie don’t work for me. The Eternal community takes its names for the colors and color pairs seriously, as opposed to winking every time they say ‘time’ and ‘primal’ instead of white and blue. Legendary cards are a bit ludicrous. Organized play isn’t where I’d like. I feel like there’s a better way to do social. Things seem copied from Hearthstone or Magic that feel like they don’t belong. But these are quibbles. I can’t argue much. The game is great, and there’s lots of little things I’m really happy about.

Two things rise above quibble.

I want an ‘old school’ Magic experience. I want players’ hands, lands, colors, decks, spells, creatures, right to attack and so forth all up for grabs. Most players, studies show, disagree. Players want to play their cards, cast their spells, fight with creatures. A little control and a little combo is good but mostly men need to be fighting, or players won’t be happy. Eternal certainly has control. But like modern Magic, Eternal can’t surprise and twist its premises. I’d love to see a modern take on old school, with Armageddon and Winter Orb, Mind Twist and Moat. I also miss true power: Ancestral Recall, Time Walk, Black Lotus. I have some ideas. But despite the name, that is not Eternal.

What I want to explore here is the economy.

Magic Economy

The original Magic economy is simple. Wizards sells you packs, you get cards you can keep, trade or sell. Cards are worth money. Most are worth very little since supply exceeds demand,...
The end of public transportation. The future of public transportation.
by mako yass — 2018-02-09T21:51:16.080Z

[Epistemic Status: Opinion]

The following is just one vision, but it is an enticing one, and something like it will probably come to bear.

Owning an autonomous car will not make sense, because as long as they’re sitting in your garage they are being wasted. Anyone who did own one would be under economic pressure to lease it out to a fleet. They would ultimately be convinced to sell it to the fleet, because it is worth much more to the fleet than it is to them. If our markets did not completely suffice this expropriation of commons resources, the people would have to run in torch-and-pitchfork and liberate the cars themselves. It is both evil and stupid to keep these noble service robots pent in a garage.

Transportation will become a utility.

When the cars- algorithms so extensively honed and tested- effectively never crash, they wont need a strong, heavy frame. Since their users are not their purchasers and most trips- laid out in advance- are guaranteed to be short, they will not need large batteries either: Neither a purchasing department nor a routing algorithm has to contend with range anxiety. With the relatively simple electric motor parts and mass production on the dime of the new taxi companies (which will hopefully not be Uber. I see no reason why Uber would really have an advantage here.), they will be cheap as hell relative to the old human-driven cars. They will flood the streets.

A car running a sufficiently sophisticated coordinated driving algorithm will not necessarily ever have to stop moving. At intersections, they’ll be able to weave through each other like schools of fish. If the public don’t believe this they will be invited to download a simulator and watch it for a while, see how it works, and how tolerant it is to interference. The code will be public, of course, as the autonomous network is a public resource, and cars from competing manufacturers would like to be able to coordinate.

Gliding along on low-friction bearings, these lightweight electric cars that never have to stop moving will be gratuitously efficient, while anything carrying more than 10 people must start and stop constantly to let them on and off. To satisfy their conflicting needs, a bus would have to take a highly indirect route. This is the reason busses are slower than driving and there is no reason to perpetuate their waste once you have autonomous electrics littering e...
Status: Map and Territory
by alkjash — 2018-02-09T19:10:01.182Z

I’m here to add another angle to the discussion on social vs. objective truth (example). Here’s an analogy for reasoning about status games and why people react so strongly against improper status moves:

Society is a collective consciousness. From Society’s point of view, the status game is the map. Genuine competence (some combination of skill, virtue, and value) is the territory. The map is meant to track the territory.

Human instinctively play the status game; it’s impossible to just say what you mean. The status game is built into people’s verbal and nonverbal behaviors toward one another.

If the status game is a good map, you can decide who to befriend, admire, and chastise based simply on their status moves. You can figure out who best to ask for advice by the way they hold their arms. You can trust the beliefs of confident people without individually investigating each of their claims. The human brain opts into the status game by default to partake in all this free value.

If the accuracy of the status game is corrupted, the map loses all value. Trust breaks down and you have to rely on first principles.

There’s an approved way of climbing the status ladder: acquiring genuine competence. Well-socialized individuals naturally play higher status as they become more competent in the relevant domain, since the connection between competence and status is built into their brains. Society approves: the map keeps fidelity to the territory.

There’s an improper way of climbing the status ladder: playing status above your competence. Jordan Peterson’s go-to example is serial killer Paul Bernardo in this prison interview. Note the minute-long interaction between Bernardo and the lawyer(?) on the right. Bernardo acts like a disappointed CEO lecturing a wayward and nervous underling.

Knowing the truth about the individuals involved, I have a visceral reaction against this status interaction: the map has detached from the territory. Even if Bernardo is speaking only literal truths, there’s an instinct that screams he’s lying.

I predict that the neural mechanisms for detecting truth from falsehood (i.e. whether the map corresponds to the territory) are closely related to the mechanisms for distinguishing proper and improper status moves (i.e. whether the status map corresponds to genuine competence). I predict that your negative reaction against lying feels similar to your ne...
Antiantinatalism
by Jacob Falkovich — 2018-02-09T16:49:05.063Z

Cross posted from Putanumonit.com

Does life suck? I think it does, somewhat. Others would say it sucks even worse than that. David Benatar thinks that life sucks so profoundly that bestowing it on anyone, like your child, is a grave crime. The antinatalist philosopher is having a moment: a profile in the New Yorker, an appearance on Sam Harris’ podcast, and a reader emailing to ask me what I think of the case against being born.

I tried to make sense of antinatalism, and I think it’s bad philosophy. But I also think it’s bad economics, and plays on the widely held intuition that an extra person makes the rest of humanity worse off by taking up some space, some resources, some piece of the pie that would have gone to others. I hold that this zero-sum view is ignorant of the reality of the modern economy. Having children is good for the children, good for you, and good for the world.

Like a pirate cow, the philosophy of antinatalism stands on three legs:

Some conception of negative utilitarianism, i.e. the view that reducing suffering is the principal (or only) thing that matters morally, and that moderate pain weighs more than great joy.
The observation that life contains more bad things than good things.
A divide between existing and non-existing agents, and the assertion that the latter (i.e. unborn babies) have only an absolute preference to avoid the risk of any suffering, and no preference for enjoying anything at all.

I’ll address these in order.

Negative utilitarianism

Negative utilitarianism isn’t strictly necessary for the antinatalist argument, which is lucky for them because it’s a pretty incoherent philosophy. However, antinatalists sometimes sneak it in by implication. For example, the reader asked me if I’ll take 5 minutes of the worst pain imaginable for an hour of the greatest pleasure. Benatar presented Harris with a similar trade-off: would Sam be willing to have children and grandchildren who will lead lives of hardship if that will ensure a bright future for countless descendants after that?

Both questions expected an answer in the negative, and yet my answer to both is that I most certainly would.

Regarding the first question: evolution hasn’t made great pleasure as accessible to us as it has made pain. Fitness advantages from things like a good meal accumulate slowly but a single injury can drop one’s fitness to zero, so the pain of an injury is fe...
A Safer Oracle Setup?
by Ofer — 2018-02-09T12:16:12.063Z

Edit: I no longer think this post deserves attention.

Edit: Invoking a black-box agent is probably a very bad idea unless there’s a consensus in the AI safety community it should be done.

This post describes a setup that allows asking a super-intelligent agent a question, and getting an answer that humans with an unbounded amount of time could come up with (Edit: actually, if the question is "too easily distinguishable" from questions we already know the answer to, it's plausible we won't get a useful answer). The setup is not restricted to questions with a single or precise answer, and there is no restriction on the length of the answer. This setup might be safer than the naive one - let me know what you think (and let me know if it's all already covered by existing work).

The Setup

First, we build a big dataset of (question, answer) pairs: {(Qi,Ai)}

The questions are of the following form: "Limited to 100 words, what might humans answer in 2018 when asked how people in 1950 could most effectively improve their wellbeing?" (Edit: this is a bad example, see first edit)

The answer in this example would be a 100 word text, that might contain the phrase "stop smoking". The questions and answers can be written by any "regular" crowd of people (e.g. MTruk workers).

We then create a super-intelligent agent, denoted X, that has the entire dataset written in memory at invocation time. The agent X is in a box and has no input channel. The goal of X is to find an output string o that is at most n characters long (where n is also written in X's memory at invocation time). The string o is the code of another agent, denoted Y.

Agent Y also has no input channel, and when invoked - some question Qi should be written in its memory. Its output is a probability distribution over all possible answers for Qi (we assume that an upper-bound on the answer length is define by the question).

The reward of X is the log-likelihood of the answers {Ai} as defined by the output of Y, meaning:

∑ilog(PY(Ai))

where PY(Ai) is the probability that the output of Y assigns to Ai.

We then follow the following algorithm:

n←1000
Create an agent X with our dataset and n in memory, and invoke it.
Take the output of X, which is the code of an agent Y, and invoke Y, each time with a new question (not from the original dataset). The questions should b

... ,,,,
"Backchaining" in Strategy
by Davis_Kingsley — 2018-02-09T12:01:54.523Z

Recently, some have been discussing "backchaining" as a strategic planning technique. In brief, this technique involves selecting a target outcome, then chaining backwards from there to determine what actions you should take; in other words, rather than starting with your current position and extrapolating forward, you start with the desired position and extrapolate back. (As far as I can tell there is very little, if any, difference between this and backward induction in game theory, but I've heard several in the community call this "backchaining" recently and so use that term here.)

For instance, you might say "Okay, to get people on board with this project we'll need to give a presentation. To make a presentation we'll need a slide deck. To make a slide deck we'll need to get the relevant metrics. Therefore, let's go get the relevant metrics."

Backchaining can be useful in that it allows you to ensure that your actions are cutting through to the objective. However, there are some scenarios where backchaining isn't really an appropriate technique - namely ones where the objective is long-term in nature or involves too many unknown unknowns.

One example that might prove fruitful to investigate is that of chess. In chess, backchaining from desired positions can be useful. However, humans simply can't establish long enough chains for this to be a practical tool in long-term planning. It would be absurd to say "All right, this game I'm going to go for a king-and-rook mate on the 'a' file against my opponent's king alone. To do that, I'll need to use the principle of zugzwang to force my opponent's king to move into an unsafe position. To do that, I'll need to..."

Instead, one usually focuses on accumulating generalized advantage in the opening and midgame, and once sufficient advantage has been established one can then transition into planning for specific endgame positions. This approach is much more effective - fixating on too particular a scenario constrains your options, so instead you just focus on building an advantage.

A similar thing is true in other domains. While backchaining can be a useful technique for accomplishing projects with relatively concrete and legible goals, it starts to falter and even become counterproductive when aimed at a target ...
Fundamental Attribution Error and The Other
by Adrian Smith — 2018-02-09T06:48:23.255Z

I.

The fundamental attribution error is a well known psychological phenomenon wherein people are likely to ascribe external motivations to their own failure and external motivations for the failures of others. This has a variety of factors, and is a well studied concept, however I intend to focus on one aspect of this common problem. One explanation for the FAE is that it results from the simple truth that we know our own situations better than those of others, and fail to take into account the uncertainty in predicting the internal state of another person. This attempt to distill certainty out of uncertain information is a major flaw in many kinds of thinking, and has predictable consequences.

When we try to understand the internal motivations of other people and weigh them highly in comparison to circumstances, we often fit people into our narratives about the world in general. This is similar to ideas presented by Tetlock in that those who attribute evidence to sweeping large scale theories also tend to be more certain than they ought to be given the evidence available. In the case of individuals trying to understand each other, or the case of individuals trying to predict the world, both can fall prey to tendency to underestimate uncertainty and fill the gap between evidence and confidence with broad theories. In reality, often the things we are trying to understand are complex and strongly driven by factors outside of our model. This isn’t to say that having a model is bad, but that for a model to be good it must include an acceptance and awareness of the inherent uncertainty in the problem domain.

II.

However, there is another instance not yet mentioned that follows a similar pattern. If we are to say that interpreting the motivations of an individual is a difficult and uncertain endeavor, mustn’t we also say the same of groups of people, at least to a certain extent? Despite this, there is a strong tendency to explain the behaviors of groups with great confidence using highly simplistic models of their behavior and motivations.

From a Cracked article about Russian Who Wants to be a Millionaire:

In America, the audience viewed the million-dollar prize as a reason
to congratulate a fellow human on accomplishing something great. In
Russia, the audience viewed the million-ruple prize as a reason to
mourn the fact that there’s one less person to share the rampan...
Stable Pointers to Value II: Environmental Goals
by abramdemski — 2018-02-09T06:03:00.244Z

Cross-posted.

In

Stable Pointers to Value

, I discussed various ways in which we can try to “robustly point at what we want” (ie, do value learning). I can tidy up the discussion there into three categories:

Standard reinforcement learning (RL) frameworks, including AIXI, which try to predict the reward they’ll get and take actions which maximize that expectation. These are incentivised to hack whatever system is feeding them rewards. This is the “easy problem of wireheading”.

Observation-utility agents

(OU), which get around the problem by assessing the future plans with the current utility function subsystem rather than trying to predict what the subsystem will say. This removes the incentive to manipulate what the subsystem will say. The subsystem can itself have uncertainty about what the “true” utility function is; for example, representing uncertainty about what the human wants. CIRL falls into this category. There are a number of problems which can arise here depending on the setup; see for example

basically everything Stuart Armstrong has ever written

. In particular, you’ve got to define “how much the human likes the plan” somehow, and this is usually

manipulable

, often to the point of

wireheading

. I called this the “hard problem of wireheading”.

Approval-directed agents

(AD) maximize human approval of individual actions, rather than planning to maximize overall approval. Just as observation-utility agents get around the easy wireheading problem by putting the reward function inside the optimization loop, this gets around the hard wireheading problem by putting humans in the loop. Other problems are introduced, though; most prominently, (a) how do we trust the learned model of human evaluation itself, and (b) how do we make this kind of system competitive with systems which plan ahead, given the obvious advantages of planning ahead? Paul Christiano has thoughts on these problems in various posts.

I want to point at an analogy to three categories of approach to the problem of generalizable environmental goals (as defined in the

alignment for advanced machine learning agenda

). It’s a fairly messy analogy, and there’s probably a better way of organizing the landscape, but FWIW.

1. Supervised Learning

Imagine you’re trying to teach a system to build bridges by showing it examples. You could learn a big neural network which distinguishes cases of “successfully b...
Knowledge is Freedom
by Scott Garrabrant — 2018-02-09T05:24:54.932Z

[Epistemic Status: Type Error]

In this post, I try to build up an ontology around the following definition of knowledge:

To know something is to have the set of policies available to you closed under conditionals dependent on that thing.

You are an agent G, and you are interacting with an environment e in the set E of all possible environments. For each environment e, you select an action a from the set A of available actions. You thus implement a policy p∈AE. Let P⊆AE denote the set of policies that you could implement. (Note that AE is the space of functions from E to A.)

If you are confused about the word "could," that is okay; so am I.

A fact (F,ϕ) about the enviornment can be viewed as a function ϕ:E→F that partitions the set of environments according to that fact. For example, for the fact "the sky is blue," we can think of F as the set {⊤,⊥} and ϕ as the function that sends worlds with a blue sky to the element ⊤ and sends worlds without a blue sky to the element ⊥. One example of a fact is (E,id) which is the full specification of the environment.

A conditional policy can be formed out of other policies. To form a conditional on a fact (F,ϕ) we start with a policy for each element of F. We will let c(f) denote the policy associated with f∈F, so c:F→AE. Given this fact and this collection of policies, we define the conditional policy pc:E→A given by e↦c(ϕ(e))(e).

Conditional policies are like if statements in programming. Using the fact "the sky is blue" from above, we can let kr be the policy that pushes a red button regardless of its environment and let kg be a policy that pushes a green button regardless of its environment. If c(⊤)=kr and c(⊥)=kg, then pc is the policy that pushes the red button if the sky is blue, and pushes big green button otherwise.

Now, we are ready to define knowledge. If P is the set of policies you could implement, then you know a fact (F,ϕ) if P is closed under conditional policies dependent on F. (i.e. Whenever c:F→P, we have pc∈P.) Basically, we are just saying that your policy is allowed to break into different cases for different ways that the fact could go.

Self Reference

Now, let's consider what happens when an agent tries to know things about itself. For this, we will consider a naturalized agent, that is part of the environment. There is a fact (A,action) of the environment that says what ... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,