trentmkelly/LessWrong-43k huggingface dataset:
Ideas for an action coordination website
by Yoav Ravid — 2019-03-08T13:32:06.756Z

inspired by Inadequate equilibria, and following the 'KickStarter for Coordinated Action' sequence.

This is an idea-dump post for a website i thought of after reading Inadequate equilibria.

Today, tools like Facebook and twitter help us coordinate better and faster. but still, that is not enough to solve problems of "high-inadequacy" - where we're stuck in bad Nash equilibrium, and moving away from it demands many things to happen together.

The idea here is to take coordination much further, so we can solve as much of the game theory problems bound in moving to other Nash equilibria.

The goal is to allow 'Exoduses' from bad Nash equilibria, to better a Nash equilibria.

Note - none of this (at least as of writing this) is being worked on, nor are there currently plans to do so. Though, of course this doesn't mean there shouldn't be, or else i wouldn't have wrote the post :)

basic concept

big problems can't be solved by a single person. but sometimes even a large group of people who agree on the problem and the solution, and are even motivated to bring the change - can't do it. That is because some problems are more complicated than that.

one such complication is coordination problems - where "everyone is taking some action A, and we’d rather all be taking action B, but it’s bad if we don’t all move to B at the same time." or each individual wouldn't want to take action A unless he knows the rest do too.

solutions to coordination problems include common knowledge (Where all know the intentions of all actors) and pre-commitment. that's the site's goal.

Coordinated actions

Any user can become a coordinator/initiator by creating a coordinated action (CA), in order to solve some problem. Each CA gets its own page (similar to kickStarter). Any user (unless otherwise specified) can obligate to the action - and here's the catch - the commitment is to be realized if and only if a certain number of other users obligated the same. In KickStarter, we commit to pay, here we commit to take action.

Versatile coordination:

Since reality is very dynamic, a rigid structure will work for few situations and be less useful. So to allow a large array of projects, options for CAs should be versatile. I see it as an ongoing project that will develop next to the community's needs. think of how tesla is implementing features in dialog with their community's wishes. here are some examples of "contr...
A defense on QI
by Billy23 — 2019-03-08T03:47:59.687Z

I dont know if this is the right place to post but i need help finding a debunking of this article
https://arxiv.org/ftp/arxiv/papers/0902/0902.0187.pdf
I am sure many worlds DOES imply immortality but i cant seem to adress all the big words the article uses
.if this is already done please just link the debunking.
Want to Know What  Time Is?
by Alexander Weinmann — 2019-03-07T19:49:29.465Z

I do not consider myself to be smart enough to answer a question that remains unanswered since thousands of years:

What is time?

Nevertheless I will answer that question now. I am asking every person smarter than me to explain to me, why my answer must be wrong. More so: There may even be some very basic mathematics below. I am very keen to find out why this mathematics is either wrong or irrelevant – or simply stupid. It sure is!

I define time as the amount of information available for a given phenomenon.

If there is a lot of information, then the phenomenon is considered to exist for a very long time. If only little information is available, the phenomenon is considered to exist for a very short time only.

For example, if a guy called Join Doe is traveling from Berlin to Moscow, there might be at least two bits of information about this:

John Doe is in Berlin
John Doe is in Moscow

If there is no other information available, then the phenomenon is very short lived, meaning John was traveling at maximum speed. That could be the speed of light.

But there might be more information available: John might take a stop in Minsk, because he loves a most beautiful Belarus girl at that place. It just means, that his travel speed is much slower. Far from the speed of light in this very case! So there is more time between his departure in Berlin and his arrival. It might even happen, because he is so very much in love with this girl, that he will never arrive in Moscow. In that case, the amount of time between departure and arrival is infinite.

Time Tcan thus be seen as function of the discrete amount N of available information: T(N) We know that for any N_1,N_2 with N_2 > N_1there will always be T(N_2) > T(N_1).

(But apart from that I do not really know a lot about the function T(N).)

Please note that by this definition there is no direction associated with time! So, time does not really tell us if Johnny starts his journey in Berlin or in Moscow. We can only be sure that he was in Minsk, and that he was having a great time there, wishing to stay for ever with the girl. (Maybe she also really cares a lot about him!)

Things are getting more interesting, if we – for example – consider a black hole. As we know by the theory of relativity, time “is slowing down” a bit, the nearer you get to a black hole. Now, let us look at John Doe again: Imagine this guy is getting really close...
Open Thread March 2019
by ryan_b — 2019-03-07T18:26:02.976Z

If it’s worth saying, but not worth its own post, you can put it here.

Also, if you are new to LessWrong and want to introduce yourself, this is the place to do it. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome. If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, and seeing if there are any meetups in your area.

The Open Thread sequence is here.
IDA 1-4/14: Problem Statement
by RAISE — 2019-03-07T14:02:01.642Z
Link: https://app.grasple.com/#/level/1506

Every Thursday for 4 weeks, we will be posting lessons about Iterated Distillation and Amplification. They're largely based on Paul Christiano's sequence here on LW. He graciously allowed us to use his work.

Note that access to the lessons requires creating an account here.

Have a nice day!
In My Culture
by Duncan Sabien (Inactive) — 2019-03-07T07:22:42.982Z
Link: https://medium.com/@ThingMaker/in-my-culture-29c6464072b2

This is a linkpost for https://medium.com/@ThingMaker/in-my-culture-29c6464072b2

One of my colleagues has a neat little trick.

Sometimes, in the middle of a conversation—especially if things are heated or tense or confusing—they’ll pause, and they’ll say something like:

“I’m sorry, but—look. In my culture…”

…and then they’ll go on to explain something that one might easily mistake for a rule, or a request, or the enforcement of a social norm, but which is actually (as far as I can tell) just a statement about the version of the world they carry around inside their head.

Because none of us quite have the same culture, after all. We all differ in different ways from the Basic Package—even those of us who’ve lived in the same towns, gone to the same schools, worked in the same industries, played the same sports, read the same books, watched the same shows—we’ve all got our own unique little takes, built up out of the odd quirks of our parents, tiny traumas and formative experiences, countless accumulated musings about how Things Could Be So Much Better If Everyone Would Just _________!

And so Your Culture, though it might match mine at a thousand different points, will also be noticeably different at a thousand others. You and I would found different churches, write different constitutions, build different schools and startups—

—and we would recognize different things as trespasses or offenses, and react to those trespasses and offenses in different ways.

“They must talk to each other directly, Ender, mind to mind. What one thinks, another can also think; what one remembers, another can also remember. Why would they ever develop language? Why would they ever learn to read and write? How would they know what reading and writing were if they saw them? Or signals? Or numbers? Or anything that we use to communicate? This isn’t just a matter of translating from one language to another. They don’t have a language at all. We used every means we could think of to communicate with them, but they don’t even have the machinery to know we’re signaling. And maybe they’ve been trying to think to us, and they can’t understand why we don’t respond.”

“So the whole war is because we can’t talk to each other.”

“If the other fellow can’t tell you his story, you can never be sure he isn’t trying to kill you.”

There’s a particular school of therapy called Internal Family Systems, which w...
Asking for help teaching a critical thinking class.
by Ronny Fernandez — 2019-03-07T02:15:38.208Z

I will soon be teaching a "critical thinking" class for undergraduates. Feel free to mentally replace "critical thinking" with "epistemic virtue". I would appreciate answers to any of these questions:

What would you do if you only had one three hour class period to teach a group of 15-30 undergrads "critical thinking"?

If you know me (Ronny Fernandez) what one educational objective would you give me, knowing it will not be my only one, to come up with a plan to achieve?

How would you given three hours or less make it so that 15-30 students are all absolutely sure of something, and then realize they were all wrong, without feeling like them feeling like you had cheated or done anything shady?

What are the most transferable rationality skills that are abundant in our community and rare elsewhere and how do you transfer them in a classroom setting?

How do you teach the things in the vicinity of scout mindset vs warrior mindset, arguments as soldiers, politics is the mind killer, without making your student temporarily dumber by giving them a fully general excuse to not engage with any disagreement they feel like ignoring?

How do you teach the difference between the kind of cognition you use to figure out how to get to your friends house and the kind of cognition Malfoy used in hpmor to think about the heritability of magic?

How do you teach the virtue of lightness, the virtue of curiosity?

How do you teach fallacies and cognitive biases without making your students temporarily dumber by giving them a fully general excuse to disregard any position or thinker they disagree with?
Do we need a high-level programming language for AI and what it could be?
by avturchin — 2019-03-06T15:39:35.158Z

While reading S.Armstrong post with some pseudocode, I asked myself: Do we plan to have some high-level programming language for AI? Not LISP or Python, but something close to natural language, but still based on formal set of instruction, like:

Go to shop.

Buy healthy food.

Bring it home in one hour.

---

If we will have such language, will it help in AI safety, as it could be less ambiguous than natural language?

UPDATED: I mean not the language needed to write first AI, but the language to give commands to the already existing AGI (when it appears).
Preferences in subpieces of hierarchical systems
by Stuart_Armstrong — 2019-03-06T15:18:21.003Z

In a previous post, I looked at hierarchical systems, and at subagents within them that could develop their own preferences. This was due to a conversation with Dylan Hadfield-Menell.

This post will ignore subagents, and answer Dylan's original question: can we deduce the goals of a hierarchical system? So, we assume the subpieces of the system are not subagents worthy of ethical consideration. Now, given that the subpieces could be inefficient at "what they're supposed to be doing", how can we assume the system itself has a goal?

Algorithms and symbols, again

The situation is as it was in the initial post, a simplified version of the task in this paper. Here an agent is moving object A to storage:

Within the hierarchy of operations needed to move object A, there is a subalgorithm tasked with gripping object B; the code for that is as follows (this is a slight variant of Algorithm 1 in the earlier post):

Algorithm 3:1:State: S={Gripped(B)=False}2:Goal: G={Gripped(B)=True}3:Suggested plans, generated by SPG(S, G):4: (Radio_On AND Grip(B))=> Gripped(B)=True5:While G=False:6: Run Suggested plans according to criteria C(G)

Here the criteria C(G) just assess whether the plan will move the situation closer to the goal G.

Let's look at the subalgorithm that is generating the suggested plans, which I labelled SPG (suggested plan generator). It takes the State S and Goal G as inputs.

It is obviously bad at its job: turning on the radio is not necessary or useful for gripping B. It's not utterly incompetent at its job, though: turning on the radio and then gripping B, will at least result in B getting gripped.

The job of an algorithm is to do its job

I've been talking about SPG's 'job'. But how do I know what this is (especially given that the term is probably undefinable in theory)?

Well, there are a number of indications. First of all, akin to "figuring out what Alice wants", we have the evocative names of the various variables: State, Goal, Suggested plans. If these labels are grounded, then they clearly indicate what the intended task of SPG is.

Note also that there is a loop where the plans of SPG are run until G is true. Even if the sy...
Leaky Concepts
by Elo — 2019-03-05T22:01:37.595Z

Original post: http://bearlamp.com.au/leaky-concepts/

When is a door not a door? When it’s Ajar.

See also:

motte and bailey fallacy

by

Scott Alexander

,

The 5-second level

,

Disguised theories

,

Neural Categories

By Eliezer Yudkowsky.

No Boundary

By Ken Wilber.

Every concept when challenged with reality is a leaky concept (even this one). The idea of a circle seems pretty great until I try to draw one in chalk on pavement. If you want to go mad, commit to drawing two lines the same length and don’t stop until you are dead from trying to line up atoms to be in the right places. There are quicker ways to go insane.

The

map-territory

distinction makes it difficult to pin down a

mapped concept in the territory

. The strange thing about reality is that despite there being a

gap between minds

, we generally have managed to communicate, to get things done, and to build a world. This world. The world in which we live in. Bricks and mortar, bits and atoms alike. We did it. We got to here, even though every concept leaks to all buggery.

Take “science man” in the prehistoric times of the savannah.

Cave man: “run it’s a lion”

Science man: “actually that’s a leopard, judging by its spots, I’d say it’s running at 40mph and will get here- augh!”

*science man gets eaten by leopard*

This silly example hopefully drives home the point of “how much does that leak matter?”. For most of history, for most conversations the difference between lion and leopard did not matter. Being right about which it was, had no effect on the basis of the following actions.

We don’t live in that world so much. We live in the world of

The Mars Climate Orbiter

, which is now the reason that all space calculations are done in metric.

“The discrepancy between calculated and measured position… had been noticed earlier by at least two navigators, whose concerns were dismissed because they "did not follow the rules about filling out [the] form to document their concerns"”

“Oh that silly space agency, I’d never make such a mistake...”

“No one in my life has ever tried to say my name and accidentally used my sibling’s name and my dogs name before finally using my name…”

The way we measure or define a concept, and rely on a shared meaning of a concept is not going to be slightly wrong enough to cause serious errors that go unnoticed right up until something terrible happens. Except for that one time, ...
Three ways that "Sufficiently optimized agents appear coherent" can be false
by Wei Dai — 2019-03-05T21:52:35.462Z

There has been a couple of recent posts suggesting that Eliezer Yudkowsky's Sufficiently optimized agents appear coherent thesis does not seem useful because it's vacuously true: one obvious way to formalize "coherent" implies that all agents can be considered coherent. In a previous comment, I suggested that we can formalize "coherent" in a different way to dodge this criticism. I believe there's reason to think that Eliezer never intended "Sufficiently optimized agents appear coherent" to have an airtight argument and be universally true. (The Arbital post contains a number of caveats, including "If there is a particular kind of optimization pressure that seems sufficient to produce a cognitively highly advanced agent, but which also seems sure to overlook some particular form of incoherence, then this would present a loophole in the overall argument and yield a route by which an advanced agent with that particular incoherence might be produced".) In this post, I suggest that considering the ways in which it could be false can be a useful way to frame some recent ideas in AI safety. (Note that this isn't intended to be an exhaustive list.)

Distributional shift

Even a very powerful optimization process cannot train or test an agent in every possible environment and for every possible scenario (by this I mean some sequence of inputs) that it might face, and some optimization processes may not care about many possible environments/scenarios. Given this, we can expect that if an agent faces a new environment/scenario that's very different from what is was optimized for, it may fail to behave coherently.

(Jessica Taylor made a related point in Modeling the capabilities of advanced AI systems as episodic reinforcement learning: "When the test episode is similar to training episodes (e.g. in an online learning context), we should expect trained policies to act like a rational agent maximizing its expected score in this test episode; otherwise, the policy that acts as a rational agent would get a higher expected test score than this one, and would therefore receive the highest training score.")

A caveat to this caveat is that if an agent is optimized for a broad enough range of environments/scenarios, it could become an explicit EU maximizer, and keep doing EU maximization even after facing a distributional shift. (In this case it may be highly unpredictable what th

...
On The London Mulligan
by Zvi — 2019-03-05T21:30:00.662Z

Original Announcement: Mythic Championship II Format and the London Test

Previously (Patrick Sullivan at Star City Games): The London Mulligan: A Game Designer’s Perspective

We’ve now had about a week to think about the London Mulligan rule. I’ve changed my mind several times in that period. There are big advantages and big disadvantages. It’s not obvious which should dominate.

My preliminary conclusion is that this is quite bad for Modern, Legacy and Vintage, and probably small positive for Limited. I’m increasingly confident it’s also bad for Standard.

Advantage 1: Less Fully Dead Games

This is the headline reason. It’s a big one.

Going to six is a big disadvantage but often survivable. Going to five and winning is a story. Going to four is so bad it often raises the question of whether it’s worth playing your first land and revealing what deck you’re running. Being down two or more cards is bad enough without the risk of not having any lands, or missing your color, or not having early spells.

Because mulligans are so bad, players often keep seven card hands that don’t have much action, or that need a land soon in order to play Magic at all. Making mulligans less hostile lets those dead games be avoided as well.

With the new rule, you get to look at all seven cards on six. You are much more likely to have a reasonable mix of lands and spells, and at least do some things. At five cards, you are probably going to have the number of lands you want, usually two or three, and a few reasonable spells. You’re down two cards, but you otherwise get to play.

This is very good. When I look back upon my experiences at the Pro Tour and Grand Prix in Cleveland, most of the games were great fun, but the dead games were the exceptions, and result in standing around for a long time waiting for the next round.

Minimizing those games is a big win. Such games can even be an exit point where people quit playing.

How many do we get to avoid? Definitely a substantial number, but also definitely not the majority.

Disadvantage 1: More Time In Mostly Dead Games

The direct danger in avoiding free wins is that we might turn games on four or five cards into long agonizing struggles that never had a chance. Whereas before, you could lose quickly and move on. This is especially true when playing online. Doubly true on the ladder in Arena.

In a weird way, the game being completely over right away ...
mAIry's room: AI reasoning to solve philosophical problems
by Stuart_Armstrong — 2019-03-05T20:24:13.056Z

This post grew out of a conversation with Laurent Orseau; we were initially going to write a paper for a consciousness/philosophy journal of some sort, but that now seems unlikely, so I thought I'd post the key ideas here.

A summary of this post can be found here - it even has some diagrams.

The central idea is that thinking in terms of AI or similar artificial agent, we can get some interesting solutions to old philosophical problems, such as the Mary's room/knowledge problem. In essence, simple agents exhibit similar features to Mary in the thought experiments, so (most) explanations of Mary's experience must also apply to simple artificial agents.

To summarise:

Artificial agents can treat certain inputs as if the input were different from mere information.

This analogises loosely to how humans "experience" certain things.

If the agent is a more limited (and more realistic) design, this analogy can get closer.

There is an artificial version of Mary, mAIry, which would plausibly have something similar to what Mary experiences within the thought experiment.

Edit: See also orthonormal's sequence here.

Mary's Room and the Knowledge problem

In this thought experiment, Mary has been confined to a grey room from birth, exploring the outside world only through a black-and-white monitor.

Though isolated, Mary is a brilliant scientist, and has learnt all there is to know about light, the eye, colour theory, human perception, and human psychology. It would seem that she has all possible knowledge that there could be about colour, despite having never seen it.

Then one day she gets out of her room, and says "wow, so that's what purple looks like!".

Has she learnt anything new here? If not, what is her exclamation about? If so, what is this knowledge - Mary was supposed to know everything there was to know about colour already?

Incidentally, I chose "purple" as the colour Mary would see, as the two colours most often used, red and blue, lead to the confusion as to what "seeing red/blue" means - is this about the brain, or is it about the cones in the eye. But seeing purple is strictly about perception in the brain.

Example in practice

Interestingly, there are real example of Mary's room-like situations. Some people with red-green colour-blindness can suddenly start seeing new colours with the right glasses. Apparently this happens because the red and green cones in their ey...