trentmkelly/LessWrong-43k huggingface dataset:
Fractional Sweets
by jefftk — 2019-12-13T01:50:02.266Z

We have a rule with the kids that they can have two sweets a day: one
before dinner and one after. They don't roll over, so you can't skip
your morning sweet to have two evening ones. You can eat just part of
a sweet, though, and Lily (5y) has gotten really into fractional
sweets.

It started with trying to compare sweets of different sizes. If a big
cookie is a sweet, how many skittles make a sweet? How many little
cookies? How many rolos? So Lily got into counting: perhaps twelve
skittles are a sweet, so she'd count them out. She was very
motivated, since she (a) wants as much candy as possible and (b) is
very into following rules exactly.

Then we got into sweet combinations. Perhaps Lily would want skittles
and ice cream. I would get a half-sized portion of ice cream, and I'd
have her figure out how many skittles were half a sweet. One way we'd
do this would be to divide them into two equal piles and pick one. We
noticed that sometimes you can't make equal piles, because there's one
left over, in which case she would generally want to give the extra
one to her sister. She got pretty comfortable with the idea of half a
sweet.

Currently we're doing a lot on a quarter basis. She doesn't totally
have the concept, but understands it well enough for "I've had half my
sweet; can I have a quarter sweet worth of chocolate chips?" Or if
she's had three of something where four make a sweet she can usually
tell me she still has one coming. Today she asked for half a sweet's
worth of nutella on waffles for breakfast, and then when they weren't
chocolatey enough asked me to bring it up to three-quarters.

One driver here has been uncertainty about what the rest of the day
will hold, and wanting to reserve portions of a sweet. If she had a
full sweet's worth of nutella on her waffle, and then in the afternoon
some tasty option came along, she would be sad not to be able to have
any. By eating only partial sweets she keeps her options open. [1]

From a perspective of keeping our kids from having too much sugar this
seems like a lot of complexity, but I think it's good complexity.
Lily's more motivated here than with anything else except maybe tablet
time [2] and that's translated into her starting to learn how to think
fractionally. One thing I'd like to get into next is how thirds and
halves interact as sixths, but I'm waiting for...
Examples of Causal Abstraction
by johnswentworth — 2019-12-12T22:54:43.565Z

I’m working on a

theory of abstraction

suitable as a

foundation for embedded agency

and specifically multi-level world models. I want to

use real-world examples

to build a fast feedback loop for theory development, so a natural first step is to build a starting list of examples which capture various relevant aspects of the problem.

These are mainly focused on causal abstraction, in which both the concrete and abstract model are causal DAGs with some natural correspondence between counterfactuals on the two. (There are some exceptions, though.) The list isn’t very long; I’ve chosen a handful of representative examples which cover qualitatively different aspects of the general problem.

I’ve grouped the examples by

symmetry class

:

Finite DAGs without any symmetries, or at least no symmetries which matter for our purposes
Plate symmetry (as in “

plate notation

”), in which there are a number of conditionally IID components
Time symmetry, in which the DAG (or some part of it) consists of one repeated subcomponent connected in a straight line (i.e. a Markov chain structure)

Note that many of the abstractions below abstract from one symmetry class to another - for example, MCMC abstracts a concrete time-symmetric model into an abstract plate-symmetric model.

I’m interested to hear more examples, especially examples which emphasize qualitative features which are absent from any of the examples here. Examples in which other symmetry classes play an important role are of particular interest, as well as examples with agenty behavior which we know how to formalize without too much mess.

Finite DAGs: Examples from Electrical Circuits

Electrical engineers rely heavily on nested layers of abstraction, of exactly the sort I’m interested in (i.e. multi-level models of the physical world). Additionally, causal models are a natural fit for digital circuits. These properties make electrical circuits ideal starting points. They’re a great conceptually-simple use case.

A few of the major abstraction layers, from lowest to highest:

Fields: the most concrete-level model used in EE
Lumped circuit abstraction: approximating the system as discrete “wires” with constant voltage connecting “circuit elements” with various voltage-current relationships and internal state.
Digital abstraction: bucket voltages into

...
Two-headed Go
by jefftk — 2019-12-12T03:30:01.526Z

I like playing Go, but of the games I play at all regularly it's
probably the one where there are the most people who are better than
me: a lot of people play go, and many of them play it a lot.

My dad and I are a similar level (~11kyu) and last week a family
friend who's substantially better (~6kyu) was looking for a game.
Instead of either of us playing him 1:1 with ~5 handicap stones, we
decided to play 2:1 even. As a casual game, we played untimed.

It was a really interesting game: my dad any I both compensated for
each others weaknesses, were more thoughtful and careful, didn't make
any major mistakes, and ended up winning by a lot. It felt a lot like
pair programming.

I'm very curious now how much better playing as a pair tends to make
people. While I expected us to do better, +5 ranks (+500 ELO) was a
much larger change than I would have guessed. It also may not be
linear: I would expect better players to generally benefit less, since
worse players are more likely to have uneven-but-complementary skills?

Comment via: facebook
The Cybersecurity Dilemma in a Nutshell
by Gentzel — 2019-12-12T03:20:01.213Z

The Cybersecurity Dilemma: Hacking, Trust, and Fear Between Nations organizes its arguments in a fairly no-nonsense, premise –> premise –> conclusion manner that I thought would be good to summarize with a blog post. You can buy the book here.

The argument:

Security dilemmas are dangerous because in seeking their own security, states often build capabilities and take actions that can directly threaten the security of other states: often creating impressions of imminent offensive intent and prompting escalation. This is especially true for cybersecurity for several main reasons:

States that desire options for future cyber operations must make intrusions in advance.

The sequential stages of cyber-attack (target acquisition, development, authorization, entry, establishing command and control (C2), pivoting, payload activation, and confirmation) can take a very long time.

E.g. time required to find zero-days, develop exploits and new tools, get authorization, get past defenses such as airgaps, set up C2 and to make pivots within networks all without revealing presence.

There is little “momentum” between stages of a cyber-attack since the attacker has to reorient themselves, and this incentivizes early intrusion.
Options for persistence within networks incentivize early intrusion.
Without moving early, states miss-out on developing economies of scale in capability from re-using code and knowledge

States that desire options purely to defend themselves also have the incentive to intrude early as the defensive process of preparation, detection, data collection, analysis, containment, and decontamination benefit from making intrusions.

Intruding to learn the strategic intentions of potential intruders
Intruding to learn potential intruder’s means of intrusion and to develop countermeasures
Intruding to find out if intrusions have already been made and to discover operations in progress

Cyber intrusions for intelligence gathering will be perceived as more threatening than past intelligence operations

They provide means of deliberate targeted attacks as well as accidental disruption, since breaking a network can be easier than espionage
They provide a beachhead for future operations and intrusions
They change the conditions of conflict and competition via:

Enabling joint operations with cyber effects
Providing insight into the defender&#x

...
Causal Abstraction Toy Model: Medical Sensor
by johnswentworth — 2019-12-11T21:12:50.845Z

Author's Note: This post is a bunch of mathy research stuff with very little explanation of context. Other posts in this sequence will provide more context, but you might want to skip this one unless you're looking for mathy details.

Suppose we have a medical sensor measuring some physiological parameter. The parameter has a constant true value X, and the sensor takes measurements M1…Mn over a short period of time. Each measurement has IID error (so the measurements are conditionally independent given X). In the end, the measurements are averaged together, and there’s a little bit of extra error as the device is started/stopped, resulting in the final estimate Y - the only part displayed to the end user. We can represent all this with a causal DAG:

Note that, conceptually, there are two main sources of error in the final estimate Y:

IID measurement noise in the M’s
Noise in Y from the starting/stopping procedure

… so the node Y is not fully deterministic. The joint distribution for the whole system is given by

P[X,M1...Mn,Y]=P[X](∏iP[Mi|X])P[Y|1n∑iMi]

Since all the measurements are to be averaged together anyway, it would be nice if we could just glom them all together and treat them as a single abstract measurement, like this:

Formally, we can do this in two steps:

Replace the nodes M1…Mn with a single node Mall=[M1,…,Mn], i.e. a list containing all the measurements. This doesn’t change the substance of the model at all, it just changes what we’re calling a “node”.
Replace the node Mall with M=1n∑Mi∈MallMi, the average of the measurements. We no longer worry about the individual measurements at all, and just directly compute the distributions P[M|X] and P[Y|M].

The second step is the interesting one, since it changes the substance of the model.

Main question

: under the abstract model, what counterfactual queries remain valid (i.e. match the corresponding concrete queries), and how do they correspond to counterfactuals on the concrete model? What about probabilistic queries, like P[X|Y]?

The concrete model supports three basic counterfactual queries:

Set the value of X
Set the value of Mi
Set the value of Y

… as well as counterfactuals built by combining multiple basic counterfactuals and possibly adding additional computation. In the abstract model:

Setting abstract&

...
[Link] (EA Podcast) Social Status: The Key to the Matrix Part III
by Daniel_Gambacorta — 2019-12-11T20:59:42.419Z

A new episode of Global Optimum has been released! Global Optimum is a podcast aimed at making altruists more effective. This episode is about social status, hypocrisy, signaling, hormones, and politics.

This episode features:

-Why does men’s testosterone go down when they fall in love?

-Does “power posing” have any psychological effects?

-What is “humblebragging” and why does it pervade social media?

-Is our preference for democracy really a preference for high status?

-What is self-esteem?

-How to increase self-esteem (the answer is disappointing)

-How to act high status (the answer is not disappointing)

Full transcript

The podcast is available on all podcast apps.

Listen here
Generalizing Experimental Results by Leveraging Knowledge of Mechanisms
by Carlos_Cinelli — 2019-12-11T20:39:08.739Z

In a recent post (and papers), Anders Huitfeldt and co-authors have discussed ways of achieving external validity in the presence of “effect heterogeneity.” These results are not immediately inferable using a standard (non-parametric) selection diagram, which has led them to conclude that selection diagrams may not be helpful for "thinking more closely about effect heterogeneity" and, thus, might be "throwing the baby out with the bathwater."

Taking a closer look at the analysis of Anders and co-authors, and using their very same examples, we came to quite different conclusions. In those cases, transportability is not immediately inferable in a fully nonparametric structural model for a simple reason: it relies on functional constraints on the structural equation of the outcome. Once these constraints are properly incorporated in the analysis, all results flow naturally from the structural model, and selection diagrams prove to be indispensable for thinking about heterogeneity, for extrapolating results across populations, and for protecting analysts from unwarranted generalizations. See details in the note we post here for discussion.
ToL: Methods and Success
by Hazard — 2019-12-10T21:17:56.779Z

(These are the touched up notes from a class I took with CMU's Kevin Kelly this past semester on the Topology of Learning. Only partially optimized for legibility)

One of the whole points of this project is to create a clear description of various forms of "success", and to be able to make claims about what is the highest form of success one can hope for given the problem that is being faced. The ultimate point of this is to have a useful frame for justifying the use of different methods. Now I'll introduce the gist of our formalization of methods so that we can get back to the good stuff.

In it's most general form, a method M is just a function from info states to hypothesis. M:I→P(W)∖∅

Often I might use notation like M(±A|E) to highlight "This method is responding to a yes or no question A given evidence E. This is mostly useful to be able to talk about certain relations between the question being asked, and your answers.

Gettier Problem

We are going to look at methods that respond with an articulation of an answer. So M(A|E)=A′ st A′⊆A. There's an interesting reason this matters, and part of it has to do with the Gettier Problem.

The Gettier problem has to do with believing the right thing for the wrong reasons. Consider this example.

There's three possible worlds, boxes are possible info states, and the two Hypothesis A and B are outlined. We haven't given a formal notion of what it means to be Occam / "to act in accord with simplicity". But pretend we have. An occam method would say:

M(E1)=A, M(E2)=B and M(E3)=A. In a bit, we're going to make a big deal about the criterion of progressive learning (never drop the truth once you have it). The method I just outline drops the truth in this problem. Suppose w3 is the true world. In E1 is says A, which is true, but then we drop the truth and say B in E2, only to return to it later. Can you see why this is sorta a gettier problem? In E1 we proclaim "A!" but we do it for inductive reasons. It's the simplest hypothesis right now. So it is true that A, but we don't have a super sure reason for saying it. That means that when we get more info, E2, we drop the truth, because our previous "bad reason" for saying A has been disconfirmed.

Our way around these sorts of gettier problems is to ...
ToL: This ONE WEIRD Trick to make you a GENIUS at Topology!
by Hazard — 2019-12-10T21:02:40.212Z

(These are the touched up notes from a class I took with CMU's Kevin Kelly this past semester on the Topology of Learning. Only partially optimized for legibility)

Time to introduce some new Topological terms. We're going to create some good intuitions around the concepts of interior, exterior, boundary, closure, and frontier. These are all operators in the sense that if you have a set S, then Int(S) is me using the $$int$$ operator to create a new set that we call "the interior of S". Ext, Bndr, Cl, and Frnt are the shorthand I will use for these operators.

Before talking about these operators in a topological sense, I want to talk about them in a metric space sense. A metric space is just some mathematical space where you have a way to specify the distance between any two points, according to a

specific definition

of distance. In the real line, the distance between any two numbers can just be the absolute value of their difference. In n-dimensional euclidean space, distance is given by the n-dimensional version of the Pythagorean Theorem. I want to start talking about interiors and boundaries and such from a metric point of view in order to contrast the way it's different from the topological view. I found that when I was trying to wrap my head around these concepts, I was implicitly assuming a metric space world view, because literally every math space I'd interacted with up to that point was a metric space.

Let's start with this picture:

The squiggly loop is our set S. In a metric space, a point is in the interior of a set if you can "draw a circle" around it, such that the circle only contains other points that are in S (formally, you talk about "balls" instead of circles. An r-ball around x is the set of all points z st d(z,x)>r). You can clearly see that I can draw a circle around x, where the circle only contains points in S, so x is in the interior of S. A boundary point like y is a point where no matter how small a circle you draw around it, the circle will contain some points in S, and some points not in S.

Likewise, the exterior of S consists of all points that you can draw a circle around such that the circle only contains points not in S. Here are definitions of our two other operators:

ClS=S∪BndrS

FrntS=BndrS∖S

Metrical to Topological

Now, here's where we shift from the met...
ToL: The Topological Connection
by Hazard — 2019-12-10T20:29:06.998Z

(These are the touched up notes from a class I took with CMU's Kevin Kelly this past semester on the Topology of Learning. Only partially optimized for legibility)

Now that we've got the basic formalism, it's time to go from our intuitive notion of verifiable to our formal notion of verifiable.

Definition: A proposition A is verifiable in (W,I)⟺∀w∈A∃e∈I(w):e⊆A

This basically lines up. Something is verifiable if no matter how it's true, there exists some information that would tell you it's true..

Now, a series of leading questions with a fun surprise at the end:

Question 1: Can you verify the contradiction?

Yes! There is no world in which contradiction is true, and with verifiability we only care about what we can do in worlds where the proposition is true.

Question 2: Can you verify the tautology?

Yes! This follows from the first axiom of our info basis. For every world, you get at least one info state. An info state is a subset of P(W), so no matter what the info state we have, it's a subset of W

Question 3: Can you verify the arbitrary finite union of verifiable propositions?

Yes! Takes a little more thought.

Question 4: Can you verify the intersection of any finite number of verifiable propositions?

Okay, get ready for this, here's The Sixth Sense "He was dead the whole time!" plot twist...

(clipped from wikipedia)

... crazy right?

If we take W to be X, and τ to be "The set of all verifiable propositions in W", then what we just proved matches up exactly with this definition. So we get to claim that verifiable propositions in W are open sets, and that they form a topology on our set of possible worlds.

What does this mean? From a practical stance, it means that we can now use all of topology, a rich and developed branch of math, to think and talk about verifiability. For any theorem that proves something about opens sets, we can make the same conclusion about verifiable propositions. Before we get into any of that, let's expand our mapping from verifiability concepts to topological concepts.

Definition: A is Closed⟺Ac is open

Okay, so we know that open is verifiable, let's see if we can connect an intuitive notion to closed. Let's look at the Halting Problem. In CS, you'd say the halting problem is semi-decidabl...
ToL: Introduction
by Hazard — 2019-12-10T20:19:06.029Z

(These are the touched up notes from a class I took with CMU's Kevin Kelly this past semester on the Topology of Learning. Only partially optimized for legibility)

Feasibility Contextualism

A large amount of philosophy has been people trying to demonstrate how you can never really know anything [citation needed]. Various forms of skepticism take the stance, "No method of inquiry get get you Real KnowledgeTM, and so no method is justified."

Despite that, it seems like (at least in terms of watching how people act in the world) everyone gets that you need some level of pragmatism. "Well I've gotta do something, and this seems like the best idea, so I'm going to do it instead of doing nothing." No one is so skeptical of knowledge that they have stayed immobile until they starved to death [citation needed].

What Kelly aims to do is create a rigorous formalism for the sort of pragmatic attitude that people take all the time. It's very much inspired by how computer scientists do things. If someone proves that a problem can't be done any faster than quadratic time, and you figure out a quadratic time algorithm, you're happy. You don't refuse to use any algorithm that doesn't run in constant time.

In a sentence, this course was about how really cool formalism to talk about "How hard is a given scientific problem" and how that effects "the best possible performance you can get given the hardness of the problem".

Sneak Peak: Induction and Metaphysics

Over the ages people have postulated what qualities scientific hypothesis should have. The logical positivists asserted that only verifiable propositions should be the domain of science (if it's true, you can do some test to demonstrate it's true). Popper wanted hypothesis to be falsifiable (if it's false, you can do some test to demonstrate it's false). Verification and falsifiability have an important connection to two other notions that philosophers of science often talk about, the problem of induction and the problem of metaphysics.

You face the problem of induction if it's the case that even if you hypothesis is true, you'll never know for sure (will the sun rise tomorrow? You can never rule out that it just won't at some point).

You face the problem of metaphysics if it's the case that even if your hypot...