trentmkelly/LessWrong-43k huggingface dataset:
Training sweetness
by KatjaGrace — 2021-02-17T07:00:14.744Z

(This is my attempt to summarize the ‘Taste & Shaping’ module in a CFAR 2018 participant handbook I have, in order to understand it better (later version available online here). It may be basically a mixture of their content and my misunderstandings. Sorry for any misunderstandings propagated. I also haven’t checked or substantially experimented with most of this, but it seems so far like a good addition to my mental library of concepts.)

Some things seem nice, and you just automatically do them (or gravitate toward them), and have to put in effort if you don’t want that to happen. Other things seem icky, and even though maybe you know they are good, you won’t get around to them for months even if they would take a minute and you spend more than that long every week glancing at them and deciding to do them later. (In my own dialect, the former are ‘delicious’. As in, ‘oh goody, my delicious book’).

How delicious things seem is caused by a kind of estimate by your brain of how good that thing will be for the goals it thinks you have.

Your brain makes these estimates in a funny way, with some non-obvious features:

The causal connections between things in the brain’s model are not the ones you would give if asked to describe the situation. For instance, you might say that practicing piano causes you to get better at piano, while in the model, practicing piano mostly causes you to be bad at the piano, since you usually experience being bad at piano immediately after you experience practicing it.

The effects of an action are based mostly on past visceral experiences with similar actions. For instance, if you usually hit your thumb when you use a hammer, then when you get out a hammer today, it might seem non-delicious. Whereas if you are just told that most people hit their thumbs when using hammers, this might not affect deliciousness as much. It is as though it is not in the right language for your brain’s model to take it in. (My guess is that it is more likely to get taken in if you translate it into ‘experience’ via imagining.)

The connection between an action and an outcomes is modeled as much weaker as more delay occurs between them. So that if you press a button which has a good effect in half a second and an equally bad effect in ten seconds, this will sum up in the estimate as good overall, because your brain will model the second effect more we

...
The feeling of breaking an Overton window
by AnnaSalamon — 2021-02-17T05:31:40.629Z

Epistemic status: real but incomplete observations.

In late February of 2020, I went to the grocery store at 2am with my husband (emptiest time), and we bought ~$1k of mostly canned or dry goods. We were the only customers in line. The cashier seemed interested in our purchases, and I felt myself stiffening as she looked. Then she asked me: “are you worried about that virus?”

And… I found myself reaching for a lie, trying to compose a lie, moving my speech-planning-bits as though the thing to do was to lie. I mean, not a technical lie. But I found myself looking for a way to camouflage or downplay my model of the virus.

Oddly, it felt more like a thing happening in me (“I found myself") than like a chosen thing. If you’ll pardon the analogy, it somehow felt at least a bit like throwing up, in that I remember once when I was trying not to throw up, and all of a sudden it was like an alien process took over my consciousness and throat, reached for the bucket, got my hair out of the way, and did the actual throwing up. And then returned my body to me when it was over. The "alien process" sensation felt a bit similar.

With the cashier, I wondered at my impulse as it was happening, but I couldn’t tell if the impulse’s source was “for the cashier's sake” (didn’t seem to make sense); “to prevent her from harming me” (didn’t seem to make sense); or … what exactly?

I forced myself to say “yes” to the cashier's question, and to elaborate a bit; to my surprise, she seemed sincerely curious, and told me several people had been in doing this and she would probably also prepare in some way. Even after this, my sentences wanted to (sound soothing? fit in? avoid disrupting others’ “normal”? I’m still not sure what), and it took me active effort to partially not do this.

—

I am somehow quite interested in what precisely was happening there, and in any related processes.

My guesses as to how to help with this puzzle-set, if you're so inclined:

Share observations (not theories) of any related-seeming things you’ve noticed (the rawer the better);

Share observations (not theories) of what it’s like to be you right now trying to look at this stuff. Do you have introspective access? Do you have sort of have introspective access, and in what way? Do you kind-of-like identify with it? Kind-of-like not-identify with it?

And okay, yes, also theories, I just hope the theory does

...
How poor is US vaccine response by comparison to other countries?
by aphyer — 2021-02-17T02:57:11.116Z

Epistemic status: unapologetically US-centric. Noticing that I am confused, and hoping the internet will explain things.

SECTION I: OBSERVATIONS

Many places I follow have been saying for a long time that US vaccine procurement and distribution is very poor, and that we could have many more people vaccinated if we would not drag our feet so much/not prosecute people for giving out vaccines when we decide they shouldn't have/etc. (I won't reiterate details. For examples, start with e.g. Zvi's post here).

I'll admit that I am predisposed to this viewpoint, and began with a very negative view of e.g. the Food and Drug Administration, but even taking that into account they seem to have very strong points that the US response has been very very bad.

However, Zvi's post included an image of a graph 'daily COVID-19 vaccines doses administered per 100 people' that confused me by showing the US very near the top:

This is only a 7-day average rather than a longer-term one, but still shows the US doing better than most countries. I believe I tracked the data source down to https://ourworldindata.org/covid-vaccinations.

When I sort a list of countries there by total # of vaccinations per 100 people, I get the following list of countries above the US:

Gibraltar	77.0
Israel	76.3
Seychelles	56.9
United Arab Emirates	51.4
Cayman Islands	23.6
United Kingdom	23.3
Jersey	20.8
Turks and Caicos Islands	16.6
Isle of Man	16.1
Bermuda	16.1
United States	15.8

followed by 75 more countries with lower numbers and a bunch more with no data.

Overall, there are 10 countries ahead of the US. One is the UK (a fairly similar country which is also facing a more dangerous local strain). One is Israel (commentary withdrawn). And the other eight, at the risk of seeming like a stereotypical American, are tiny places I didn't even think were countries. (Isn't the Isle of Man part of the United Kingdom? Why does it get its own row?)

I notice that I am confused. If the US rollout of vaccines has been this botched, why are we so far ahead of, say, Germany (5.0)? Or Singapore (4.4)? Or Switzerland (5.6)?

SECTION II: EXPLANATIONS

Five explanations spring to mind:

The data for the US is mistaken (too high). Perhaps we are fraudulently inflating our numbers to look good.
The data for other nations is mistaken (too low). Perhaps they are not publiciz

...
How Strong is Our Connection to Truth?
by anorangicc — 2021-02-17T00:10:16.174Z

[Slider thankfully has pointed out some flaws in parts of my argumentation and I recommend you to also read Slider's comment.]

Everybody who wants to have agency over one's live should believe two things. Firstly, there is an objective truth about the world in which we live. Secondly, we can partially discover this world: There are ways in which we can get an approximate understanding of what's true.

However, questions remain even if these things are taken for granted. My which methods can we get to this truth, and how precise can our approximation get?

It's quite clear for me (and I did not think of these issues myself but believe that they are well-covered in classical philosophy) that any method that uses all available information in the most effective and sound way should have logic as one of its tools (without the advantages of logical inference, the method is not effective; if it does not avoid contradictions, it is not sound). Furthermore, assuming that we don't think we already know the truth (which seems not plausible), we should look out for hints from the environment, evidence, which help us to get better insights into the real world.

There are some more things which basically all human beings who are able to survive assume: that we can partially trust our senses, except when they contradict each other or we have other reasons to doubt. These are basically axioms; just as the mathematician has to assume some axioms before beginning to work, nothing about the real world can be derived without axioms (but as soon as we have those axioms, together with our sensory experiences, we can infer much more conclusions about the real world using our brains). Having nothing but logic or mathematics, there are still many possible ways the world might interact with us. So we need these axioms, or if you wish, priors to trust our sensations.

However, choosing those axioms can be tricky. To get information about the whole of our planetary society, our senses are not enough. We need to extend their range using communication technology and media: phones, newspapers, the internet. To be able to talk about what happens on the other side of the earth, we have to extend our trust in our personal sensations to trust in some newspapers or that telecommunication cables faithfully transport information between different places.

So as soon as people started to globalize the...
Weirdly Long DNS Switchover?
by jefftk — 2021-02-16T20:00:05.905Z

[EDIT: as Jai points out in the FB comments, the problem was likely forgetting IPv6. The
AAAA record is now updated, and I should be able to tell in a few
hours whether that fixed it.]

I'm helping my dad migrate some code to a new server. It was at

162.209.99.139 for years, but ten days ago he changed
his DNS settings to point to

34.199.143.13. Everything
looks good to me:

$ dig notes.billingadvantage.com
notes.billingadvantage.com. 1800
IN A 34.199.143.13

The TTL is 1800s, or 30min, which agrees with

dns.google.
I expected everyone would be moved over within a couple hours, but a
week later the old server is still receiving traffic nearly as much as
the new:

date
old server
new server

2021-02-05
943
0

2021-02-06
201
127

2021-02-07
17
108

2021-02-08
364
423

2021-02-09
488
448

2021-02-10
255
503

2021-02-11
281
345

2021-02-12
250
248

2021-02-13
0
88

2021-02-14
0
78

2021-02-15
217
262

2021-02-16
202
287

The old server getting no traffic on the 13th and 14th is probably
because that's the weekend, and the users who happen to be still stuck
on the old site aren't using it on the weekend. I asked one of the
users still getting the old server to try rebooting, to no effect.

I thought maybe something was misconfigured with the name servers, but
it looks fine:

$ whois billingadvantage.com \
| grep 'Name Server'
Name Server: NS1.ZEROLAG.COM
Name Server: NS2.ZEROLAG.COM

$ dig notes.billingadvantage.com \
@ns1.zerolag.com
notes.billingadvantage.com. 1800
IN A 34.199.143.13

$ dig notes.billingadvantage.com \
@ns2.zerolag.com
notes.billingadvantage.com. 1800
IN A 34.199.143.13

I'm not seeing any references to the old IP address anywhere.

Any guesses about why the traffic isn't moving over?
Reasonable ways for an average LW retail investor to get upside risk?
by Randomized, Controlled — 2021-02-16T19:44:17.277Z

By "upside risk" I mean: more upside risk than just holding the market.
By "reasonable" I mean: will not require an arbitrarily long education process to understand/carry out. I work a day job and have other commitments in my life, and I'm not that fundamentally interested in finance/investing. I'm not trying to get anywhere near an efficient frontier, I just want to be able to get upside risk more systematically than "some smart seeming EA/LWer drops a stock/crypto tip in a facebook group".

By "average": I assume I'm close to the LW median: I'm comfortable making kelly bets in the $1k - $100k range, depending on details. Mostly in the $10k range.

Are IPO ETFs a good candidate for this? SPACE SPAC ETFs? Buy and hold cryptocurrencies that aren't BTC/ETH? Buy and hold BTC/ETH? Sell random call and put options? Something else? Emerging markets ETFs?
Second Citizenships, Residencies, and/or Temporary Relocation
by Josh Jacobson — 2021-02-16T19:24:14.598Z

EDIT (4/20/21): I am occasionally updating this post at this linked Google Doc with new information. You may wish to instead read this post there.

Introduction

Over the past few years, I’ve gained an interest in securing options for residency outside of my home country, in particular via second citizenship.

There are a number of potential benefits that can arise from having a second residency option. These include professional and education opportunities, other economic opportunities, the ability to mitigate a variety of potential risks via relocation, travel benefits, and the potential extendability of these benefits to a spouse, descendants, or in rare cases friends and colleagues.

In this post I:

Share the reasons I’ve identified for securing a second residency
Provide thoughts on planning for a soon upcoming, largely unexpected departure
Describe long-term options for alternative residency
Share my knowledge and experience toward securing a second citizenship
Provide the lessons I’ve learned in pursuing genealogy for the purposes of securing a second residency
Share some other related topics I may write about in the future as an extension of this post and offer to help or connect people interested in securing second residencies

Notes and Disclaimers

The original version of this post was written in early October 2020 and commissioned by the Center For Applied Rationality (

CFAR

). It has been somewhat reorganized for publication on this forum.
This document is written nearly entirely off memory. As a result, it is likely that there are significant mistakes. It’s worth validating anything in here that you plan to act on.
It is also written as a first-pass, optimizing for sharing the information with minimal time-investment. There may be cases of imprecise language, and inconsistencies in presentation or organization, as a result.
I’ve pursued this topic out of personal interest, without an expectation that I would end up sharing my knowledge of it. As a result, I typically only looked into things to the degree necessary for my personal interest. I therefore expect this document to not be comprehensive and for there to be a high level of subjectivity in my account. Many opportunities are limited on the basis of current citizenship or ancestral background.
You may want to navigate to sections of interest using the outline on the left rather than read the post as a whole. The

...
Disentangling Corrigibility: 2015-2021
by Koen.Holtman — 2021-02-16T18:01:27.952Z

Since the term corrigibility
was introduced in
2015,
there has been a lot of discussion about corrigibility, on this
forum and elsewhere.

In this post, I have tied to disentangle the many forms of
corrigibility which have been identified and discussed so far. My aim
is to offer a general map for anybody who wants to understand and
navigate the current body of work and opinion on corrigibility.

[This is a stand-alone post in the counterfactual planning
sequence. My original plan was to write only about how
counterfactual planning was related to corrigibility, but
it snowballed from there.]

The 2015 paper

The technical term corrigibility, a name suggested by
Robert Miles to denote concepts previously discussed at MIRI, was
introduced to the AGI safety/alignment community in the 2015 paper
MIRI/FHI paper titled
Corrigibility.

An open-ended list of corrigibility desiderata

The 2015 paper does not define corrigibility in full: instead the
authors present initial lists of corrigibility desiderata. If the
agent fails on one of these desiderata, it is definitely not
corrigible.

But even if it provably satisfies all of the desiderata included in
the paper, the authors allow for the possibility that the agent might
not be fully corrigible.

The paper extends an open invitation to identify more corrigibility
desiderata, and many more have been identified since. Some of them
look nothing like the original desiderata proposed in the paper.
Opinions have occasionally been mixed on whether some specific
desiderata are related to the intuitive notion of corrigibility at
all.

Corrigibility desiderata as provable safety properties

The most detailed list of desiderata in the 2015 paper applies to
agents that have a physical shutdown button. The paper made the
important contribution of mapping most of these desiderata to
equivalent mathematical statements, so that one might prove that a
particular agent design would meet these desiderata.

The paper proved a negative result: it considered a proposed agent
design that provably failed to meet some of the desiderata. Agent
designs that provably meet more of them have since been developed, for
example here. There has also been
a lot of work on developing and understanding the type of mathematics
that might be used for stating desiderata.

Corrigibility as a lack of resistance to shutdown

Say that an agent has been equipped with a ph...
Cartesian frames as generalised models
by Stuart_Armstrong — 2021-02-16T16:09:20.496Z

Scott presented Cartesian frames/Chu spaces as follows:

Let W be a set of possible worlds. A Cartesian frame C over W is a triple C=(A,D,⋅), where A represents a set of possible ways the agent can be, D represents a set of possible ways the environment can be, and ⋅:A×D→W is an evaluation function that returns a possible world given an element of A and an element of D.

In a previous post, I defined GM, the category of generalised models.

In this post, I'll try and see how these two formalisms relate to each other.

Equivalence with Cartesian frames

We'll now demonstrate the equivalence of Cartesian frames morphisms with the morphisms of generalised models. To do so, and avoid a collision of symbols, I've slightly tweaked the notation for Cartesian frames.

Equivalence of morphisms

Let C0=(A0,D0,⋆0) and C1=(A1,D1,⋆1) be Cartesian frames over W: thus there are relations ⋆0:A0×D0→W (written as a0⋆0d0=w) and ⋆1:A1×D1→W (written as a1⋆1d1=w′).

A morphism between them is a pair of maps (g0:A0→A1,h1:D1→D0), such that, for all a0∈A0 and d1∈D1, g0(a0)⋆1d1=a0⋆0h1(d1).

How can we express this in the generalised model formalism?

First, let Ei=Ai×Di×W. In terms of features, this can be defined by setting ¯¯¯fAi=Ai, ¯¯¯fDi=D and ¯¯¯fW=W. Then Fi={fAi,fDi,fW}, and Mi=(Fi,Ei,Qi) is the feature-split generalised model with Ai⊂2¯¯¯fAi=2Ai, Di⊂2¯¯¯fDi=2Di, and W⊂2¯¯¯fW=2W.

As we'll see in the bears example, there can be more interesting ways of defining the feature split Mi.

Then the map pair (g0,h1) is equivalent to the (feature-split) relation r, defined such that (a0,d0,w)∼r(a1,d1,w′) iff:

g0(a0)=a1,

h1(d1)=d0,

and w=w′.

Without loss of clarity, we can thus write r as the feature-split relation (g0,h0,IdW).

Composing (g0,h1) and (g1,h2) generates (g1∘g0,h1∘h2). Take r as the relation defined by (g0,h1) and q as the relation defined by (g1,h2). Then if (a0,d0,w)∼pr(a2,d2,w′′), there must exist an (a1,d1,w′) with (a0,d0,w)∼r(a1,d1,w′)∼p(a2,d2,w′′). Then:

g1g0(a0)=g1(a1)=a2,

h1h2(d2)=h1(d1)=d0,

w=w′=w′′.

So composition of morphisms (g,h) for Cartesian frames is the same as the composition of corresponding relations (g,h,IdW).

The extra structure

We have two structures to add: Cartesian frames have the ⋆ map, while generalised models have the probability measures Q; we need to relate them.

One natural way to relate them is to consider that if a⋆d=w, then we should get Q(w∣a,d)=1 an...
Generalised models as a category
by Stuart_Armstrong — 2021-02-16T16:08:27.774Z

Naming the "generalised" models

In this post, I'll apply some mathematical rigour to my ideas of model splintering, and see what they are as a category[1].

And the first question is... what to call them? I can't refer to them as 'the models I use in model splintering'. After a bit of reflection, I decided to call them 'generalised models'. Though that's a bit vague, it does describe well what they are, and what I hope to use them for: a formalism to cover all sorts of models.

The generalised models

A generalised model M is given by three objects:

M=(F,E,Q).

Here F is a set of features. Each feature f consists of a name or label, and a set in which the feature takes values. For example, we might have the feature "room empty?" with values "true" and "false", or the feature "room temperature?" with values in R+, the positive reals.

We allow these features to sometimes take no values at all (such as the above two features if the room doesn't exist) or multiple values (such as "potential running speed of person X" which includes the maximal speed and any speed below it).

Define ¯¯¯f as the set component of the feature, and ¯¯¯¯¯F as disjoint union of all the sets of the different features - ie ¯¯¯¯¯F=⊔f∈F¯¯¯f.

A world, in the most general sense, is defined by all the values that the different features could take (including situations where features take multiple values and none at all). So the set of worlds, W, is the set of functions from ¯¯¯¯¯F to {0,1}, with 1 representing the fact that that feature takes that value, and 0 the opposite. Hence W=2¯¯¯¯F, the power set of ¯¯¯¯¯F.

The set of environments is a specific subset of these worlds: E⊂W. The choice of E is actually more important than that of W, as that establishes which values of the features we are modelling.

The Q is a partial probability distribution. In general, we won't worry as to whether Q is normalised (ie whether Q(E)=1) or not; we'll even allow Qs with Q(E)>1. So Q could be more properly be defined as a partial weight distribution. As long as we consider terms like Q(A∣B), then the normalisation doesn't matter.

Morphisms: relations

For simplicity, assume there are finitely many features taking values in finite sets, making all sets in the generalised model finite.

If M0=(F0,E0,Q0) and M1(F1,E1,Q1) are generalised models, then we want to use binary relations between E0 and E1 as morphisms between th...