trentmkelly/LessWrong-43k huggingface dataset:
The generalized Sierpinski-Mazurkiewicz theorem.
by Donald Hobson — 2022-07-29T00:12:18.763Z

Suppose any uncountable set S and any countable set of bijective transformations T={T1,T2,...} ∀i Ti:S→S bijective.

Then assuming that for all pairs of distinct finite sequences of transformations [p1,p2,...pm] and [q1,q2...qn] that the set {s∈S:p1∘p2..pm(s)=q1∘q2..qn(s)} is countable.

Then there exists a set K⊂S,K≠∅ partitioned into K1,K2,... such that Ki=Ti(K)..

Proof.

Consider the set of all sequences of transformations (type signature N→T)

Form the equivalence relation ∼ by a∼b⟺∃m,n∈N:∀i∈N:am+i=bn+i

This is the sequences that are the same up to a finite number of insertions, deletions and replacements. Discard all the sequences equivalent to any cyclic sequence. (A sequence is cyclic if ∃m∀i:ai=ai+m. Think if these like the digits of rationals, they might do all sorts of things at first, but eventually they repeat the same thing over and over) Call the set of remaining sequences Q.

Pick a representative of each equivalence class. Call this set of representatives R.

Note that each equivalence class is countable, as for any a∼r∈R then a can be represented by (m,n,a1,a2,...am) as the rest of the sequence can be filled in using the equivalence relation ∀i:am+i=rn+i. Use countable union of countable sets a lot.

Note that, given any function f:R→S can be uniquely extended into a function f:Q→S such that f([a1,a2,...])=a1(f([a2,...]). Use the rule f(a)=a1∘a2..am∘r−1n∘r−1n−1..r−11(f(r)). This is well defined because, if we increment both m and n, then we are just applying an extra function and inverse. But those cancel each other out, by the condition in the equivalence relation. Also m−n is a fixed constant for particular a and r, because we ruled out all eventually cyclic sequences.

Pick any r∈R. For any a,b∼r, there must be only countably many s∈S such that the extension of f(r)=s has f(a)=f(b). Looking at the definition of the generalization above, we see its a finite composition of bijections. So Generalize: f(r)→f(a) is a bijection (type S→S). Let c=[am+1,am+2,...] where m,n are such that ∀i:am+i=bn+i. Then f(a)=am∘..a1(f(c)) and f(b)=bn∘...
The Conversations We Make Space For
by Severin T. Seehrich — 2022-07-28T21:37:00.404Z

(Cross-posted to EA Forum)

While developing the necessary skills and models for making

The Berlin Hub

as great as it can be, I have had countless opportunities to talk and think about the norms, values, and rituals of the EA and rationalist communities. So here is a list of three types of conversations that we tend to make space for, sometimes in unique ways. I hope that making these explicit will help community builders reflect on which kinds of events to run, why to run them, and how.

1) Truth-seeking conversations

For rationalists, truth-seeking conversations are a way to cultivate epistemic rationality. For EAs, they are a crucial part of “using evidence and reason” to pursue their altruistic goals — according to

CEA’s definition

of effective altruism as “using evidence and reason to figure out how to benefit others as much as possible, and taking action on that basis.”

We can divide truth-seeking conversations into three types:

1. Didactic: one person knows something the others don’t. The person with the knowledge teaches; the others listen, ask questions, and disagree if they think something is off.

Example:

talks

.

2. Symmetrical - assisting: one person listens to the other to help them think through an issue. Can happen with or without repeated targeted questions.

Examples:

socratic ducking

,

socratic questioning

,

Street Epistemology

.

3. Symmetrical - collaborative: two people disagree with one another about an issue. They sit down and talk things through in the spirit of collaborative truth-seeking.

Example: the archetypal method for collaborative truth-seeking is

Double Crux

.

The key virtues of truth-seeking conversations are:

to be aware of what you know and what not

,
to

be curious

,
to admit when you are wrong,
to share what you know, if and only if it is wanted.

2) Impact-focused conversations

For rationalists, impact-focused conversations are a way of cultivating instrumental rationality. For EAs, these conversations put the “figure out how to benefit others as much as possible” into “using evidence and reason to figure out how to benefit others as much as possible, and taking action on that basis.”

Once again, we can distinguish one asymmetrical and two symmetrical types:

1. Didactic: one person has crucial information about relevant parts of the world. They teach their mental models and give advice to others in the community.

Examples:

talks

, certain kinds of...
Announcing the AI Safety Field Building Hub, a new effort to provide AISFB projects, mentorship, and funding
by Vael Gates — 2022-07-28T21:29:52.424Z

Tldr: If you’re interested in working on an AI safety field building project similar to those listed below (e.g. researcher outreach): please fill out

this application form

or

nominate someone

($200 bounty)! Hiring is ongoing indefinitely. If you’re an EA org that has an AI safety field building project, please

submit your project idea

, and if our priorities and people align sufficiently we’ll try to get it done!

[Crossposted to the EA Forum]

Update from 10/31/22: I'm still working on similar projects and looking for people, and encourage you to apply!
Update from 11/12/22: I'm currently not hiring new people, due to the new funding situation. You're still welcome to fill out the application form, but unfortunately I'm focusing on existing projects at the moment.

Update from 2/1/23: AISFB has ended! See the Retrospective here.

When individual EAs are interested in working on AI safety research, they can apply to many programs designed to give mentorship, funding, and project ideas (SERI MATS, MLAB, AI Safety Camp, SERI / CERI / CHERI summer programs, etc). This is amazing and speaks to our growth as a community.

However, I think there’s a noticeable lack of such structure in field building. For EAs interested in AI safety field building, how do they know which projects are most promising? Do they have to make them up from scratch? Where do they get collaborators, how do they get access to mentorship, how do they show a good track record to apply for funding?

To fill this gap, I’m starting the “AI Safety Field Building Hub”, a new effort to provide projects, mentorship, and funding to individuals interested in working in AI safety field building. If you’re an EA looking for a full-time or part-time project, I’m hiring project-specific contractors. I have a list of projects I’m interested in working on, have been sent more shovel-ready projects by others, and hope that other organizations will continue to send me great project ideas they don’t have capacity to manage. In the future, this “hub”-like functionality may spin into its own organization; at the moment, I’m looking to hire people to work for me.

Of note, my projects tend to be aimed more at outreach and at older populations– AI researchers, academia and industry, rather than university populations or high schoolers— which I think is relatively neglected right now. I also think there ar...
Defining Optimization in a Deeper Way Part 4
by J Bostock — 2022-07-28T17:02:33.411Z

In the last post I introduced a potential measure for optimization, and applied it to a very simple system. In this post I will show how it applies to some more complex systems. My five takeaways so far are:

We can recover an intuitive measure of optimization
Even around a stable equilibrium, Op(A;n,m) can be negative
Our measures throw up issues in some cases
Our measures are very messy in chaotic environments
Op seems to be defined even in chaotic systems

It's good to be precise with our language, so let's be precise. Remember our model system which looks like this:

In this network, each node is represented by a real number. We'll use superscript notation to notate the value of a node: wn is the value of node n in the world W.

The heart of this is a quantity I'll call Comp, which is:

Comp(A; n, m)=limxm→wn[xm−wmym−wm]

Which is equivalent to.

Comp(A:n,m)=∂sm∂sn∣∣A varies/∂sm∂sn∣∣A constant

(sn is the generic version of wn, xn, yn)

Our current measure for optimization is the following value:

Op(A;n,m)=limxm→wn[−log|Comp(A:n,m)|]

Op is positive when the nodes in A are doing something optimizer-ish towards the node m. This corresponds when Comp is < 1. We can understand this as when A is allowed to vary with respect to changes in sn, the change that propagates forwards to sm is smaller.

Op is negative when the nodes in A are doing something like "amplification" of the variance in m. Specifically, we refer to A optimizing m with respect to n around the specific trajectory W, by an amount of nats equal to Op(A;n,m). We'll investigate this measure in a few different systems.

A Better Thermostat Model

Our old thermostat was not a particularly good model of a thermostat. Realistically a thermostat cannot apply infinite heating or cooling to a system. For a better model let's consider the function

Therm(θ, p; sT)⎧⎪⎨⎪⎩p θ≤sTpθsT −θ<sT<θ−p sT≤−θ

Now imagine we redefine our continuous thermostat like this:

sTt+δt=sRt−d

sRt+δt=sRt−δt×Therm(θ, p; sTt)

Within the narrow "basin" of −θ≤sT≤θ, it behaves like before. But outside the change in temperature over time is constant. This looks like the following:

When we look at our optimizing measure, we can s...
Covid 7/28/22: Ruining It For Everyone
by Zvi — 2022-07-28T15:10:00.680Z

There are lots of ways to try and ruin it for everyone. Covid restrictions that make no sense are a proven successful strategy for this, demanding the Doing of More. Monkeypox offers exciting additional ruining potential.

The surprise winner, however, is an old Alzheimer’s paper that it seems the entire field of drug development and diagnosis was based around, and it turned out it’s not only fraud, it’s very obvious fraud once you actually look at the manipulated images?

The full implications of this are being insufficiently explored.

Executive Summary

Case levels are steady and should remain so.
Whole field of Alzheimer’s has been based on a fraud for decades.
Various monkeypox news got split off into another post.

Also yes, Biden got Covid, but it doesn’t even make a bullet point, he’s fine.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 780k cases (+1%) and 2,450 deaths (+0%)

Results: 807k cases (+5%) and 2,964 deaths (+21%).

Prediction for next week: 850k cases (+5%) and 2,850 deaths (-3%).

Last week Colorado reported -158 deaths, which I didn’t notice, after 234 the previous week and now 234 again this week. Three of the previous four weeks had zero. That explains most of the prediction error and why the numbers keep flopping back and forth, as the real number should probably be more like 50. Thus I predict a moderate decline. There’s no real uncertainty here that’s worth much, the game is predicting reporting errors.

Deaths

Cases

Biden got Covid, but he’s feeling much better now, he’s tested negative twice.

Physical World Modeling

The headline: White House to Launch Effort to Develop Next Generation of Covid Vaccines. More accurate would be ‘White House to hold meeting to ask how development is going on next generation of Covid vaccines.’ The work is being done by Pfizer and Moderna. Calling them in to talk about it is not launching an effort. Giving them additional funding for it would speed things along, but the White House already knows this and Congress is continuing to not provide the funds. I do think it is a worthwhile use of time to hold the meetings.

Study finds three shots plus an Omicron infection are 94% effective against BA.5.

Should hospitals continue to screen everyone for Covid, given that most Covid cases in hospitals are there for unrelated reasons? ...
Monkeypox Post #2
by Zvi — 2022-07-28T13:20:01.260Z

Enough distinct things happened this week in Monkeypox that it made sense to split off the information on it into a distinct post. This is probably slightly over the threshold required for doing that.

Executive Summary

WHO declares Monkeypox a public health emergency of international concern.

One could define of ‘keeping up with the news’ is when the WHO declares an emergency and you become more worried only to the extent you worry about what they are going to do next.

Thus, this does not indicate a crisis situation, or even one worse than one would have expected a few weeks ago. It is, like most monkeypox news this week (and also the majority in general), an acknowledgement of the inevitable.

Will we turn the tide soon? It briefly looked like this might have already happened, but it turned out not to be the case. I give us a reasonable chance to indeed turn the tide relatively soon, and without a broad spillover into the non-MSM community. There will of course be some cases of spread elsewhere, but my (not too confident) hunch is that the disease won’t robustly spread in most other places, and will remain mostly within MSM.

The overall case numbers are still quite low. If you are not in contact with the places this is concentrated, the whole thing is in practice fully ignorable, and would be for several additional doublings. Even after that, it would be mostly ignorable. You’d want to be aware of what to look for, but talk of major behavioral adjustments (or even speculations on things like shutting daycare centers) seem at worst highly unlikely.

Various News

When all the available vaccine slots are taken within seven minutes (as 300k doses remain parked overseas) the problem likely is not a lack of keeping up with the news. That sounds bad. Some needed perspective, however, is what our health officials do best.

Yes, I imagine not counting days makes the count smaller.

CDC numbers on case growth.

My understanding is these are cumulative numbers, which means the lack of acceleration this past week looked like excellent news. That is especially true given the expansion of testing capacity.

Later, on July 26, we had this.

That’s 700 in 4 days, versus 1k in 7 days. We continue to see a leveling off in case growth. All responses are things like ‘sigh’ but this was excellent news.

Until the next day, and, well, whoops.

Onc...
For Better Commenting, Stop Out Loud
by DirectedEvolution — 2022-07-28T01:39:35.009Z

"Man stands face to face with the irrational. He feels within him his longing for happiness and for reason. The absurd is born of this confrontation between the human need and the unreasonable silence of the world." - Albert Camus, The Myth of Sisyphus and Other Essays

These days, a little unreasonable silence of the world actually sounds kind of nice! With apologies to Camus, I think a 21st-century Sisyphus would have been condemned to endlessly respond to annoying internet comments.

I'm not talking about outright trolling or doxxing. That's 21st-century Prometheus.

Sisyphus isn't tormented by trollish insults, but by the compulsion to engage in ambiguous forms of effortful and frustrating dialog that he knows won't lead anywhere but feels like he has to participate in anyway. He's responding to comments full of ill-informed overconfidence, self-righteous activist monoperspective, and the crappiest of drive-by criticisms, which I call PONDS ("prickly, opaque, nitpicky, disengaged, and shallow"). He himself, at some point in his life, has written a comment that's just as bad as the stupidity he's responding to, and he angrily empathizes with the other person's state of mind while wishing they'd just listen and learn! Every time he perfects his reply and hits "submit," the conversation rolls back downhill again.

Stopping Out Loud is a way of wedging the boulder of a bad conversation in place. Sisyphus declares that he will no longer be reading or replying to responses to the comment he's about to post. He's leaving the boulder halfway up the hill. Anyone trying to drag him down into a pointless fight or fruitless back-and-forth is, well, S.O.L.

Stopping Out Loud helps Sisyphus avoid the perception of social defeat that comes with letting the other person "have the last word." It also avoids disincentivizes further attention-getting and nastiness, and helps him feel like he's in control of his actions.

Sisyphus sometimes includes extra information when he Stops Out Loud.

Why he's stopping.
A clear, final statement of the point he wanted to make.
Conditions under which he'd revisit the thread.
An invitation or disinvitation to get in touch when he and the other person have had a chance to cool off.
A suggestion to continue the debate via PM, where it's sometimes easier to speak freely without feeling like the whole internet is looking on.

This is also helpful to the other ...
Seeking beta readers who are ignorant of biology but knowledgeable about AI safety
by Holly_Elmore — 2022-07-27T23:02:57.192Z

Seeking the biology-ignorant! Come and do your part!
I am working on a post with Beth Barnes that will explain meiotic drive to AI safety people so they can determine if there are any useful analogies to gradient hacking. If you don't know what "meiotic drive" is, you might be perfect! I need beta readers who can tell me if they don't understand the biology stuff, so that I can get an idea of how they interpret the things they think they do understand, and so I know if what I'm saying is really making sense to them and seems related to AI safety concepts. I want this paper to be maximally accessible to AI safety researchers, so I want to really be sure I'm minimizing my biology jargon and explaining the necessary biology concepts well in language they understand.
This would involve looking over a ~5-15 page google doc, flagging jargon you don't know or understand, and commenting on your confusions. Ideally I'd also find people who were willing to look at successive drafts. Ideally ideally I'd find people who were willing to have a brief call to talk about their interpretation of what they had read. The timeframe is within the next month (until end of August 2022).
Comment below or PM me if you are interested :)
Principles of Privacy for Alignment Research
by johnswentworth — 2022-07-27T19:53:28.209Z

The hard/useful parts of alignment research are largely about understanding agency/intelligence/etc. That sort of understanding naturally yields capabilities-relevant insights. So, alignment researchers naturally run into decisions about how private to keep their work.

This post is a bunch of models which I use to think about that decision.

I am not very confident that my thinking on the matter is very good; in general I do not much trust my own judgment on security matters. I’d be more-than-usually interested to hear others’ thoughts/critiques.

The “Nobody Cares” Model

By default, nobody cares. Memetic reproduction rate is less than 1. Median number of citations (not counting self-citations) is approximately zero, and most citations are from people who didn’t actually read the whole paper but just noticed that it’s vaguely related to their own work. Median number of people who will actually go to any effort whatsoever to use your thing is zero. Getting other people to notice your work at all takes significant effort and is hard even when the work is pretty good. “Nobody cares” is a very strong default, the very large majority of the time.

Privacy, under this model, is very easy. You need to make a very large active effort in order for your research to not end up de-facto limited to you and maybe a few friends/coworkers.

This is the de-facto mechanism by which most theoretical work on alignment avoids advancing capabilities most of the time, and I think it is the main mechanism by which most theoretical work on alignment should aim to avoid advancing capabilities most of the time. It should be the default. But obviously there will be exceptions; when does the “nobody cares” model fail?

Theory-Practice Gap and Flashy Demos

Why, as a general rule, does nobody care? In particular, why does nobody working on AI capabilities care about most work on alignment theory most of the time, given that a lot of it is capabilities-relevant?

Well, even ignoring the (large) chunk of theoretical research which turns out to be useless, the theory-practice gap is a thing. Most theoretical ideas don’t really do much when you translate them to practice. This includes most ideas which sound good to intelligent people. Even those theoretical ideas which do turn out to be useful are typically quite hard to translate into practice. It takes months of work (at least), often additiona...
Moral strategies at different capability levels
by Richard_Ngo — 2022-07-27T18:50:05.366Z

Let’s consider three ways you can be altruistic towards another agent:

You care about their welfare: some metric of how good their life is (as defined by you). I’ll call this care-morality - it endorses things like promoting their happiness, reducing their suffering, and hedonic utilitarian behavior (if you care about many agents).
You care about their agency: their ability to achieve their goals (as defined by them). I’ll call this cooperation-morality - it endorses things like honesty, fairness, deontological behavior towards others, and some virtues (like honor).
You care about obedience to them. I’ll call this deference-morality - it endorses things like loyalty, humility, and respect for authority.

I think a lot of unresolved tensions in ethics comes from seeing these types of morality as in opposition to each other, when they’re actually complementary:

Care-morality mainly makes sense as an attitude towards agents who are much less capable than you, and/or can't make decisions for themselves - for example animals, future people, and infants.

In these cases, you don’t have to think much about what the other agents are doing, or what they think of you; you can just aim to produce good outcomes in the world. Indeed, trying to be cooperative or deferential towards these agents is hard, because their thinking may be much less sophisticated than yours, and you might even get to choose what their goals are.
Applying only care-morality in multi-agent contexts can easily lead to conflict with other agents around you, even when you care about their welfare, because:

You each value (different) other things in addition to their welfare.
They may have a different conception of welfare than you do.
They can’t fully trust your motivations.

Care morality doesn’t focus much on the act-omission distinction. Arbitrarily scalable care-morality looks like maximizing resources until the returns to further investment are low, then converting them into happy lives.

Cooperation-morality mainly makes sense as an attitude towards agents whose capabilities are comparable to yours - for example others around us who are trying to influence the world.

Cooperation-morality can be seen as the “rational” thing to do even from a selfish perspective (e.g. as discussed here), but in practice it’s difficult to robustly reason through the consequences of being cooperative without relying on ingrained co

...