trentmkelly/LessWrong-43k huggingface dataset:
Bear Surprise Freedom Network
by CraigMichael — 2022-03-12T10:15:36.107Z

Here's some crazy nascent ideas I have for getting information in and out of Russia when other methods fail. From what I'm seeing, current Internet censorship efforts focus on... the Internet.

It's worth remembering that there are plenty of other ways to use technology to communicate that don't use the Internet as we generally think of it.

Bear Surprise Introduction

Most of what I know about Russian and Russian-adjacent culture is from using LiveJournal in the early-to-mid 2000s. I know Bear Surprise and Как пропатчить KDE2 под FreeBSD. So, I'm hijacking the former college-aged meme for my oddball collection of censorship-subverting telecommunication ideas, and dubbing them the Bear Surprise Freedom Network. (If you have similar ideas, please feel free to reuse this term—it would be cool if someone could search “bear surprise freedom network _____” and find potential technology workarounds to whatever the Russian government might prevent them from doing).

It's likely I'm missing a lot of political and cultural subtleties related to Padonkaffsky jargon. I'm not trying to make some kind of weird point with it. The bear stuff just always made me laugh, even though I'm sure I was/am missing a lot of context.

I'm talking about a lot of radio stuff, and I'm something of a radio dilettante. If you're an expert here, I would appreciate any feedback you have.

This article is short on implementation details, because I'm short on them. These are all things I've seen work in one form or another at different points in my life. You might well ask, "how do you expect people who don't know where their next meal will come from to care about this?" That's a good question, and I don't have a good answer to it. I can say if there are interested parties looking for novel ideas to implement ways to exchange information that will be difficult to censor, there are some here.

Here's eight ideas.

1. Medium: Landlines, Technology: FidoNet

It looks like landlines still exist in Russia, and there's still BBSes in FidoNet Region 50 (Russia) as of March 11, 2022.

What is FidoNet?

FidoNet was started in the 1980s by young boys and girls with dogs in the United States as an attempt to imitate IP over Avian Carriers, except with more reliable canines rather than pigeons. Sort of like Stranger Things meets Weird Science meets The Dog Whisperer. Hence, the FidoNet name and logo us...
Book review of "Mengzi"
by Anonymous — 2022-03-12T04:54:01.801Z

Book review of “Mengzi: With Selections From Traditional Commentaries”, trans. Bryan W. Van Norden

This is a review of a classical Chinese philosophical text with a heavy focus on virtue ethics and politics. I don’t have any knowledge of Classical Chinese, I’m not a philosopher, and I’m not very virtuous. You may instead want to read the SEP entry on Mengzi, or just buy the book and read it – 2300 years of Confucian scholars can’t be wrong!

Introduction

The Confucian or Ruist[1] tradition within Chinese philosophy seems underrated in the West. Everybody likes Buddhism and Daoism. People particularly love pretty-sounding mistranslations of the Daodejing by authors who don’t know any Chinese. People love Zen (or Chan, in the Chinese reading of the character). People have even heard of Zhuangzi and/or a butterfly.

But almost nothing from Confucianism has percolated into the popular culture. Everyone knows there was a guy named Confucius who went around saying wise stuff, but nobody even knows any of the aphorisms (even though they're often great), let alone actually reading any of the Confucian canon.

I think the explanation for this is pretty obvious. Eastern philosophy really started taking off in the West during the 60s. But at that time, there was just no market for a philosophy saying you should obey your parents and elder siblings, be loyal to the state, patiently study old books, and cultivate temperance and good manners.

That's still going to be unappealing to a lot of people today (especially to LessWrong readers, who are probably more likely than average to distrust authority, to dislike formal schooling, and to think there is little value in tradition).

But I think Confucian philosophy does offer an interesting perspective that is of increasing value. After all, it was developed by and for highly educated scholars who also wanted to have a real-world impact.

These were people who were carefully trained from a young age to excel at high-stakes standardized tests that determined their entire future. They studied hard to become intellectuals, yet at the same time wanted to do good in the real world. It even turns out many of them (increasingly so in the Song-Ming period) were interested in mindfulness meditation, with a focus on personal effectiveness rather than the Buddhist focus on enlightenment. After their years of study, they ended up in practi...
If someone with sufficient capabilities decides to deliberately create an Unaligned AGI, is there anything anyone can actually DO, to stop them?
by  — 2022-03-12T00:01:03.158Z

The scenario is simple, some unassuming programmer creates a DAO on a Blockchain that is the seed AI, with the single purpose gaining political, economic, and military power to create a new world order with the DAO at the top of the proverbial food chain. The question becomes, what do the bloggers/posters of LessWrong.org actually DO to stop the AI DAO?
Your Future Self's Credences Should Be Unpredictable to You
by David Udell — 2022-03-11T23:33:38.448Z

My exposition of Yudkowsky's below idea about skipping unnecessary studying:

I think that to contain the concept of Utility as it exists in me, you would have to do homework exercises I don't know how to prescribe. Maybe one set of homework exercises like that would be showing you an agent, including a human, making some set of choices that allegedly couldn't obey expected utility, and having you figure out how to pump money from that agent (or present it with money that it would pass up).

Like, just actually doing that a few dozen times.

Maybe it's not helpful for me to say this? If you say it to Eliezer, he immediately goes, "Ah, yes, I could see how I would update that way after doing the homework, so I will save myself some time and effort and just make that update now without the homework", but this kind of jumping-ahead-to-the-destination is something that seems to me to be... dramatically missing from many non-Eliezers.

If you already know how your future self's credences (to be arrived at using your own reasoning process) will differ from your current credences, you should just trust your expected-future-self's reasoning and immediately adopt his expected conclusions. If you can foresee how a reasoner like yourself will be led to think in some particular way, then you also have the ability to put yourself in that agent's shoes and be so led right now. Whatever considerations will foreseeably later motivate you are equally moving considerations now.

In deep stock-markets, many traders are trying to make a buck by knowing something that no one else yet does. Because the current prices of various assets already take into consideration all public knowledge about them, you can't expect to reliably make money as a trader unless you know something that the other traders don't. Here, "knowing something" is basically synonymous with "being able to predict how public knowledge about the assets in question will evolve." If a trader can foresee what everyone will believe in the near future, he can beat everyone else to it by betting on those expected beliefs now. Because of this, whenever a trader in a deep stock-market comes to know something about a relevant future belief, prices today are pushed around to reflect that belief that would've been held tomorrow. The deep stock-market is thus epistemically efficient: it factors in every relevant credence that any trade...
Arguments are good for helping people reason about things
by AprilSR — 2022-03-11T23:02:33.158Z

Yudkowsky recently posted a Twitter thread about how ideal reasoners respond to arguments. My understanding of his reasoning is:

The more smart/rational/whatever you are, the better you are at figuring out what is true
Thus, whether you believe the conclusion of an argument should be based primarily on whether the conclusion is true, rather than on how effectively the argument was presented

This principle seems valid to me.

Valentine, in a LessWrong comment thread, used Yudkowsky's thread to draw the conclusion that a healthy rationalist community should not make arguments. I think this is silly. We are not, unfortunately, logically omniscient; we cannot just look at data and draw all the correct conclusions from it. The purpose of an argument is to help people realize what conclusions they should draw from data without having to figure it all out on their own.
Is There a Valley of Bad Civilizational Adequacy?
by lbThingrb — 2022-03-11T19:49:49.049Z

This post is my attempt to think through an idea that I’ve been mulling over since this discussion on Twitter last November prompted by a question of Matthew Barnett, which I was reminded of while reading the section on energy in Zvi’s recent post on the war in Ukraine. The meaning of the title, “valley of bad civilizational adequacy,” is the idea that as one relaxes constraints on the bounded rationality of hypothetical future collective human policy decisions, the result may initially be a decrease in expected utility captured by humans, due to increased existential risk from unaligned AGI, before one starts the ascent to the peak of optimal rationality.

Preliminary definition: By pro-growth policy I mean a certain set of public policy proposals that aren’t directly about AI, but could shorten the timeline to AGI: less immigration restriction, particularly for high-skill workers; cheaper, denser housing, especially in the SF Bay Area; and cheap energy, by building out lots of nuclear and renewable generation capacity. (Is there anything else that fits this category?)

Main argument

Pro-growth policy can be expected to accelerate AI capabilities research and therefore shorten the timeline to AGI, via the agglomeration effects of more smart people in dense urban tech hubs, decreased energy cost of running ML experiments, and overall economic growth leading to more lucrative investment opportunities and therefore more research funding.
Having less time to solve AI safety would cause a greater increase in AI X-risk than any decrease in AI X-risk resulting from pro-growth policy also accelerating AI-alignment research.
AI X-risk dominates all other considerations.
Therefore pro-growth policy is bad, and AI-risk-alarmist rationalists should not support it, and perhaps should actively oppose it.

Possible counterarguments

(Other than against step 3; the intended audience of this post is people who already accept that.)

The main argument depends on

the amount that AI-risk alarmists can affect pro-growth policy,
the effect of such changes in pro-growth policy on the timeline to AGI,
and the effect of such changes in the AGI timeline on our chances of solving alignment.

One or more of these could be small enough that the AI-risk community’s stance on pro-growth policy is of negligible consequence.

Perhaps pro-growth policy won’t matter because the AGI timeline will be very short, n...
Rhythm Stage Setup v4
by jefftk — 2022-03-11T19:40:02.850Z

I've recently finished a major reworking of my rhythm stage setup,
and wanted to sit down where things are now. As before, this is a
system that takes MIDI from a few sources and generates audio:

Piano: this is my main instrument in Kingfisher, and you
primarily hear it directly through its own output. But it can control
other sounds, either by selecting a bass note (whatever I have most
recently played on the left side of the piano) or as a full keyboard
(everything I'm currently playing).

Drums: a foot-controlled electronic
drumkit. Like piano, you primarily hear it through its own
output. I feed it into the system so that I can determine the current tempo.

Breath: a breath controller, which detects how hard I am
currently blowing. This can be used to directly to control volume
like a wind instrument, or slowly ("inflating a virtual bag") to let
sound gradually swell and decay.

High-level changes since my last overview
two years ago:

I'm no longer playing the Axis
49 jammer. I still really like it, and the feeling of the Wicki-Hayden
layout under my key fingers is great, but I'm too tempted to play fast
melody and hurt my (not working great and I'm
still not sure why; started on Meloxicam last week) hands.

I'm no longer playing wooden foot drums that I
made, and bought four Yamaha
KU-100s. This also means I can use both my heel and toe, and play
more complex patterns.

I've moved my code from Mac to
Linux, and moved from VSTs in Reaper to fluidsynth for everything.
This doesn't sound quite as nice, but at least it means I'll never
need to go through manual Mac
setup again.

I gave up on the toe
buttons; my feet are busier now.

Some videos showing how things sound now:

Sandy Boys (Mandolin, Footbass, Whistle Synth):

All the Rage (Piano, Footbass, Footdrums, Whistle Synth):

Tamlin (Mandolin, Footbass):

Playing around in D (Piano, Footbass, Footdrums, Organs):

Here's what the controls look like:

This is a small cheap USB keyboard, with stickers on it so I can
remember what I have set each of the keys to do. It's connected to a
program
that listens for keyboard events, presents as a virtual midi device,
and generates midi signals, which means my core
code can work just with MIDI.

The interface isn't really designed for anyone but me, but talking
through it is a good illustrat...
Deep Dives: My Advice for Pursuing Work in Research 
by scasper — 2022-03-11T17:56:28.857Z

Context [optional]:

I’m a Ph.D student at MIT doing research in technical AI safety. I mostly do work related to interpretability, adversaries, and robust reinforcement learning (see more

here

). Often, I am referred either by some Effective Altruist friend of mine or 80k to talk with someone who is interested in technical AI safety research – usually an undergrad (feel free to email me if you’d like to talk – scasper@mit.edu). One piece of advice that I give to almost everyone is to do a “deep dive.”

What’s a “Deep Dive?”

Sometimes there is a catch-22 when it comes to pursuing research work – you need opportunities to gain experience, and you need experience to get opportunities. A deep dive is one way I recommend to gain experience and demonstrate initiative on one’s own. It’s also a good way to explore an area if you’re not sure that you’re interested in it or not. Having done several of them in the past, I believe that they are the best academic experiences I’ve had aside from research projects themselves.

More concretely, a “deep dive” is a procedure that I recommend for reading papers that I think allows you to learn a lot about an area of research that you are interested in. There are 5 steps.

Get a broad level of familiarity with active research in the field that you’re interested in. This could include looking through high-level survey papers, survey posts, reading lists, etc. For AI safety, there is no shortage of resources like this. Textbooks usually won’t be so useful for this because they typically cover established concepts in a field rather than active areas of work.
Pick a fairly niche subfield of active research that you want to learn a lot about. The exact scope should be up to you. One possibility could be to let the citations of a survey paper define your subfield. Another could be to focus on the research area that a particular professor or organization you’re interested in works in. Getting advice on subfields from someone experienced could be useful at this stage.
Read a ton of papers from the subfield. Ideally, you’d be able to read 50 or more. This will probably take over a month. I think that a pace of approximately one paper per day is good. When looking for papers, the “cited by” button on Google Scholar will be your friend.
Take notes on all of the papers as you read them. The goal is to take a set of no

...
If your solution doesn't work, make it work
by Ege Erdil — 2022-03-11T16:10:51.479Z

Consider the following problem:

There are an unspecified but finite number of chips on a table. Each chip is one of three colors: white, black or red. Define a "move" to be the act of replacing two chips of different colors with one chip of the third color. Prove that if after some sequence of moves we end up with a single chip on the table, the color of this chip can't depend on the exact sequence of moves that we made. In other words, if there are two distinct sequences of moves we could make such that in both cases we end up with a single chip left on the table, these chips must be of the same color.

So this is a game in which we have a state that can be represented by an ordered triplet of natural numbers (x1,x2,x3)∈N3 where each coordinate corresponds to the number of the chips of each of the three colors currently on the table. We're allowed to make any one of the following three moves:

(x1+1,x2+1,x3)→(x1,x2,x3+1)(x1+1,x2,x3+1)→(x1,x2+1,x3)(x1,x2+1,x3+1)→(x1+1,x2,x3)

For the moment, let's assume that the claim we're being asked to prove is indeed correct. The first question we can ask is whether it's even always possible to end up with only one chip, and here the answer is obviously no: if we start with, say, (2,0,0) then we have no legal moves and therefore no way to get down to only one chip.

Therefore, if we rephrase the claim, we can say that the claim asks us to show that there is a function

f:N3→{white,black,red,none}

which sends each state to a particular color if we can end up with a single chip of that color, and sends a state to "none" if there's no way to end up with a single chip from that initial state. This is the part of the claim which says that the color of the final chip should be independent of the sequence of moves we choose to make: it can only be a function of the current state of the game, not of what the players choose to do later on.

Furthermore, we can notice that we lose nothing by allowing moves to be inverted in this game (why?). This means f actually has to be invariant under all legal moves.

What is a good guess for what this function could be? The simplest guess is that it's a linear function: we have some expression like

f(x1,x2,x3)=x1v1+x2v2+x3v3

where the vi are some numbers, and these outputs correspond in some way to the four possible values of the function. If we impose the requirement that f ought to be invariant under o...
Ways to invest your wealth if you believe in a high-variance future?
by Alex_Altair — 2022-03-11T16:07:49.302Z

I largely follow standard financial advice, for example I have a moderate fraction of my wealth in retirement accounts. But I also put substantial weight on the future being... extremely high variance, such that I think there's a pretty good chance that "putting money away" will end up having been useless or at least not very useful. Generally I think it would be a pretty bad idea to have a specific belief about how the future is going to be extremely different and therefore wildly go against currently accepted financial best practices. But it seems highly reasonable, if one believes in the most important century hypothesis, to somehow change your investment plans for that. Here are some relevant futures I can imagine;

A straight-up fast-take-off singularity.
There's a mini-armageddon that causes some societal infrastructure to collapse. Maybe banks fail and I lose that money, or maybe people stop using USD and start bartering with petroleum and gold.
Society moves toward post-scarcity in a way that makes money less useful.
The legislative landscape changes. Maybe congress appropriates retirement accounts for emergency use. Or maybe longevity tech is going really well, and people's lifespans are longer congress extends the age at which you can take the money out. Maybe they retroactively change capital gains such that investing money was useless.

One problem I'm having is that while it's easy to come up with ideas that are useful in some of these scenarios (e.g. build a nuclear bunker) they're totally wasted in the other scenarios. It feels like there should be some way to convert current money into a non-abstract asset that would be always useful, and also fungible in the future. Like can I... somehow buy and store a ton of joules? Or can I somehow... buy 10 years worth of non-perishable food, to make sure I don't starve? When I try to think through these, the logistics seem to be more costly than any expected benefit.

The category that feels most promising to me is something like, invest in making myself a person more able to handle those high-variance scenarios. Gain skills in operations, negotiation, maintain good health, etc. But like, "become a stronger person" is just the kind of thing I'm always trying to do anyway, and it mostly seems to require time and cognitive resources. The thing I'm trying to figure out is how to convert money into being a stronger ...
A Longlist of Theories of Impact for Interpretability
by Neel Nanda — 2022-03-11T14:55:35.356Z

I hear a lot of different arguments floating around for exactly how mechanistically interpretability research will reduce x-risk. As an interpretability researcher, forming clearer thoughts on this is pretty important to me! As a preliminary step, I've compiled a list with a longlist of 19 different arguments I've heard for why interpretability matters. These are pretty scattered and early stage thoughts (and emphatically my personal opinion than the official opinion of Anthropic!), but I'm sharing them in the hopes that this is interesting to people

(Note: I have not thought hard about this categorisation! Some of these overlap substantially, but feel subtly different in my head. I was not optimising for concision and having few categories, and expect I could cut this down substantially with effort)

Credit to Evan Hubinger for writing the excellent Chris Olah's Views on AGI Safety, which was the source of several of these arguments!

Force-multiplier on alignment research: We can analyse a model to see why it gives misaligned answers, and what's going wrong. This gets much richer data on empirical alignment work, and lets it progress faster
Better prediction of future systems: Interpretability may enable a better mechanistic understanding of the principles of how ML systems and work, and how they change with scale, analogous to scientific laws. This allows us to better extrapolate from current systems to future systems, in a similar sense to scaling laws.

Eg, observing phase changes a la induction heads shows us that models may rapidly gain capabilities during training

Auditing: We get a Mulligan. After training a system, we can check for misalignment, and only deploy if we're confident it's safe
Auditing for deception: Similar to auditing, we may be able detect deception in a model

This is a much lower bar than fully auditing a model, and is plausibly something we could do with just the ability to look at random bits of the model and identify circuits/features - I see this more as a theory of change for 'worlds where interpretability is harder than I hope'

Enabling coordination/cooperation: If different actors can interpret each other's systems, it's much easier to trust other actors to behave sensibly and coordinate better
Empirical evidence for/against threat models: We can look for empirical examples of theorised future threat models, eg inner misalignment

Coordina

...