trentmkelly/LessWrong-43k huggingface dataset:
Beyond micromarriages
by Richard_Ngo — 2022-03-11T03:40:01.463Z

tl;dr micromarriages aren't fully analogous to micromorts, which makes it tricky to define them satisfactorily. I introduce an alternative unit: QAWYs (Quality-Adjusted Wife Years), where 1 QAWY is an additional year of happy marriage.

I once compiled a list of concepts which I’d discovered were less well-defined than I originally thought. I’m sad to say that I now have to add Chris Olah’s micromarriages to the list. In his words: “Micromarriages are essentially micromorts, but for marriage instead of death. A micromarriage is a one in a million chance that an action will lead to you getting married, relative to your default policy.”

It’s a fun idea, and helpful in making small probabilities feel more compelling. But upon thinking about it more, I’ve realised that the analogy doesn’t quite work. The key difference is that micromorts are a measure of acute risk - i.e. immediate death. For activities like skydiving, this is the main thing to worry about, so it’s a pretty good metric. But most actions we’d like to measure using micromarriages (going to a party, say, or working out more) won’t lead you to get married immediately - instead they flow through to affect marriages that might happen at some later point.

So how can we measure the extent to which an action affects your future marriages, even in theory? One option is to track how it changes the likelihood you’ll get married eventually. But this is pretty unhelpful. By analogy, if micromorts measured an action’s effect on the probability that you’d die eventually, then all actions would have almost zero micromorts (with the possible exception of some life-extension and existential risk work during the last few decades). Similarly, under this definition the micromarriages you gain from starting a new relationship (or even from literally getting married) could be mostly cancelled out by the fact that this relationship cuts off other potential relationships.

An alternative is to measure actions not by how much they change the probability that you’ll get married eventually, but by how much you expect them to causally contribute to an eventual marriage. The problem there is that many actions can causally contribute to a marriage (meeting someone, asking them out, proposing, etc) and there’s no principled way of splitting the credit between them. I won’t go into the details here, but the basic problem is the same a...
On continuous decision boundaries
by eeegnu — 2022-03-11T03:36:01.860Z

I've always had a fascination with decisions that rely on accepting/declining some continuous value. For instance, you might receive an unexpected email offering to purchase your domain name e.g. eeegnu.com, and depending on some huge number of factors, (i.e. how attached am I to the domain name, is the offer enticing enough, do I want to go through the hassle, etc.) you'll arrive at a decision. Intuitively, we'd think that there's some hypothetical $ value X where every offer < X yields No, and every offer ≥ X yields Yes, or framing this probabilistically, we'd expect a plot of $ from 0 to infinity against the probability of choosing yes to be non-decreasing. This might look something like a sigmoid, centered at the point where you're 50/50 on whether you should accept or not. Instead I argue that the actual plot would look allot more chaotic, with various regions and points that are wildly different that neighboring points.

To really discuss this, it'll help to fixate on a particular example that controls for some unknown factors on the part of the person making the decision. Assume that they've received this offer for their domain name from a futuristic blockchain trading post, a pre-signed contract whereby clicking Yes, they'll immediately receive $X worth of crypto in their bank/wallet account (and this is a currency they're happy to keep things in), and the domain name will transfer ownership to some unknown figure. Assume also that they were not planning on selling this domain name, and that the offer rather mysteriously appeared.

Now we can think about different regions on the graph. The most natural place to start is around the neighborhood of our decision makers internal expected value of their domain name. This might be ~$100 given the amount they've already spent in yearly renewal fees, but for someone who randomly received an offer, the probability of accepting such an offer will be quite low, and similarly, values less than this can be expected to be upper bounded by its probability. Another natural region is where the offer becomes enticing (assuming it exists): the smallest X where the probability is ≥ 0.5. This might be ~$10k, a few orders of magnitude higher than our expected valuation, and not something too unusual in the space of domain resale. Ignoring human psychology factors for preferring some...
Book review: The Dawn of Everything
by PeterMcCluskey — 2022-03-11T01:56:13.494Z
Link: http://www.bayesianinvestor.com/blog/index.php/2022/03/10/the-dawn-of-everything/

Book review: The Dawn of Everything: A New History of Humanity by David
Graeber and David Wengrow.

This book is about narratives of human progress. I.e. the natural
progression from egalitarian bands of maybe 20 people, to tribes, to
chiefdoms, to states, with increasing inequality and domination by
centralized bureaucracy. That progress is usually presumed to be driven
by changes in occupations from foragers, to gardeners, to farmers, to
industry.

Western intellectuals focus on debates between two narratives:
Hobbesians, who see this mostly as advances from a nasty state of
nature, and those following in Rousseau's footsteps, who imagine early
human societies as somewhat closer to a Garden of Eden. Both narratives
suggest that farming societies were miserable places that were either
small advances or unavoidable tragedies, depending on what you think
they replaced.

Graeber and Wengrow dispute multiple aspects of these narratives. The
book isn't quite organized enough for me to boil their message down to
a single sentence. But I'll focus on what I consider to be the most
valuable thread: we should be uncertain about whether humanity made (is
making?) a big mistake by accepting oppression as an inevitable price of
material wealth.

The Dawn of Everything asks us to imagine that humans could build (and
may have been building) sophisticated civilizations without domination
by powerful states, and maybe without depending on farming.

The book doesn't mention Burning Man, possibly to appeal to a broader
audience than those who have accurate information about it. But I'll
assume my readers understand Burning Man culture, and use it as an
example of the quasi-anarchist culture that Graeber and Wengrow hope
for.

Occupy Wall Street
(in which Graeber was active) shared some of this culture, but mixed
with too much statist framing to be a clean example.

Freedom and Equality

Graeber and Wengrow lament the loss of freedom that's associated with
modern society. They use a somewhat nonstandard notion of freedom,
focused on:

(1) the freedom to move away or relocate from one's surroundings;

(2) the freedom to ignore or disobey commands issued by others; and

(3) the freedom to shape entirely new social realities, or shift back
and forth between different ones.\

Nothing here about being secure in one's property or livelihood. E.g.
at Burning Man, almost all order is maintained via p...
Are there any impossibility theorems for strong and safe AI?
by David Johnston — 2022-03-11T01:41:01.184Z

Are there any impossibility theorems for the existence of AI that is both strong and safe? I think such theorems would be interesting because they could help to evaluate proposals for safe AI: we could ask "which assumption does this proposal break?"
I have a vague sense that a theorem of this sort might be able to be developed along the following lines:
1. The kind of strong AI that we want is a technological tool such that it's easy to tell it what to do, and it can successfully do a wide variety of complex things when told
2. Simple instructions + complex results -> AI has a lot of flexibility in its action
3. There are only a few ways to reliably achieve goals requiring complex behaviour e.g. something approximating expected utility maximisation
4. 2+3 + instrumental convergence -> flexibility is likely to be exploited in dangerous ways

Do fleshed out versions of this argument exist? Do you have any other ideas about impossibility theorems?
Algorithmic Measure of Emergence v2.0
by interstice — 2022-03-10T20:26:26.996Z

It has occasionally been proposed that we need a new algorithmic definition of 'complexity', different from Kolmogorov complexity, that better captures our intuitive sense of which structures in the world are truly complex. Such definitions are typically designed with the goal of assigning a low complexity to extremely simple structures such as a perfectly ordered lattice or an endless string of '0's, but also to totally random objects, unlike Kolmogorov complexity. Instead, one hopes, the objects assigned high complexity will be (binary representations of) things like cathedrals, butterflies, or other objects which we think of as truly possessing a intricate internal structure. Examples of these sorts of proposed definitions include sophistication and logical depth(see here for a nice survey and discussion).

What's the point of such definitions? There's a great tradition in math and science of quantifying vague but conceptually useful notions. Good definitions give you a new way of thinking about the world, and allow you to bring the powerful tools of math and science to bear upon a problem. It's difficult to find powerful ideas like this; attempting to capture our intuitive impressions formally can be a useful first step.

I'd like to introduce a new definition of this sort, with the aim of quantifying the number of different levels on which an object can be usefully described. For example, fluids can admit a microscopic description in terms of individual molecules or a coarse-grained description which specifies the average density in a given region. Planets can be described as collections of vast collections of atoms or as single points(for the purposes of orbital mechanics calculations) Butterflies can be described as bio-chemical systems or as optimizers trying to increase their fitness. An interesting feature of our world is the presence of many such different levels of description; and the existence of individual objects which can be described well in many different ways: we could say that such objects display a high degree of 'emergence'. I think this is one aspect of what causes us to consider objects to be 'truly complex'.

But what does it mean, in algorithmic terms, for there to be more than one way to describe an object?[1] In algorithmic information theory, a description of an object is a program which outputs that object. There are infinitely many ...
I left Russia on March 8
by avturchin — 2022-03-10T20:05:59.650Z

I moved to Bishkek in Kyrgyzstan, a small country in Central Asia. I am travelling with family and a toddler. The town is lovely and has many nice cafes.

International travel from Russia is still possible here, as many work migrants in Russia are from Kyrgyzstan. So planes are flying and money sending services seems to work.

There was some kind of interview on the Russian side of the border: when you will return? why your vacation is so long? Some people said that their phones were checked and border guards read their message history. My message history was clean.

Note that any non-official opinion about the current Russian operation could be punished by 15 years in jail according to the new law. It is also illegal to call it "war". So far no one was jailed for 15 years, but some people were fired or arrested for a shorter time after protests.
What’s the chance a smart London resident dies of a Russian nuke in the next month?
by DanielFilan — 2022-03-10T19:20:01.434Z

Credit allocation: the structure of this estimate, and some of the numbers, are basically lifted from a talk Nuño Sempere gave, but most of the numbers aren’t.

Epistemic status: I spent maybe an hour putting this together. I spent more time thinking about the bit I wrote about more, and less time thinking about the bit I wrote about less. For everything but the prior, you might want to check out the Samotsvety Group’s forecast here.

tl;dr About 0.01%.

As I write this, there’s a lot of tension between the West and Russia over Russia’s invasion of Ukraine. If you’re living in a major city like London, you might wonder whether you should move to the countryside in case your city is the target of a nuclear strike. Here’s my estimate of the chances of that, as well as some context for how bad those odds are.

First of all: what’s the prior odds for this sort of thing? Let’s say we’re talking about a NATO nuclear exchange with Russia. Since the US and Russia have co-existed as nuclear powers for about 70 years, you would think that Laplace’s law of succession would give you about 70:1 odds against a nuclear strike in a normal year. That being said, I think this is conservative: Laplace’s law assumes that in the absence of historical data, you’d have even odds on a nuclear strike. But if I didn’t know any history, I’d think the odds of intense conflict in a given year would be like 20:1 against, not 1:1, since countries don’t want to have this sort of conflict and have ways of avoiding it. The way we adjust for that is that we add that 20 to the 70 years to get a prior odds of 90:1 - see this Wikipedia page on pseudocounts.

I then think we should update down a bit because there’s a relatively wide class of intense NATO-Russia conflicts, only some of which would lead to nuclear war. I’m going to say that that’s a 1.5:1 update against (a mild update because it’s easy to see how intense NATO-Russia conflict leads to a nuclear exchange), and then get 135:1 odds against, on priors. Note that this is a different kind of update to adding pseudocounts that produces different effects, and I’m not totally sure it’s legit, but I’m going with it.

Now: we should probably update on the fact that Russia’s invading Ukraine, and the West is sanctioning Russia over that. The question is, how big an update is that? Bayes’ rule says we multiply the odds by the likelihood ratio: that is, the ...
What's the best ratio for Africans to starve compared to Ukrainians not dying in the war?
by ChristianKl — 2022-03-10T18:52:47.004Z

Actions have consequences. One of the consequences of our trade sanctions is to increase wheat and corn prices significantly. There are currently Africans who are on the brink of starvation and the increased food prices will increase starvation?

Is our current position "It's okay if 1,000,000 Africans starve if we can prevent 100,000 Ukrainians from not dying in the war"?

Are there models that tell us how much people are likely to starve as a result of our actions? Are there wheat futures that we can use to tell us future wheat prices and economic models that estimate how many people will starve?
Infohazards, hacking, and Bricking—how to formalize these concepts?
by Yitz — 2022-03-10T16:42:35.929Z

This is an attempt to formulate a thought that’s been rolling around in my brain for a few years now. I want to formalize a hypothesis, but I’m not sure how to convey what the hypothesis even is. Hopefully by writing this out I’ll help make that happen (spoiler alert: I partially succeeded?), or someone reading this will be able to help me here.

Working towards the theme:
The idea starts with the classic LessWrong meme of an infohazard (defined here as some form of external sensory “input”) that when received by its “target” causes that target harm. This could be something wholly internal, like increasing depression, but might have external effects as well. A victim might either do something they wouldn’t otherwise do of their own volition, or could even collapse completely, in the case of infohazards invoking seizure or death. The idea of an infohazard is not merely theoretical. We know from experience that visual and auditory infohazards exist for individual human beings. Examples range from sad stories which make people temporarily depressed, to people “brainwashed” by cults to do stuff they would otherwise never have done, and even to people who get seizures or death at certain inputs, like flashing lights. From that basis in fact, one can generalize to arrive at the hypothesis that for every individual human, such an infohazard exists (even if unconstructable in practice). An even stronger (but somewhat less plausible) hypothesis would be that there exists singular infohazards which would work for most or even all of humanity. On a more(?) theoretical level, it seems reasonable that a being with complete knowledge of a human brain and unlimited compute power could “trivially” construct an infohazard tailored to that brain. Could such a being control the brain’s output in some arbitrary manner by giving it the right inputs, or are limitations on how far even an optimal infohazard could go?

We are far from understanding the human brain deeply enough to even begin to answer these questions, so let’s try to abstract the situation a bit by considering the field of computing, where we have (arguably) greater scientific and mathematical understanding of the field. The best analogy to an infohazard in computer science would seem to be found in the practice of hacking.

“Hacking” as a term covers many different modes of action, but one popular method of hacking can...
Ukraine Post #2: Options
by Zvi — 2022-03-10T16:10:03.951Z

Or: Why the current situation once again proves the need for all of my pre-existing policy preferences.

Writing about the war is continuing to prove difficult, as things move quickly, the epistemic environment continues to be almost maximally hostile and a lot of elements are things where I was not previously up to speed.

I want to build a model of the situation, which leads to the problem of where even to begin, yet the speed premium beckons and although having a newborn is great as you can imagine I’m on less than ideal amounts of sleep and free time.

Thus it seems worthwhile to skip ahead a bit and ask the practical question first. Assume for the sake of discussion that we prefer people doing better and not dying to people doing worse and dying, and agree that we do not want Russia to win the war.

What would then be our options, either individually or collectively? What can we do?

(Where the collective ‘we’ here is some combination of ‘America’, ‘America + EU/NATO’ and ‘everyone everywhere who wants this same general set of outcomes.’)

I indeed strongly prefer such outcomes.

If you don’t prefer such outcomes enough to care, this post is not trying to convince you to change your mind and is probably not for you. That would be a very different post, and I don’t know what the True Objection would be of most people open to being convinced.

If you do prefer such outcomes, there are a lot of different potential approaches.

On a personal level, one could:

Provide military aid (e.g. by giving money directly to Ukraine).
Provide humanitarian aid (e.g. by giving money to those that provide it).
Lobby or advocate for better government policies or corporate actions.
Attempt to build (and ideally share) a better understanding of the situation.
Convince others to also prefer such outcomes, ideally via #4.

I am going to go ahead and advocate here for the policy responses I think are appropriate. The first ones are direct actions, which are pretty obviously:

Provide military assistance to the extent we can without too much escalation.
Humanitarian aid.
Economic sanctions.

The rest fall under the category of ‘things worth doing anyway.’ This stuff is win-win. This means doing things like:

Nuclear power.
Additional other energy production of various kinds.
Reconciling with other oil producers to extent possible.
Carbon tax.
Taking in as many Russians and Ukrainians as possible w

...
Covid 3/10/22: We Have a Plan
by Zvi — 2022-03-10T16:10:01.037Z

Covid situation continues to improve, although Covid remains a thing. There’s a new executive plan that I did my best to read with incomplete success, that checks off lots of boxes and doesn’t have anything horribly wrong with it.

What’s less great is that Congress passed an omnibus bill that cut out all pandemic funding and most already allocated funds have been spent, so we are failing to pay for our Paxlovid purchases and threaten to run into a wide range of other problems as well if this isn’t addressed, such as us no longer paying for the uninsured to get treatment and running out of monoclonal antibodies. It would be quite the disaster, although scaled to the extent of cases at the time.

In the meantime, slowly but surely less and less masking is required and vaccine cards are checked less, and people start to see each other more often the way people used to. We presume that like most such things the snafus with pandemic funding will get sufficiently worked out that it won’t actually cause us to lose access to treatments, although I wouldn’t fully count on it. Life returns, and we can shift our doom scrolling to the invasion of Ukraine.

On that front, writing continues to be difficult. I’ve written a post focused on what options are available, both on an individual and collective level, on the continued theory of ‘this is the piece of the puzzle I know how to write right now’ and writing each piece even out of order makes other pieces easier.

Meanwhile, back to the Covid show.

Executive Summary

Conditions are safe and one can return to living life.
Congress withholds pandemic funding, endangering treatment if not fixed.
New executive plan for Covid seems fine, so long as they get their funding.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 247k cases (-25%) and 9.200 deaths (-19%).

Results: 226k cases (-32%) and 9,138 deaths (-20%).

Prediction for next week: 170k cases (-25%) and 7,050 deaths (-22%).

The decline in deaths finally picked up a bit, which I expect to accelerate a bit more. Cases pleasantly surprised, as it seems behavioral changes haven’t slowed the decline much yet, although I will continue to predict that those changes will soon have some impact. I definitely see substantial changes taking place around me.

Deaths

I found the log scale button, so here’s the new deaths graph.

S...