trentmkelly/LessWrong-43k huggingface dataset:
Her Internal
by matejsuchy — 2021-05-12T02:05:30.126Z
Link: https://matejsuchy.substack.com/p/finding-her-pt-1

In my

previous

blog post, I explicated my current model for the origin and function of

prior

clusters, which I associate with Jung’s archetypes and the general idea of cognitive subagents. In this post and the following, I want to examine a specific archetypal dyad in the context of an enchanting Spike Jonze film.

The Classical Anima and Animus

The anima is a number of things, but the most expressive compression seems to be to identify it with a prototype of the opposite gender’s psyche. Jung writes:

Every man carries within him the eternal image of woman, not the image of this or that particular woman, but a definite feminine image. This image is fundamentally unconscious, an hereditary factor of primordial origin engraved in the living organic system of the man, an imprint or "archetype" of all the ancestral experiences of the female, a deposit, as it were, of all the impressions ever made by woman-in short, an inherited system of psychic adaptation. Even if no women existed, it would still be possible, at any given time, to deduce from this unconscious image exactly how a woman would have to be constituted psychically. The same is true of the woman: she too has her inborn image of man.

Now, you might be asking yourself: ok, so is the anima just a device for predicting the behavior of the opposite sex? If so, it would certainly be useful, especially in societies with significant

preference

divergence across genders. However, this probably doesn’t compress losslessly enough. Many Jungians seem to

think

that the anima has much greater impact on a man’s preference; an impact one can purportedly infer from

inadequate equilibria

in his rationalized policy.

What would this look like? Well, in my previous post I hinted at my hypothesis that a top-level agentive preference can generally be decomposed as a preference over rotations between subagents. If we keep around a model of the opposite gender in order to improve fitness in games with real instantiations of that gender, why not incorporate it in the rotation? It would allow for other models to become much sparser, which should be optimal as long as subagent-switching costs are sufficiently low.

An anima, then, would be a feminine subagent whom we could rotate into a scenario for one of two reasons: to provide insight or to model action. The latter case seems to have some empirical evidence; I can’t seem to remember ...
Louisiana Gets Vaccine Passports Right
by tkpwaeub — 2021-05-12T01:23:26.615Z

Quietly, with very little fanfare, Louisiana got vaccine passports right.
https://lawallet.com/covid19/

What's not to like?
First off, let's take a peek at their guiding principles:

These principles look a lot like what Jeffrey Zients was hinting at when asked at a White House press briefing about vaccine passports:

Our role is to help ensure that any solutions in this area should be simple, free, open source, accessible to people both digitally and on paper, and designed from the start to protect people’s privacy.

I love that they're building on existing infrastructure - digital drivers licenses. When one person scans another person's DDL, they instantly pull up the person's photo from the state's DMV. That all but eliminates the need for forgery.
It's completely optional - no mandates as to exactly how the credential is to be used. Louisiana trusts its citizens to figure out how best to use it.

A smartphone isn't necessary. The person doing the verification simply needs to scan the barcode on the back of the license. That pulls up the person's official DMV photo, along with their vaccination status, if the cardholder has opted to disclose this..

Sure - it requires a drivers license. But so far every passport requires auxiliary forms of ID. Otherwise, people can just pass QR codes to one another. Verifiers end up having to check multiple documents. Interrupted workflows are known to be far more error prone and tiring ("alert fatigue").

And sure, the app costs money (unlike the Excelsior pass). But it's a modest $5.99 and the app both stores your card and verifies other cards. It's "peer to peer." The money is used in part to pay for necessary enforcement (inevitably that means undercover operations).

I'm not sure what it was that put Louisiana at the forefront of digital driver's licenses, but I have some guesses. New Orleans is home to Mardi Gras, so perhaps underage drinking is a common problem. There have been more than a few serious hurricanes - so the state has an interest in digitizing as much as possible, since maintaining paper records can be difficult during floods. Louisiana is far from the most populous state, so it's not as risky to experiment. Whatever the reason - kudos to Louisiana. The rest of us should be taking notes.
Thoughts on Iterated Distillation and Amplification
by Waddington — 2021-05-11T21:32:27.104Z

Alignment researcher Paul F. Christiano has written several posts on what he refers to as Iterated Distillation and Amplification (IDA). In this post, I will argue that IDA is a general method of adaptation and that it can be found in various different guises in a wide range of contexts. Its inherent flexibility mirrors its inherent lack of controllability.

Summary: The point of IDA is to train an AI the same way society trains individuals. What we are hoping to do is to make an AI extract the scaffold of human morality and internalize human moral boundaries. This project might, unfortunately, be doomed to failure: for an AI to be able to adapt to moral norms over time, it must also be capable of breaking them. If it is able to break moral norms, alignment cannot be guaranteed. Therefore, IDA might not be a viable path to AI alignment.

Apollonians versus Dionysians

In science the Apollonian tends to develop established lines to perfection, while the Dionysian rather relies on intuition and is more likely to open new, unexpected alleys for research.

So writes Hungarian biochemist Albert Szent-Györgyi in a 1972 letter to Science magazine[1]. 35 years earlier, he had won the Nobel Prize in Physiology or Medicine. To his eyes, he had his Dionysian nature to thank: they are the only ones capable of finding something new.

It's fairly obvious when you think about it. If you are spending your time developing what is already known, you're not out there exploring the vast unknown. You are doing something safe and sensible instead; cleaning up the mess left behind by past Dionysians.

What the Apollonian is doing can be considered distillation or compression. He exploits conventional knowledge. What the Dionysian is doing can be thought of as amplification or learning. He attempts to go beyond conventional knowledge.

Christiano's dual-phase model is, in fact, very similar in nature to a whole bunch of other ones. And if we take a look at these we might get a better sense of what IDA is really all about (filtered, of course, through my limited understanding).

But first:

Edgework and Nodework

What's the deal with edgy behavior, really? The wording implies the answer: it's about edges. More specifically, it's about finding them. To the people who already know where the edges (of acceptable behavior) are, edgy people are incredibly annoying. They break rules and conventions an...
Massachusetts Tick-Borne Disease Distribution
by jefftk — 2021-05-11T18:40:08.363Z

As I wrote yesterday,

MA is
no longer reporting good data to the CDC on Lyme disease.
Starting in 2019, however, MA has been publishing

tick-borne
disease reports. They include this misleading chart:

The caption includes "Although there are differences in the rate of patient
visits, this shows that people are exposed to ticks throughout all of
Massachusetts and should take recommended steps to reduce the chance
of being bitten."

One issue is that they've chosen to chart the rate of visits and not
the rate of diagnoses, but the main issue is that the scale hides the
degree of variation between counties. Pulling the numbers and
manually dividing by population estimates, I see:

The rate in the
Islands (Dukes and Nantucket Counties) is 10 times any other MA
county!

(Even though these are also vacation areas, people are categorized by
their home county not their infection county.)

Now, I agree with them that taking precautions is worth it when
walking through tick-friendly terrain anywhere in the state. But
there are still degrees of precautions, from clothes, to tick checks,
to choosing different activities, to choosing other counties, and the
report should emphasize the difference in risk so people can make good
decisions.

Comment via: facebook
Yampolskiy on AI Risk Skepticism
by Gordon Seidoh Worley — 2021-05-11T14:50:38.500Z
Link: https://www.researchgate.net/publication/351368775_AI_Risk_Skepticism

Roman Yampolskiy posted a preprint for "AI Risk Skepticism". Here's the abstract:

In this work, we survey skepticism regarding AI risk and show parallels with other types of scientific skepticism. We start by classifying different types of AI Risk skepticism and analyze their root causes. We conclude by suggesting some intervention approaches, which may be successful in reducing AI risk skepticism, at least amongst artificial intelligence researchers.

Nothing really new in there to anyone familiar with the field, but seems like a potentially useful list of citations for people coming up to speed on AI safety, and perhaps especially AI policy, and a good summary paper you can reference as evidence that not everyone takes AI risks seriously.
Challenge: know everything that the best go bot knows about go
by DanielFilan — 2021-05-11T05:10:01.163Z

On a few different views, understanding the computation done by neural networks is crucial to building neural networks that constitute human-level artificial intelligence that doesn’t destroy all value in the universe. Given that many people are trying to build neural networks that constitute artificial general intelligence, it seems important to understand the computation in cutting-edge neural networks, and we basically do not.

So, how should we go from here to there? One way is to try hard to think about understanding, until you understand understanding well enough to reliably build understandable AGI. But that seems hard and abstract. A better path would be something more concrete.

Therefore, I set this challenge: know everything that the best go bot knows about go. At the moment, the best publicly available bot is KataGo, if you’re at DeepMind or OpenAI and have access to a better go bot, I guess you should use that instead. If you think those bots are too hard to understand, you’re allowed to make your own easier-to-understand bot, as long as it’s the best.

What constitutes success?

You have to be able to know literally everything that the best go bot that you have access to knows about go.

It has to be applicable to the current best go bot (or a bot that is essentially as good - e.g. you’re allowed to pick one of the versions of KataGo whose elo is statistically hard-to-distinguish from the best version), not the best go bot as of one year ago.

That being said, I think you get a ‘silver medal’ if you understand any go bot that was the best at some point from today on.

Why do I think this is a good challenge?

To understand these bots, you need to understand planning behaviour, not just pick up on various visual detectors.

In order to solve this challenge, you need to actually understand what it means for models to know something.

There’s a time limit: your understanding has to keep up with the pace of AI development.

We already know some things about these bots based on how they play and evaluate positions, but obviously not everything.

We have some theory about go: e.g. we know that certain symmetries exist, we understand optimal play in the late endgame, we have some neat analysis techniques.

I would like to play go as well as the best go bot. Or at least to learn some things from it.

Corollaries of success (non-exhaustive):

You should be able t

...
Is driving worth the risk?
by Adam Zerner — 2021-05-11T05:04:47.935Z

− The Thinking Ladder

Harry was wondering if he could even get a Bayesian calculation out of this. Of course, the point of a subjective Bayesian calculation wasn't that, after you made up a bunch of numbers, multiplying them out would give you an exactly right answer. The real point was that the process of making up numbers would force you to tally all the relevant facts and weigh all the relative probabilities. Like realizing, as soon as you actually thought about the probability of the Dark Mark not-fading if You-Know-Who was dead, that the probability wasn't low enough for the observation to count as strong evidence.

− HPMOR Chapter 86: Multiple Hypothesis Testing

I'm 28 years old and have never had a drivers license. At some point earlier on in my life I decided that driving is something that has a bad expected value (EV) due to the risk of death and the massive value of life, but at the same time, the EV isn't so bad where I will go out of my way to avoid it. Since deciding this, the belief has become cached. However, various things have recently prompted me to reconsider the belief.

I'm looking to buy a home and am leaning towards making it a requirement that the place is fully walkable (as opposed to mostly walkable but sometimes requiring a car).

The topic of MIRI relocating came up and the badness of cars is relevant to that.

I have a wedding coming up where I have to travel from Vegas to SF. I have the option of getting a ride from my girlfriend's mom, or taking a flight. I'm tempted to go by car so I don't have to pay the money for a flight, but I don't actually think that is the right decision. ("No thanks Sandra. I think that computers are going to take over the world and make us all immortal. You have a slightly higher chance of dying in a car than a plane, so I'd rather pay for a flight.")

Covid has prompted me to explore the EV of doing things. Eg. looking at the chances of catching covid and dying, trying to put a dollar amount on that, and then asking whether such an activity is worth it. In doing so for covid, it naturally leads to asking the question for activities such as driving as well.

Perhaps this is a good starting point. As I started to get into in my comment on the MIRI relocation post, in 2018, there were 11.18 deaths per 100k people in the US, or a 0.01118% chance of dying. If you value life at $10M, 0.0001118 * $10,000,000 =...
A Wiki for Questions
by weathersystems — 2021-05-11T02:31:03.124Z

Q&A sites

Sites like stackexchange, r/askscience, quora, and lesswrong.com/questions all have a similar format. One person asks a question. Everyone gives their own answer. You can vote on answers to (hopefully) raise the best answer to the top. When the answer becomes out of date, if you're lucky, the original author will update their comment.

There are some benefits with this approach.

Competition

Each answer competes with the other answers for top position. This competition does a decent job at sorting out the good from the bad answers.

Reward

These sites usually have some sort of karma system where you get cred for writing better answers. Maybe that reward system motivates people to answer questions more often? Seems at least possible.

Ease

The format is very easy to understand. If you have a question, ask it. If you know an answer to a question and want to help out, you give your answer. There is no complicated coordination required. If you're reading through the answers and think one is best, upvote.

There are also some key problems with this approach.

Duplicate questions

All those sites have many duplicate questions. Even stackexchange only flags duplicates without removing them. Duplicate questions make searching for answers inefficient. They also make writing answers inefficient, as you may be covering the same ground someone else has on a duplicate question.

Duplicate answers

Even on a single question post, you'll have many different answers. One per commenter. The asker of the question or (more importantly) the person who finds this post from google, is left to sort through the answers and make a judgement call on which one is best. That shouldn't be the job of the person who writes or reads the question. Presumably they're not experts on the topic... as they're the ones who don't know the answer.

There is the voting mechanism, true, but it's not perfect. I often find the best answer on a stackoverflow post is the 5th one down, mostly because it's more up to date. Which brings me to...

Stale questions and answers

Answerers often can update their answers over time, but most don't (god bless the few who do). But it's not the answerers' fault for not keeping their comment up to date. They just tried to answer someone's question, they didn't sign on for maintaining their post for the rest of time. Because answers are very often stale, you can't ...
Massachusetts Lyme Reporting
by jefftk — 2021-05-10T22:40:05.857Z

In putting together yesterday's post on

Lyme disease prevalence I
noticed something very strange with Massachusetts:

It's even more striking if you look at a map (2018):

There's no way Lyme follows state borders like this; it has to be some
sort of data issue.

Looking over several years of CDC maps, the year that Massachusetts
starts looking different matches up with the huge drop in the chart
above. Compare 2015:

To 2016:

The only public explanation there seems to be is this short
news article. In 2016 the MA Department of
Public Health "stopped spending time and resources trying to track
down the clinical information, instead relying solely on positive lab
results to give a more accurate estimate of Lyme disease case
numbers."

The CDC gets their information from the states, and in this case
requires both
a clinical diagnosis and a positive test. MA, apparently uniquely
among the states, has decided that when compiling statistics
confirming a clinical diagnosis is extra work that doesn't improve
accuracy. So MA doesn't generally have the stats it would need to
file with the CDC, and instead of marking MA as "no data", the CDC
publishes the trickle of reports that it does get from MA.

I don't know whether the CDC or MA is being more unreasonable here,
but the effect is pretty bad: user-facing websites like TickCheck show
MA as now having low Lyme levels. The CDC does not even include a
note in their FAQ to
say MA data shouldn't be trusted.

Comment via: facebook
Human priors, features and models, languages, and Solmonoff induction
by Stuart_Armstrong — 2021-05-10T10:55:12.078Z

This post was originally inspired by Robin Hanson's publication "Uncommon Priors Require Origin Disputes, but it took a left turn and brought in Solmonoff induction, Shannon entropy, and model splintering.

Priors, features, models, Shannon entropy, and computer languages

Let's weave a simplified tale about how humans might generate their priors. Humans grow up in the world, getting used to operating within it, long before they learn about Bayesian reasoning. Indeed, some humans never learn explicit Bayesian reasoning at all! Though it seems they often use Bayesian reasoning implicitly.

Let's imagine our Hero or Heroine, living their lives and functioning in society. They need a mental apparatus that allows them to cope well with what they encounter everyday, especially what's important to them.

They might have a generalised model whose features represent what they care about - and what's important for them to know. If all their friends are music fans, then, to function socially, they may need to distinguish between Ava Max and Masked Wolf. If they work in a kitchen, then it is important for them not to confuse boning knives and filleting knives. In prison, it's lethal to confuse Nuestra Familia with the Aryan Brotherhood.

But if they're neither involved nor interested in those various worlds, they can get away with grouping those pairs together as "modern singers", "kitchen knives", and "prison gangs". So people's mental models will contain features that matter to them - actionable features. It's vitally important for all life on Earth the the sun not go red giant tomorrow; however, since there's very little anyone could do about it if it did, there's no need for most people to know anything about red giants or the life cycle of stars. This leads to the famous New Yorker cover, which not only encodes how many New Yorkers do see the world, but also how it is useful for New Yorkers to see the world:

We can think of this in terms of Shannon entropy and compressed communication. People need to be able to swiftly call to mind any key features about their situation, and rapidly combine this with other knowledge to reach a decision - such as when their hand is reaching into the knife drawer. So, roughly at least, their minds will encode important features about their lives and the relationships between these features, with ease of retrieval being important.

This is an e...
Biological Holism: A New Paradigm?
by Waddington — 2021-05-09T22:42:31.231Z

Biologist Michael Levin was recently featured in an article by The New Yorker with a captivating headline: Is Bioelectricity the Key to Limb Regeneration?

The thought struck me that Levin's work is a great example of what you might call a novel paradigm in biology that people here might not be entirely familiar with. Now is as good a time as ever to make your acquaintance. Let's dive straight in.

You've probably heard complaints about 'reductionism' and 'mechanistic' views of life. These complaints are usually followed by a slogan: the whole is more than the sum of its parts. Other words that tend to crop up: emergence, self-organization, complexity, and so on. A recurring theme is that we are missing out on something important when we break a living thing down into parts. This perspective on life—which we can call holism—might strike you as wishy-washy New Age-nonsense. I want to capture your attention with a surprising notion: it emerged from the Manhattan Project.

Los Alamos National Laboratory became a site of cross-fertilization during WWII where great minds exchanged ideas and discussed the grandest challenge faced by science at large: predicting (and controlling) the behavior of nonlinear systems. It was an important challenge, as that included most things. As Manhattan Project participant Stanislaw Ulam quipped:

Using a term like nonlinear science is like referring to the bulk of zoology as the study of non-elephant animals.

Of these great minds, John von Neumann was probably the greatest. One of his contributions to the challenge, game theory, deals with the dynamics of interacting agents who make rational decisions. The assumption of rationality can be very useful, even if it doesn't literally apply in real-life circumstances. This is because it converts the impossible task of predicting the nonlinear dynamics of interpersonal interactions into an optimization problem.

But even more important in our biological context is the cellular automaton, discovered along with Ulam. Chiara Marletto, working with David Deutsch within the framework of constructor theory, has shown that you need the logic of cellular automata in order to get the appearance of design without actual design[1]. In other words: von Neumann and Ulam explained why life is something that can just happen. You don't need a divine entity. You don't need fine-tuned initial conditions. All yo...