trentmkelly/LessWrong-43k huggingface dataset:
Fixing The Good Regulator Theorem
by johnswentworth — 2021-02-09T20:30:16.888Z

Conant & Ashby’s “

Every Good Regulator Of A System Must Be A Model Of That System

” opens with:

The design of a complex regulator often includes the making of a model of the system to be regulated. The making of such a model has hitherto been regarded as optional, as merely one of many possible ways.

In this paper a theorem is presented which shows, under very broad conditions, that any regulator that is maximally both successful and simple must be isomorphic with the system being regulated. (The exact assumptions are given.) Making a model is thus necessary.

This may be the most misleading title and summary I have ever seen on a math paper. If by “making a model” one means the sort of thing people usually do when model-making - i.e. reconstruct a system’s variables/parameters/structure from some information about them - then Conant & Ashby’s claim is simply false.

What they actually prove is that every regulator which is optimal and contains no unnecessary noise is equivalent to a regulator which first reconstructs the variable-values of the system it’s controlling, then chooses its output as a function of those values (ignoring the original inputs). This does not mean that every such regulator actually reconstructs the variable-values internally. And Ashby & Conant’s proof has several shortcomings even for this more modest claim.

This post presents a modification of the Good Regulator Theorem, and provides a reasonably-general condition under which any optimal minimal regulator must actually construct a model of the controlled system internally. The key idea is conceptually similar to

some of the pieces from Risks From Learned Optimization

. Basically: an information bottleneck can force the use of a model, in much the same way that an information bottleneck can force the use of a mesa-optimizer. Along the way, we’ll also review the original Good Regulator Theorem and a few minor variants which fix some other problems with the original theorem.

The Original Good Regulator Theorem

We’re interested mainly in this causal diagram:

The main goal is to choose the regulator policy P[R|X] to minimize the entropy of outcome Z. Later sections will show that this is (roughly) equivalent to expected utility maximization.

After explaining this problem, Conant & Ashby replace it with a different problem, which is not equivalent, and they do not bothe...
Book review: The Geography of Thought
by PeterMcCluskey — 2021-02-09T18:47:35.781Z
Link: http://www.bayesianinvestor.com/blog/index.php/2021/02/06/geography-of-thought/

Book review: The Geography of Thought: How Asians and Westerners Think
Differently... and Why, by Richard E. Nisbett.

It is often said that travel is a good way to improve one's
understanding of other cultures.

The Geography of Thought discredits that saying, by being full of
examples of cultural differences that 99.9% of travelers will overlook.

Here are a few of the insights I got from the book, but I'm pretty sure
I wouldn't have gotten from visiting Asia frequently:

There's no Chinese word for individualism - selfish seems to be the
closest equivalent.

Infants in the US are often forced to sleep in a separate bed, often in
a separate room. That's rather uncommon in Asia. Does this contribute
to US individualism? Or is it just a symptom?

There are no Asians in Lake
Wobegon.
I.e. Asians are rather reluctant to rate themselves as above average.

Westerners want contracts to be unconditionally binding, whereas Asians
want contracts to change in response to unexpected contexts.

Asians are likely to consider justice in the abstract, by-the-book
Western sense to be rigid and unfeeling.

Chinese justice is an art, not a science.

Origins of Western Culture

Those cultural differences provide hints about why science as we know it
developed in the West, and not in Asia.

I read Geography of Thought in order to expand my understanding of some
ideas in Henrich's WEIRDest
People.

Nisbett disagrees somewhat with Henrich about when WEIRD culture arose,
writing a fair amount about the Western features of ancient Greek
culture.

Nisbett traces some of the east-west differences to the likelihood that
the Greeks met more apparent contradiction than did Asians, via trade
with other cultures. That led them to devote more attention to logical
thought. (Here's an odd claim from Nisbett: ancient Greeks were
unwilling to adopt the concept of zero, because "it represented a
contradiction").

Nisbett agrees with Henrich that there was some sort of gap between
ancient Greek culture and the Reformation, but believes the gap came
later than Henrich does. These two quotes are about all that Nisbett has
to say about the gap:

As the West became primarily agricultural in the Middle Ages, it
became less individualistic.

The Romans brought a gift for rational organization and something
resembling the Chinese genius for technological achievement, and -
after a trough lasting almost a millennium - thei

...
Still Not in Charge
by Zvi — 2021-02-09T16:00:01.474Z

Epistemic Status: Speed premium, will hopefully flesh out more carefully in future

Previously: Why I Am Not in Charge

A brief update about my exchange with Scott from this past week.

After Scott Alexander wrote a post about why WebMD is terrible and it would be good but impossible to put me in charge of the CDC, which is super flattering and unsurprisingly blew up my inbox, I wrote a quick response post to expand on things, and give more color on my model of the dynamics involved in organizations like the CDC and FDA, my view of how people in my position can often get reliably ahead of such organizations, and what would happen if one tried to change them and get them to do the most useful things. That required tying things back to several key past posts, including Zeroing Out, Leaders of Men, Asymmetric Justice, Motive Ambiguity, and the sequences on Simulacra and Moral Mazes.

The disagreements between my model and Scott’s, and the places in which my communication attempts fell short, are broadly in two (closely linked) categories, which his Reddit response captured well and made clearer to me, as did other reactions on the same thread.

The first category is where claims about how perverse things are get rounded down and not seen. Scott is advocating here for what we might call the Utility Function Hypothesis (UFH).

The second is (as Scott explicitly notes) the generalized form of the Efficient Market Hypothesis, which one might call the Efficient Action Hypothesis (EAH).

UFH and EAH are linked.

If UFH is impactfully false, then EAH is also false. If you’re acting meaningfully distinctly from how you would act with a coherent utility function, you are leaving money on the table.

If UFH is effectively true (e.g. it is not impactfully false), then EAH is plausible, but can still either be true or false. EAH could still be false if either one could have a better model of the impact of decisions than those making the decisions (this could be better political sense, better physical-impact sense that then impacts the political calculus, or both). EAH could also still be false if the utility function being maximized doesn’t square up with what we’d mean by politically successful.

If the EAH is true, then UFH is effectively true as well. Anyone acting in a way that can’t be improved upon isn’t meaningfully different from how they’d be with a utilit...
We Need Browsers as Platforms
by jefftk — 2021-02-09T15:40:09.452Z

It's fashionable to say that the web is bloated, and that the features
built to support webapps make it too complex. You

can
divide

the
web into:

Documents: providing information. News, blog posts,
documentation.

Apps: doing things. Email, spreadsheets, games.

(This is really a continuum, where a blog post with a comment
section is pretty documenty but also a bit appy.)

On one hand, I completely agree that supporting apps makes the
platform complex: browsers are incredibly complicated to build and
work on, with such a high ongoing maintenance cost that we only have
three rendering engines: Firefox's Gecko, Chrome's Blink, and Safari's
Webkit (Blink is a fork of Webkit). On the other, supporting apps is
much better than the alternatives.

Outside of a browser, there are essentially two models for getting
applications:

Independent installation. You download the program for the
manufacturers website, or loaded off a CD. The desktop model.

App store. Your OS has a list of programs that can be
installed, and which have gone through some amount of review. The
smartphone model.

Independent installation is decentralized, but also a security
nightmare. Random users installing random software gives you botnets.
App stores are centralized, which puts their operators in a position
of enormous power over what users can run on their devices, and means
governments can require them to

take
down apps.

The web has developed with the principle that it should always be safe
to visit a site. As new capabilities have been added this has
been critical to maintain. This means you don't need an app store,
with power to reject your app.

Ten years ago Mozilla posted Booting to the
Web:

Mozilla believes that the web can displace proprietary, single-vendor
stacks for application development. To make open web technologies a
better basis for future applications on mobile and desktop alike, we
need to keep pushing the envelope of the web to include—and in
places exceed—the capabilities of the competing stacks in
question.

...

We want to take a bigger step now, and find the gaps that keep web
developers from being able to build apps that are—in every
way—the equals of native apps built for the iPhone, Android, and
WP7.

The web platform has come so far in supporting apps over this decade;
we couldn't have made

Bucket
Brigade with...
Epistemology of HCH
by adamShimi — 2021-02-09T11:46:28.598Z

Introduction

HCH is a recursive acronym meaning “Humans consulting HCH”. Coincidentally, It’s also a concept coined by Paul Christiano, central in much of the reasoning around

Prosaic AI Alignment

. Yet for many, me included, the various ways in which it is used are sometimes confusing.

I believe that the tools of Epistemology and Philosophy of Science can help understand it better, and push further the research around it. So this post doesn’t give yet another explanation of HCH; instead, it asks about the different perspectives we can take on it. These perspectives capture the form of knowledge that HCH is, what it tells us about AI Alignment, and how to expand, judge and interpret this knowledge. I then apply these perspectives to examples of research on HCH, to show the usefulness of the different frames.

Thanks to Joe Collman, Jérémy Perret, Richard Ngo, Evan Hubinger and Paul Christiano for feedback on this post.

Is it a scientific explanation? Is it a model of computation? No, it’s HCH!

HCH was originally defined in

Humans Consulting HCH

:

Consider a human Hugh who has access to a question-answering machine. Suppose the machine answers question Q by perfectly imitating how Hugh would answer question Q, if Hugh had access to the question-answering machine.

That is, Hugh is able to consult a copy of Hugh, who is able to consult a copy of Hugh, who is able to consult a copy of Hugh…

Let’s call this process HCH, for “Humans Consulting HCH.”

Nowadays, this is actually called weak HCH, after the

Strong HCH

post which extended the definition. That being said, I’m only interested in perspective about HCH, which includes the questions asked about it and how to answer them. Although the difference between Weak and Strong HCH matters for the answers, the questions and perspective stay the same. I’ll thus use HCH to mean one or the other interchangeably.

The main use of HCH is as an ideal for what a question-answerer aligned with a given human should be like. This in turn serves as the aim of schemes like

IDA

and

Debate

. But thinking about HCH entangles many different angles. For example, what can HCH do? One can interpret this question as “What is the power of the HCH scheme?” or “What questions can HCH answer for a given human?” or “Is HCH aligned with the human parametrizing it?” or “Is HCH competitive?”. Each one of these questions requires different assumptions and focus...
How would free prediction markets have altered the pandemic?
by mike_hawke — 2021-02-09T10:55:43.987Z

In WebMD and the Tragedy of Legible Expertise, Scott Alexander ends by saying that we should be grateful for the experts we have, but that prediction markets would still be better.

Is he right? If so, how much better would prediction markets be? Let's do some counterfactual history. Conditional on large, liquid, free prediction markets being made suddenly available to the public in Nov 2019 (or April 2020, or any other date), what would have been the most likely pandemic outcomes? I'm interested in both vague and specific answers. Even better if you can estimate a quantity of money or life-years saved.
Against butterfly effect
by ForensicOceanography — 2021-02-09T07:46:44.918Z
Link: https://forensicoceanography.wordpress.com/2021/02/09/against-butterfly-effect/

It is known that, when you simulate a deterministic system described by nonlinear differential equations, small differences in the initial conditions can be exponentially amplified, resulting in huge differences in the final result.

To describe this phenomenon Edward Lorentz famously said that “a butterfly flapping its wings in Brazil can produce a tornado in Texas”. This quote, popularized by Gleick’s 1987 bestseller on chaos theory, came to mean that small events and small decisions can have huge and unpredictable consequences.

The problem with this conception is that it is extrapolating from only two data points a correlation that (almost surely) does not exist.

Let us suppose that some aliens run a simulation of our universe, starting in 1 January, with our present initial conditions x(0). This simulation could be deterministic or probabilistic, depending on your philosophical standpoint on how our universe works. The aliens go on simulating until 1 July, and observe that on 1 July there is no tornado in Texas. Then they run again the simulation, but this time they slightly modify the initial condition x(0) (a butterfly flips its wings). This time, on 1 July there is a tornado in Texas. Does this couple of observation mean, in any meaningful way, that the butterfly caused the tornado?

To answer this question, we must run many simulations, sampling all the possibile initial conditions. If our universe is not deterministic, it would make sense also to repeat many times the simulation for each initial condition. Then we could measure the correlation between the correlation between the event “butterfly flips wings on 1 january” and the event “tornado in Texas on 1 july”. But this correlation will be almost likely 0. The more the system is chaotic, the more correlations will decay exponentially with time.

There are systems (like human history) in which small decisions can have big consequences. For example, I guess that the aliens simulating our universe could detect some positive correlation between the events “Francis Ferdinand gets shot in 1914” and “New Zealand is at war in 1917”. I do not think that this correlation is very big, but it could be fairly greater than 0. But this is because Francis Ferdinand heir to the throne was a very special person, whose life had big and predictable correlations with the lives of millions of other people.

If the system ...
Evolution from distinction difference
by KatjaGrace — 2021-02-09T05:20:14.733Z

If we have norms such that each copy of a common behavior must to be a tiny step away from from its parent, rather than a giant step or no step, this would seem to make culture much more amenable to gradient descent via evolution than it otherwise would be.

Is the latter somehow reason for us seeing the former? For instance, did ancient groups who frowned on really weird people and who felt awkward being too conformist outflourish other groups with their better evolved cultural norms and artifacts?

Also, is this picture true and applicable to the real world? Is the size of these steps in human culture such that culture learns well? Can you see such effects in the world?

Is this why women’s clothing—which seems expected to vary more between women—also changes faster over time than men’s clothing? (Is that even true?)
Promoting Prediction Markets With Meaningless Internet-Point Badges
by 1a3orn — 2021-02-08T19:03:31.837Z

Intro

(x-post)

I'd like to live in a world where prediction-market use is common and high-prestige.

The easiest way for this to happen is for prediction markets with money to be legal.

In the absence of this, there might nevertheless be some potential low-hanging fruit for a point-based prediction market -- Metaculus, or some unidentified contender -- to promote the wide acceptance of prediction markets. The same action might also improve the general quality of journalism, potentially.

Proposal

The prediction market creates a new feature. The feature allows a user of the market to create a small badge, displayable on the user's blog, Medium, Substack, or elsewhere, that displays the person's username and a score measuring the accuracy of their predictions.

The score could be an absolute measure such as Brier score, or a relative measure such as the percentile that the person occupies within the market. It could also be colored according to the number of predictions the person has made; or it could have a tag indicating that this accuracy only obtains within a particular subject or field; or it could indicate the time horizon with which they typically make predictions; generally, there are numerous addendums that could be added.

All of the above details are important, but for the moment I put them to the side.

The badge could be displayed at the head of every article by the author, potentially.

Codewise, this would work similarly to any front-end widget managed by another server, i.e., like a commenting system, like a Twitter embed, and so on and so forth. So of course it would update live as the predictions by the author came true / did not come true, even on older articles.

(This badge could be supplemented, of course, by embedded questions from prediction markets, which could be placed in articles. Metaculus already has these.)

Possible Points in Favor

People are tired of shitty media. There's an enormous groundswell of media distrust from many angles, as far as I can tell. A measure like this is easy to understand, at least in the basics, and provides clear evidence of credibility for those who use it, entirely independent of trust.

It also evens the credibility playing-field between individuals and large agencies, which could be popular.

People like little badges if they grant status. If the first users of this are sufficiently high-prestige, or if predictions / articl...
Micromorts vs. Life Expectancy
by Young Buck — 2021-02-08T19:03:11.770Z

I am trying to get a solid grip on how micromorts work (quick intro for those not familiar with the concept at https://en.wikipedia.org/wiki/Micromort). I have been doing some calculations, and there is one result I am seeing that I haven't been able to resolve or explain.

I attempted to reconstruct the "22 micromorts per day" value for deaths from all causes in the U.S. for 2010 (as listed on the Wikipedia page), and was able to do that, so I updated using 2019 data to get an estimate of 24 micromorts per day. I got this as follows: if you take the total number of deaths from 2019 in the U.S. (2,854,838) divided by the population of the U.S. in 2019 (328,239,523), that estimates the probability that someone dies in a year in the U.S. Divide by the number of days (365.25) to get a probability of death per day, and then multiply by 1 million to get 24 micromorts per day for deaths from all causes in the U.S. in 2019. So far, so good.

Then I decided to look at the number a different way: if the risk of death is 24 micromorts per day, we should be able to convert that to an estimated lifespan, because on average we get 1 million micromorts total per person (obviously some people get fewer and some get more, but by definition it has to average out). But if we take 1 million micromorts divided by 24 micromorts per day, we get 45,454 days, or 124 years. That is significantly longer than the 2018 CDC estimated human lifespan in the U.S. of 78.7 years (although at least it is within an order of magnitude).

Can anyone explain why there is a difference between the two estimates for human lifespan? I feel like there is a simple explanation lurking somewhere, but I haven't been able to figure out what it is yet. Thanks for your help!
What metrics should I track when experimenting with sleep?
by  — 2021-02-08T17:22:41.628Z

I used to sleep very little (6 hours or so) and function well with extremely good academic achievement. Instead of sleeping 8 hours now, I would love those extra 2 hours each day again but I'm not sure it won't affect my cognitive ability in the short term. I don't trust my impressions and would love to have some quantitative data.

What are areas that might be affected the most (e.g., long-term memory, working memory, speed of arithmetic calculations, vocabulary, reasoning, mood, etc.)?
What are some tasks that would measure my performance in those areas? Ideally those tasks should be:

quick to perform
easy to implement (I could write a quick program to run those but if there is a web app somewhere that does that already, that would be ideal)
fine with the subject knowing what is being measured (I'm going to apply them to myself, after all)
produce similar results regardless of how much experience I have with them (I don't want to wonder whether improvement/regression is due to the changes in my sleep or getting better/tired with the task.) It's ok if the skill builds over a week or so and then stays constant.

I'm not attempting to prevent long-term negative consequences since AFAICT nobody knows what those might be. Besides, there are significant changes in my lifestyle every month or so and so there would be too much noise in the data.
The distinction distance
by KatjaGrace — 2021-02-08T07:40:14.341Z

People have a strong tendency to be different from one another (e.g. are horrified to be caught in the same dress, find it weird to order the same dishes as their companion without comment or to choose the same art for their living room). Yet they also have a strong tendency to conform.

These are even in the same areas, and the best behavior seems to be balancing on an edge between the two forces. You don’t want to wear either a dress that someone else is wearing, nor a dress in a style that hasn’t been worn since the 1600s.

I have noticed both of these human forces before, but I hadn’t seen them so vividly as acting in the same realm. You don’t want your essay to be on an identical topic to another student’s, but you also don’t want it to be outside the bounds of what the professor thinks of as an essay, or expressing views beyond a short hop from those others would endorse.

This makes me imagine the curlicues of culture as growing in the fertile interstitial zone between options too conformist to consider and options too wild to consider. Kind of like a Mandelbrot set or a tidal flat or a cellular automaton. There’s a similar pattern in the space of ways the whole of culture could have been: if everyone was very conformist about everything, it would be monotony, and if everyone immediately flung themselves as far away from anyone else as they could on every axis, it would be another kind of monotony. But with this balance of effects as it is, we get some complicated spiraling evolution of art movements and attitudes, trousers and tools. Each idea bringing forth riffs of of it in every direction.

Inspired by a conversation with Robin Hanson, where he probably basically said the main point here, that these two forces act in opposition.