trentmkelly/LessWrong-43k huggingface dataset:
Why Huntington-Hill?
by jefftk — 2020-10-25T00:50:33.884Z

The US House of Representatives uses the

Huntington-Hill
method to figure out how many representatives each state should
have. First you give each state one representative, because even the
smallest state is guaranteed one, and then you assign the remaining
spots, one at a time, to whichever state has the highest "priority".
Priority being ratio of its population to the geometric mean of the
number of seats it currently holds and the number it would hold if it
received this extra seat:

Ps = state population
Rs = state reps

Ps
---------------
sqrt(Rs*(Rs+1))

Where does this come from? I had a shot at deriving it, and it
actually makes a lot of sense. First, we restate the problem has one
of error minimization. At every stage, we want to assign the next seat
wherever it would most minimize representational inaccuracy.
Current error is, summed over all states:

Pt = total population
Rt = total (target) reps

| Pt Ps |
| -- - -- | * Ps
| Rt Rs |

For each state we might give a seat to, the effect that would have on
total error is:

| Pt Ps | | Pt Ps |
| -- - ---- | * Ps - | -- - -- | * Ps
| Rt Rs+1 | | Rt Rs |

We would like to identify the state that minimizes this quantity.
Since we are adding representatives one by one, Pt/Rt
will always be greater than Ps/Rs [1] and we can remove
the absolute value and distribute the Ps.

PtPs PsPs PtPs PsPs
---- - ---- - ---- + ----
Rt Rs+1 Rt Rs

Cancel the PtPs/Rt and we have:

PsPs PsPs
---- - ----
Rs Rs+1

Combine the two fractions and cancel again:

PsPs
---------
Rs*(Rs+1)

Since we're trying to identify the state that minimizes the quantity,
we can instead identify the state that minimizes its square root:

Ps
---------------
sqrt(Rs*(Rs+1))

Which is in the prioritization of Huntington-Hill.

I initially tried to derive this from squared error, which did not
work and ended up with an enormous amount of scribbles on paper.

[1] This is not quite true, as we get to assigning the very last
representatives, but I think it still works?

Comment via: facebook
Notes on Courage
by David Gross — 2020-10-24T22:32:25.531Z

This post examines the virtue of courage and explores some practical strategies for becoming more courageous.

Courage (sometimes “bravery” or the closely-related virtue of “valor”) is one of the most frequently mentioned virtues in virtue-oriented traditions. It was one of the four “cardinal virtues” of ancient Greece, for example.

Courage also undergirds other virtues:

“Courage is not simply one of the virtues but the form of every virtue at the testing point, which means at the point of highest reality.” ―C.S. Lewis[1]

“Courage is the most important of all the virtues because without courage, you can’t practice any other virtue consistently.” ―Maya Angelou[2]

Fear and Our Response to It

Courage has to do with our response to risk and fear. This response has at least three components:

The way we judge how threatening a situation is—how easily spooked we are (emotional) and how sensible our risk assessment is (cognitive).
How we act when we are immediately confronted with a frightening scenario—how well we think and perform while afraid.
How we respond to the risk of being in a fearful scenario at some future time (sometimes “fear” in this anticipatory context is called “anxiety,” “worry,” or “dread”)—whether our risk-aversion is well-honed or whether we are overly risk averse because we “fear fear itself.”

Fear is an unpleasant good in the same sort of way that pain and nausea are: Such things are no fun, but they are useful. Fear (when it is operating properly) informs you that you have put yourself in a situation in which you run the risk of harm. The unpleasantness of the sensation of fear prompts you to be averse to doing it again. Fear also can prepare you for an immediate, protective fight-or-flight response.

(Although we are averse to fear, we sometimes also perversely seek it out. Similar to how some people crave the pain of ghost chilies or spankings, some people crave the fright of horror movies and roller-coasters. Is this perhaps a way of helping to regulate our fear response through practice or inoculation?)

The visceral fear response is adaptive and it’s no surprise that we see it in other animals and that it seems to be to some extent a “deep,” subconscious part of our mental make-up. This can also make our fears difficult to work with on a conscious, rational level, as the experiences of people with phobias, panic disorders, anxiety disorders, or post-t...
“Prediction” and “explanation” are not causation
by jasoncrawford — 2020-10-24T18:55:55.767Z
Link: https://twitter.com/jasoncrawford/status/1320068209916035072

Everyone knows that correlation is not causation. Many people don't know that in scientific jargon, “predict” and “explain” are also not causation. They are forms of correlation.

(Technically, “association” might be a better term than “correlation”, which can have a narrower technical meaning in statistics. But since I'm writing this for non-experts, I'm going to use the term “correlation” in the colloquial, wider sense.)

These terms can cause extreme miscommunication:

In lay usage, “X predicts Y” implies that X comes before Y. Predictions are about the future. In statistics, there is no time implication at all. It is just a type of correlation. If I said that I could use 2020 data to “predict” things that happened in 2019 (or 1920), most people would laugh at me. But this is a perfectly legitimate usage of statistical “prediction”.

Similarly, the general sense of “explanation” means a conceptual understanding of a phenomenon. In statistics, “explanation” implies no such understanding, only a certain type of correlation.

Because of the time-implication of “prediction” and the conceptual-understanding implication of “explanation”, most people are likely to interpret these as evidence for or even proof of causation. But in many cases, they merely mean correlation.

If you are reading scientific papers, be careful with how you interpret these terms. If you are reading reports about science, know that the reporter might not be clear on this point. If you are reporting science yourself, be responsible with what you write.

(By the way, without @NeuroStats I wouldn't know this stuff either. Any value in this post is thanks to her; errors are mine alone.)
The Darwin Game - Rounds 0 to 10
by lsusr — 2020-10-24T02:17:43.343Z

The most important change between my game and Zvi's original is that bots can read each others' source code. They can simulate each other and predict each others' behavior. Within a day of the tournament launching—and an entire week before entries closed—Zack_M_Davis had already written a bot to simulate opponents and then open-sourced it to everyone.

That's what happens when a significant contributor to an open source Lisp dialect participates in a software competition.

Taleuntum wanted to write an even better simulator but was informed that it would take too many years to run.

Muticore solved the limited compute problem and wrote a safe, effective, obfuscated simulator, with randomization.

The Phantom Menace

Three separate people asked me what happens if a bot crashes the game while simulating an opponent with malware in it. This turned out not to matter because nobody deployed malware to destroy simulators. Only one player, Measure, deployed malware—and the malware didn't crash the game. Instead, it attempted to replace its opponent's move method with a method that returned 0 instead. But the threat of getting disqualified seemed to scare away other potential simulators.

Taleuntum did write a MatrixCrashingBot that crashes simulators but did not submit it. This is a disappointment, as Taleuntum would have been allowed to submit this bot as a separate entry on the grounds that it does not coordinate with Taleuntum's CloneBot. To my knowledge, nobody else took advantage of this deliberate loophole in the rules either.

RaterBot safely combed through its opponent's source code for "2"s and "3"s to estimate aggression without the dangers associated with running untrusted code.

Computer programs attempting to simulate each other can produce complex behavior. The behavior is so complex it is provably undecidable—and that's totally ignoring the real-world sandboxing problem.

Nevertheless, two contestants requested I write code to simulate their opponents. I refused these requests. Zvi[1] accepted a simpler bot and the other contestant dropped out.

I'm surprised running the enemy is complicated though—it should just be a function call.

―quote from the contestant who dropped out

The most significant use of an opponent's source code came from Vanilla_cabs.

Attack of the Clones

Zvi's original game was dominated by a clique of players who coordinated out-of-game to defeat the non-c...
Babble and Prune : 5 new ideas
by  — 2020-10-23T23:42:50.540Z

I did something similar to the recent babble challenges here on Lesswrong over the past week. I was looking for five brand new ideas, like reverse all advice by SSC. The criterion was that each idea should be at least as interesting as the question "Should you reverse any advice you hear?"

Infinite Minds

According to the MUH, everything mathematically possible must exist. So surely infinite minds with infinite computational power exist.
This brings us to something like a Fermi Paradox. Where are the infinite minds?

Air Conditioning

For a brain emulation, fluctuations in the ambient temperature are easy to create.
These fluctuations could be used as thermal music.

Antigraph

Capture the LessWrong social graph.
Make it more strongly connected, via Zoom chats etc. Actively reduce the degree of separation from 6 to 2, by video chats among random LessWrong members.

AntiSignalling

Intentionally underperforms in exams and walks out of the exam hall early because it is a waste of time to answer questions that ve already knows how to solve

Noticing Confusion

Why are there no non-profit phone/laptop manufacturing organizations? Where all the hardware design etc is open source? This is especially strange since they are not making much money in the first place. As far as I can tell, this is mostly due to social inertia and norms in the industry. Please correct me if I'm wrong.
What is our true life expectancy?
by Adam Zerner — 2020-10-23T23:17:13.414Z

I recently learned that when they talk about a life expectancy of eg. 80 years, it's assuming that the future looks similar to the past. But with exponential progress in technology, that doesn't seem like a good assumption. For example, according to Bostrom's survey (90% likelihood), the median pessimistic year for AGI is 2075.

So then, taking into account technological progress, what do we expect life expectancy to be?
No Causation without Reification
by Gordon Seidoh Worley — 2020-10-23T20:28:51.831Z

I don't know how inspiring "no causation without reification" is as a rallying cry. Probably not very. But as a pithy description of an important insight, I think it's quite nice.

There's lots of reason to care about causation. In fact, it's hard to talk about much of anything without causation showing up. We need some way to talk about the fact that stuff happens, then other stuff happens, and that other stuff happens only because the first stuff happened, or so we hope to claim. The idea that one event causes another seems fairly fundamental to how humans perceive the world.

Formally, we often model causation as the action of one thing implying another, and we might formalize this with mathematical notation like A⟹B to mean some event or thing A causes some other event or thing B to happen.

Hopefully that all sounds boring and obvious to you, and has you wondering when I'm going to get to the point.

My point is that the entirety of what we think we mean when we say A⟹B is ontological, i.e it's in the map. Not once ever has causality existed in the territory.

Okay, I know, time to smash the downvote button, but hear me out.

Try to imagine what the world is like if you're not modeling it. Are you picturing atoms? Particles? Wave functions? Strings?

Wrong!

Those are all models we impose on the world to make sense of it. Useful ones, usually, but still models.

The territory, noting that the notion that such a thing exists is itself another model, is a kind of soup of stuff that's all mixed up and irreducible. You either have everything or nothing, for all time or no time. If you want anything else, you have to draw a map, make distinctions, and make choices.

Those distinctions are hopefully correlated with how the territory works, because that makes them useful for things like predicting what will happen (and we often call this correspondence "truth"), but they are not the territory itself. The territory just is; always has been, always will be.

This means that there's no aspect of the territory that is causality. There's no A, there's no B, there's no ⟹, there's just "is".

But if it's not in the territory, why is causality so useful that it's largely invisible to us, and why is it so hard for us to wrap our minds around what the world would be like if we didn't perceive its presence?

Well, there clearly is some featur...
Desperately looking for the right person to discuss an alignment related idea with. (and some general thoughts for others with similar problems)
by Rasmus Eide — 2020-10-23T19:42:41.567Z

I have no idea how to start this post. I don't even have the resources (mostly health-like ones lacking) to figure out the meta-strategy to reach out for help, and this is like the billionth place I've tried these last weeks since things became to urgent to do things the right way. My instincts and social anxiety are screaming at me that I am burning bridges and am going to be downvoted and hated and booed by everyone I consider high status for posting this mess for the rest of eternity but no matter how I run the numbers over and over again to try to find an excuse not to post this they still come out saying I have to write this even if it ruins my life. So here goes nothing please don't downvote.

There's a topic that really deserves a long article that is beyond my resources to create but I'd probably call something like "The expert, the hobbyist, and the 998 crackpots", the idea being a simplified version of the real scenario that will probably happen at some point and might be happening right here. The idea is basically that a hobbyist has had a critical insight, and needs to get it to the expert, and the expert desperately needs that insight, and everyone knows this, but there are also 998 Crackpots that each have their own useless garbage insight and the expert needs to spend a significant amount of time to determine if a given contact is the hobbyist or one of the crackpots, and due to the nature of crackpots and impostor syndrome, the hobbyist or crackpot cannot tell from the inside which one it is they are. As such, even though each one (me) knows they are far more likely to be a crackpot than the real hobbyist they all need to transmit the insight they have and treat it as potentially important, and all these players together needs to find a way for the expert to find the useful needle insight in the 999 haystack.

Now, there is a partial solution we've been using; this website, with community vetted articles and votes to filter out and iterate on ideas and the experts only reading the ones filtered through this process, but it fails in two important cases that both apply here (and potentially others that I haven't thought of):

One, the hobbyist ends up being someone like me; someone who can't write articles to the needed standard, doesn't have the psychology needed to participate in the social context, and is simply to ill and lacking capa...
Local Solar Time
by jefftk — 2020-10-23T19:40:06.406Z

Sometimes we talk about how society is "path dependent": what things
would be different if history had not followed its particular path?
One place that's fun to think about is time. Until the
development of

railways,
places used local solar time. Washington DC is at 77°W while NYC
is at 74°, so clocks in NYC would be set ~12min ahead of those in
DC.

Initially, this didn't matter: traveling from NYC to DC was a multi
day endeavor, so a 12 minute difference in clocks was trivial. As
roads got better and then railroads were built, however, travel times
decreased enormously:

Atlas of the Historical Geography of the United States, 1932

If you're running a railway system, it's really very useful to choose
a single location's time to use internally. Your workers are moving
East and West, fast enough for this sort of discrepancy to become a
problem, and they certainly don't want to be continuously changing
their watches as they move from town to town. Railway schedules are
much easier to work with if they use railway time, people start using
railway time in their regular life, the government adopts a system of
time zones.

Imagine, however, that things had gone differently and we were still
on local solar time. At this point it would not be hard to make our
phones always report the correct time as we move east and west, since
our phones know where we are. Automated calendar software could deal
with local solar time with about as much effort as it could with time
zones. Do all routes to the present pass through a system of time
zones, or is there an alternative history where we really could be on
local solar time today?

Railroads need shared time much more than other forms of
transportation. There are a small number of high-capacity tracks, and
the same with vehicles. Efficient use requires central organization,
coordinated operation, and shared scheduling, which is far easier with
a shared time system. If instead of railroads we had developed cars,
it would still be annoying to need to reset your watch as you move
between places, but it would be much less of an issue. Perhaps we
could have had political systems which made private railways
impractical, combined with governments that were not competent enough
to run their own? Or progress in making lighter engines could have
been much faster?

Alternatively, if you had lots of Nort...
Adjusting probabilities for the passage of time, using Squiggle
by NunoSempere — 2020-10-23T18:55:30.860Z

Most times I go to update a forecast, nothing has happened. There are several ways to update on this, which I explore below.

This post was written after Ozzie Gooen suggested I play around with Squiggle, following up on his earlier Multivariate estimation & the Squiggly language. Squiggle is an experimental domain-specific language for working with probabilities, which is currently in the very early stages of its existence. It can probably do anything Guesstimate can do, while being more easily copied and pasted, at the cost of not having a graphical interface. It currently uses vim as a text editor, so the user has to press i to switch to insert mode and edit the code, and in general it's on the very early stages, e.g., syntax and functionality are still being figured out.

1. Naïve decay

Perhaps the most straightforward way of updating on the passage of time is as follows: If one assigns a probability pyearly=0.9 to an event happening in a year, then the monthly probability pmonthly is such that (1−pmonthly)12=1−0.9, or pmonthly=1−(1−0.9)1/12. Now, if a month has passed, then the remaining probability is pupdated=1−(1−pmonthly)11.

In Squiggle, this would look as follows:

## Press i to edit.
## Initial setup
yearly_probability=0.9
period_probability_function(epsilon)=1-(1-yearly_probability)^(1/epsilon)
probability_decayed(t, total_time, period_probability) = 1-(1-period_probability)^(total_time-t)

## Monthly decomposition
total_time_monthly=12 # months in a year
monthly_probability=period_probability_function(total_time_monthly)
probability_decayed_monthly(t)=probability_decayed(t, total_time_monthly, monthly_probability)
probability_decayed_monthly
### probability_decayed_monthly(6)

And this would look like so:

The type signature of this function is p→(t→p). That is, I take a probability (0.9), and output a function that goes from time passed, t, to probability remaining after time t. If I wanted to know what the probability is after 6 months, I'd uncomment the last line, probability_decayed_monthly(6), in this case equal to 0.684.

2. Unprincipled decay.

A strategy which works well as a forecasting technique in practice, but which is totally unprincipled, is to consider probabilities over probabilities. I can ask myself what the lowest and upper bets I'd be willing to take would be, and they might be something like

1:9, where I receive 9 if the event

...
Humans are stunningly rational and stunningly irrational
by Stuart_Armstrong — 2020-10-23T14:13:59.956Z

Are humans mainly rational or mainly irrational? This is an old debate, with some political implications.

But the question is ill posed, because it elides the important detail: compared with what?

Humans are stunningly irrational

Let V be some reward/utility function that encodes human preferences - including a mix of wealth, security, status, happiness, pleasant lives, good friends and experiences, romantic connections, sex, etc... It doesn't matter too much which of the adequate V's this is.

Then it's clear that humans are stunningly irrational as compared with a fully rational V-maximiser. As I mentioned before, one of the reasons that we think we're rational is because we only contemplate a tiny slice of the action space.

But if I were unboundedly rational, I could do many things. I could re-engineer biology from physics and some research articles, and create cures for multiple diseases, becoming incredibly rich and respected. I could launch a media campaign to make myself very popular. I could create tailored drugs to make myself maximally happy and extremely productive.

Hell, I could just program a superintelligence to optimise the world and maximise V. Unboundedly rational means that I'd be more effective than any such superintelligence, so I'd achieve a lot.

So it seems that, if a random agent gets V=0, and an unboundedly rational V-maximiser would expect to get V=1, then a human would get a value of V that is utterly tiny - maybe 1/10100 or smaller. On the scale of randomness to full rationality, humans are almost indistinguishable from the randomness. We're stunningly irrational.

Humans are stunningly rational

To see how rational humans are, let's zoom in on that difference between humans and randomness. Suppose that it takes twenty actions to walk to a door (it actually takes far more). Then every time a human walks to a door, they add at least twenty bits of information as compared with a random agent. Add in the movements that humans do every day, the planning, the eating, the conversations, and we have thousands of bits of information; we occupy a tiny slice of possibility space, and have extreme optimisation powers.

And that degree of optimisation is similar to that of animals. Humans have longer term planning, rather extreme social skills, the ability to integrate vast amount of new information and come up with new ideas and designs. We direct the ...