trentmkelly/LessWrong-43k huggingface dataset:
Slack Has Positive Externalities For Groups
by johnswentworth — 2021-07-29T15:03:25.929Z

You ever have one of those group projects where you all need to find a time to meet, but some or all of the group members have packed schedules, so you pull out the whenisgood and it turns out the only times that work are 2-2:15 pm on Thursday or 6:30-7:30 am on Tuesday?

For this sort of scheduling problem, the best group members are those with lots of

slack

in their schedule - people who either have lots of time available, or have very flexible time commitments which can move around to accommodate a group meeting. But if my schedule is flexible, note that most of the benefits of that flexibility are captured by the group as a whole, not by me: my flexibility mostly allows the group to accommodate less-flexible members.

The slack in my schedule creates positive externalities for the group. I mostly control how much slack to create/maintain in my schedule, but a large chunk of the benefit goes to other people. This means I’m incentivized to create/maintain less-than-optimal slack in my schedule.

Once you look for it, this shows up in other guises too: many different flavors of slack create positive externalities for groups. In general, we should expect people to create/maintain less slack than would be socially optimal, and this in turn will make groups less functional. What do other forms of this look like, and what can we do about it?

Many Flavors of Slack

A few common forms of slack:

Financial: money not budgeted for anything in particular, or which can easily be spent on something else instead, is financial slack.
Time: time not scheduled for anything in particular, or which can easily be rescheduled, is time slack.
Space: space not used for anything in particular, or which can easily be used for something else, is space slack.
Emotional: capacity for excess stress is emotional slack.
Social: multiple social groups which one can fall back on, or the ability to make new friends quickly, provide social slack.

We can also separate short-term vs long-term slack for each of these. For instance, a bank may have lots of capital to invest, but limited liquidity, so they can’t move their capital around quickly: high long-term financial slack but limited short-term financial slack. Conversely, someone who has some savings on hand but is spending as much as they earn has short-term financial slack, but not long-term financial slack. Exercise for the reader: what would short-te...
Gaming Incentives
by Jozdien — 2021-07-29T13:51:05.459Z

I played chess for a while earlier this year. I never played or watched any sports when I was younger, so sport forums were weird to me. One thing that kept coming up led to a long meandering stream of thought I tried in vain to edit together into a coherent narrative. It’s probably easy to guess what the

most

frequent

and

hotly

debated

question would be: who’s the greatest to ever play the sport?

My standard response to this kind of question is to say it’s a question

of semantics

, or specific definitions. But this is a place where we reward players for greatness anyway, what with World Champions and player rankings, so there's something being incentivized.

I don't think many try to think about what they mean when they talk about the quality of a player - probably because it's more fun to not. But I'm under no such obligation. For most people, what it means is a varying combination of three factors - natural talent, dedicated effort, and actual skill level.

Depending on your sensibilities, it may seem strange at first to claim that people try to use these to defend their favourite choice. Going back to chess, there are many names that come up in these arguments, but the three most prominent serve as a good microcosm:

Bobby Fischer

, eleventh World Chess Champion, who single-handedly ended Soviet chess supremacy and had an unparalleled dominance over his opponents at the time;

Garry Kasparov

, thirteenth World Chess Champion, who was the highest rated player in the world for over 21 years and had an unparalleled dominance over his opponents in his time; and

Magnus Carlsen

, sixteenth and current World Chess Champion, the highest rated player in history, and who doesn't have unparalleled dominance over his opponents, but for defensible reasons.

To me at first glance, it seemed obvious that Magnus fits the title of best player the most - ELO ratings being a metric of relative playing level, and the quality of the average GM

increasing over time

.

Proponents

of Fischer or Kasparov mainly sound the argument that Magnus reigns in an era with advanced chess AI that any player can use to train and prepare with. That's true, professional chess players make heavy use of engines to create and test their strategies. This is also partly why Magnus doesn't dominate this age like his predecessors - the improvement of engi...
Did they or didn't they learn tool use?
by Daniel Kokotajlo — 2021-07-29T13:26:32.031Z

DeepMind claims that their recent agents learned tool use; in particular, they learned to build ramps to access inaccessible areas even though none of their training environments contained any inaccessible areas!

This is the result that impresses me most. However, I'm uncertain whether it's really true, or just cherry-picked.

Their results showreel depicts multiple instances of the AIs accessing inaccessible areas via ramps and other clever methods. However, in the actual paper page 51, they list all the hand-authored tasks they used to test the agents on, and mention that for some of them the agents did not get >0 reward. One of the tasks that the agents got 0 reward on is:

Tool Use Climb 1: The agent must use the objects to reach a higher floor with the target object.

So... what gives? They have video of the agents using objects to reach a higher floor containing the target object, multiple separate times. But then in the chart it says failed to achieve reward >0.

EDIT: Maybe the paper just has a typo? The blog post contains this image, which appears to show non-zero reward for the "tool use" task, zero-shot:
How to bayesian update on multiple covid tests?
by rchplg — 2021-07-29T13:15:09.804Z

I'm curious how I should think about the risk of hanging out with a friend (who is 2x pfizer vaccinated). It seems like a good opportunity for bayesian thinking in the real world, but I'm really unclear how to think about it.
Info: he tested positive on 4 UK lateral flow tests (LFTs), all from the same box (on 2 different days). After this, his roommates took two tests from the same box & both were negative.
He has subsequently taken 3 PCR tests + a LFT each day, which have been negative.
However, false positives seem to be very rare even for LFTs. They're ~1/1000 (number range from .9968 in original studies to .9997 more recently)
https://www.gov.uk/government/news/new-analysis-of-lateral-flow-tests-shows-specificity-of-at-least-999
https://www.bmj.com/content/372/bmj.n706
But false negatives seem common for everything, including PCRs. It seems there's a 20-67% false negatives (20% being best it ever gets, on day 8 of infection)
https://www.acpjournals.org/doi/10.7326/m20-1495
https://pubmed.ncbi.nlm.nih.gov/33301459/
Given this, what are the chances he had covid (from maybe 10-20x lower-than-average-risk prior, but in oxford/England)?
Importance of Ideas and People We Disagree With
by Olitness — 2021-07-29T11:50:56.346Z
Link: https://olitness.wordpress.com/2021/07/29/importance-of-ideas-and-people-we-disagree-with/

True Diversity as Source of Innovation and Resilience

It's important to have true diversity among people. For there to be people who vehemently disagree with you, your methods, your ways, your goals. And you actually want that! Mere tolerance of them is not realising its true importance and how they benefit you.

You want people spread out on the spectrum, from one extreme to another, with most people somewhere around the middle.

Why would you want that if your way is, of course, "the right way"?

One big reason is that such diversity creates most resilient, robust and balanced society that flourishes with new ideas, options and innovations. You like those things, options, innovations and not living in collapsed society or being dead, right?

And believe or not, there is rarely just one way to do things. You always want there to be extra options and ways. You might love going to work through forest, but in a case that there is a wildfire, you want there to be another way like a road through a city for example!

If we all go to the right, and there is no one going to the left, then in very unlikely event that going right way proves to be fatal (black swans are still a thing), then there will be no one remaining! But if we diversify, then we survive.

This doesn't mean you should change your opinion (unless you are really compelled to). But you should want there to be someone who opposes your opinion, even if they are wrong.

True vs. Fake Diversity

Diversity among people isn't based on race, ethnicity or gender or any arbitrary group like that because people aren't their race, ethnicity or gender. What makes a person are ideas they enact on! That's where true and productive and more enjoyable diversity comes from.

Compare it with fake diversity where you separate people based on gender or race. This is mere discrimination and/or virtue signalling, not a diversity because two people can be same in all other respects except for their race or gender, or whatever.

In most contexts, diversifying among those variables makes no difference. But when diversifying among ideas, that makes almost always meaningful difference.

Diversifying Isn't Always Ignorance

Diversifying in investing can be thought of as a sign of ignorance or lack of expertise and is at best considered way to preserve the wealth, not to grow it.

This might be true also on individual level. More focused you a...
How much compute was used to train DeepMind's generally capable agents?
by Daniel Kokotajlo — 2021-07-29T11:34:10.615Z

I'm talking about these agents (LW thread here)

I'd love an answer either in operations (MIPS, FLOPS, whatever) or in dollars.

Follow-up question: How many parameters did their agents have?

I just read the paper (incl. appendix) but didn't see them list the answer anywhere. I suspect I could figure it out from information in the paper, e.g. by adding up how many neurons are in their LSTMs, their various other bits, etc. and then multiplying by how long they said they trained for, but I lack the ML knowledge to do this correctly.

Some tidbits from the paper:

For multi-agent analysis we took the final generation of the agent(generation5)andcreatedequallyspacedcheckpoints (copies of the neural network parameters) every 10 billion steps, creating a collection of 13 checkpoints.

This suggests 120 billion steps of training for the final agents. But elsewhere in the post they state each agent in the final generation experienced 200 billion training steps, so.... huh?

Anyhow. Another tidbit:

In addition to the agent exhibiting zero-shot capabilities across a wide eval-uation space, we show that finetuning on a new task for just 100 million steps (around 30 minutes of compute in our setup) can lead to drastic increases in performance relative to zero-shot, and relative to training from scratch which often fails completely.

So, if 100 million steps takes 30min in their setup, and they did 200 billion steps for the final generation, that means the final generation took 30 x 2,000 = 41 days. Makes sense. So the whole project probably took something like 100 - 200 days, depending on whether generations 1 - 4 were quicker.

How much does that cost though??? In dollars or FLOPs? I have no idea.

EDIT: It says each agent was trained on 8 TPUv3's. But how many agents were there? I can't find anything about the population size. Maybe I'm not looking hard enough.
Violent Unraveling: Suicidal Majoritarianism
by mako yass — 2021-07-29T09:29:05.182Z

First they came for the Communists
And I did not speak out
Because I was not a Communist

Then they came for the Socialists
And I did not speak out
Because I was not a Socialist

Then they came for the trade unionists
And I did not speak out
Because I was not a trade unionist

Then they came for the Jews
And I did not speak out
Because I was not a Jew

Then they came for me
And there was no one left
To speak out for me

There has long been a poem about Suicidal Majoritarianism, and its history still resounds, but this is not enough. It should be named, and someone should describe its essential pattern:

A cruel logic advises the majority to disenfranchise, exile or kill some minority that it finds burdensome. The majority knows that they can easily dominate this minority, as they are many. Their mistake, the reason this should be treated as a disease even by those who sit in the majority, is that this logic will be applied again to the next minority, continuously, by induction: eventually this cruel logic will be extended to us.

It isn't obvious to the perpetrators of a Violent Unraveling that it is a failure of rationality. A causal decision theorist might reason, "The purges will continue with or without me. It is external to me. My complicity now in this purge does not cause the purges to continue and to expand their scope until my own family is taken by them. I should accept this war of all against all, and strive against the odds to stand as the lone victor" (Perhaps I should try to be less judgemental towards CDT agents. I barely understand them any more.)

I am a logical decision theorist. For me the logic here is much easier to dismiss. I see that the rationale of the purges would abstract beyond their present target. I see that there are ways in which I am a minority, and so one day the logic will apply to me. I decide not to implement the logic, knowing that many other LDT-like agents will stand firmly beside me. Not every person will move with us, but enough will, that the additional safety we gain by denying the cruel logic from our domains shall be worth the blood that we are forgoing.

Culture, writing and enforcing norms, is closer in character to LDT than CDT (though it is perhaps implementable on top of a medium of CDT agents woven into contracts). Cultures can coordinate to prevent a behaviors that are individually rational but harmful when reflected throughout...
Do we have a term for the issue with quantifying policy effect Scott Alexander stumbled on multiple times?
by yhoiseth — 2021-07-29T08:45:29.636Z

In Things I learned Writing The Lockdown Post, Scott Alexander describes a really tricky issue when trying to quantify the effects of some policies:

This question was too multi-dimensional. As in, you could calculate everything right according to some model, and then someone else could say "but actually none of that matters, the real issue is X", and you would have a hard time proving it wasn't.

A long time ago, I remember being asked whether banning marijuana was good or bad. I spent a long time figuring out the side effects of marijuana, how addictive it was, how many people got pain relief from it, how many people were harmed by the War on Drugs, etc - and it turned out all of this was completely overwhelmed by the effects of deaths from intoxicated driving. If even a few people drove on marijuana and crashed and killed people, that erased all its gains; if even a few people used marijuana instead of alcohol and then drove while less intoxicated than they would have been otherwise, that erased all its losses. This was - "annoying" is exactly the right word - because what I (and everyone else) wanted was a story about how dangerous and addictive marijuana was vs. how many people were helped by its medical benefits, and none of that turned out to matter at all compared to some question about stoned driving vs. substituting-for-drunk-driving, which nobody started out caring about.

It might actually be even worse than that, because there was some hard-to-quantify chance that marijuana decreased IQ, and you could make an argument that if there was a 5% chance it decreased IQ by let's say 2 points across the 50% of the population who smoked pot the most, and you took studies about IQ vs. job success, criminality, etc, really seriously, then lowering the national IQ 1 point might have been more important than anything else. But this would be super-annoying, because the studies showing that it decreased IQ were weak (and you would have to rely on a sort of Pascal-type reasoning) and people reading something on the costs/benefits of marijuana definitely don't want to read something mildly politically incorrect trying to convince them that IQ is super important. And if there are twenty things like this, then all the actually interesting stuff people care about is less important than figuring out which of the twenty 5%-chance-it-matters things actually matters, and i

...
An Apprentice Experiment in Python Programming, Part 2
by konstell — 2021-07-29T07:39:47.468Z

[Epistemic status: me trying to recall what happened in a 4.5-hour pair-programming session, during which I was focused on solving problems. This post was reconstructed from scratch code files, written messages (the session was over voice call) and my python shell history.]

Previously: https://www.lesswrong.com/posts/kv3RG7Ax8sgn2eog7/an-apprentice-experiment-in-python-programming

Three days ago gilch and I had another session on Python Programming, where we continued talking about decorators.

Lambda decorator

So far, all of the decorators we've seen have been functions that take in another function as parameter, then used by the syntax @<decorator_name>. Turns out we can use a lambda function as a decorator too:

@lambda f: print("hi")
def add(a, b):
return a + b

When this file gets run in Python 3.9 interactive mode, the output is

hi

and when I call the add function:

>>> add(1, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not callable

Gilch asked me to write out the desugared version of this code. I wrote add = print("hi"), which was not correct. Gilch gave me the correct desugared code:

def add(a, b):
return a + b

add = (lambda f: print("hi"))(add)

Partial

We did a brief introduction to partial. We had code from last time:

def subtract(a, b, c):
return a - b - c

Accessing doc from within python shell:

>>> from functools import partial
>>> help(partial)

Using partial to create a function with the first and second arguments passed in:

>>> partial(subtract, 1, 2)
functools.partial(<function subtract at 0x7f34649af3a0>, 1, 2)
>>> partial_sub = partial(subtract, 1, 2)

Gilch pointed out that the line above, partial_sub = partial(subtract, 1, 2), could've been written as partial_sub = _ because in python shell, _ returns the last value that's not None.

Testing the partial function:

>>> partial_sub(3)
-4

1 - 2 - 3 equals -4, so we're getting what we expected.

Gilch then pointed out that arguments can be passed into partial as keyword arguments:

>>> partial(subtract, b=0, c=2)
functools.partial(<function subtract at 0x7f34649af3a0>, b=0, c=2)
>>> _(10)
8

Finally, there's a way to implement partial functions using lambda:

>>> lambda x: subtract(x, 0, 2)
<function <lambda

...
Evidence that adds up
by TekhneMakre — 2021-07-29T03:27:34.676Z

(Is there already a name and/or clearer analysis for this?)
Some kinds of evidence add up more than others.
E.g. say someone gives me a recommendation like "do X". (Taking this as evidence of "doing X will get you what you want", rather than as a command.) Then someone also recommends "do X", and someone else, and so on. I'll only get somewhat convinced to do X. Maybe lots of people are telling me to do X because X is good for me to do, but maybe there's some other explanation, like an information cascade, or they're just signaling, or it's a "belief" computed by deference not truth, or I give off falsely strong superficial cues of really needing to X. (And, decision theoretically, how I decide to process the recommendation might be best thought of as affecting what's already the case about why I'm given the recommendations.) These other hypotheses predict the same observation--people telling me to do X--so in this scenario they have probability bounded away (well away, given my priors) from 0. In particular, each repeated observation of the same sort of thing--each "do X"--imparts less and less information.
On the other hand, say someone tells me "do X; my friend did X and then Y happened", and then someone else tells me "...and then Z happened", and "do X; this study showed that W tends to happen", and so on. In this scenario, I eventually get much more convinced to do X than in the previous scenario (assuming that I want to do X when I'm confident of what would happen if I did X). There's fewer hypotheses that predict this sort of sequence of observations, than hypotheses that predict just the less specific sequence "do X do X do X". We could call this "evidence that adds up".
This is different from giving abstracted reasons / justifications. "Do X because R" doesn't add up as well as an anecdote: it punts to the proposition R. If everyone gives R as a reason, that only adds up so much. (Of course, giving reasons is useful in other ways than adding up; maybe R is more easily verified, maybe I believe but don't actually care about R and so can ignore the recommendation.)
To be more precise, we would say, "evidence that doesn't add up about X", since there might be some other proposition about which we keep gaining much information from repeated observations of people saying "do X". Maybe this is already implicit in the word "evidence", rather than "observat...
Risk Premiums vs Prediction Markets
by SimonM — 2021-07-28T23:03:50.398Z

aka Why (some) market forecasts are biased and what can we do about them?

tldr; When you care about an outcome, it will skew the odds you'll bet at. (To insure against bad outcomes). When everyone cares about an outcome, it skews the market odds.
People (including myself) often point at market forecasts as being the best predictors for some variable (inflation, the economy, whether England will win the Euros).

However, market prices are more than just expectations. In the words of an anonymous twitter account:

Market price = Expectations + Risk Premium + Idiosyncratic Issues

This is a somewhat tautological framing however (since we don't know what risk premium or the idiosyncratic issues are other than the things which make this equality true). I'm going to try and explain what risk premium is, since people often focus too much on expectations and use idiosyncratic issues an excuse (fees are too high on PredictIt, you can't trade large size, etc..). Risk premium is important because unlike idiosyncratic issues which can be fixed with good market structure, risk premium will not go away.

This matters, because if we can't remove risk premiums and the idiosyncratic issues, then prediction markets are less accurate and less valuable for decision making.

To this end, I'm going to start with some definitions:

What are risk premiums? Risk premium is excess returns which results from the payout being correlated with the rest of the world.
What are excess returns? Excess returns are returns higher than 0 (or the risk-free rate depending on context) after accounting for the riskiness of the payout.
Some examples:

a 1-1 bet on a 50/50 coin flip doesn't have excess returns
a 2-1 bet on a 50/50 coin flip has excess returns
a 10% 1y bond on a risky company may or may not have excess returns, depending on the default rate of the company and the recovery rate in event of default
the stock market has excess returns (probably)

What is correlation with the rest of the world?
When an event happens, that says something about the world state we're in as well as the payout we get from the market. If the world state affects our utility in other ways than just the market, that is the correlation we're talking about.
Some examples:

If I bet on a coin flip, aside from the money I have bet on the event, it has no effect on my well being.
If I bet on England to win the Euros, then I have some

...