trentmkelly/LessWrong-43k huggingface dataset:
Good books about overcoming coordination problems?
by Sean Aubin — 2022-02-02T19:09:37.815Z

The questions I mostly frequently ask myself are "How does anything good scale?" and "How is anything good and large-scale sustained?"

The most trivial answer to this is "Markets and incentives!" which is not wrong, but also seems like an underwhelming answer. Markets and incentives do solve many coordination problems and have reduced large amounts of suffering. However, they also seem to have limits in terms of their predictive power. I would like to read books which discuss how cooperation can be leveraged when incentivized markets (free from rent-seeking!) are insufficient.

In "Mission Economy", Marianna Mazzucato claims cooperation flourishes outside of markets when the government needs to explicitly define missions. She uses The Apollo Mission as an example, but this answer felt slightly under-specified. By the end of the book, I couldn't distinguish a good "mission" from a bad one. Even worse, I can't even tell if a mission is being carried out badly, other than ham-fistedly comparing it to The Apollo Mission, even though there's a ton of vague prose spent on the topic.

What books should I read to get a better pragmatic understanding? Is there a “handbook for social change” regardless of scale?

Some candidates I'm currently considering:

The Box: How the Shipping Container Made the World Smaller and the World Economy Bigger by Mark Levinson

A historical book about water treatment, electricity distribution or libraries?

A historical overview of unions. Supposedly unions have driven a number of beneficial labour reforms, but they also can result in stagnation and corruption?
February 2022 Open Thread
by Raemon — 2022-02-02T18:38:28.703Z

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the new Concepts section.

The Open Thread tag is here. The Open Thread sequence is here.
Voting Results for the 2020 Review
by Raemon — 2022-02-02T18:37:32.139Z

Full voting results here. Original 2020 Review announcement
here.

That's it folks! The votes are finalized! The Annual Review of 2020 has come to a close. So ends this yearly tradition that we use to take stock of the progress made on LessWrong, and to provide reward and feedback to the writers and researchers who produced such great works.

Donate to thank the authors (matching funds until Feb 15th 11:59pm)

Speaking of reward and feedback, this year we're doing something new with the Review. Like normal, the LessWrong team will awarding prizes to top posts. But this year we'll be allocating prizes from two different pools of money – the Review Vote pool, and the Unit of Caring pool.

For each pool, the review panel will be using moderator discretion. We'll be ensuring the prizes go to posts which we believe further our cause of developing the art of rationality and intellectual progress. But for the Review Vote prize pool, our judgment will be strongly informed by the results of the vote. For the Unit of Caring prize pool, our judgment will strongly be informed by the opinions expressed by donors who contribute to the prize pool.

For the Review Vote prize, we will allocate $10,000.

For the Unit of Caring prize, we will allocate up to $5000, matching the total amount that other LessWrong users contribute to the pool. (i.e. if LessWrong users donate $4000, the pool will be $8000. If users donate $6000, then the total prize pool will be $11,000).

[Update: the donation period is now over

If you want to donate while signaling support for particular posts, you can do so using the buttons for individual posts further down the page. Here is your opportunity to not just spend internet points, but to actually spend a costly signal of support for the authors and posts you found valuable!

Donations must be made by February 10th to contribute to the matching pool.

EDIT: deadline extended to the end of February 15th

Complete Voting Results (1000+ Karma)

You can see more detailed results, including non-1000+ karma votes, here.

A total of 400 posts were nominated. 121 got at least one review, bringing them into the final voting phase. 211 users cast a total of 2877 votes. Users were asked to vote on posts they thought made a significant intellectual contribution.

Voting is visualized here with dots of varying sizes (roughly indicating that a user thoug

...
How would you go about testing a political theory like Neofeudalism?
by FinalFormal2 — 2022-02-02T17:09:05.354Z

I recently read a book called "The Coming of Neo-Feudalism: A Warning to the Global Middle Class" that speculated that a large drive for democracy was the equalizing power of guns.

The idea is that feudalism was sustainable because the military technology of the time (calvary) allowed for a warrior class to support the nobility, and that modern technologies are once again allowing for this concentration of power. AI and drones are much less democratic forms of weaponry and allow for more complicated controls of the populace like what China is doing with their algorithms.

I read a similar book called "The Dictator's Handbook" which put democracies and autocracies along a spectrum of what percentage of the populace's consent you need to rule, and I also read Robin Hanson's "Elephant in the Brain" which similarly tends to attribute everything to power and self-interest.

What are the limits of this style of "cynical-economic" political theory? Do you think the neo-feudal assessment is accurate?

I think if the neo-feudal theory turned out to be untrue, it would be most likely to be because it's incorrect in its assessment of the democratic leanings of new technologies and not because it misunderstood human psychology.
An Experiment in Habit Changing - 5 Week Retrospective
by Willa — 2022-02-02T17:00:39.047Z

Introduction

I committed internally and externally to following self-created rules for specific parts of my life to change habits and improve my production to consumption ratio. Here's what I did, why I did it, and what I learned after 5 weeks!

Reasoning

My use of time was bad, I had a very unbalanced production vs consumption ratio and spent the majority of my time passively consuming content or otherwise not doing much outside of work hours. Changing habits is hard, so I wanted help and made an external commitment to help with accountability and improve my likelihood of following the rules I created. The goal was and is to increase my deliberate use of time and productivity, with addon benefits of improving my mood, health, and general well-being.

External Help

I filled out lsusr's commitment form and submitted once per week progress reports, this was good accountability and helped me quite a bit with habit changing experiment.

Timespan

3 Months was too long, a few days much too short, so I settled on about 5 weeks, this experiment ran from: 27 December 2021 through 30 January 2022

The Rules, My Goals for Each Rule, and What Actually Happened + My Thoughts

Exceptions

Banned or limited activities are fine if done in-person with friends / family unless excessive; e.g. watching 2 episodes of TV or a movie in-person with others or going to a theater are all fine, but no binge watching. LAN parties and social gaming are fine for the most part, but gaming online even while on comms with friends for more than 3 hours is excessive.

YouTube

Write a minimum 300 word review for each video watched.

Goals

Reduce passive consumption of YouTube content

Improve my categorization of videos and channels, that way I know more readily what videos and types of videos from each channel I watch are more often not-worthwhile product placements, not very fun entertainment, ads, or some combination thereof

Contribute to building my knowledgebase via content from a unique platform that may not be found elsewhere

What Actually Happened

I watched way fewer YouTube videos, and switch to 2x speed for many of the ones that I did watch.

My video consumption became more deliberate and my categorization of videos improved: I would typically quit a video within the first 30 seconds if it didn't seem worthwhile, was just an ad, wasn't very entertaining, or some combination thereof. This

...
Competitive programming with AlphaCode
by Algon — 2022-02-02T16:49:09.443Z
Link: https://deepmind.com/blog/article/Competitive-programming-with-AlphaCode

Note: I'm totally going to alter the article based on the comments. Though of course I'll give credit where it is due.

Edit: Apparently there is a post by one of the authors going in depth on the paper. Here you go if you don't want to read this. He gives more details than the blog, and gives a neat example of one of the questions AlphaCode solves. But he doesn't give a lot of details on what AlphaCode actually is/

Predict the headline result. Go on, I dare you.

Deepmind released a new Transformer model which produced code in response to questions from contests on Codeforce. They simulated participation in 10 recent contests, and produced less than 10 potential solutions for each problem.

Any predictions on how likely people thought this was? Submit your answers below[1].

Elicit Prediction (forecast.elicit.org/binary/questions/YcRTy-tLR)

Elicit Prediction (forecast.elicit.org/binary/questions/BPG9K9fsj)

Elicit Prediction (forecast.elicit.org/binary/questions/gcTIyAIwJ)

And the actual answer is... they ranked within the top 54% of the contestants[2]. I am personally impressed by this. What's more impressive is that they substantially outperform SOTA on prior datasets (3% was SOTA on APPS, AlphaCode got 7%). Oh, they also made a new dataset because apparently the old ones sucked (34% of AlphaCode submissiond were correct). And since they built their model and techniques for the new one, they can't compare the full model to the old, so they had to use a slightly crippled version.

Sidenote: You know what's cool about Alphacode? They show you all the questions Alphacode attempted, and let you see all the changes it made to its code before it submitted it. Click here for a link. They also link to the paper on that page.

They made a new dataset because the old one had too many false positives?

The model was pre-trained on Github code, and finetuned on CodeContests, which was critical to performance. Each contest had newly made problems. Test/validation questions were not present in the training set.

A new dataset (called CodeContests) was created for fine-tuning with a lot more hidden test cases than is normal for for competetive programming datasets[3]. Prior models would often pass test cases whilst being incorrect (30% false positive rate)[4]. Incorrect solutions were...
[Intro to brain-like-AGI safety] 2. “Learning from scratch” in the brain
by Steven Byrnes — 2022-02-02T13:22:58.562Z

(Last revised: October 2024. See changelog at the bottom.)

2.1 Post summary / Table of contents

Part of the

“Intro to brain-like-AGI safety” post series

.

Having introduced the “brain-like-AGI safety” problem in the

previous post

, the next 6 posts (#2–#7) are primarily about neuroscience, building up to a more nuts-and-bolts understanding of what a brain-like AGI might look like (at least, in its safety-relevant aspects).

This post focuses on a concept that I call “learning from scratch”, and I will offer a hypothesized breakdown in which >90% of the human brain (including the cortex) “learns from scratch”, and the other <10% (including the brainstem) doesn’t. This hypothesis is central to how I think the brain works, and hence will be a key prerequisite for the whole rest of the series.

In Section 2.2, I’ll define the concept of “learning from scratch”. As an example, if I assert that the cortex “learns from scratch”, then I’m claiming that the cortex starts out totally useless to the organism—outputting fitness-improving signals no more often than chance—until it starts learning things (within the individual's lifetime). Here are a couple everyday examples of things that “learn from scratch”:

In most deep learning papers, the trained model “learns from scratch”—the model is initialized from random weights, and hence the model outputs are random garbage at first. But during training, the weights are updated, and the model outputs eventually become very useful.
An initially-blank flash drive also “learns from scratch”—you can't pull useful information out of it until after you’ve written information into it.

In Section 2.3, I will clarify some frequent confusions:

“Learning from scratch” is different from “blank slate”, because there is an innate learning algorithm, innate neural architecture, innate hyperparameters, etc.
“Learning from scratch” is different from “nurture-not-nature”, because (1) only some parts of the brain learn from scratch, while other parts don’t, and (2) the learning algorithms are not necessarily learning about the external environment—they could also be learning e.g. how to control one’s own body.
“Learning from scratch” is different from (and more specific than) “brain plasticity”, because the latter can also include (for example) a genetically-hardwired circuit with just one specific adjustable parameter, and that parameter changes semi-

...
Are there alternative routes to variolate COVID instead of the respiratory tract?
by df fd — 2022-02-02T10:45:17.465Z

So we know that there are COVID cases that only have gastrointestinal symptoms [nausea, vomiting, diarrhea]. Presumably, the virus only managed to infect gastrointestinal cells, either due to primary exposure site or innate immunology peculiarities. When these patients recovered we assumed that they are now immune to COVID.
We are primarily concerned about the effect of COVID on the respiratory system. Is it possible for us to give variolation in the form of a small, precise amount of virus in a pill that is only released in the small or large intestine?
Sure, you would have vomiting, diarrhea, fever but it would be much safer than respiratory tract variolation. Dehydration is a risk but it can be fixed much easier than whatever COVID is doing to the respiratory system.
if we are concerned about the virus contamination even then [e.g. when you vomit or fecal contamination], would intra-muscular injection of live virus work? if not why not?
Help me understand the rationale of the NIH to not recommend Fluvoxamine for COVID treatment.
by df fd — 2022-02-02T10:23:36.309Z

So the TOGETHER trial signal boosted by Scott of slate star codex found Fluvoxamine to be effective at reducing 30% of COVID hospitalisation and fatality.

The NIH looked at the study and found it unconvicing, I am a bit confused as to the rationale.

I'll list it out as I understand it:

the primary outcomes [retention in the emergency department for >6 hours or admission to a tertiary hospital] was chosen without rationale
There was no significant difference in mortality between study arms in the intention-to-treat (ITT) population [however, it's 2% in treatment arm and 3% in placebo arm as expected of the 30% reduction expectation]
significant difference was only found in patients who have persisted in taking >80% of fluvoxamine doses, however there were also improved outcome for patients who have persisted in taking >80% of placebo dose, suggesting that another mechanism [e.g. conscientiousness] to be resposible for [most? all?] the improvement in outcome.

Is my understanding correct and does NIH's critiques of the study hold merrit?
Autopilot Ethics And The Illusory Self
by George3d6 — 2022-02-02T10:11:16.215Z
Link: https://www.epistem.ink/p/autopilot-ethics-and-the-illusory

I think most people reading this blog agree that the folk idea of a "self" is bunk. There is no ghost or homunculus unconstrained by physical reality controlling our body, nor are our future or past actions understandable or predictable by our current self.

I assume this seems unimportant to you, it seemed unimportant to me. But when I was trying to steelman Harris' case for harping on so much about free will, I think I accidentally managed to convince myself I was wrong in thinking it so irrelevant a matter.

Among philosophers, a guy like Chalmers or Nagle is actually fairly niche and often cited just for the association. Which are the philosophers’ people actually read, say the one that the vast majority of politicians have read? Well, for example, John Rawls.

Remember Rawls' "original condition," thought experiment? What most people take to be the underpinning of our commonly agreed upon political philosophy?... yeah, you're right, that makes no sense unless you assume you are a magical ghost that's just so happening to control a human body.

And suddenly you realize that a very very large swath of philosophy and thought in general, both modern and historical, might be laboring under this completely nonsensical model of the mind/self that we simply fail to notice.

Since I wrote that, I kept thinking of more and more situations where laboring under this model seems to cause a great amount of confusion.

I think one is people thinking about algorithms used to control things like cars making decisions of ethical importance. The very idea of an autopilot having to make an ethical decision is kind of silly, given how rarely such a situation occurs on the road in which someone must be harmed. But let me grant that there are situations in which an autopilot might be forced to make a split-second decision with ethical import.

Maybe not quite a trolley problem, something more realistic, like breaking quickly enough to endanger the driver or bumping slightly into the car in front, thus damaging it and putting its passengers in slight danger. Or deciding whether to veer off-road into a ditch or hit a child that jumped in the middle of the street. You get the point, take your pick of a semi-reasonable thought experiment where one way or another the autopilot is likely to cause harm to someone

The "real" answer to this question is that nobody has the ability to give a general a...
Impossibility results for unbounded utilities
by paulfchristiano — 2022-02-02T03:52:18.780Z

Some people think that they have unbounded utility functions. This isn't necessarily crazy, but it presents serious challenges to conventional decision theory. I think it probably leads to abandoning probability itself as a representation of uncertainty (or at least any hope of basing decision theory on such probabilities). This may seem like a drastic response, but we are talking about some pretty drastic inconsistencies.

This result is closely related to standard impossibility results in infinite ethics. I assume it has appeared in the philosophy literature, but I couldn't find it in the SEP entry on the St. Petersburg paradox so I'm posting it here. (Even if it's well known, I want something simple to link to.)

(ETA: this argument is extremely similar to Beckstead and Thomas' argument against Recklessness in A paradox for tiny probabilities and enormous values. The main difference is that they use transitivity +"recklessness" to get a contradiction whereas I argue directly from "non-timidity." I also end up violating a dominance principle which seems even more surprising to violate, but at this point it's kind of like splitting hairs. I give a slightly stronger set of arguments in Better impossibility results for unbounded utilities.)

Weak version

We'll think of preferences as relations < over probability distributions over some implicit space of outcomes Ω (and we'll identify outcomes with the constant probability distribution). We'll show that there is no relation < which satisfies three properties: Antisymmetry, Unbounded Utilities, and Dominance.

Note that we assume nothing about the existence of an underlying utility function. We don't even assume that the preference relation is complete or transitive.

The properties

Antisymmetry: It's never the case that both A<B and B<A.

Unbounded Utilities: there is an infinite sequence of outcomes X1,X2,X4,X8,… each "more than twice as good" as the last.[1] More formally, there exists an outcome X0 such that:

X2k<12X2k+1+12X0 for every k.
X0<12X1+12X0 [2]

That is, X1 is not as good as a 12 chance of X2, which is not as good as a 14 chance of X4, which is not as good as a 18 chance of X8... This is nearly the weakest possible version of unbounded utilities.[3]

Dominance:...
Solar Snow Redirector?
by jefftk — 2022-02-02T02:20:05.925Z

Three years ago we put solar panels
on the only part of our roof that got sunlight, which unfortunately
mostly faces WNW. Since then, however, our neighbors have removed a
large tree just east of us, and now the part of our roof that faces
ESE gets lots of light. So I'm back to thinking about solar.

This roof is a good candidate for solar, with one problem:
the panels might avalanche snow onto the neighbor's car:

The standard approach here is to put snow guards on the panels or the roof
with the goal of holding back, slowing down, or breaking up the snow.
In this case, however, I don't want the snow to stay on the roof and I
don't mind it falling next to the house. I just need it not to fall
out from the house, into the neighbor's driveway.

Is this a solved problem? Alternatively, is there something you can
attach to the roof to redirect the snow straight down? Something like:

Alternatively, people who've tried snow guards, how well do they work?
Are they going to keep the snow from squashing the neighbor's car, and
the neighbor?