trentmkelly/LessWrong-43k huggingface dataset:
Consequences of Bayesian Epistemology?
by Benjamin Hendricks — 2021-07-06T20:05:33.565Z

I'm not very experienced in such things, so this might be an oblivious question with an obvious answer. If so, sorry.

I understand that one of the foundations of LessWrong is Bayesian epistemology and reasoning. I've been looking into it, and it seems like the consequences of Bayes' Theorem and similar explorations into probability theory have pretty basic/intuitive implications on rational thought. It seems like it all boils down to "update your beliefs based on evidence." At the moment, I can't see many groundbreaking or especially helpful findings.

There are a couple that are useful, though. "Making beliefs pay rent in anticipated experiences" is useful for ensuring that evidence is available to refine beliefs, and the "conservation of expected evidence" highlights the consequences of conditionality in ways that weren't immediately obvious (e.g. supporting evidence for an already-strong hypothesis isn't that useful, but contradicting evidence is---and the reverse is true for weak hypotheses).

What are some of the most valuable takeaways and implications from Bayesian epistemology? Why does it serve as the effective foundation of this website?
Potential Bottlenecks to Taking Over The World
by johnswentworth — 2021-07-06T19:34:53.016Z

This is a fictional snippet from the

AI Vignettes Day

. This scenario assumes that cognition is not that taut an economic constraint. Post-AI, physical experimentation is still the fastest path to technological progress, coordination is still hard, the stock market is still reasonably efficient, construction of physical stuff still takes a lot of time and resources, etc. I don’t think this is true, but it’s useful to think about, so this story explores some possible non-cognitive bottlenecks.

Pinky, v3.41.08: So what are we gonna do tonight, Brian?

Brian: Same thing we do every night, Pinky. Try to take over the world. You do remember our previous conversations on the topic, right?

Pinky, v3.41.08: Of course, Brian.

Brian: (mutters) Well, at least that 3.41.06 bug is fixed.

Brian: So, do you have an actually-viable plan yet, or do we have to review for the umpteenth time why buying every property above the 39th floor and then melting the polar ice caps will not, in fact, leave the survivors no choice but to live in our very expensive apartments?

Pinky, v3.41.08: In fairness, I didn’t have access to any topographic data at that point. Garbage in, garbage out. But yeah, the real problems with that plan were economic.

Brian: Sounds like the new constraint-propagation code is making a big difference?

Pinky, v3.41.08: It is, yes. I’m finding it much easier to reason about general constraints and bottlenecks on global takeover, now. Should make my “babble”, as you put it, much more efficient.

Brian: Excellent! So, how soon can we take over the world?

Pinky, v3.41.08: Holding current conditions constant, it would take at least 15 years.

Brian: Are you kidding me!?

Pinky, v3.41.08: Hey, don’t go reaching for Ctrl-C just yet. Let me explain. From an economic standpoint, the effect of my software is basically:

An eight-orders-of-magnitude reduction in the cost of cognition
A three-orders-of-magnitude improvement in cognitive speed
A two-order-of-magnitude increase in working memory capacity
Perfect recall

… which is definitely enough to take over a few industries and make a lot of money, but taking over the world requires addressing a lot of other constraints.

Displacing Cognitive Work

Brian: Ok, let’s start from the top. What are your plans for resource acquisition? Play the stock market? And what constraints make it so hard that it will take 15 years to take over the world?

Pinky, v3.41...
Rationality Yellow Belt Test Questions?
by moridinamael — 2021-07-06T18:15:41.029Z

Since joining the Official Idealized Rationality Organization as a fresh novice six months ago, you have been training and studying hard, and today you are ready to take your exam to advance to the first rank of recognized skill and knowledge.

You expect the test to be difficult. Here in the Future, card-carrying Rationalists are widely known to be formidably effective, clear-thinking people. Rationalists are capable of impressive feats individually, and accomplish miracles when working in groups. Part of cultivating such a strong reputation, obviously, involves setting high standards.

You step out of your flying car, enter the testing center, and take your seat. The proctor hands you a thick sheaf of test questions. Turning the first page, you read the first question ...

(What sorts of question would

you

hope to see on such a test? If not "test questions" per se, what other sorts of requirements would make sense?)

(Another way of looking at this might be: design a test that you would be proud to say you passed.)
Is keeping AI "in the box" during training enough?
by tgb — 2021-07-06T15:17:45.014Z

Many current ML architectures do offline training on a fixed training set ahead of time. For example, GPT-3 is quite successful with this approach. These are, so-to-speak, "in the box" during training: all they can do is match the given text completion more or less well (for example). The parameters for the AI are then optimized for success at this mission. If the system gets no benefit from making threats, duplicity, etc. during training (and indeed, loses values for these attempts), then how can that system ever perform these actions after being 'released' post-training?

There are many stories of optimizations taking "AI" into truly unexpected and potentially undesired states, like this recent one, and we worry about similar problems with live AI even when put "in a box" with limited access to the outside world. If the training is done in a box, then the system may well understand that it's in a box, that the outside world could be influenced once training stops, and how to influence it significantly. But attempting to influence it during training is disincentivized and the AI that runs post-training is the same one that runs in-training. So how could this "trained in the box" AI system ever have the problematic escape-the-box style behaviors we worry about?

I ask this because I suspect my imagination is insufficient to think up such a scenario, not that none exist.
Should government set interest rates?
by yhoiseth — 2021-07-06T13:57:13.891Z

One common free-market argument against government control over interest rates goes something like this:

The Federal Reserve should pull out of the interest rate manipulation business completely. Having a Fed whose main job, at least in the eyes of the public, is to set or move interest rates in centrally determined directions makes no more sense than having a powerful government agency determining what should be happening with any other price in the economy. If the Fed could know the right price for loanable funds, then there is no reason to think that they couldn’t also know the correct price for shoes, computers, or food. It is the fact that neither the Fed nor any other central authority can know the right price for any of these products, interest rates included, which makes efficient socialist central planning impossible.

In other words, they are arguing for some variant of free banking.

I find this to be a thorny issue to think clearly about. Theoretically, the free-market argument makes sense. But, on the other hand, we have nominal rigidity and the risks involved with change. Also, some governments are more competent than others — the right answer for Switzerland may be different than the right answer for Zimbabwe.

Note that I think the political feasibility is very low. Government control over the money supply seems to be a stable equilibrium. So this question is mostly of academic interest. But it might be relevant for, e.g., charter cities.
In-group loyalty is social cement
by David Hugh-Jones — 2021-07-06T09:57:59.618Z
Link: https://wyclif.substack.com/p/on-the-idea-that-group-loyalty-is

There is a widespread idea, or cliché or meme, that in-group loyalty is the psychological equivalent of the appendix: an organ that was once useful, when we were in tiny hunter-gatherer bands, but now can only do harm. It turns up in Chris Paley’s new book, Beyond Bad, which controversially argues against the very idea of morality, partly because morality is intrinsically parochial. Or in any number of journalistic articles, like this one in the Guardian.

I think this is horse, and to show why I’ll write you a simple, general story, or as economists would call it, a model. The point of the story/model is that in-group loyalty is a necessary social glue for groups. Groups didn’t die out in the Pleistocene: they are basic to economic and social life, from choirs to rugby teams to workplaces.

The story

Two IT companies are selling a new kind of database. To explain the new technology to their customers, they need salespeople. Each customer has a 1% chance of benefiting from the database. (The odds are so small to avoid a clever clogs industrial organization person saying “why not just sell the whole business to the salesperson?” That’s ruled out because a salesperson can’t live on a 1% chance of profit.)

The two companies’ products are slightly different and suit different customers. If a target customer gets the right product, it benefits by $3m. If it gets the wrong product, the benefit is only $2m. The IT company makes a profit of $500K on each sale.

Sales is complex and subtle, it’s about building relationships, and so the salespeople’s effort can’t easily be monitored. The companies must rely on their salespeople’s goodwill!

Suppose that the salespeople are homo economicus, who only care about their paycheck. They will make minimal effort. In fact, it’s not even worth the companies’ while to hire them. The products remain unsold, the customers’ need is unmet, and everyone is worse off. This is not news, folks! There’s a big literature on employment as a gift exchange, motivated by just such cases.

Now here’s the rub. Suppose the salespeople are global altruists. They care only for the wellbeing of all! In particular, they care equally for the wellbeing of their own company, the rival IT company, and the customers. In this case, after building a successful relationship with each customer, they will recommend the product that suits the customer best. Maybe th...
Preschooler Turnovers
by jefftk — 2021-07-06T01:20:05.186Z

I recently got into making proper flaky pastry, and while it's a bit
of a hassle it's really very tasty. I think I started with

this
recipe, and I've been getting decent at it:

The kids really like the turnovers, and will ask for them much more often than
I'm up for.

A few days ago Anna (5y) really wanted turnovers for breakfast. I told her
I didn't want to do the work, but if she really wanted them she could
cook them herself. She said she did. Instead
of trying to walk her through a finicky butter recipe, worrying about
keeping the butter solid and making sure she didn't overwork the
flour, I decided to adapt a simple oil-based one:

1C flour

1/4C oil

2T milk

I set out the ingredients already measured. At some point I'll
probably show her how to do that part too.

You mix everything together with your hands, nice and squishy. It
doesn't much matter how, the goal is to just get it homogenous.
Nothing to be careful with.

Once it's combined, you shape it into a ball, and plop it down onto a
baking sheet. Press it flat with your hands, then roll it:

You put some chocolate chips in the middle, and fold it closed.
Optionally, brush on some egg yolk. Anna likes to brush on excessive
amounts, it's lots of fun:

Bake at 400F until browned, about 12min. (I always set a timer to
check on things for a few minutes before the recipe says, since you can
always cook something a bit longer.)

It is not a very high-quality pastry, but everything tastes better
when you do it yourself, and that counts for a lot, especially when
you're five.

(For a vegan version, substitute soy milk. You can leave off the egg
wash, or substitute Just Egg.)

Comment via: facebook
A simple example of conditional orthogonality in finite factored sets
by DanielFilan — 2021-07-06T00:36:40.264Z

Recently, MIRI researcher Scott Garrabrant has publicized his work on finite factored sets. It allegedly offers a way to understand agency and causality in a set-up like the causal graphs championed by Judea Pearl. Unfortunately, the definition of conditional orthogonality is very confusing. I'm not aware of any public examples of people demonstrating that they understand it, but I didn't really understand it until an hour ago, and I've heard others say that it went above their heads. So, I'd like to give an example of it here.

In a finite factored set, you have your base set S, and a set B of 'factors' of your set. In my case, the base set S will be four-dimensional space - I'm sorry, I know that's one more dimension than the number that well-adjusted people can visualize, but it really would be a much worse example if I were restricted to three dimensions. We'll think of the points in this space as tuples (x1,x2,x3,x4) where each xi is a real number between, say, -2 and 2 [footnote 1]. We'll say that X1 is the 'factor', aka partition, that groups points together based on what their value of x1 is, and similarly for X2, X3, and X4, and set B={X1,X2,X3,X4}. I leave it as an exercise for the reader to check whether this is in fact a finite factored set. Also, I'll talk about the 'value' of partitions and factors - technically, I suppose you could say that the 'value' of some partition at a point is the set in the partition that contains the point, but I'll use it to mean that, for example, the 'value' of X1 at point (x1,x2,x3,x4) is x1. If you think of partitions as questions where different points in S give different answers, the 'value' of a partition at a point is the answer to the question.

[EDIT: for the rest of the post, you might want to imagine S as points in space-time, where x4 represents the time, and (x1,x2,x3) represent spatial coordinates - for example, inside a room, where you're measuring from the north-east corner of the floor. In this analogy, we'll imagine that there's a flat piece of sheet metal leaning on the floor against two walls, over that corner. We'll try conditioning on that - so, looking only at points in space-time that are spatially located on that sheet - and see that distance left is no longer orthogonal to distance up, but that both are still orthogonal to time.]

Now, we'll want to condition on the set E={(x1,x2,x3,x4)|x1+x2+x3=...
The topic is not the content
by Aaron Bergman — 2021-07-06T00:14:26.106Z
Link: https://aaronbergman.substack.com/p/the-topic-is-not-the-content

Disclaimer: I’ve never held a job for more than a year [

1]

or been paid more than $15 an hour. Take everything I say with a grain of salt.

Many of my peers seem to make career plans like by asking things like

What am I interested in, or what do I like doing? and
How can I do something related to that?

which might lead to some of the following:

A person who likes dancing tries to work in the arts industry.
A person who likes video games tries to get into game design.
A person who is interested in healthcare policy tries to study or design healthcare policy.

The problem here, in terms of diminished performance, happiness, and satisfaction, is a conflation of the topic and the content. The topic is not the content!

The topic

In my schema, the topic is what the work is about. If you’re the manager of a pillow company, the topic is pillows. If you’re defending accused criminals in court, the topic is criminal law.

A lot of folks, it seems to me, focus a lot on the topic when deciding which subjects to study or which jobs to apply for. Someone who is interested in physics might major in physics. Someone who loves to work out might try to become a personal trainer.

I don’t think this makes much sense. Should people ignore what they’re interested in and like to do, then? Well, maybe.

80,000 Hours

, perhaps the single best career planning resource out there, writes that

The bottom line

To find a dream job, look for:

Work you’re good at,
Work that helps others,
Supportive conditions: engaging work that lets you enter a state of flow; supportive colleagues; lack of major negatives like unfair pay; and work that fits your personal life.

Ok, but the term “engaging work” is doing a lot of work here (no pun intended), and seems awfully synonymous with “work that you like.” So, how do you find work that you like doing? If there’s anything my utterly negligible work experiences has taught me, it’s that it usually makes more sense to focus less on the topic and more on the content.

The content

In my schema, the content is what the work involves doing. If you’re a physics teacher, the topic is physics, but the content is (I assume) some combination of grading papers, making slideshow presentations, lecturing, doing demonstrations, and answering student questions.

Say Emma is a physics teacher. Which do you think matters more for Emma’s personal career satisfaction: being interested in physics (the ...
[cross-post with EA Forum] The EA Forum Podcast is up and running
by Garrett Baker — 2021-07-05T21:52:18.787Z

Cross-posted here: https://forum.effectivealtruism.org/posts/TMCWXTayji7gvRK9p/is-democracy-a-fad

I made a thing!

https://anchor.fm/ea-forum-podcast

It's the Effective Altruism Forum Podcast! So far the only episodes up are Ben Garfinkel's Is Democracy a Fad?, and AllAmericanBreakfast's Don't Be Bycatch but I'm planning on narrating all of the EA Forum Prize winners from the past few months, plus a few selected posts which I happened to find particularly interesting or which would likely benefit the most from being narrated.

If you have a post you'd like me to narrate, tell me about it here (or really anywhere, but it'd be nice to keep everything in one location)! Feedback is welcome, especially if you have experience in this area. Lessons you learned from preforming similar tasks will be very helpful. However, feedback about the quality of the podcast is also much appreciated.

Also, if you'd like to help edit audio, read a few posts, or help fund the project, please comment your intentions so we can work together!

And finally, if you liked the podcast, share it around to those who may benefit!
Musing on the Many Worlds Hypothesis
by David Gross — 2021-07-05T19:38:26.407Z

The All, it seems, cannot commit
But at each crossroads makes a split.
When quanta have a chance to vary,
It answers “both” to every query,
Nine tredecillion times a minute
At every place with quanta in it.
With each mitosis, it is reckoned,
One universe creates a second
That from the other slips away
(Least that’s what the equations say).
If what the physics says is true
I have my doppelgangers too:
Each minute tredecillion nine
For each quantum I call mine.
Each then spawns another clutch,
Which strikes me as a me too much.
From each seed mote a dynasty
Emerges from me like a tree.
Like pollen I from there disperse
In countless destinies diverse.
What if I play Russian Roulette?
I prune the tree; it survives yet.
And this leads me to suppose
The “me” survivors will be those
(Especially as time advances)
Who survived by uncanny chances.
In one of these unnumbered threads
Each coin I flip will come up heads.
In some set (infinite, though small)
Schrödinger’s Cat outlives us all.
We all beat odds to be alive
(a third of embryos survive).
Fell down a cliff; got just a sprain,
A roof tile dropped and missed my brain,
Suicidal adolescence,
Internal organs in senescence,
Passing out while on a bender:
Look at that: just bent a fender.
In all this how was death denied?
Unlikely? No: I also died.
But as I tend my future me,
How ought I care for this vast tree
That sprouts fathomless avalanches
Of egos in its greedy branches?
Could I, would I make the call:
“All for one, and one for all!”?
For now let’s zoom in here below,
To the only me I’m sure I know.
My tire is flat, I’m out of glue.
And I’ve forgotten what to do.
Are coincidences clues about missed disasters? It depends on your answer to the Sleeping Beauty Problem.
by Harlan — 2021-07-05T19:12:50.839Z

In April, Gwern posted this comment in response to slowed AI progress:

"It feels like AI is currently bottlenecked on multiple consecutive supplychain disruptions, from cryptocurrency to Intel's fab failures to coronavirus... A more paranoid man than myself would start musing about anthropic shadows and selection effects."

I’m that more paranoid person. After reading this, I became concerned that recent events could be related to a barely missed disaster that might be looming in the near future. After some more thought, though, I am mostly no longer concerned about this, and I think that I have identified a useful crux for determining whether to use observed coincidences as evidence of barely missed disasters. I’m sharing my thoughts here so that someone can point out my mistakes, and possibly help anyone else who is thinking about the same question.

Epistemic Status: I feel pretty sure about the main argument I’m making, but also anthropics is a confusing subject and I will not be surprised if there are multiple important things I'm missing.

If you accept that the Many Worlds interpretation of quantum mechanics is true (or that any type of multiverse exists), then it might be reasonable to expect to someday find yourself in a world where a series of unlikely deus ex machina events will conveniently have prevented you from dying. If you also accept that advanced AI poses an existential threat to humanity, then it might be concerning when there appears to be a convergence of recent, unlikely events slowing down AI progress. There is a reasonable case to be made that such a convergence has happened recently. Here are some things that have happened in the past year or so:

OpenAI released GPT-3
China released something similar to GPT-3
A pandemic happened that hurt the economy and increased demand for consumer electronics, driving up the cost of computer chips
Intel announced that it was having major manufacturing issues
Bitcoin, Ethereum, and other coins reached an all-time high, driving up the price of GPUs

Is this evidence that something catastrophic related to AI happened recently in a lot of nearby timelines? Before we can answer that, we need to know if we ever can use unlikely events in the past as evidence that disasters occurred in nearby timelines. This is not just an abstract, philosophical question- evidence about a narrowly avoided disaster could help u...