trentmkelly/LessWrong-43k huggingface dataset:
Gravity Turn
by alkjash — 2021-08-16T19:20:06.748Z
Link: https://radimentary.wordpress.com/2021/08/16/gravity-turn/

[The first in a sequence of retrospective essays on my five years in math graduate school.]

My favorite analogy for graduate school is the gravity turn: the maneuver a rocket performs to get from the launch pad to orbit. I like to imagine a first-year graduate student as a Falcon X rocket, newly-constructed and tasked with delivering a six-ton payload into low Earth orbit.

Picture this: you begin graduate school, fresh as a rocket arriving at Cape Canaveral and bubbling with excitement for your maiden voyage. Your PhD adviser, on the other hand, is the Hubble Space Telescope. Let's call her Dr. Hubble (not to be confused with the astronomer of the same name). Dr. Hubble is ostensibly the ideal guide for your first orbit insertion. After all, she is famously good at staying in orbit - she’s been up there since 1990.

But problems quickly arise as you probe Dr. Hubble for advice on how to approach the launch. Namely:

She left Earth more than thirty years ago, and space technology has since been completely revolutionized.
She states all advice at an extremely high level with birds-eye-view detachment, observing, as she is, from a vantage point a thousand miles overhead.
Most fatally, the Hubble Space Telescope vessel does not include the lower-stage rockets that brought her into space. In fact, she doesn’t include large engines of any kind. Her thirty years of experience free-falling in orbit will do you very little good until you break out of the stratosphere.

The problem is even worse than this, however. It is not that Dr. Hubble, despite her best intentions, gives outdated advice. It is not even that Dr. Hubble cannot consciously articulate all the illegible skills she’s reflexively performing to stay in orbit. The problem is that even if you could perfectly imitate what Dr. Hubble is doing right now, you would likely still crash and burn.

What I didn’t understand going into graduate school is that academic mathematicians are often working in a state akin to the free-fall of orbit. The Hubble Space Telescope remains in orbit around Earth because it travels horizontally so quickly that, even as it’s continuously accelerating towards the Earth, it continually misses. The laws of physics have arranged it so that it is not possible - barring deliberate sabotage - for her to fall back into a sub-orbital trajectory.

Similarly, a successful research professor is ...
Framing Practicum: Dynamic Equilibrium
by johnswentworth — 2021-08-16T18:52:00.632Z

This is a

framing practicum

post. We’ll talk about what dynamic equilibrium is, how to recognize dynamic equilibrium in the wild, and what questions to ask when you find it. Then, we’ll have a challenge to apply the idea.

Today’s challenge: come up with 3 examples of dynamic equilibrium which do not resemble any you’ve seen before. They don’t need to be good, they don’t need to be useful, they just need to be novel (to you).

Expected time: ~15-30 minutes at most, including the Bonus Exercise.

What’s Dynamic Equilibrium?

Our picture of stable equilibrium was a marble sitting at the bottom of a bowl. This is a static equilibrium: the system’s state doesn’t change.

Picture instead a box full of air molecules, all bouncing around all over the place. The system’s state constantly changes as the molecules move around. But if we look at the box at large scale, we don’t really care about the motions of individual molecules, we just care about the overall distribution of molecule positions and velocities - the number of molecules in the upper-left quadrant, for instance. And this distribution may be (approximately) constant, even though the individual molecules are bouncing around.

This is dynamic equilibrium: even though the states of the system’s components are constantly changing, the distribution of states of the system’s components has a stable equilibrium.

The box-of-air example involves a distribution of similar physical parts (i.e. molecules), but we can also have a dynamic equilibrium with a Bayesian belief-distribution. For instance, I can think about which of my dishes I expect to be dirty tomorrow or next week or next month. I don’t run the dishwasher every day, so in the short term I expect dirty dishes to pile up - a non-equilibrium expectation. But in the long run, I generally expect the distribution of dirty dishes to be roughly steady - I don’t know exactly which days I’ll wash them, but my expectations for 100 days from now are basically the same as my expectations for 101 days from now.

The dirty dishes themselves may pile up and then be cleaned and then pile up again, never reaching an equilibrium state. But my expectations or forecasts about the dishes do reach an equilibrium.

What To Look For

In general, dynamic equilibrium should spring to mind in two situations:

A system has parts which are constantly changing/moving, but some aggregate summary of

...
Power vs Precision
by abramdemski — 2021-08-16T18:34:42.287Z

I've written a couple of posts (1,2) about General Semantics and precision (IE, non-equivocation). These posts basically argue for developing superpowers of precision. In addition, I suspect there are far more pro-precision posts on LessWrong than the opposite (although most of these would be about specific examples, rather than general calls for precision, so it's hard to search for this to confirm).

A comment on my more recent general-semantics-inspired posts applied the law of equal and opposite advice, pointing out that there are many advantages to abstraction/vagueness as well; so, why should we favor precision? At least, we should chesterton-fence the default level of precision before we assume that turning the dial up higher is better.

I totally agree that there are advantages to fuzzy clusters as well; what I actually think is that we should level up both specificity-superpowers and vagueness-superpowers. However, it does seem to me like there's something special about specificity. I think I would prefer a variant of LessWrong which valued specificity more to one which valued it less.

To possibly-somewhat explain why this might be, I want to back up a bit. I've been saying "specificity" or "precision", but what I actually mean is specificity/precision in language and thought. However, I think it will be useful to think about physical precision as an analogy.

Physical Power vs Precision

Power comes from muscles, bones, lungs, and hearts. Precision comes from eyes, brains, and fingers. (Speaking broadly.)

Probably the most broadly important determiner of what you can accomplish in the physical world is power, by which I mean the amount of physical force you can bring to bear. This determines whether you can move the rock, open the jar, etc. In the animal kingdom, physical power determines who will win a fight.

However, precision is also incredibly important, especially for humans. Precision starts out being very important for the use of projectiles, a type of attack which is not unique to humans, but nearly so. It then becomes critical for crafting and tool use.

Precision is about creating very specific configurations of matter, often with very little force (large amounts of force are often harder to exercise precision with).

We could say that power increases the range of accessible states (physical configurations which we could realize), while prec...
Metaforecast update: Better search, capture functionality, more platforms.
by NunoSempere — 2021-08-16T18:31:08.932Z

Metaforecast

is a search tool for probabilities. Since the

last public update

, we have made several improvements.

tl;dr:

Search is much better, also faster
Capture functionality now makes it more convenient to save predictions as images.
There is now a database open to the public.
Questions now have quality indicators like number of forecasters, number of forecasts, volume traded, liquidity, etc.
Added new platforms: Kalshi, Betfair, Rootclaim and CoupCast
Website initial load is faster

The rest of the post outlines the details of the above improvements. However, we expect that most people will just find it more fun to

try it out

or to see all Metaforecast COVID predictions in a LessWrong post.

Details

Capture forecasts to use in blog posts

By clicking on "Capture" and then on "Capture image and generate code", you can create an image out of any question, and save it on imgur. The Markdown and HTML snippets to display the captured image—which include a link back to the original question—are automatically generated.

This makes it easier to add questions to documents, e.g., LessWrong posts. This is equivalent to just taking a screenshot, but perhaps faster and more convenient, so people might actually use it to add probabilities to their own posts.

Imgur has rate limits for each IP of around 50 images per hour, which sounds reasonable. Note that these are per user, rather than for Metaforecast as a whole, so feel free to experiment with it.

The alternative to this would be a "true embed", where question images are generated on the fly by a server, and updated with the passage of time as the crowd forecast changes. This would have some advantages, but would require further effort.

Much better search

Initially, Metaforecast used a custom search script on top of

Fuse.js

, an open source fuzzy-search library. This was simple to implement, but resulted in a search that was fairly slow and suboptimal. We switched to Algolia, which has built in support for synonyms, plurals, removing stop words, and generally some very nice default search capabilities. It also does some indexing in the background so that searches for more complex queries aren't noticeably slower.

Personally, I'm sad to move to a non-open source option, but Algolia is very noticeably better. It's also free for under 10k records, which Metaforecast fits under. ElasticSearch is an open source alternative, but it has a re...
All Metaforecast COVID predictions
by NunoSempere — 2021-08-16T18:30:36.851Z

These are taken from Metaforecast, and are provided without comment. To do this yourself, make a search query on Metaforecast, then click on "Capture" on the top right corner, then on "Capture image and generate code".

My hope is that after reading this post, other people might do interesting analytical work on top of these predictions, or, more likely, use Metaforecast to capture predictions and include them in their posts when analyzing other topics.
1h-volunteers needed for a small AI Safety-related research project 
by PabloAMC — 2021-08-16T17:53:12.866Z

Hi everyone!

I'm currently doing an AI Safety internship where we're carrying out a research study about "multimodal models: usefulness and safety". In this project, we want to analyse how people can directly use language models (and multimodal models in the future) to help them solve everyday tasks. This is conducted by researchers from the Centre for the Study of Existential Risk (CSER), the Leverhulme Centre for the Future of Intelligence (CFI), the University Complutense de Madrid (UCM), the Psychometrics Centre at the Cambridge Judge Business School (JBS) and the Valencian Institute for AI Research (VRAIN-UPV).

To understand this question better, we need volunteers to complete one or two questionnaires during the next two weeks. If you want to help us please register yourself in https://forms.gle/ZgVppnt2xWsEBNJd9 and we will promptly send you further instructions.

Thanks a lot!
Some phrases in The Map that... Confuse me- help please, to make my review of it better!
by William Gasarch — 2021-08-16T16:56:47.520Z

I recently read

A Map that Reflect the Territory: Essays by the Less Wrong Community

I will be posting a review of it on the lesswrong site soon.

But I want to improve it first and need your help!

There were some words or phrases that were used that either confused me
or I could not find a definition on the web anywhere! So please clarify if you can.

-------------------------------
In the article

What Motivated Rescuers During the Holocaust? by Martin Sustrik,

is the following:

As I already said, I am not an expert on the topic, but if what we see
here is not an instance of the bystander effect, I'll eat my hat.

He is referring to that people who begin helping one Jew escape the Nazi's
end up helping more.

The phrase Bystander Effect is on the web! A lot!
It seems to be that

the more people that are bystanders who could prevent something bad from
happenings the less likely someone really will.

This seems different from how its used in the essay.
(Side note- I couldn't seem to find this essay on the less wrong website.)

-----------------------
Yed graphs.
In the article

What Makes People Intellectually Active? by Abram Demski

https://www.lesswrong.com/posts/XYYyzgyuRH5rFN64K/what-makes-people-intellectually-active

is the following:

I might write one day on topics that interest me, and have sprawling Yed graphs
in which I'm trying to make sense of confusing and conflicting evidence.

What are Yed Graphs?

----------------------
In the article

What Makes People Intellectually Active? by Abram Demski

https://www.lesswrong.com/posts/XYYyzgyuRH5rFN64K/what-makes-people-intellectually-active

is the following:

Its like the only rationality technique is TAPs, and you only set up taps of the form
``resemblance to rationality concept'' IMPLIES ``think of rationality concept''.

What is TAP?
Zen and Rationality: Equanimity
by Gordon Seidoh Worley — 2021-08-16T16:51:02.116Z

This is post 8/? about the intersection of my decades of LW-style rationality practice and my several years of Zen practice.

In today's installment, I look at equanimity from a rationalist perspective.

In Zen in particular and various Buddhist lineages in general there's a lot of talk about equanimity. Sometimes it refers to a particular meditative state, other times to a more general virtue of meeting the world on equal footing with no particular preconceptions about what one will believe.

It has a so-called near enemy, though, which is indifference, and this is central to the straw stereotype of the advanced Buddhist practitioner you've likely encountered. Think of the unflappable monk who continues to meditate while the building burns down around them, or more likely the idea that through meditation you should become a person who can suppress their emotional responses and never smile, frown, laugh, or cry no matter what the world throws at you. I don't know much about stoicism, but indifference is the same kind of thing people seem to mean when they say that a person is stoic.

Equanimity is something quite different. From the outside it might produce behaviors you could interpret as indifference, especially in training environments like a meditation center or monastery, but in fact it's something more nuanced than that, and something rationalists are quite familiar with.

In everyday language we might say equanimity is about being open to the possibility that the world is just as it is rather than how you think it is, and that whatever thoughts and beliefs you form should not be formed on the basis of what you wish were true but instead on the basis of what you actually observe.

If we focus only on beliefs, equanimity is about being a good Bayesian reasoner, updating fluidly and proportionately in response to evidence. The only thing to watch out for here is that we often model Bayesian reasoning in toy environments where physical reality is constrained to match the model, and Bayesian updating out in the real world includes a lot more uncertainty about the correctness of the fundamental assumptions the model makes than you'll encounter in worked examples.

Thus it may not surprise you that we find equanimity in the Litany of Tarski:

If the box contains a diamond,
I desire to believe that the box contains a diamond;
If the box does not contain a diamond,
I desire to b

...
Why must plausibilities be represented using real numbers?
by  — 2021-08-16T16:31:51.516Z

Professor Jaynes writes:

Page 17 of his book

Page 656 of his book

Why must degrees of plausibility be represented using real numbers when there are alternative number systems, which don’t have 1-to-1 correspondence to the reals, that satisfy transitivity and universal comparability?

Kevin S. Van Horn writes in his paper, “Constructing a Logic of Plausible Inference: A Guide to Cox’s Theorem”:

Page 10 of the paper

What’s so special about R? If we want our theory to have no holes then why don’t we use hyperreals or complex numbers? They also satisfy transitivity and universal comparability and I would argue that they have less holes than R
Is it worth making a database for moral predictions?
by Jonas Hallgren — 2021-08-16T14:51:54.609Z

Epistemic status: Confused about what probabilities people assert to different alignment theories working.

I essentially want to know whether it would be worth spending some of my time working on conceiving a database like https://www.moralmachine.net/ with more complexity as a benchmark for moral predictions in future ai systems. It, of course, highly depends on if it would be useful. According to my own probability distribution of potential future alignment structures, something like a complex moral database would seem to be worth it. First and foremost, alignment testing will be useful, and you would need a moral database for this. Secondly, it could be used as a groundwork for a friendly AI.

I've had some different ideas about how to construct it, and its usefulness will depend on its design. The basic architecture I have in mind is the following: Start with a basic scenario such as the trolley problem (moral machines) with the number of people and happiness among others as variables and add a Bayesian reasoner that gets the input of a situation where all variables are pseudo-randomized according to an "importance" algorithm described below. (Maybe one can get an average happiness vs. population ratio as a numerical "solution" to The Repugnant Conclusion using this?) The Bayesian reasoner is then asked what confidence they have to predict the current situation, and if this confidence is under a certain percentage, we send out an inquiry about the situation it currently is in. We also assign an "importance" rating to each problem, mainly determined by the number of lives affected. This "importance" rating would, in turn, determine the confidence required and the priority ranking of that situation being lifted to the system's attention. This is akin to a sorting system where the most frequently used items are put on top, but it's the most "important" situation instead. This should allow the moral database to focus on the most "important" decisions that we have and, therefore, be more efficient than random sampling.

The biggest problem with human sampling is our plentitude of cognitive biases, and that's usually where the concept of CEV comes in. The first approach would then be to say, "fuck that, I'll let the programmers do the heavy lifting," and just go ahead and create a moral database with all problems of cognitive biases existent within it. This, howeve...
Modelling Transformative AI Risks (MTAIR) Project: Introduction
by Davidmanheim — 2021-08-16T07:12:22.277Z

Numerous books, articles, and blog posts have laid out reasons to think that AI might pose catastrophic or existential risks for the future of humanity. However, these reasons often differ from each other both in details and in main conceptual arguments, and other researchers have questioned or disputed many of the key assumptions and arguments.

The disputes and associated discussions can often become quite long and complex, and they can involve many different arguments, counter-arguments, sub-arguments, implicit assumptions, and references to other discussions or debated positions. Many of the relevant debates and hypotheses are also subtly related to each other.

Two years ago, Ben Cottier and Rohin Shah created a

hypothesis map

, shown below, which provided a useful starting point for untangling and clarifying some of these interrelated hypotheses and disputes.

The MTAIR project is an attempt to build on this earlier work by including additional hypotheses, debates, and uncertainties, and by including more recent research. We are also attempting to convert Cottier and Shah’s informal diagram style into a quantitative model that can incorporate explicit probability estimates, measures of uncertainty, relevant data, and other quantitative factors or analysis, in a way that might be useful for planning or decision-making purposes.

Cottier and Shah's 2019 Hypothesis Map for AI Alignment

This post is the first in a series which presents our preliminary outputs from this project, along with some of our plans going forward. Although the project is still a work in progress, we believe that we are now at a stage where we can productively engage the community, both to contribute to the relevant discourse and to solicit feedback, critiques, and suggestions.

This introductory post gives a brief conceptual overview of our approach and a high-level walkthrough of the hypothesis map that we have developed. Subsequent posts will go into much more detail on different parts of this model. We are primarily interested in feedback on the portions of the model that we are presenting in detail. In the final posts of this sequence we will describe some of our plans going forward.

Conceptual Approach

There are two primary parts to the MTAIR project. The first part, which is still ongoing, involves creating a qualitative map (“model”) of key hypotheses, cruxes, and relationships, as describ...
Mildly against COVID risk budgets
by Optimization Process — 2021-08-16T05:53:27.066Z

A friend is hosting a party tonight! It'll cost me 200 microCOVIDs, which, as a healthy thirty-something, I very cautiously estimate to cost about 2 micromorts,[1] which is roughly equivalent to 1 hour out of my remaining life expectancy. But I'm super excited for this party; I'd happily burn an hour of my life driving there and back. Should I go?

Unless something subtle is going on... obviously yes, right?

"What about your visit to that coffee shop earlier today? Don't you think that's decision-relevant?"

The coffee shop? I'm confused. Why would that be decision-relevant?

"Well, you accumulated about 150 microCOVIDs there."

Sure, but that's not really relevant to the decision theory of whether to attend the party, is it? The party is equally enjoyable either way, and the costs of multiple COVID exposures add pretty much linearly by the axiom of independence, so previously-incurred risks are irrelevant to my future decisions. (If the coffee shop had been a week ago, sure, I'd be inflicting some of those microCOVIDs on my fellow partygoers, which, sure, could be decision-relevant, I haven't done the math; but it seems very unlikely to me that I'll become a full-fledged germ factory between this morning and this evening, so I think that consideration is insignificant in this case.)

"But you try to maintain a 200-microCOVID-per-week risk budget, don't you?"

Sure, but... hmm.

"So there's something wrong with your assertion that previously-incurred risks are irrelevant to future decisions."

...or something is wrong with the idea of risk budgeting.

"...hmm."

...so, which is it?

Zero externalities

Maybe the answer will be clearer if we simplify the problem: let's get rid of externalities. Suppose I am the world's most boring superhero, Captain Can't-Transmit-COVID. I can still catch it just like anybody else, but the cost is borne by me alone, not my friends or housemates.

Now is my coffee-shop visit decision-relevant for whether I should attend the party?

I can't think of a single reason it would be.[2] This strongly suggests that "risk budgeting" has something to do with the risk of transmitting COVID to other people.

Externalities to an enthusiastic optimizer

Let's add an externality back in: suppose I can only transmit COVID to my partner. Perhaps risk budgets will fall naturally out of the arrangements we make.

Me: So. My best estimate for partner-to-vaccinated-partner tr

...