trentmkelly/LessWrong-43k huggingface dataset:
Notes on Moderation, Balance, & Harmony
by David Gross — 2020-12-25T02:44:55.342Z

This post examines the virtues of moderation, balance, and harmony. I wrote this not as an expert, but as someone who wants to learn more. I hope it helps people who want to know more about these virtues and how to nurture them.

I also use this opportunity to go on a (related) tangent about the VIA Institute on Character, and its questionable focus on playing to character strengths over strengthening character weaknesses.

What are these virtues?

If you practice these virtues, you respect that each part of your life is meant to serve the whole of it, not the other way around.

Even with generally beneficial or neutral things, it’s possible to take them too far. Moderation means you are on guard against this. Take fitness, for example: How can anyone complain about being more fit? But a compulsive focus on fitness as an end in itself can make you forget the point of being fit. “People who are always taking care of their health are like misers, who are hoarding a treasure which they have never spirit enough to enjoy.”[1]

If you are immoderate, you risk becoming a fanatic, an obsessive, a monomaniac. Such things in themselves might be mere eccentricities. But they are often accompanied by harmful neglect of other facets of life. Balance means that you have a good perspective on your life as a whole, and that you use this perspective to attend to the points that are most in need of attention, rather than letting habit or lack of moderation constrain your effort to where it’s less helpful. Work/life balance, balance between focus on self and others, and balance between living-in-the-present, learning-from-the-past, and preparing-for-the-future are some common challenges.

Harmony is closely related to integrity. Pursuing the musical analogy, when your life is harmonious, the different parts of it are attuned to each other; they are different instruments but playing the same symphony. Harmony is also not mere melody or monotone: being a “Johnny One-Note” isn’t where it’s at. To everything there is a season, as the song goes.

Some related virtues include temperance (moderation in pursuit of pleasure), conciliation / cooperation / compromise (social harmony advanced through moderation in the pursuit of one’s priorities), perspective, and flexibility.

Avoiding burnout by knowing how to pace yourself is one application of these virtues.

Aristotle made moderation a keystone of his...
A few predictions
by Tim Liptrot — 2020-12-25T01:27:17.917Z

Will the new, more infectious UK strain impact virus reproduction in the US?

A new Covid-19 strain is rapidly expanding in the UK and has increased reproduction. Several pieces of evidence all point toward the new strain increasing virus reprodcution by 50-80%. My favorite piece of evidence is the regression of new vs. old proportion on number of weekly cases, seen here

Among 40 local authorities in East and South East England with more than five VOC
samples there is a significant trend of increasing reported cases with increasing
frequency of N501Y (Figure 1, weighted linear regression p=10-6). A 10% difference in
VOC frequency in mid-November corresponds to approximately 50 more weekly cases
per 100 thousand in early December. Local authorities with few VOCsamples have
similar reported cases as the rest of the UK (linear regression intercept = 137 cases per
100k versus UK median 130.4 per 100k).

Question

My question asks - will the new strain migrate to the US and produce an increase in reproduction? I will use the same regression method to answer. Specifically, will a regression between districts show a statistically and practically significant difference in cases associated with the new strain proportion. Statistical significance is p =.05. Practical significance means an association of a 1% change in proportion with an increase of 1% x mean cases per person. For example, if the model assigns all-old-strain districts one case per thousand, an all-new-strain district should have two per thousand. This is much smaller than the UK currently observes.

Null if no studies ever done.

Negative if media reports new strain still not in US on February 28th.

Probablity

The probability that the strain does not reach the US is very low, may .05%. If the strain gets here I expect a similar result to the UK. If anything the US has a weaker control mechanism. OTOH, model uncertainty is huge.

Conclusion: 80%

Note: This is just a calibration exercise for me.
What are intuitive ways for presenting certainty/confidence in continuous variable inferences (i.e. numerical predictions)?
by George3d6 — 2020-12-25T00:55:39.451Z

I enjoy the idea of thinking in probabilities, and also, as part of my job, I have to deal with the hellish task of figuring out how to present probabilities to people in a simple way.

It seems fairly easy to reason about probabilities for discrete events, there's a whole apparatus of reasoning regarding binary outcomes that's more or less "made it into the zeitgeist" at this point.

For > 2 discrete outcomes you can pretty much reason in terms of accuracy + balanced accuracy + (in specific cases) a custom-weighted accuracy.

Again, these seem intuitive to me for two reasons:

you can present them to intelligent people that aren't too familiar with mathematics overall and they "click" fairly quickly.
they are simple enough that our monkey brains can use them, to some extent, in day-to-day reasoning

On the other hand, I can't find a solution that's as intuitive for numerical predictions, the two most obvious are:

r2 score, which is a "scientific" context will usually make sense, to some extent, under certain assumptions about the distribution of your data
a "percentage accuracy" score (e.g. abs(inferred - observed)/observed ) which indicates the error as some percentage of the original value... however this gets really tricky when trying to sum up more than one observation and in many situations, it seems to scale oddly.
a "difference accuracy" score (e.g. abs(inferred - observed) )

Even worst, given all those 3 scores combined, you're still left with many problem types for which they are unrepresentative.

Granted, one can throw 20 other error measures at the problem and paint a basically clear picture, but then the "understandable" bit is lost. Even an r2 score is not really obvious unless you work in specific domains. The "percentage" and "difference" based accuracy scores, on the other hand, behave horribly chaotic even with seemingly small inferential differences and can often be completely misleading even when taken together ... hence why I assume, they are never used ... but they are the easiest to explain to epople.

The alternative I currently have is always having confidence intervals, and thinking about accuracy as being bound by those. But that just tosses you into a whole different problem space regarding figuring out the acceptable distance between the inferential bonds.

Obviously, there's no solution here, even the combination of balanced + unbalanced accurac...
A breakdown of priors and posteriors - an example from medicine
by masasin — 2020-12-24T23:35:15.461Z
Link: https://old.reddit.com/r/UpliftingNews/comments/7v99xo

This post is lifted (with edits) from my comment on a thread on Reddit that talked about a blood test that detected amyloid plaque, a potential precursor to Alzheimers. A top-level commenter felt that the fact that the test couldn't tell for certain whether someone had Alzheimers meant that it was useless.

"Not everyone with amyloid in their brains will turn out to have dementia, and not everyone who has dementia will be found to have amyloid in their brains."

So what is the point of this article again? To get a load of clicks I assume, cos there's no real content here.

I attempted to correct that misconception.

(If anyone knows how to do the whole thing with odds ratios rather than probabilities, please let me know! We can't update as normal here.)

tl;dr: No test is ever 100% certain, but it can still constrain your probabilities, making you more certain than you were before. Not everyone who is outside during a thunderstorm gets hit by lightning, and not everyone who gets hit by lightning was outside during a thunderstorm. Going outside in a thunderstorm is still much more dangerous than staying inside.

Notation: P(dementia|amyloid) is read as the probability of having dementia given that you have amyloid. To make things fit in a LessWrong post, I'll abbreviate having dementia to D, and having amyloid to A. A positive test result is +, and a negative test result is −. Finally, a negation is ¬, so ¬A would be no amyloid.

O(D:¬D), read as the odds of dementia to no dementia, is the odds ratio that D is true compared to the odds ratio that D is false. O(D:¬D)=3:1 means that it's 3 times as likely that somebody has dementia than that they don't. In general, odds doesn't say anything about the magnitude of the probabilities, so they could be small, like 3% and 1%, or big, like 60% and 20%. (Here, of course, P(D)+P(¬D)=1 because the two choices represent all possibilities, so the probabilities have to be 75% and 25% respectively.)

Imagine you have ten thousand people in the right demographics. How about these (the numbers are made up):

Dementia status
Has amyloid
Does not have amyloid
Total

Has dementia
1000
300
1300

Does not have dementia
100
8600
8700

Total
1100
8900
10000

So, a person chosen at random from that population has a P(D)=13% chance of having dementia, and a P(¬D)=87% chance of not having dementia. Their odds ratio for dementia i...
Defusing AGI Danger
by Mark Xu — 2020-12-24T22:58:18.802Z

This represents thinking about AGI safety done under mentorship by Evan Hubinger. Thanks also to Buck Shlegeris, Noa Nabeshima, Thomas Kwa, Sydney Von Arx and Jack Ryan for helpful discussion and comments.

tl;dr

A common perspective to take with respect to AI safety to try and construct a story in which artificial general intelligence (AGI) goes right. I have recently acquired another perspective that I find helpful, which is AGI safety as disrupting stories for why AGI will go poorly. I will start by presenting an analogy to convey intuition, then apply this perspective to concrete examples of current alignment strategies.

Introduction

Suppose you have a bomb that is about to explode. Your task is to make sure no one gets hurt. How might you reason about how to do this?

One way to do this is backward chaining, or backchaining, which involves identifying a sub-goal that, if achieved, will accomplish your main goal. This process is then recursively applied to generate a concrete next action.

Applied to the situation at hand, I might identify that defusing the bomb would prevent everyone from getting hurt. However, I do not know how to defuse a bomb. A sub-goal that would help me defuse a bomb is learning about bombs, which might suggest that I take a class on bomb-defusal. This is not helpful because I do not have time to take such a class. In general, backchaining might fail when there are many possible ways to accomplish a goal because it might cause you to fixate on one possible path.

Suppose that instead I ask “why do I think the bomb explosion will go poorly?”. The answer I get seems much more useful: the bomb will throw lots of little bits everywhere and those bits will hit people. This gives me two possible subquestions to ask: “why will the bomb throw bits everywhere?” and “why will those bits hit people?”. Answering the second question: the people are close by and there is nothing stopping the bits. Again, this yields two possible recursive subquestions. Notice how this framing forces me to consider many possible paths towards solving my goal simultaneously.

The point of the second perspective is to construct a specific story for why the bomb explosion will hurt people. The goal is to eventually arrive at something that looks like “the timer will hit zero, which will activate this diode, allowing electricity to flow from the battery into the explosive. T...
Operationalizing compatibility with strategy-stealing
by evhub — 2020-12-24T22:36:28.870Z

Thanks to Noa Nabeshima and Kate Woolverton for helpful comments and feedback.

Defining optimization power

One of Eliezer's old posts which I think has stood the test of time the best is his “Measuring Optimization Power.” In it, Eliezer defines optimization power as follows.[1] Let A be some action space and p be some probability measure over actions. Then, for some utility function U:A→R and particular action a∗∈A, Eliezer defines the bits of optimization power in a∗ as

optp,U(a∗)=−log2∫{a∈A | U(a)≥U(a∗)}p(a)da

which, intuitively, is the number of times that you have to cut the space in half before you get an action as good according to U as a∗.

In my opinion, however, a better, more intuitive version of the above definition can be obtained by using quantilizers. A q-quantilizer relative to some utility function U and base distribution over actions p is a system which randomly selects an action from the top q fraction of actions from p sorted by U. Thus, a 0.1-quantilizer selects actions randomly from the top 10% of actions according to U. Intuitively, you can think about this procedure as being basically equivalent to randomly sampling 1q actions from p and picking the best according to U.

Now, using quantilizers, we can give a nice definition of optimization power for an entire model. That is, given a model M, let q∗∈(0,1) be the smallest fraction[2] such that a q∗-quantilizer with base distribution p is at least as good[3] at satisfying U as M. Then, let optp,U(M)=−log2q∗. What's nice about this is that it gives us a measure of optimization power for a whole model and a nice intuitive picture of what it would look like for a model to have that much optimization power—it would look like a q∗-quantilizer.

Both of these definitions do still leave the distribution p unspecified, but if we want a very general notion of optimization power then I would say that p should probably be some sort of universal prior such that simple policies are weighted more heavily than their more complex counterparts. If we use the universal prior, we get the nice property that the more complex the policy needed to optimize some utility function, the more optimization power is needed. Thus, we can replace optp,U with just optU where p is assumed to be some universal prior.

Compatibility with strategy-stealing

Now, given such a definition of optimization power, I think we can...
The medical test paradox: Can redesigning Bayes rule help?
by habryka — 2020-12-24T22:08:30.602Z
Link: https://www.youtube.com/watch?v=lG4VkPoG3ko

A great coverage of one of my favorite Youtuber's of the (on LessWrong) classical reformulation of Bayes rule in terms of odds instead of probabilities. I expect to link a bunch of people to this, in particular if they get stuck reading Eliezer's intuitive explanation of Bayes.
Epistemology Volume of "A Map That Reflects the Territory" Set - My Personal Commentary
by hamnox — 2020-12-24T21:34:33.135Z

Intro

This post compiles my personal comments on the Epistemology book from the Lesswrong 2018 bookset. I put almost no effort into making it legible or relevant to anyone else.

A sketch of good communication

For research, we want to exchange deep models
For logistics team, okay to err on side of comparing shallower nodes just to come to agreement.
Has implications for #sbub . Centralized and federated systems look an awful lot like the bad case of communication. The only way to converse is to straight up adopt the output. If I want #sbub to empower people trying to deeply understand the world, there has to be a lot of space just for interacting with other people's models and drawing intricate connections to your own.

Babble and Prune Sequence

Babble and Prune

Currently I need to tune down my prune step and increase my babble. A lot. #habits
#sbub in my current imagination is extremely babble and prune. Some concretizations include:
Computers are capable of producing babble. That's an idea for a feature right there: integrated gpt-2.

Run another Lesswrong poll. Seed statements from babble and prune article. Seed claims of which the reader uses and how well this serves them, what others use and need. Seed claims of good sources of babble, good algorithms of prune, good combinations.

ask LW team about make a post with the convo embedded
make a link post, send delayed invitations to answer and comment
advise "if you notice a hint of confusion or not-rightness, sit with it for a moment. Say a short sentence out loud expressing some aspect of the intuition, write it down and submit it if you will."
provide a submission form for seed statement ideas, things you don't automatically want to vote in favor of.

More Babble

Akjash is a graph theorist, good to know
Scrabble babble as an example of the kind of play I want to do with my language generation project.
Advice in this article seems geared from an assumption of the brain as an implicit graph, namely that individual edges and neighborhoods can be computed in reasonable time. This assumption may not hold for me. I really struggle to generate and re-compact local planning. Expansion may be related? Author clarifies this is not the same thing as connectivity, and doesn't clarify which actions in their example address which metric. I find myself trying to capture any babble I have 2 steps ahead
Better if this essay had offered its

...
TAPs for Tutoring
by Mark Xu — 2020-12-24T20:46:50.034Z

Introduction

Scott Alexander:

Architects. Teachers. Teachers of teachers, but the art of teaching teaching is much the same as the art of teaching. Three levels is enough. Though the levels have to mix. The teacher who trains the next architect must be a master both of teaching and of architecture.

Paraphrased for my own purposes:

Tutors. Tutees. Tutors tutor tutees, but the art of being taught is much the same as the art of teaching. The levels have to mix. The tutor who tutors the tutee must be a master both of teaching and of learning.

Much has been written about the Feynman Technique as a method of learning that involves teaching. The main problem with the Feynman technique is that it takes too much time to ever actually execute in practice. Instead, I will describe two TAPs that are my 80/20 for both learning and teaching.

TAPs

TAP 1: Examples

Trigger: Notice you made a claim with some amount of vagueness.

Action: Start saying "For example, ..."

For example (see what I'm doing), I might say "regular exercise is good for you" and notice that "regular" and "good" are words with multiple possible interpretations, so I would follow it up with "For example, walking for 10 minutes a day makes you happier". Another example: I might say "functions map inputs to outputs, for example, the square function takes numbers and maps them to the square of that number, e.g. it would take 2 to 4, 3 to 9, etc." Notice that sometimes your example itself can have some vagueness, so the TAP fires recursively. This is the key TAP for teaching.

There is also a dual TAP for learning; trigger: notice someone else made a claim with some amount of vagueness, action: say "for example?". For example, someone might tell me that they "dislike mundane activities", to which I would say "for example?" They then might tell me that "scheduling Facebook events" is a mundane activity, giving me a much better understanding of what they had in mind when they said "mundane". This TAP is especially helpful because your interlocutor is likely to give you a particularly representative example. For example (this TAP is strong), if I asked for an example of a bird, the person I'm asking is much more likely to say "robin" than "penguin".

Asking for examples is extremely powerful because it encourages specificity. It is much easier to reason about The Feynman Lectures on Physics than the abstra...
The First Sample Gives the Most Information
by Mark Xu — 2020-12-24T20:39:04.936Z
Link: https://markxu.com/first-sample

I originally heard this point made by Ben Pace in Episode 126 of the Bayesian Conspiracy Podcast. Ben claimed that he learned this from the book How to Measure Anything, but I think I identified the relevant section, and this point wasn't made explicitly.

Suppose that I came up to you and asked you for a 90% confidence interval for the weight of a wazlot. I'm guessing you would not really know where to start. However, suppose that I randomly sampled a wazlot and told you it weighed 142 grams. I'm guessing you would now have a much better idea of your 90% confidence interval (although you still wouldn't have that good a guess at the width).

In general, if you are very ignorant about something, the first instance of that thing will tell you what domain you're operating in. If you have no idea how much something weighs, knowing the weight tells you the reasonable orders of magnitude are. Things that sometimes weigh 142 grams don't typically also sometimes weigh 12 solar masses. Similarly, things that take 5 minutes don't typically also take 5 days, and things that are 5 cm long aren't typically also 5 km long.

For more abstract concepts, having a single sample allows you to locate the concept in concept space by anchoring it to thing space. "Redness" cannot be properly understood until it is known that "apples are red". "Functions" are incomprehensible until you know "adding one to a number" is a function. "Resources" are vague until you learn that "money is a resource".

In reality, the first sample often gives you more information than a random sample. If I ask a friend for an example of a snack, they're not going to randomly sample a snack and tell me about it; they're probably going to pick a snack that is at the center of the space of all snacks, like potato chips.

From an information-theoretic perspective, the expected amount of information gained from the first sample must be the highest. If the sampling process is independently and identically distributed, the 2nd sample is expected to be more predictable given knowledge of the first sample. There is some chance that the first sample is misleading, but the probability that it's misleading goes down the more misleading the sample is, so you don't expect the first sample to be misleading. If you're very ignorant, your best guess for the mean of a distribution is pretty close to the mean of the samples you have,...