trentmkelly/LessWrong-43k huggingface dataset:
The Power to Teach Concepts Better
by Liron — 2019-09-23T00:21:55.849Z

This is Part VIII of the Specificity Sequence

When you teach someone a concept, you’re building a structure in their mind by connecting up some of their mental concepts in a certain way. But you have to go in through their ears. It’s kind of like building this ship-in-a-bottle LEGO set.

In this post, we’ll visualize what’s happening in a learner’s brain and see how a teacher can wield their specificity powers to teach concepts better.

Mind-Hanging A Concept

Reading a startup’s pitch begins as a learning exercise: learning what the startup does. In How to Apply to Y Combinator, Paul Graham writes:

We have to read about 100 [applications] a day. That means a YC partner who reads your application will on average have already read 50 that day and have 50 more to go. Yours has to stand out. So you have to be exceptionally clear and concise. Whatever you have to say, give it to us right in the first sentence, in the simplest possible terms. […]

The first question I look at is, “What is your company going to make?” This isn’t the question I care most about, but I look at it first because I need something to hang the application on in my mind.

It’s worth unpacking and visualizing this part of PG’s advice, because we’ll see that the power to mind-hang the concept you’re trying to communicate is closely related to the power of specificity. Stay tuned for that.

First, one more snippet from PG:

The best answers are the most matter of fact. It’s a mistake to use marketing-speak to make your idea sound more exciting. We’re immune to marketing-speak; to us it’s just noise. So don’t begin your answer with something like “We are going to transform the relationship between individuals and information.” That sounds impressive, but it conveys nothing. It could be a description of any technology company. Are you going to build a search engine? Database software? A router? I have no idea. […]

One good trick for describing a project concisely is to explain it as a variant of something the audience already knows. It’s like Wikipedia, but within an organization. It’s like an answering service, but for email.

I recently helped the founder of PierShare craft his YC application. For the “What is your company going to make?...
Five Minute Beans
by jefftk — 2019-09-22T13:10:01.611Z

At the

EA
dinner last night more people showed up than had RSVP'd, and we
ran out of vegan protein. Julia was still running the intro Q&A, so I
made more beans. Someone asked for the recipe, which in this case is
optimized for being fast while still being pretty tasty:

1 can black beans (15.5oz)

~1/4 C cooking wine or other acid

~1/4 C olive oil or other fat

Salt, pepper, garlic powder, and smoked paprika, or other
spices

This is about two people's worth of beans, so multiply quantities up
accordingly.

Put a skillet on the stove on high, and put the wine into it. The
goal is to give it a chance to reduce slightly while you're doing
everything else, though since we're going to be making this very
quickly it's not going to get to do much.

Open the can(s) of beans and drain them. Normally I wouldn't drain
the beans and instead let them reduce, but we're trying to be fast.
Microwave the beans, since that will get them tolerably hot faster
than putting them right in the pan.

Once the beans are good and hot, put them in the pan with the wine.
Stir. It will thicken slightly, but we don't have time to let it
thicken fully. Add the oil and spices to taste. Without enough oil
and salt it is edible but not tasty, so it's more common for people to
use too little than too much.

While it would benefit from a bit more time to cook together and
combine flavors it's still very good at this point and something I was
happy to serve.

(The combination of skillet and microwave is also what I do with two-minute quesadillas. More
cooking thoughts here.)
Timer Toxicities
by Zvi — 2019-09-22T12:10:00.701Z

Follow-up to: Free-to-Play Games: Three Key Trade-Offs

The central free-to-play mechanic is to ration action and resources via real world time. This leads to two of the three key trade offs. Players are prevented from having fun because they are time restricted, either unable to play or unable to have the resources to play the way they would like, allowing the game to sell a solution to these problems. More perniciously, players become trained to constantly check in with the game in order to claim rewards and keep their resources from becoming idle. This can warp a person’s life more than one would think, changing behavior to allow timely access, and preventing focus on other subjects.

This obsession effect, and the ability of real world time delays to be an interesting resource to include in trade-offs, have also caused these mechanics to seep into non-free games, especially RPGs.

Resource rationing takes the form of timers. The form of the timer does a lot to determine how toxic the rationing will be to the player. There are several knobs one can turn.

The Knobs

Many of these knobs represent related aspects, and thus are closely intertwined, but listing them out still seems useful. In each case, moving towards the first named end will reduce toxicity.

Steady versus Sudden: If a resource accumulates over time, does the resource accumulate gradually with no limit, gradually up to a limit, or all at once?

Fixed versus Delayed: Does the resource replenish at a fixed time, or at a time after it is used?

Slow versus Rapid: How frequently must one check-in to maximize results?

Batched versus Disjoint: Are there multiple timers running simultaneously? If so, how hard are they to line up?

Tracked versus Lost: How easy is it to track when accumulation is complete?

Forgiving versus Punishing: How punishing is it to fail to check in?

Isolated versus Cumulative: Are there cumulative rewards for reliably checking in? How important are they?

Progressive versus Competitive: Are you in competition with others? If so, what kind?

Queued versus Idle: Can you give orders in advance?

Explained versus Mysterious: Can you tell what matters?

Annoying versus Limiting: Are the things you get from timers the long-term limiting factor in your ability to do things?

Incidental versus Central: Is timing the key determinant of your success?

Knob 1: Steady versus Sudden

If you have a resour...
Taxing investment income is complicated
by paulfchristiano — 2019-09-22T01:30:01.242Z

How should a state tax investment income if it wants to maximize its citizens’ welfare? This sounds like a simple question but I find it surprisingly hard to think about. Here are some of the positions I’ve moved through over the last few years:

Taxing investment has distortionary effects, but we should have non-zero investment taxes for the same reasons we want a broad tax base in general—doubling the size of a tax quadruples its social cost, so it’s better to have lots of small taxes rather than a few big taxes. Whether we should have lower taxes on investment income or ordinary income mostly depends on whether taxes have a bigger impact on decisions about working or about saving.

Using invest income taxes to fund redistribution only makes sense if they are disproportionately paid by richer people. But if they are, then by exactly the same token they will also reduce the incentives to become rich—in fact they have just as large a disincentive effect as an equally-progressive income tax. And that’s on top of disincentivizing and distorting saving. So we should just raise income taxes instead.

We can decompose investment income into two parts—interested earned by capital at the risk-free rate, and an excess return from taking on risk (or illiquidity, or whatever). Taxing the risk-free returns seems bad, but taxing excess returns seems like it’s almost a free lunch: it reduces an investor’s losses as well as their gains, so they can just lever up their investments to offset the effect of taxes. The net effect is similar to implementing a large sovereign wealth fund, with the government and private investors splitting both the risk and the returns from productive investments. It’s probably preferable to a sovereign wealth fund, because it doesn’t require the state to literally own huge quantities of assets. Since risk compensation is the large majority of investment returns, it seems like capital gains taxes are good on balance.

All of the risk incurred by a sovereign wealth fund or capital gains tax is ultimately passed back on to citizens, whether in the form of taxes or reduced public services or inflation. So a capital gains tax or sovereign wealth fund is actually only a good idea if citizens are systematically under-exposed to market risk. But most people invest relatively conservatively. So in fact these policies can mostly be justified as either corre

...
Meetups: Climbing uphill, flowing downhill, and the Uncanny Summit
by Raemon — 2019-09-21T22:48:56.004Z

Epistemic status: rough partial-model formed from... well, basically a single conversation, but which resonated with several years of vague accumulated impressions. Curious if it resonates or anti-resonates with others.

In the comments of Mingyuan's "Meetups as Institutions for Intellectual Progress", Bendini notes:

The current format is not ideal for accomplishing anything, so much so that I've stepped down from organising mine because they were providing so little value. It's a sad state of affairs, but from what I can tell the majority are content with them being low-effort social groups.

In my experience this is fairly common – most people who show up to meetups are just sorta there to hangout. If you get 20 people, maybe 3 of them will be driven to proactively "do stuff."

I was chatting with a friend recently who said [paraphrased] "Honestly I'd rather go to a social gathering where I can just hang out than one where we're doing a bunch of effortful stuff... but I enjoy those meetups more when they're filled with the sort of people who do do effortful stuff (even if they're not right now). They're more... alive and interesting."

And this matched a couple years experience running the NY meetups – it seemed like what most people wanted, most of the time, was fun social gatherings.

On any given meetup night, most of the value was not in "the presentation on Bayes Theorem" or whatever that gathered people together – it was the chatting afterwards and talking about whatever else was interesting. If you ask someone "would you rather just skip to the part where we hang out and chat, rather than do a workshop or listen to a presentation", many people would say "yes."

Nonetheless... in eras when the NYC meetup swerved towards "mostly social hangouts", attendance dropped, and the vibrance faded. There's a lot to do in NYC and "just hang out" isn't an exciting enough option to compete with many alternatives (including "rousing yourself out of your house in the first place", which requires activation energy).

And I suspect this might be fairly common... at any given level of "amount of effortful stuff you do." [1]

"Hanging out" is a valuable, important part of a healthy lifestyle...
Welcome to Rational Ottawa
by Thewal — 2019-09-21T04:32:30.817Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
What are the differences between all the iterative/recursive approaches to AI alignment?
by riceissa — 2019-09-21T02:09:13.410Z

I have been trying to understand all the iterative/recursive approaches to AI alignment. The approaches I am aware of are:

ALBA (I get the vague impression that this has been superseded by iterated amplification, so I haven't looked into it)

HCH (weak and strong)

Iterated amplification (IDA)

Debate

Meta-execution

(Recursive) reward modeling

Factored cognition

Factored evaluation

(I think that some of these, like HCH, aren't strictly speaking an approach to AI alignment, but they are still iterative/recursive things discussed in the context of AI alignment, so I want to better understand them.)

One way of phrasing what I am trying to do is to come up with a "minimal set" of parameters/dimensions along which to compare these different approaches, so that I can take a basic template, then set the parameters to obtain each of the above approaches as an instance.

Here are the parameters/dimensions that I have come up with so far:

capability of agents: I think in HCH, the agents are human-level. In the other approaches, my understanding is that the capability of the agents increases as more and more rounds of amplification/distillation take place.

allowed communication: It seems like weak and strong HCH differ in the kind of communication that is allowed between the assistants (with strong HCH allowing more flexible communication). Within IDA, there is low bandwidth vs high bandwidth oversight, which seems like a similar parameter. I'm not sure what the other approaches allow.

training method during distillation step: I think IDA leaves the training method flexible. According to this post, factored cognition seems to use imitation learning and factored evaluation seems to use reinforcement learning. I think recursive reward modeling also uses reinforcement learning. HCH seems to be just about the amplification step (?), so no training method is used. I'm not sure about the others.

entity who "splits the questions", coordinates everything during amplification, or selects the branches: In factored cognition, factored evaluation, IDA, and HCH, it seems like the human splits the questions. In Debate, the branches are chosen by the two AIs in the debate (who are in an adversarial relationship).

entity who does the evaluation/gives feedback ("the overseer"): It seems like in factored evaluation, the human gives feedback. In Debate, the final judgment is provided by the

...
Finding Cruxes
by Logan Riggs — 2019-09-20T23:54:47.532Z

This is the third post in the Arguing Well sequence, but it can be understood on its own. This post is influenced by double crux, is that your true rejection, and this one really nice street epistemology guy.

The Problem

Al the atheist tells you, “I don’t believe in the Bible, I mean, there’s no way they could fit all the animals on the boat”.

So you sit Al down and give the most convincing argument on it indeed being perfectly plausible, answering every counterpoint he could throw at you, and providing mountains of evidence in favor. After a few days you actually managed to convince Al that it’s plausible. Triumphantly you say, “So you believe in the Bible now, right?”

Al replies, “Oh no, there’s no evidence that a great flood even happened on Earth”

"..."

Sometimes when you ask someone why they believe something, they’ll give you a fake reason. They’ll do this without even realizing they gave you fake reason! Instead of wasting time arguing points that would never end up convincing them, you can discuss their cruxes.

Before going too deep, here’s a shortcut: ask “if this fact wasn’t true, would you still be just as sure about your belief?”

Ex. 1: Al the atheist tells you, “I don’t believe in the Bible, I mean, there’s no way they could fit all the animals on the boat”

Instead of the wasted argument from before, I’d ask, “If I somehow convinced that there was a perfectly plausible way to fit all the animals on the boat, would you believe in the Bible?” (This is effective, but there’s something subtly wrong with it discussed later. Can you guess it?).

Ex. 2: It’s a historical fact that Jesus existed and died on the cross. Josephus and other historical writers wrote about it and they weren’t Christians!

If you didn’t know about those sources, would you still be just as sure that Jesus existed?

General Frame

A crux is an important reason for believing a claim. Everything else doesn’t really carry any weight. How would you generalize/frame the common problem in the above two examples? You have 3 minutes

Using the frame of probability theory, each crux would have a percent of the reason why you believe that claim. For example, say I’m very sure (95%) my friend Bob

...
Reframing Impact
by TurnTrout — 2019-09-20T19:03:27.898Z

​

​

​

Technical Appendix:

First

safeguard?

This sequence is written to be broadly accessible, although perhaps its focus on capable AI systems assumes familiarity with basic arguments for the importance of AI alignment. The technical appendices are an exception, targeting the technically inclined.

Why do I claim that an impact measure would be "the

first

proposed safeguard which maybe actually stops a powerful agent with an imperfect objective from ruining things – without assuming anything about the objective"?

The safeguard proposal shouldn't have to say "and here we solve this opaque, hard problem, and then it works". If we have the impact measure, we have the math, and then we have the code.

So what about:

Quantilizers? This seems to be the most plausible alternative; mild optimization and impact measurement share many properties. But

What happens if the agent is already powerful? A greater proportion of plans could be catastrophic, since the agent is in a better position to cause them.
Where does the base distribution come from (opaque, hard problem?), and how do we know it's safe to sample from?

In the linked paper, Jessica Taylor suggests the idea of learning a human distribution over actions – how robustly would we need to learn this distribution? How numerous are catastrophic plans, and what is a catastrophe, defined without reference to our values in particular? (That definition requires understanding impact!)

Value learning? But

We only want this if our (human) values are learned!

Value learning is impossible without assumptions, and getting good enough assumptions could be really hard. If we don't know if we can get value learning / reward specification right, we'd like safeguards which don't fail because value learning goes wrong. The point of a safeguard is that it can catch you if the main thing falls through; if the safeguard fails because the main thing does, that's pointless.

Corrigibility? At present, I'm excited about this property because I suspect it has a simple core principle. But

Even if the system is responsive to correction (and non-manipulative, and whatever other properties we associate with corrigibility), what if we become unable to correct it as a result of early actions (if the agent "moves too quickly", so to speak)?

Paul Christiano's take on corrigibility is much broader and an exception to this critique.

What is the core principle?

Notes

The

...
Synthesis of subagents: exercise
by Julija Kobrinovich — 2019-09-20T17:24:21.433Z

What topics is the exercise related to:

1. Mitigation of cognitive distortions such as the stereotype of physical attractiveness

2. Sequence "Multiagent models of mind", article “Why Subagents?”

3. article "Being the (Pareto) Best in the World"

Two sources are the basis for this exercise:

- the idea (and practices) of Western esoterics on “biners neutralization” (in particular, quite widely and extensively it is described in the book “The Great Arcana of the Tarot” by Vladimir Shmakov, I don’t know if the book is translated into English)

- the method for developing the theme in screenwriting (in particular, “Substance, Style and the Principles of Screenwriting” by Robert McKee)

Here, I describe the exercise as I practice it – a cross between the two initial variants, and intended for self-development.

Two words about what esoterics has to do with it:

1. The academic study of esoterics - one of the areas of my interests.

2. If we consider the Western esoteric tradition from a psychological point of view, there are many methods and techniques aimed at “convergence” and “neutralization” of opposing personality traits/emotions/ ideas. It is enough to recall the alchemical procedures and visualizations, where the elements of fire and water “merge” together. It seems to me that in this body of knowledge, there is a lot of useful and undeservedly forgotten things.

By the way, is it rational to have prejudices against “esoterics”? :) And do you have it?

I use the exercise in two versions.

The purpose of the

first one

:

remove the severity of the confrontation between the opposing ideas or subagents
if possible - to achieve their synthesis.

It is like a Double Crux. With the difference that:

exercise is purely intellectual, without focusing technic. It does not require you to merge completely with the role and feelings of the investigated subagent
unlike the Double Crux, the opposing subagents do not ‘negotiate,’ but merge, becoming the expression of some new value that they ‘did not understand’ before.
it reduces the initial “weight “of any arguments of this subagent (both” for “and” against”).

I use the same version of the exercise to mitigate cognitive biases – such as, for example, the physical attractiveness stereotype; good mood in sunny weather and a bad one in rainy weather; distortions of self-esteem.

I don’t know how well it works for everyone. I tested it only on ...
Everyday belief in diminishing returns is resistant to diminishing returns
by aaq — 2019-09-20T13:44:13.597Z

Epistemic status: A mental piton, to hang your climbing rope on

The second half hour of cardio you do in a day matters less to your overall happiness and health than the first half hour you do, the second quarter-hour, the first quarter-hour; knitting this into our base mental framework is why the law of diminishing returns is such a "Wow!" moment for a lot of beginner econ students.

The problem with consistently applying the law of diminishing returns to everyday life, however, is that it's boring. People don't become socially-renowned Zen masters through 1 pomodoro of samadhi a day. All they get are improvements to their overall mood and the occasional boring state of pure pleasure and equanimity. People don't become [in]famous computer hackers by spending 2, not-especially-well-planned hours a week learning how to automate the boring stuff with Python; they just learn enough to be a little dangerous.

It doesn't feel like you're using a glitch in the Matrix disguised as a psuedoscientific law. But that's exactly what you're doing.

Understand: Most people do not have the discipline to do just 15 minutes of something. They have to do 0, or 45. This is your actual competition.

You can routinely beat the average, in almost every everyday domain, by applying the law of diminishing returns to your own life ruthlessly and consistently. Or perhaps I should say, ruthlessly and consistently enough?