trentmkelly/LessWrong-43k huggingface dataset:
What is the connection between these two definitions of ascription universality?
by Liam Donovan — 2019-10-27T18:03:20.369Z

Definition A:

a system is ascription universal if, relative to our current epistemic state, its explicit beliefs contain just as much information as any other way of ascribing beliefs to it.

Definition B:

a family of question-answering systems A[·] are ascription universal (w.r.t. 𝔼 and L) if A[C] epistemically dominates C for every computation C.

This definition is more technical, but Christiano describes the underlying intuition as follows:

"ascription universality,” which tries to capture the property that a question-answering system A is better-informed than any particular simpler computation C.

I'm totally failing to see the connection between these two definitions.

A few things I find especially confusing:

1. Definition A seems to measure the transparency/honesty of a single system, whereas definition B seems to measure the knowledge of one system relative to another. Why are these concepts related?

2. Wouldn't an automatic photosensitive streetlight be "ascription universal" by definition A? The explicit way of assigning beliefs to the streetlight "if the light is on, the system believes it's dark outside; otherwise the system believes it's light outside". From our epistemic standpoint, there's no way to ascribe other beliefs to it that contain more information than the explicit beliefs, so it appears to satisfy the criterion. However, I don't think a streetlight has any of the nice properties you would want an "ascription universal" system to have (in particular, it wouldn't be an effective overseer in IDA...)

Edit: After thinking about this some more, my best guess is that definition A is meant to describe the epistemically dominated computation from definition B. I think a computation satisfying definition A would be epistemically dominated by our current epistemic state (though I'm not sure if that's a coherent thing to say given that epistemic dominance is also defined relative to our current epistemic state). Similarly, if a particular computation C satisfies definition A, then an overseer that can understand A's explicit beliefs can (I think??) epistemically dominate C.

If this line of thinking is correct, I'm marginally less confused...
The Missing Piece
by Martin Sustrik — 2019-10-27T05:50:00.824Z

In computer science, compiler is a program that translates things that programmers write, such as "if CloseButtonPressed then CloseApplication", to things that computers are able to understand and execute, such as "0001010111001010111011000111100010101011110".

When the first compiler was written, someone had to write it in the machine code, the zeroes and ones that computers understand. They had to type in the numbers, or, more likely, punch holes into punchcards.

But from that point on, programmers were able to write human-readable programs instead of the inscrutable machine code. They just had to use the existing compiler to translate from the former to the latter.

Now imagine you are tasked with writing the second, improved, version of the compiler.

You can, of course, do the same thing again. You can type in the numbers, or maybe punch zeroes and ones into punchcards. But writing machine code is hard.

And now you have a better option at your disposal: You can write the new version of the compiler in a human-readable language and use the old version of the compiler to translate it into machine code!

Similarly, the third version of the compiler can be compiled by the second version of the compiler, the fourth by the third and so on.

That, by the way is not a hypothetical scenario. That's how the compilers are written nowadays.

Now, here comes the question: What if all the software disappeared overnight? What if the only thing we were left with was the human-readable source code of the latest version of the compiler? Would we be able to reconstruct the language and the compiler and get everything running again?

On one hand, the source code contains all the relevant information. It contains the full description of the language, its syntax and its semantics. It contains all the instructions needed to translate it into the machine code. It should be therefore possible to use it to create a functional compiler again.

But on the other hand, how exactly would you do that? The source code is, after all, just a pile of text. And there's no compiler to turn it into the machine code, into something that computer understands. It sounds like you are faced with a kind of chicken-and-egg problem.

Clearly, the source code is not enough. There's some information missing. But it's hard to pinpoint what exactly the missing part may be.

***

What if we foun...
Jacy Reese (born Jacy Anthis)?
by jefftk — 2019-10-26T21:00:01.838Z

Update 2020-05-08: Jacy wrote to me to tell me that his full
name is "Jacy Reese Anthis", and that he had been using "Jacy Reese"
in his public writing. Since getting married he's switched to using
his full name exclusively. I looked at Wikipedia, and I noticed that
since I wrote this six months ago, other people had continued trying
to edit the name: I see a 1/12
edit by Lockedinthebox reverted
the next day by Bodole and a 1/19
edit by Lockedinthebox again, again promptly
reverted by Bodole. The back-and-forth ends with a 4/29
Bodole edit to put in the full name, citing Jacy's website and
Twitter.

Yesterday I noticed that Wikipedia:Jacy
Reese doesn't mention anywhere that until recently he went by Jacy
Anthis. Thinking this was an oversight, I looked up the relevant
section in the

Manual
of Style and saw:

In some cases, subjects have had their full names changed at some
point after birth. In these cases the birth name should be given in
the lead as well.

So I edited
the page to add it, changing it from:

To:

Then I thought to look at the talk page, to
see whether maybe there was a reason it didn't already give a birth
name. It turns out there's been extensive discussion on this, as
people keep noticing that his birth name is missing, adding it, and
then others removing it for not being verifiable. Looking back I see:

December
2017: article created, with "Jacy
Reese".

August
2018: an anonymous user changes it to "Jacy Reese Anthis, known
professionally as Jacy Reese".

Three
days later: reverted by DominikPeters.

March
2019: user MoonStarDolphin changes it to "Jacy Reese (born Jacy Anthis)".

Two
days later: reverted by Utsill.

October
2019: I change it to "Jacy Reese (born Jacy Anthis)".

Six
hours later: reverted by Bodole.

The people adding his birth name are doing it because it's "obvious",
sometimes giving citations of his earlier use of "Jacy Anthis", and
the people reverting it are pointing out that Wikipedia's rules for Biographies
of living persons are really very strict. These rules began after
the Seigenthaler
biography incident where someone put false information in Seigenthaler's
wikipedia article, and are a sensible response to how false
information about someone on Wikipedia can be very damaging.

After my edit was reverted I looked around to see if I could find
better sources for this, ...
Fetch The Coffee!
by matthewp — 2019-10-26T20:41:27.191Z

This is a reaction to a specific point in the "Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More"

The Disagreement

"Stuart Russell: It is trivial to construct a toy MDP in which the agent's only reward comes from fetching the coffee. If, in that MDP, there is another "human" who has some probability, however small, of switching the agent off, and if the agent has available a button that switches off that human, the agent will necessarily press that button as part of the optimal solution for fetching the coffee. No hatred, no desire for power, no built-in emotions, no built-in survival instinct, nothing except the desire to fetch the coffee successfully. This point cannot be addressed because it's a simple mathematical observation."

"Yann LeCun: [...] I think it would only be relevant in a fantasy world in which people would be smart enough to design super-intelligent machines, yet ridiculously stupid to the point of giving it moronic objectives with no safeguards."""

Now, I think the coffee argument was the highlight of [1]; a debate which I thoroughly enjoyed. It does a reasonable job of encapsulating the main concern around alignment. The fielded defence was not compelling.

However, defences should be reinforced and explored before scoring the body-blow. There is the stub of a defence in LeCun's line of thought.

In particular, I think we need to go some way towards cashing out exactly what kind of robot we're instructing to 'Fetch the coffee'.

The Javan Roomba?

Indulge, for a moment, a little philosophising: what does it even mean to 'fetch the coffee'?

Let's unpack some of the questions whose answers must be specified in order to produce anything like the behaviour you'd get from a human.

Whose coffee?

When? Now, or when the others arrive?

What is coffee?

How much coffee?

Should the coffee be fetched in solution or dry?

Is there coffee available?

Should the sugar also be fetched? Cups?

Should anything else be done along the way?

What is fetching?

What path should be taken?

Is fetching satisfied by a terminal distance of one metre or more?

Is a successful fetching zone be a sphere centred on the requestor's centre of gravity, or an arm length cone on the requestor's main shoulder?

Is fetching coffee like fetching a stick?

Can the coffee be frozen to avoid spillages?

Is the request satisfied by $wget https://upload.wikimedia.org

...
The Pit
by alkjash — 2019-10-26T05:28:07.610Z
Link: https://radimentary.wordpress.com/2019/10/25/the-pit/

If you brought a man with keen ears to the edge of the pit and dropped a quarter over exactly the right spot, you could count to eleven before he heard it hit the ground. If you next told the man that a sliver of sunlight was visible from the very bottom of said pit, he might have squinted at you skeptically. If you proceeded to say that the bowels of this same pit were inhabited by twenty-odd live human beings, he would certainly have slapped you across the side of the head and called you a shameless liar. But you wouldn’t have lied once.

The inhabitants of the pit – the pitfolk – were frail people, bone-pale from the perennial lack of sunlight, all taut skin wrapped about wan elbows. However they shifted their bodies to and fro, they were bound – as if by cowed by that measly sliver of sunlight – to walk hunched over, keeping their faces downcast.

Through the decades, the pitfolk developed an extraordinary black and white vision, as all they could see were the meager shadows which shifted around their ankles. From these faded images the pitfolk deduced the whole of their reality. This made for a rather miserable experience, but it did not stop the pitfolk from building an entire way of life about the dance of dim shadows – black silhouettes against grey stone – that was their everything.

Each day at noon, when the dim light against their backs shone brightest, the pitfolk gathered in a large circle with one tribe member or another in the center, and that chosen member would play out a long and complex dance with the long and dextrous fingers attached to his long and sinewy limbs. The whole circle would watch intently as the shadows cast from the dance leaped across the ground, swaying to a silent rhythm. The full performance, which lasted nearly two hours, had been passed down from parent to child as long as any living tribe member could remember, and probably longer. And although each generation brought into it their own unique flicks of the finger and twirls of the elbow, the dance remained remarkably unchanged through the years. As everyone knew, the shadow dance was the story of their tribe.

Of course, each tribe member was free to form their own opinion about what exactly that story was. By some miracle of memory the pitfolk retained the faintest inkling of the great goings-on in the outside world, and thro...
Climate technology primer (1/3): basics
by amarbles — 2019-10-25T23:18:09.351Z

This is my first post here, cross-posted from

https://longitudinal.blog/co2-series-part-1-review-of-basics/

Posting was suggested by Vaniver, and the actual work of cross-posting was done by Benito.

Comments appreciated. (Will only cross post the first of the 3 climate posts here.)

This is the first of a series of three blog posts intended as a primer on how technology can help to address climate change.

Note: you can annotate this page in Hypothes.is here: https://via.hypothes.is/https://longitudinal.blog/co2-series-part-1-review-of-basics/

Excellent documents and courses are available online on what we are doing to the climate — such as the IPCC reports or Prof. Michael Mann’s online course or this National Academies report. Even more so, this type of open source analysis is what the field needs to look like. But I had to dig a bit to see even the beginnings of a quantitative picture of how technologies can contribute to solving the problems.

There are integrative assessments of solutions, such as Project Drawdown, and a huge proliferation of innovative technologies being proposed and developed in all sectors. There are also some superb review papers that treat the need for new technologies. Bret Victor has a beautiful overview emphasizing the role of general design, modeling and implementation capacity, and there is now a climate change for computer scientists course at MIT. But, as a scientifically literate (PhD in Biophysics) outsider to the climate and energy fields, I was looking to first understand the very basics:

How can I estimate the world’s remaining carbon budget for various levels of warming? What are the uncertainties in this?
What would happen if Greenland melted?
What are the fundamental physical limits on, say, the potential for sucking carbon dioxide out of the atmosphere? How low could the cost theoretically go?
How many trees would you need to plant, and are you going to run out of water, land or fertilizer for them?
If we did manage to remove a given amount of CO2, how would the climate respond? How would the climate respond if we magically ceased all emissions today?
What would be the real impact if we could electrify all cars?
Are batteries going to be able to store all the variable renewable energy we’ll need? Is there enough Uranium obtainable to power the world with nuclear? Order of magnitude, how much would it cost to just replace our entir

...
Two explanations for variation in human abilities
by Matthew Barnett — 2019-10-25T22:06:26.329Z

In My Childhood Role Model, Eliezer Yudkowsky argues that people often think about intelligence like this, with village idiot and chimps on the left, and Einstein on the right.

However, he says, this view is too narrow. All humans have nearly identical hardware. Therefore, the true range of variation looks something like this instead:

This alternative view has implications for an AI takeoff duration. If you imagine that AI will slowly crawl from village idiot to Einstein, then presumably we will have ample time to see powerful AI coming in advance. On the other hand, if the second view is correct, then the intelligence of computers is more likely to swoosh right past human level once it reaches the village idiot stage. Or as Nick Bostrom put it, "The train doesn't stop at Humanville Station."

Katja Grace disagrees, finding that there isn't much reason to believe in a small variation in human abilities. Her evidence comes from measuring human performance on various strategically relevant indicators: chess, go, checkers, physical manipulation, and Jeopardy.

In this post, I argue that the debate is partly based on a misunderstanding: in particular, a conflation of learning ability and competence. I further posit that when this distinction is unraveled, the remaining variation we observe isn't that surprising. Similar machines regularly demonstrate large variation in performance if some parts are broken, despite having nearly identical components.

These ideas are not original to me (see the Appendix). I have simply put two explanations together, which in my opinion, explain a large fraction of the observed variation.

Explanation 1: Distinguish learning from competence

Humans, despite our great differences in other regards, still mostly learn things the same way. We listen to lectures at roughly the same pace, we read at mostly the same speeds, and we process thoughts in similar sized chunks. To the extent that people do speed up lectures by 2x on Youtube, or speed read, they lose retention.

And putting aside tall tales of people learning quantum mechanics over a weekend, it turns out to be surprisingly difficult for humans to beat the strategy of long-term focused practice for becoming an expert at some task.

From this, we have an important insight: the range of learning abilities in humans is relatively small. There don't really seem to be humans who tower above us in ...
Effect heterogeneity and external validity in medicine
by Anders_H — 2019-10-25T20:53:53.012Z

Our paper "Effect heterogeneity and variable selection for standardizing causal effects to a target population" has just been publised in the European Journal of Epidemiology at https://link.springer.com/article/10.1007/s10654-019-00571-w . While the journal's version of record is behind a paywall, a preprint is available on arXiv at https://arxiv.org/pdf/1610.00068.pdf.

This paper argues for my very deeply held belief that we can make significant advances in quantitative reasoning for medical decision making by thinking more closely about effect heterogeneity and how this relates to the choice of effect scale.

Over the course of the last 7 years, external validity and generalizability have become increasingly hot topics in statistical methodology and computer science. In particular, a lot of progress has been made by Judea Pearl and Elias Bareinboim, who introduced a framework based on causal diagrams that can be used to reason about how to take causal information from one setting (for example: a randomized trial) and apply it in a different setting (for example: a clinically relevant target population).

The key questions of interest are: How do we know whether such extrapolation is even possible? How do we determine what information we need from the study, and what information we need from the target population, in order to extrapolate the findings? How do we put this information together in order to obtain a valid prediction for what happens if the intervention is implemented in the target population?

Pearl and Bareinboim's framework for answering these questions is, of course, mathematically valid. However, in my opinion, their approach also throws the baby out with the bathwater. In particular, we argue that instead of attempting to extrapolate the magnitude of the effect (i.e. a measure of the "size" of the difference between what happens if the drug is taken, and what happens if the drug is not taken), they attempt to look at the people who were assigned to receive the intervention in the study, and extrapolate their distribution of outcomes to the target population without any reference to how that distribution differs from what happened to the people who were randomized to the control condition.

Theoretically, this approach will work if the extrapolation procedure can account for every cause of the outcome whose distribution differs between the ...
Door Ideas
by jefftk — 2019-10-25T12:40:01.627Z

The front door to our apartment doesn't really have space to swing.
When we first moved in it opened to the right, blocking part of the
hallway and keeping you from carrying groceries straight through to
the kitchen:

This was annoying, so after some discussion with housemates I turned
it around:

In practice this is maybe better, maybe worse, still pretty annoying.

We've considered removing the door:

But this would let kid-noise carry more down to Julia's office,
wouldn't work if we had a housemate with a small child or cat, and if
at some point the people downstairs were not close friends it would be
a little weird.

We could have the door open out instead:

But this would be too awkward to open, since the landing is so
small that the door kind of needs to swing through you.

So, it currently still looks like:

One idea is I could cut it down the middle, and let it fold a second
time:

This gets it out of the way when it's open, but at the cost of having
a really very large swing.

Perhaps the door should open in the middle?

Very narrow for a double door, and it still sticks out a bit on the
left. There's extra space on the right, though, so you could make it
asymmetrical?

Or, even one door going in and one going out?

This would be a bit more work, but we could put a track on the ceiling that allowed the door to slide around a curve, opening outwards but with a much reduced sweep:

Predictions on what we'd end up liking best?
Iron: From mythical to mundane
by jasoncrawford — 2019-10-24T22:43:45.898Z
Link: https://rootsofprogress.org/iron-from-mythical-to-mundane

Metalworking is one of the oldest crafts, going back far beyond recorded history. But until a few thousand years ago, one of the most abundant metals—iron—was virtually unknown. The ancient Egyptians and Sumerians knew iron only from meteors, and considered it heavenly, a gift from the gods. Later civilizations discovered how to smelt it, and their period came to be known as the Iron Age, but they didn’t know how to consistently make it strong, and not brittle. During the Industrial Revolution, we discovered how to make it strong, consistent, and even cheap.

Now high-quality steel is everywhere around us: holding up our buildings (in both girders and reinforced concrete), in our cars and ships, in trains and their rails, in bridges and towers, in electrical infrastructure, in refrigerators and washing machines, in pots and pans, in forks and knives, in hammers and saws, in nails and screws, in tables and chairs.

The wonder of the metal is gone, but the wonder of the process remains. A steel foundry is an amazing sight, with enormous blast furnaces several stories high, raging at thousands of degrees 24/7, attended by a swarm of workers, like a great beast with an insatiable hunger for ore, coke, and lime.

The product they put out is of such purity and consistency that it would be hailed by the ancients as a miracle, a feat of craftsmanship possible only through divine intervention. But in the modern economy, it is a commodity.

What is this material? Why is it so hard to make? And how did it go from mythical to mundane?

Full post with lots of images: https://rootsofprogress.org/iron-from-mythical-to-mundane