trentmkelly/LessWrong-43k huggingface dataset:
Debate update: Obfuscated arguments problem
by Beth Barnes — 2020-12-23T03:24:38.191Z

This is an update on the work on AI Safety via Debate that we previously wrote about here.

Authors and Acknowledgements

The researchers on this project were Elizabeth Barnes and Paul Christiano, with substantial help from William Saunders (who built the current web interface as well as other help), Joe Collman (who helped develop the structured debate mechanisms), and Mark Xu, Chris Painter, Mihnea Maftei and Ronny Fernandez (who took part in many debates as well as helping think through problems). We're also grateful to Geoffrey Irving and Evan Hubinger for feedback on drafts, and for helpful conversations, along with Richard Ngo, Daniel Ziegler, John Schulman, Amanda Askell and Jeff Wu. Finally, we're grateful to our contractors who participated in experiments, including Adam Scherlis, Kevin Liu, Rohan Kapoor and Kunal Sharda.

What we did

We tested the debate protocol introduced in AI Safety via Debate with human judges and debaters. We found various problems and improved the mechanism to fix these issues (details of these are in the appendix). However, we discovered that a dishonest debater can often create arguments that have a fatal error, but where it is very hard to locate the error. We don’t have a fix for this “obfuscated argument” problem, and believe it might be an important quantitative limitation for both IDA and Debate.

Key takeaways and relevance for alignment

Our ultimate goal is to find a mechanism that allows us to learn anything that a machine learning model knows: if the model can efficiently find the correct answer to some problem, our mechanism should favor the correct answer while only requiring a tractable number of human judgements and a reasonable number of computation steps for the model. [1]

We’re working under a hypothesis that there are broadly two ways to know things: via step-by-step reasoning about implications (logic, computation…), and by learning and generalizing from data (pattern matching, bayesian updating…).

Debate focuses on verifying things via step-by-step reasoning. It seems plausible that a substantial proportion of the things a model ‘knows’ will have some long but locally human-understandable argument for their correctness. [2] Previously we hoped that debate/IDA could verify any knowledge for which such human-understandable arguments exist, even if these arguments are intractably large. We hoped the debaters ...
Wholehearted choices and “morality as taxes”
by Joe Carlsmith — 2020-12-23T02:21:36.392Z

(Cross-posted from Hands and Cities)

In Peter Singer’s classic thought experiment, you can save a drowning child, but only by ruining your clothes. Singer’s argument is that (a) you’re obligated to save the child, and that (b) many modern humans are in a morally analogous relationship to the world’s needy. (Note that on GiveWell’s numbers, the clothes in question need to cost thousands of dollars; but the numbers aren’t my point at present).

I was first exposed to arguments like this in high school, and they left a lasting impression on me, as they do on many. But I also think they tend to leave some people with a reluctant, defensive, and quasi-adversarial relationship to the omnipresence of Singerian decisions of this kind, once this omnipresence is made clear. See, for example, the large literature on moral “demandingness,” which focuses heavily on the question of what costs you “have” to pay to help others, and what you are “allowed” not to pay.

Morality, in this conception, is sort of like taxes; it invades your personal domain — the place where you make choices based on what you want and care about — and then takes some amount of stuff for its own ends. One hopes it isn’t too much; one wonders how little one can get away with paying, without breaking “the rules.”

I’m not going to evaluate this conception directly here. But here’s a version of Singer’s thought experiment that I sometimes imagine, which lands in a different register in my mind (your mileage may vary).

Suppose that I am setting off to walk in the forest on a crisp fall afternoon — something I’ve been looking forward to all week. As I near the forest, I notice, far away, what looks like some sort of commotion down by the river, off of my walking route, though I can’t see very clearly. I consider going to see what’s going on, but the light is fading, so I decide to continue on my way.

I learn later that while I was walking, a man in his early forties drowned in that river. He was pinned under some sort of machinery. Five other people were there, including his wife and son, but they weren’t strong enough to lift the machinery by themselves. One extra person might have made the difference.

(Here I try to imagine vividly what it was like trying to save him — his wife desperate, weeping, pulling at him, his own eyes frantic, the fear and chaos, the eventual despair– and the pain of his absence afterwa...
An appeal for vitamin D supplementation as a prophylactic for coronaviruses and influenza and a simple evolutionary theory for why this is plausible.
by Michael A — 2020-12-22T19:40:08.731Z

I'd like to bring home a case for vitamin D as a probable prophylactic for seasonal respiratory viruses like influenza or coronaviruses. Medical authorities already advise supplementing vitamin D in the winter but they haven't emphasized why it especially matters now. This is a no risk, low cost, low effort intervention so our standard of evidence doesn't need to be all that high here. We only need to have good cause to suspect that it helps, even if only just a little.

There quite a few studies showing that vitamin D deficiency is strongly associated with severity of infection.

Here

is a nice one. It found "VitD deficiency was associated with a 6-fold higher hazard of severe course of disease and a ~15-fold higher risk of death." in its decently sized cohort of 185 patients.

These are very large numbers. They really are not the sort you expect to see from a parameter that is merely correlated with the true causes and not itself causative. So perhaps there is something to this? A

meta-analysis

of studies relevant to severity paints a similar, but less stark, picture.

Influenza and coronaviruses are both enveloped respiratory viruses and both are prevalent in fall and winter, precisely when vitamin D levels are low and lowering. In tropical countries, the rainy season brings clouds and drives people indoors. It also brings their flu season, but its not as stark a difference as it is for temperate climes. As a spot check, let's look at the most populous nation with a rainy season. India's season generally ends in September which is when they peaked at ~100k cases per day. This is the month vitamin D levels would be expected to start to increase. It is down to ~25k now.

India isn't alone. Broadly speaking, COVID-19 appears so far to have seasonality that syncs up closely with that of influenza. We have been seeing a large increase in cases as we enter the dead of winter across Europe and USA. But perhaps this is due to the cold outside and dry conditions inside. Our protective mucus dries up when humidity is below ~40%, making us more vulnerable to infection. As temperatures lower, the virus's outer envelope solidifies like butter. This obstructs ozone and other reactive chemicals from penetrating inside and damaging its vulnerable interior. However, these two commonly cited factors obviously can't explain why it has long been observed that influenza pr...
TAI Safety Bibliographic Database
by JessRiedel — 2020-12-22T17:42:05.595Z

Authors: Jess Riedel and Angelica Deibel
Cross-posted to EA Forum

In this post we present the first public version of our bibliographic database of research on the safety of transformative artificial intelligence (TAI). The primary motivations for assembling this database were to:

Aid potential donors in assessing organizations focusing on TAI safety by collecting and analyzing their research output.
Assemble a comprehensive bibliographic database that can be used as a base for future projects, such as a living review of the field.

The database contains research works motivated by, and substantively informing, the challenge of ensuring the safety of TAI, including both technical and meta topics. This initial version of the database has attempted comprehensive coverage only for traditionally formatted research produced in 2016-2020 by organizations with a significant safety focus (~360 items). The database also has significant but non-comprehensive coverage (~570 items) of earlier years, less traditional formats (e.g., blog posts), and non-safety-focused organizations. Usefully, we also have citation counts for essentially all the items for which that is applicable.

The core database takes the form of a

Zotero library

. Snapshots are also available as Google Sheet, CSV, and Zotero RDF. (Compact version for easier human reading: Google Sheet, CSV.)

The rest of this post describes the composition of the database in more detail and presents some high-level quantitative analysis of the contents. In particular, our analysis includes:

Lists of the most cited TAI safety research for each of the past few years (Tables 2 and 3)
A chart showing how written TAI safety research output has changed since 2016 (Figure 1).
A visualization of the degree of collaboration on TAI safety between different research organizations (Table 4).
A chart showing how the format of written research varied between organizations, e.g., manuscripts vs. journal articles vs. white papers (Figure 2).
A comparison of the number of citations that different organizations have accumulated (Figure 4).

In 2020 we observe a year-over-year drop in technical safety research, but not meta safety research, which we do not understand (Figure 1). We suggest some possible causes, but without a convincing explanation we must caution against drawing strong conclusions from any of our data.

If you...
100 Tips for a Better Life
by Ideopunk — 2020-12-22T14:30:12.756Z

(Cross-posted from my blog)
The other day I made an advice thread based on Jacobian’s from last year! If you know a source for one of these, shout and I’ll edit it in.

Possessions

1. If you want to find out about people’s opinions on a product, google <product> reddit. You’ll get real people arguing, as compared to the SEO’d Google results.

2. Some banks charge you $20 a month for an account, others charge you 0. If you’re with one of the former, have a good explanation for what those $20 are buying.

3.

Things you use for a significant fraction of your life (bed: 1/3rd, office-chair: 1/4th) are worth investing in.

4. “Where is the good knife?” If you’re looking for your good X, you have bad Xs. Throw those out.

5. If your work is done on a computer, get a second monitor. Less time navigating between windows means more time for thinking.

6. Establish clear rules about when to throw out old junk. Once clear rules are established, junk will probably cease to be a problem. This is because any rule would be superior to our implicit rules (“keep this broken stereo for five years in case I learn how to fix it”).

7. Don’t buy CDs for people. They have Spotify. Buy them merch from a band they like instead. It’s more personal and the band gets more money.

8. When buying things, time and money trade-off against each other. If you’re low on money, take more time to find deals. If you’re low on time, stop looking for great deals and just buy things quickly online.

Cooking

9.

Steeping minutes: Green at 3, black at 4, herbal at 5. Good tea is that simple!

10. Food actually can be both cheap, healthy, tasty, and relatively quick to prepare. All it requires is a few hours one day to prepare many meals for the week.

11. Cooking pollutes the air. Opening windows for a few minutes after cooking can dramatically improve air quality.

12. Food taste can be made much more exciting through simple seasoning. It’s also an opportunity for expression. Buy a few herbs and spices and experiment away.

13. When googling a recipe, precede it with ‘best’. You’ll find better recipes.

Productivity

14. Advanced search features are a fast way to create tighter search statements. For example:
img html
will return inferior results compared to:
img html -w3

15.

You can automate mundane computer tasks with Autohotkey (or ...
What is food like?
by KatjaGrace — 2020-12-22T12:30:17.591Z

I have an Anki1 deck of things I feel like a failure regarding. Instead of each card having a question that I see if I can remember the answer to, it has a potentially shameful thing that I see if I still feel bad about. Each time I look at one, as well as marking it correct to the extent that I no longer feel bad about it, I briefly do a little bit to make it better. (Learn about the thing I’m embarrassed to not know about, practice the skill that I don’t have, think about whether it’s a real problem, etc). My sense is that one can often feel bad about something for a long time which one could alternatively make marked progress on in a very short time.

This time I drew a card marked, ‘can’t critique food’. Admittedly not the most horrifying of failures, and I don’t currently feel too bad about it, or remember writing it down. But I suppose that what I had in mind is that when people discuss the merits or subtleties of different foods and food establishments, I often feel like the topic is outside my domain, and furthermore suspect that if my strongest views of the moment were revealed—e.g. “good fries > bad fries » not fries”, or “I want to eat something cool and wet and I prefer it involve parsley”, or “pea protein is the worst”—I would seem childish.

So I read online briefly and found that Jonathan Gold is a famous food critic, then read a very small amount of his writing. Now I will have a go at critiquing food at all, which I expect is a decent step toward being passable at it (while also fulfilling my intention to occasionally do things I haven’t done).

*

On an almost empty teacup of Mary’s Gone crackers I found in my bedroom while cleaning up recently.

Food that you know has been sitting open on your desk for a week can be tainted by a foul flavor of unease. But my expectations for these crackers were cleansed with the fresh-toasted snap and delicate flavor of my first nibble of one.

Crispy things are often light and insubstantial—crunch and heft seem to often overlap in coarse, unyielding foods. Yet this Mary’s Gone cracker had both a pleasing, easy crackle and a real density. And while in no way chewy, it was good to chew: coming apart not into a wheat slurry, but into a textured rubble of seed meal, satisfying and nutty. I wanted to find the fragments lost in the corners of my mouth. I wanted to crush the last seeds with my teeth. All the while the br...
The Best Visualizations on Every Subject
by ryan_b — 2020-12-21T22:51:54.665Z

Edit: the list is now a public GitHub repository, with all that implies. Added sections by media type. Last update: 6 Jan 2021

Motivation

This is The Best Textbooks on Every Subject, but for visualizations. I greatly adore good visualizations, chiefly because there are so many visualizations that are so terrible. I have seen many such tools mentioned here, but always in passing.

The actual motivator is re-reading the posts Exercises in Comprehensive Information Gathering and Fact Posts: How and Why. While there is no substitute for the wrench-time they recommend, I think these kinds of tools make the process more efficient and lend themselves to insights which are difficult to acquire through reading alone; in my experience scale and distance are both easier to grasp in a visual medium, for example.

Also there is a non-trivial sense in which they are beautiful in their own right. If we are able to compare many examples, people in the community might even be able to help advance the art.

Submission Rules

One nomination per comment; please include an explanation of why you nominated it. Contra the best textbooks list we won't require comparison with other visualizations because there are so few authoritative ones.

Current List

WEB:

History

ORBIS: The Geospatial Network Model of the Roman World: https://orbis.stanford.edu/
Data Visualization and the Modern Imagination: https://exhibits.stanford.edu/dataviz
A Simulated Dendrochronology of Immigration 1790-2016: https://web.northeastern.edu/naturalizing-immigration-dataviz/

Math

Byrne's Euclid: The First Six Books of the Elements of Euclid With Coloured Diagrams and Symbols: https://www.c82.net/euclid/
The Empirical MetaMathematics of Euclid and Beyond: https://writings.stephenwolfram.com/2020/09/the-empirical-metamathematics-of-euclid-and-beyond/
An Interactive Introduction to Fourier Transforms: http://www.jezzamon.com/fourier/
Better Explained: https://betterexplained.com/
Jason Davies: Math and Cartography: https://www.jasondavies.com/
Bit-player: "An amateur’s outlook on computation and mathematics": http://bit-player.org/

Economics

The Observatory of Economic Complexity: https://oec.world/
Our World in Data: https://ourworldindata.org/

Machine Learning

Distill: https://distill.pub/

Miscellaneous

Explorable Explanations: https://explorabl.es/
Complexity Explorables: https://complexity-explorables.org/
The Pudding Visual Essays: https:/

...
Consciousness of abstraction
by Richard_Kennaway — 2020-12-21T21:35:31.001Z

Preface

I expect that many people on LessWrong have heard of Korzybski, of General Semantics (K's name for his system), and of some of the phrases he coined, such as "the map is not the territory". abramdemski has recently posted on an aspect of his system that I want to communicate more fully here. I wrote a comment on this rather a long time ago. This is the long version.

Korzybski invented the term "consciousness of abstraction", which has to do with being aware of the "ladder of abstraction" (I think another of his coinages). It seems to me that this skill covers a lot of other things that have been spoken of on LessWrong and elsewhere, such as "Looking", the leakiness of abstractions, the nature of definitions, and abramdemski's article. It relates to a lot of the series "A user's guide to words". It is a fundamental aspect of rationality, but it has not yet been the primary focus of an article.

Korzybski is the source where I encountered it, but I can only recommend reading the original work ("Science and Sanity", 4th or 5th edition[1]) to those who specifically want to see what the original said, and are willing to deal with its great length, turgid style, and outdated science. Instead I would recommend S. I. Hayakawa's "Language in Thought and Action", which gives the ideas in briefer and more readable form.

Consciousness of abstraction

There is a world out there. (Those inclined to dispute this are invited to do so elsewhere, after putting this to the test by leaping in front of a train.) Our senses give us the naive impression that we have some sort of direct knowledge of that world, but a minimal familiarity with the mechanisms of perception show that that is an illusion. We have physical sense organs that interact with the outside world. These generate electrochemical signals in nerve fibres, and somehow (I shall not attempt to unpack the "somehow") we experience perceptions. Put an external obstacle in the way and the perception goes away. (The sun is always shining, even at night.) Interrupt the nerve signals and the perception goes away. (Your mouth is still there, before the dentist's anaesthetic wears off.) The thing out there and its effect on the sense organs are physically different things. The sense organs and the signals they send are physically different things. The signals and what we make of them are physically different things. And th...
Intake Rumble Spew
by Willa — 2020-12-21T21:25:35.804Z

Too much reading. Eyes unfocused and tired

Too little writing. Fingers still and ready

Libraries of words. Burst

Across the fibre. Pulse

Down the copper. Flux.

Signal demuxed at the demarc

Thus the Internet was routed

The LAN shouts, packets whorl and twirl

Dancing into my desktop's NIC

96 Firefox tabs later

Where has my time gone?

What did I learn?

Why?

Right...Write. Write!
When life was literally full of crap
by jasoncrawford — 2020-12-21T21:16:04.831Z
Link: https://rootsofprogress.org/when-life-was-literally-full-of-crap

If you want a single, vivid, and frankly disgusting example to hold in mind to remember how much our lives have improved over the last ~150 years…

Consider shit.

Literally, excrement. How much previous generations had to think about it, be around it, even handle it.

Before the automobile, horses flooded the streets, and cities were mired in their muck. According to Richard Rhodes’s Energy: A Human History, in New York City, horses dropped 4 million pounds of manure and 100 thousand gallons of urine on the streets every day. (!) And in The Rise and Fall of American Growth, Robert Gordon quotes this passage from The Horse in the City:

The smell of manure heaps and the clouds of flies they attracted were some of the most objectionable nuisances stables created. The nuisance was especially bad in the summer…. On New York’s Liberty Street there was a manure heap seven feet high. New York streets … were often covered with layers of manure.

Shoveling shit was literally a full-time job. And one of the key uses of horses was to pull the wagons that carried away horse droppings.

But excrement wasn’t just something to be cleared away. Oh no, that would be a profligate waste! Shit was valuable. It was a treasure. It was gold.

It was fertilizer.

For thousands of years, manure, mainly from livestock, was the primary way to renew the fertility of the soil after each harvest. Agricultural systems were defined in large part by how much manure they could effectively use on arable land. The transition from slash-and-burn to the two-field rotation of antiquity depended on livestock that would graze in pasture, then be penned in on fallow fields at night to fertilize it with their droppings. The transition to the three-field rotation depended on the plow, which among other things allowed manure to be buried under the soil; and the wagon, which allowed more manure to be transported.

Dung was also used as fuel, in areas that didn’t have much firewood, and even as a building material. The poorest areas today still burn dung for fuel, contributing to indoor air pollution:

The energy ladder. WHO, via Our World in Data

Our World in Data

By the 1800s, the manure trade had gone global. The Chincha Islands, off the coast of Peru, where it rarely rained, had been accumulating seagull droppings for thousands of years. The accumulated guano, stories high, was mined like coal or lime and exported to Eur...