trentmkelly/LessWrong-43k huggingface dataset:
Models, myths, dreams, and Cheshire cat grins
by Stuart_Armstrong — 2020-06-24T10:50:57.683Z

"she has often seen a cat without a grin but never a grin without a cat"

Alice in Alice in Wonderland, about the Cheshire cat (also known as the Unitary Authority of Warrington Cat).

Let's have a very simple model. There's a boolean, C, which measures whether there's a cat around. There's a natural number N, which counts the number of legs on the cat, and a boolean G, which checks whether the cat is grinning (or not).

There are a few obvious rules in the model, to make it compatible with real life:

¬C → (N=0).

¬C → ¬G.

Or, in other words, if there's no cat, then there are zero cat legs and no grin.

And that's true about reality. But suppose we have trained a neural net to automatically find the values of C, N, and G. Then it's perfectly conceivable that something might trigger the outputs ¬C and G simultaneously: a grin without any cat to hang it on.

Adversarial examples

Adversarial examples often seem to behave this way. Take for example this adversarial example of a pig classified as an airliner:

Imagine that the neural net was not only classifying "pig" and "airliner", but other things like "has wings" and "has fur".

Then the "pig-airliner" doesn't have wings, and has fur, which are features of pigs but not airliners. Of course, you could build an adversarial model that also breaks "has wings" and "has fur", but, hopefully, the more features that need to be faked, the harder it would become.

This suggests that, as algorithms get smarter, they will become more adept at avoiding adversarial examples - as long as the ultimate question is clear. In our real world, the categories of pigs and airliners are pretty sharply distinct.

We run into problems, though, if the concepts are less clear - such as what might happens to pigs and airliners if the algorithm optimises them, or how the algorithm might classify underdefined concepts like "human happiness".

Myths and dreams

Define the following booleans: HH detects the presence of a living human head, HB a living human body, JH a living jackal head, JB a living jackal body.

In our world real world we generally have HH↔HB and JH↔JB. But set the following values:

¬HH,HB,JH,¬JB,

and you have the god Anubis.

Similarly, what is a dragon? Well, it's an entity such that the following are all true:

{is lizard, is flying, is huge, breath is fire, intelligence is human level, ...}

And, even though those featur

...
Does NYT have policies?
by Douglas_Knight — 2020-06-24T04:06:57.556Z

Does the New York Times have written policies? Does it publish them? Have they leaked?

Here is a list of six public documents. Most interesting are the Ethical Journalism Guidebook/Handbook and the Guidelines on [Our] Integrity. The first mentions three documents: (A) "the Newsroom Integrity Statement, promulgated in 1999"; is this the Guidelines linked above? (B) "the Policy on Confidential Sources, issued in 2004," archived here. Do they still publish it? and (C) "the Rules of the Road," which sounds like a private document not specific to journalism.

Are there other private written policies? Have they leaked? Are there rumors about them?

I don't mean to imply that policies are an unalloyed good. At some level of detail or disorganization, people simply don't learn them. I have a largely unjustified intuition that lying is bad and lying about policies is particularly bad, seeming to exist to diffuse responsibility.
The Dark Miracle of Optics
by Suspended Reason — 2020-06-24T03:09:29.874Z

Alternate titles:

The Public-Private Information Gap Rules Everything Around Me
Baudrillard’s Simulacra, Steelmanned
“Have your cake and eating it too”
The Precarity of Prestige Economies
“Goodhart’s is just a subset, mannn.”
“Costly signals are just a subset, mannn.”
The Tragedy of Appearances
On Truth & Lies in a Nonmoral Sense

Epistemic status: no idea how original any of this is; it just connects a lot of nodes in my brain. I’ve been told there’s a real debt to Robert Trivers, which I hope to educate myself on shortly. I may just be reinventing signal theory.

In the beginning there was defection.

We can treat a prisoner’s dilemma as an elegant stand-in for coordination more generally. A one-off dilemma has as its ideal solution defection. Bellum omnium contra omnes: the war of all against all, or, “hyper-individualism.”

At the same time, it is clear that many of the “benefits sought by living things”[1]—which is to say, that which assists survival—are more readily reached by group effort.

Crucially, an iterated prisoner’s dilemma has the opposite optimal equilibrium: tit-for-tat, or cooperation, in its many variations, its various guises. And so the trick becomes how to switch individuals over from one-offs onto iterated dilemmas. The technology which achieves this is reputation, allowing formation of a ledger anchored to names[2], faces, identities. Individuals sharing an ecosystem continually run into each other, and given a reputation ledger, cannot defect and “get away” with it, continuing to freeride into oblivion.[3]

Tit-for-tat is exceedingly simple. It enables mutualism (in symbiosis[4]) and is practiced internally to species as diverse as stickleback fish, tree swallows, all primates, bats. All it requires is a sense of continuous identity and tracking of that identity’s (recent-)historical actions. We can take bats as an example: Since mothers’ hunting returns are unequally distributed, but bat babies do better when consistently fed, mothers communally share food. But should a researcher pump up a mother’s gullet full of air, so it appears she had a strong return but is refusing to share, suddenly the other mothers will no longer feed her children, will allow them to starve.

We can read human pr...
Half-Baked Products and Idea Kernels
by Liron — 2020-06-24T01:00:20.466Z

When I ask someone at work for a project proposal, I never want the person to go silent on me and put in 100 hours of solitary work, and then finally show me something and ask for my feedback. I always want to see a half-baked product.

You can half-bake something in an hour or two, or even in a few minutes.

The advantage of half-baking is that you get a quick feedback loop. The more you think there’s a possibility that I’ll say “no, that’s not what I wanted”, the more half-baked you should make your first effort before getting my feedback.

When brainstorming ideas, my term for a half-baked idea is a kernel. A kernel is usually a crappy idea on its own, but there’s “something to it” that could make it the seed of a better idea. I encourage people to toss out kernels.

There are two reasons why operating this way is efficient:

Diminishing Returns on Time Spent

Say you work on something for 100 hours. While each hour adds value, typically the highest-value hour is the first hour and the lowest-value hour is the last hour, and it follows a curve like this:

For example, if you're going to spend 100 hours writing a long report, spending one hour to brain-dump the key bullet points would give a reader a lot more than 1% of the final value of your report. Realistically, it’s probably more like 20%.

So the less time you spend working before getting feedback, the higher your productivity was in that time.

Efficient Course Correction

When you’re starting out on a new project that isn’t well-understood, you’re unlikely to go in the exact right direction. So you don’t want to go too far before getting a course correction, or you’ll waste time.

The top path shows how most people waste their time by investing too much effort between course corrections. The bottom path shows the efficient approach: you do a small chunk of work, then get feedback from your boss or your customers to correct your course, then do the next small chunk of work.

Once your course corrections become small, you can do larger chunks of work between course corrections. Until then, take small steps that produce half-baked products and idea kernels.
How do you Murphyjitsu essentially risky activities?
by 9eB1 — 2020-06-23T21:09:54.593Z

In the CFAR Handbook they have the following process instructions for Murphyjitsu:

Select a goal. A habit you want to install, or a plan you’d like to execute, or a project you want to complete.

Outline your plan. Be sure to list next actions, concrete steps, and specific deadlines or benchmarks. It’s important that you can actually visualize yourself moving through your plan, rather than having something vague like work out more.

Surprise-o-meter. It’s been months, and you’ve made little or no progress! Where are you, on the scale from yeah, that sounds right to I literally don’t understand what happened? If you’re completely shocked—good job, your inner sim endorses your plan! If you’re not, though, go to Step 4.

Pre-hindsight. Try to construct a plausible narrative for what kept you from succeeding. Remember to look at both internal and external factors.

Bulletproofing. What actions can you take to prevent these hypothetical failure modes? Visualize taking those preemptive actions and then ask your inner sim “What comes next?” Have you successfully defused the danger?

Iterate steps 3-5. That’s right—it’s not over yet! Even with your new failsafes, your plan still failed. Are you shocked? If so, victory! If not—keep going.

It seems like this process presumes that mitigations are low-cost and that the project you are trying to achieve is fundamentally achievable according to your inner sim. Most of this is presumption is contained in step 3. I've been thinking about how to apply this process to projects in a professional context (rather than a "self-help" context I guess) and in many cases you face costly tradeoffs regarding derisking mitigations. Also, sometimes your project may just be a big bet.

How do you change Murphyjitsu to work in such situations? Also, if people have experiences using Murphyjitsu in projects (e.g. a 1-3 month project involving a small team people), I'd be interested in learning how it's different.
What is "Instrumental Corrigibility"?
by joebernstein — 2020-06-23T20:24:16.371Z

In his article here, Paul Christiano mentions the idea that instrumentally corrigible agents are not robust, as slight misalignments between their values and the overseer values would lead to an adversarial relationship. While the intuition that slight deviations in values potentially results in adversarial behavior makes sense, I am not sure if I quite understand the meaning behind what an instrumentally corrigible agent is.

I would presume it relates to how corrigibility arises, that is different from Paul's amplification approach.
Modelling Continuous Progress
by Sammy Martin — 2020-06-23T18:06:47.474Z

I have previously argued for two claims about AI takeoff speeds. First, almost everyone agrees that if we had AGI, progress would be very fast. Second, the major disagreement is between those who think progress will be discontinuous and sudden (such as Eliezer Yudkowsky, MIRI) and those who think progress will be very fast by normal historical standards but continuous (Paul Chrisiano, Robin Hanson).

What do I mean by ‘discontinuous’? If we were to graph world GDP over the last 10,000 years, it fits onto a

hyperbolic growth pattern

. We could call this ‘continuous’ since it is following a single trend, or we could call it ‘discontinuous’ because, on the scale of millennia, the industrial revolution exploded out of nowhere. I will call these sorts of hyperbolic trends ‘continuous, but fast’, in line with Paul Christiano, who argued for continuous takeoff, defining it this way:

AI is just another, faster step in the

hyperbolic growth we are currently experiencing

, which corresponds to a further increase in rate but not a discontinuity (or even a discontinuity in rate).

I’ll be using Paul’s understanding of ‘discontinuous’ and ‘fast’ here. For progress in AI to be discontinuous, we need a switch to a new growth mode, which will show up as a step function in the capability of AI or in the rate of change of the capability of the AI over time. For takeoff to be fast, it is enough that there is one single growth mode that is hyperbolic or some other function that is very fast-growing.

This post tries to build on a simplified mathematical model of takeoff which was first put forward by Eliezer Yudkowsky and then refined by Bostrom in Superintelligence, modifying it to account for the different assumptions behind continuous, fast progress as opposed to discontinuous progress. As far as I can tell, few people have touched these sorts of simple models since the early 2010’s, and no-one has tried to formalize how newer notions of continuous takeoff fit into them. I find that it is surprisingly easy to accommodate continuous progress and that the results are intuitive and fit with what has already been said qualitatively about continuous progress.

The code for the model can be found here.

The Model

The original mathematical model was put forward

here

by Eliezer Yudk...
Requesting feedback/advice: what Type Theory to study for AI safety?
by rvnnt — 2020-06-23T17:03:32.452Z

TL;DR:

Requesting feedback or advice on questions of what math to study (or what projects to engage in) in order to learn how to prove things about programs and to analyze the difficulty/possibility of proving things about programs. Ulterior motive: AI safety (and also progress towards a university degree).

Motivation, Goal, Background

To my current understanding, any endeavor to figure out how to program[1] safe AI would probably eventually[2] benefit a lot from an ability to prove various properties of programs:

The AI will have to self-modify, or create successor agents, i.e. (re)write (parts of) new AI-software; and we'd probably want the AI to be able to prove[3] that said AI-software satisfies various "safety properties" (which I would not yet even know how to specify).

I'd like to get closer to being able to design the above kind of AI, and to better understand (the limits of) proving things about programs. To my current tentative understanding, the knowledge I'm looking for probably lies mostly under the umbrella of type theory.

(Also, I'm currently studying for a degree at a university and have the opportunity to choose a project for myself; and I think learning about type theory, lambda calculus, and/or automated theorem proving might be the most useful way to use that opportunity. (Since I don't see a way to pass deconfusion work off as a university project.))

Sub-goals, skills

To clarify the top-level goal here (viz. being able to design AIs that are able to prove stuff about AI-programs), I've listed below some (semi-)concrete examples of skills/abilities which I imagine would constitute progress towards said goal:

Given a type system S and some property P of programs/terms, be able to
(dis)prove that some, all, or no terms of S have property P.

As simple concrete examples:

Are programs in a given language guaranteed to terminate? (Do all terms in the given type system have normal forms?)

Given some function F: Can some term in the given type system implement F?

Does a given type system allow general recursion?

Be able to write a program R that, given encodings of S, P, and a term T of S,

finds a (dis)proof that T has property P (if doing so is possible).

Given a property P of programs, and a type system S,

design a (minimal) type system S' that admits terms/programs R that are able to prove (as above) properties P of some/all terms of S

...
[META] Building a rationalist communication system to avoid censorship
by Donald Hobson — 2020-06-23T14:12:49.354Z

The recent disappearance of Star Slate Codex made me realise that censorship is a real threat to the rationalist community. Not hard, government mandated censorship, but censorship in the form of online mobs prepared to harass and threaten those seen to say the wrong thing.

The current choice for a rationalist with a controversial idea seems to be to publish it online, where the most angry mobs from around the world can access it easily, or not to publish at all.

My solution, digital infrastructure for a properly anonymous, hidden rationalist community.

Related to kolmogorov-complicity-and-the-parable-of-lightning (Now also deleted, but here are a few people discussing it)

https://www.quora.com/How-has-religion-hindered-scientific-progress-during-the-reformation-period

https://www.reddit.com/r/slatestarcodex/comments/78d8co/kolmogorov_complicity_and_the_parable_of_lightning/

So we need to create the social norms and digital technologies to allow good rationalist content to be created without fear of mobs. My current suggestions include.

Security

1) Properly anonymous publishing. Each post that is put into this system is anonymous. If a rationalist posts many posts, then subtle clues about their identity could add up, so make each post independently anonymous. Given a specific post, you can't find others by the same author. Record nothing more than the actual post. With many rationalists putting posts into this system, and none of the posts attributable to a specific person, mobs won't be able to find a target. And noone knows who is putting posts into the pool at all.

2) Delay all published posts by a random time up to a week, we don't want to give info away about your timezone, do we.

3) Only certain people can access the content. Maybe restrict viewing to people with enough less wrong karma. Maybe rate limit it to 4 posts a day or something, to make it harder to scrape the whole anonymous site. (Unrestricted anonymous posting, restricted viewing is an unusual dynamic)

4) Of course only some posts will warrant such levels of paranoia, so maybe these features could be something that can be turned on and off independently.

My current model of online mobs is that they are not particularly good at updating on subtle strands of evidence and digging around online. One person who wants to stir up a mob does the digging, and then posts the result somewhere o...
Old-world Politics Fallacy
by Rudi C — 2020-06-23T12:32:55.946Z

When a nasty political problem (like the current SSC situation) hits my consciousness, I'm habituated to Do Something About It. I feel an urge to investigate the political climate, find allies, and fight back against the threat. In the current Internet age, and with my nonexistent political clout and social influence (especially considering I live in Iran and people here generally can't be counted on to know or care about global politics), the substitution bias kicks in; I substitute doing The Real Thing (to which my contribution would be very meager if any) with reading online forums (Reddit, Twitter, Lesswrong, SSC, Hackernews, ...)(substituted for "investigate the political climate"), voting the Correct posts in said forums and possibly writing some low-effort answers to particularly egregious pieces ("find allies, and fight back"), and generally feeling bad that I have failed the Mission (and that the Society is broken).

I speculate that this fallacy probably has some evolutionary roots. In a hunter-gatherer tribe, a person such as myself (I estimate myself to be upper-mid status.) would have had a fair chance of affecting political change against causes that mostly hurt everyone but a minority of politicians who don't produce much value anyways. Especially since I (and my family) have been quite morally upstanding and honest, in a small community, we would have had a reputation to draw on. Even if not, engaging with the community would have been the essential first step in being part of a coalition, necessary for survival.

Obviously, in the 21st century, all this is moot for political stuff that actually matters. Most people are quite powerless in affecting those matters, one reason being that the important issues now affect orders of magnitude more people. I don't know what the optimal strategy currently is. My gut feeling is that a lot of the good people are un-politicizing themselves and simply giving up. (Hasn't Scott's default defensive strategy been more (self)censorship?) Individual contributory power being what it is, this might actually be the best heuristic. If political "activism" consists mostly of low-skill low-reputation noise-making, a game of quantity over quality, then "good" people will immediately lose the comparative advantage. In fact, in Iran, the situation seems to be that the only marginally effective activism is violence. Which, i

...
New York Times, Please Do Not Threaten The Safety of Scott Alexander By Revealing His True Name
by Zvi — 2020-06-23T12:20:00.788Z

In reaction to (Now the entirety of SlateStarCodex): NYT Is Threatening My Safety By Revealing My Real Name, So I Am Deleting The Blog

I have sent the following to New York Times technology editor Pui-Wing Tam, whose email is pui-wing.tam@nytimes.com:

My name is Zvi Mowshowitz. I am a friend of Scott Alexander. I grew up with The New York Times as my central source of news and greatly value that tradition.

Your paper has declared that you intend to publish, in The New York Times, the true name of Scott Alexander. Please reconsider this deeply harmful and unnecessary action. If Scott’s name were well-known, it would likely make it more difficult or even impossible for him to make a living as a psychiatrist, which he has devoted many years of his life to being able to do. He has received death threats, and would likely not feel safe enough to continue living with other people he cares about. This may well ruin his life.

At a minimum, and most importantly for the world, it has already taken down his blog. In addition to this massive direct loss, those who know what happened will know that this happened as a direct result of the irresponsible actions of The New York Times. The bulk of the best bloggers and content creators on the internet read Scott’s blog, and this will create large-scale permanent hostility to reporters in general and the Times in particular across the board.

I do not understand what purpose this revelation is intended to serve. What benefit does the public get from this information?

This is not news that is fit to print.

If, as your reporter who has this intention claims, you believe that Scott provides a valuable resource that enhances the quality of our discourse, scientific understanding and lives, please reverse this decision before it is too late.

If you don’t believe this, I still urge you to reconsider your decision in light of its other likely consequences.

We should hope it is not too late to fix this.

I will be publishing this email as an open letter.

Regards,

Zvi Mowshowitz

PS for internet: If you wish to help, here is Scott’s word on how to help:

There is no comments section for this post. The appropriate comments section is the feedback page of the New York Times. You may also want to email the New York Times technology editor Pui-Wing Tam at pui-wing.tam@nytimes.com, contact her on Twitter at @puiwingtam, or phone the New York Times at 844-

...