trentmkelly/LessWrong-43k huggingface dataset:
Convict Conditioning Book Review
by Borasko — 2021-04-09T22:10:58.939Z

The Book:

In 1979 Paul Wade was admitted into San Quentin State Prison, being locked up he discovered things lost to the outside world. Wade ended up despising most things preached about the industry from the outside, from the top level of media all the way down to to the laypersons' knowledge on the subject. After 23 years in prison he wrote this book to show how he survived all that time with his sanity intact, share the knowledge he gained, and correct the narratives that he sees as destroying people's ability to gain functional strength.

This is a strength training book, but one that might be different than most. In this book the prison system and his survival in it is a supporting character, to the main star that is calisthenic strength training. Paul Wade, or as he said he was known in prison, Paul “Coach” Wade, discovered “old school” calisthenics from the other older inmates he interacted with. Over time he learned to love strength training and the art of making one’s self stronger, passionate about the craft and the history of how humans were able to become strong throughout the ages. This was definitely influenced by him needing to survive in prison, where fights were explosive and only the strongest were to survive.

“For more years than I dare to count, my training system has kept me physically tougher and head-and-shoulders stronger than the vast majority of psychos, veteranos, and other vicious nutjobs I’ve been forced to rub shoulders with for two decades” -Page 3

Wade goes on a fun few paragraph rant about how calisthenics as a word has been bastardized by the pubic school gym teachers and fitness media to mean “do a couple of pushups and squats” to paraphrase. He thinks that those are better than nothing but they are doing it all wrong. He thinks all of current strength based media is doing it all wrong. He’s pissed at the workout industry because, “The average gym junkie today is all about appearance, not ability”(page 2). Wade’s whole book is made off the thesis of training for strength and function, being able to have a shredded body and use it too. The “use it too” part is especially important, because in the latter part of the first chapter, he claims in prison that being perceived as weak could literally mean death for the individual.

”... other prisoners exploit weakness like they breath air; intimidation is the daily current in...
Identifiability Problem for Superrational Decision Theories
by Bunthut — 2021-04-09T20:33:33.374Z

Superrationality, and generalizations of it, must treat options differently depending on how they're named.

Consider the penny correlation game: Both players decide independently on either head or tails. Then if they decided on the same thing, they each get one util, otherwise they get nothing. You play this game with an exact copy of yourself. You reason: since the other guy is an exact copy of me, whatever I do he will do the same thing. So we will get the util. Then you pick heads because its first alphabetically or some other silly consideration, and then you win. How good that you got to play with a copy, otherwise you would have only gotten half an util.

Now consider the penny anti-correlation game: Both players decide independently on either head or tails. If they decided the same thing, they get nothing, otherwise they get one util each. You play this game with an exact copy of yourself. You reason: since the other guy is an exact copy of me, whatever I do he will do the same thing. So the best I can do is to pick randomly with 50% chance, that way I get half an util. Thats if the gamemaster is nice. If he isnt nice, then identical copies in identical environments get the same result from their RNG. In that case you lose the game whatever you do. How bad that you had to play with a copy, otherwise you could have gotten half an util.

The problem is that these are the same game, only with the labels for one players actions switched (or for the other player. which is exactly the problem). Despite this, superrational reasoning gives us different results. This could happen because we have taken a theoretical symmetry from a physical one, and physical symmetry can pay attention to the detailed makeup of symbols. If we are aware of the difference, it should not be too surprising that there is a setup exploiting it.

Now I think the reasoning presented is correct in both cases, and the lesson here is for our expectations of rationality. From what I've seen people formed their expectations about what algorithm-controlling decision theories will do when they are worked out largely around rationality conditions, and what ought to be the outcome. In this way, common knowledge of rationality was supposed to guarantee various things, like pareto-optimal equilibria or an existent and unique solution concept. This has taken a blow here. Superrationality, from which...
Opinions on Interpretable Machine Learning and 70 Summaries of Recent Papers
by  — 2021-04-09T19:19:42.826Z

Peter Hase

UNC Chapel Hill

Owen Shen

UC San Diego

With thanks to Robert Kirk and Mohit Bansal for helpful feedback on this post.

Introduction

Model interpretability was a bullet point in Concrete Problems in AI Safety (2016). Since then, interpretability has come to comprise entire research directions in technical safety agendas (2020); model transparency appears throughout An overview of 11 proposals for building safe advanced AI (2020); and explainable AI has a Twitter hashtag, #XAI. (For more on how interpretability is relevant to AI safety, see here or here.) Interpretability is now a very popular area of research. The interpretability area was the most popular in terms of video views at ACL last year. Model interpretability is now so mainstream there are books on the topic and corporate services promising it.

So what's the state of research on this topic? What does progress in interpretability look like, and are we making progress?

What is this post? This post summarizes 70 recent papers on model transparency, interpretability, and explainability, limited to a non-random subset of papers from the past 3 years or so. We also give opinions on several active areas of research, and collate another 90 papers that are not summarized.

How to read this post. If you want to see high-level opinions on several areas of interpretability research, just read the opinion section, which is organized according to our very ad-hoc set of topic areas. If you want to learn more about what work looks like in a particular area, you can read the summaries of papers in that area. For a quick glance at each area, we highlight one standout paper per area, so you can just check out that summary. If you want to see more work that has come out in an area, look at the non-summarized papers at the end of the post (organized with the same areas as the summarized papers).

We assume readers are familiar with basic aspects of interpretability research, i.e. the kinds of concepts in The Mythos of Model Interpretability and Towards A Rigorous Science of Interpretable Machine Learning. We recommend looking at either of these papers if you want a primer on interpretability. We also assume that readers are familiar with older, foundational works like "Why Should I Trust You?: Explaining the Predictions of Any Classifier."

Disclaimer: This post is written by a team of two people, and hence its breadt...
My Current Take on Counterfactuals
by abramdemski — 2021-04-09T17:51:06.528Z

[Epistemic status: somewhat lower confidence based on the fact that I haven't worked out a detailed theory along the lines I've suggested, yet.]

I've felt like the problem of counterfactuals is "mostly settled" (modulo some math working out) for about a year, but I don't think I've really communicated this online. Partly, I've been waiting to write up more formal results. But other research has taken up most of my time, so I'm not sure when I would get to it.

So, the following contains some "shovel-ready" problems. If you're convinced by my overall perspective, you may be interested in pursuing some of them. I think these directions have a high chance of basically solving the problem of counterfactuals (including logical counterfactuals).

Another reason for posting this rough write-up is to get feedback: am I missing the mark? Is this not what counterfactual reasoning is about? Can you illustrate remaining problems with decision problems?

I expect this to be much more difficult to read than my usual posts. It's a brain-dump. I make a lot of points which I have not thought through sufficiently. Think of it as a frozen snapshot of a work in progress.

Summary.

I can Dutch-book any agent whose subjective counterfactual expectations don't equal their conditional expectations. I conclude that counterfactual expectations should equal conditional probabilities. IE, evidential decision theory (EDT) gives the correct counterfactuals.
However, the Troll Bridge problem is real and concerning: EDT agents are doing silly things here.
Fortunately, there appear to be ways out. One way out is to maintain that subjective counterfactual expectations should equal conditional expectations while also maintaining a distinction between those two things: counterfactuals are not computed from conditionals. As we shall see, this allows us to ensure that the two are always equal in real situations, while strategically allowing them to differ in some hypothetical situations (such as Troll Bridge). This seems to solve all the problems!
However, I have not yet concretely constructed any way out. I include detailed notes on open questions and potential avenues for success.

What does it mean to pass Troll Bridge?

It’s important to briefly clarify the nature of the goal. First of all, Troll Bridge really relies on the agent “respecting logic” in a particular sense.

Learning agents who don’t reason logic

...
[AN #145]: Our three year anniversary!
by Rohin Shah — 2021-04-09T17:48:21.841Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer.

HIGHLIGHTS

Alignment Newsletter Three Year Retrospective (Rohin Shah) (summarized by Rohin): It’s (two days until) the third birthday of this newsletter! In this post, I reflect on the two years since the previous retrospective (AN #53). There aren’t any major takeaways, so I won’t summarize all of it here. Please do take this 2 minute survey though. I’ll also copy over the “Advice to readers” section from the post:

Don’t treat [newsletter entries] as an evaluation of people’s work. As I mentioned above, I’m selecting articles based in part on how well they fit into my understanding of AI alignment. This is a poor method for evaluating other people’s work. Even if you defer to me completely and ignore everyone else’s views, it still would not be a good method, because often I am mistaken about how important the work is even on my own understanding of AI alignment. Almost always, my opinion about a paper I feel meh about will go up after talking to the authors about the work.

I also select articles based on how useful I think it would be for other AI alignment researchers to learn about the ideas presented. (This is especially true for the choice of what to highlight.) This can be very different from how useful the ideas are to the world (which is what I’d want out of an evaluation): incremental progress on some known subproblem like learning from human feedback could be very important, but still not worth telling other AI alignment researchers about.

Consider reading just the highlights section. If you’re very busy, or you find yourself just not reading the newsletter each week because it’s too long, I recommend just reading the highlights section. I select pretty strongly for “does this seem good for researchers to know?” when choosing the highlight(s).

If you’re busy, consider using the spreadsheet data...
Why unriggable *almost* implies uninfluenceable
by Stuart_Armstrong — 2021-04-09T17:07:07.016Z

When talking about learning processes, I've said things like "every uninfluenceable learning process is unriggable" and "every unriggable learning process is uninfluenceable, if the set of environments is large enough".

In this post, I want to clarify the relationship between the two, in ways that are hopefully understandable. I won't be using much math, but several hopefully intuitive examples.

The main points I hope to get across are:

Uninfluenceable and unriggable are of different types. Unriggable is algebra about your learning process; uninfluenceable is about learning independent facts about the environment.

Nevertheless, these are almost equivalent.

Unriggable process that are not uninfluenceable are specific and unstable, and generally collapse if the agent can gain extra information.

The third point is new, so will be of interest even to those who already understand these concepts.

Informal definitions

Here, it suffices to think of uninfluenceable as meaning "learning about values/preferences is the same as learning objective facts about the universe" and unriggable as "you can't push the expected probability of a value/preference in a particular direction".

Unriggable-but-influenceable

Parents guide you

I've used an example involving a robot agent asking its parents, so let's continue with that. There is a robot who doesn't know whether to become a banker or a doctor. To figure this out, it can ask either of its parents:

Its possible actions are M (ask mother) and F (ask father). When it does so, the relevant parent will "banker" (B) or "doctor" (D), which will fully resolve the issue.

So, given the possible histories, this is what the robot will learn, with actions in red and observations in blue:

[M or F]B→ become a banker [M or F]D→ become a doctor

Let's μBB be the world in which both parents will say "banker", μBD the world in which the mother says "banker" and the father says "doctor", μDB the world in which they will say the opposite, and μDD the world in which both will say "doctor".

Then the learning process is unriggable if the robot cannot push towards "doctor" or "banker" via its actions. This means that the probabilities of D/B are the same whichever parent the robot asked. This happens when the robots (subjective) probabilities of μDB and μBD are the same[1].

Unriggable-but-influenceable is unstable

If the...
Covid 4/9: Another Vaccine Passport Objection
by Zvi — 2021-04-09T14:50:01.628Z

I’ve been travelling to New York City once again, and it’s been a busy week, including putting in bids on multiple different apartments. Developments are ongoing, so I’ve been even busier than usual and it’s likely some stuff has slipped through the cracks.

The overall situation seems unchanged. As people take more risks and the new strains dominate, we’ve settled into about a 10% week over week growth in positive rates, but due to vaccinations death rates continue to decline. That means that one’s risk as an unvaccinated person continues to rise, even as risk on a population level goes down. With everyone eligible now or at least soon, I’d urge once again for everyone who isn’t yet vaccinated to schedule an appointment as soon as possible.

Many places that have less robust vaccination efforts than America are seeing record high case numbers and hospitalization numbers, as the new strains overwhelm efforts that were previously calibrated to let things stabilize. Things are quite bad out there in many countries. Check your local situation, and act accordingly.

Vaccination rates have stalled out in America, but at 3 million shots a day, which will still get us there pretty soon. Unless something unexpectedly makes our situation worse, we should reach the tipping point on vaccinations within a month or two, at most within three, after which we see a steady and accelerating improvement across the board.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: Positivity rate of 5.3% (up 0.4%) and deaths are unchanged.

Result (on Friday):

Deaths falling a lot indicates that last week’s failure to drop was a measurement error, and the numbers from Wikipedia were more accurate. On reflection, I should have realized this and made a better prediction. The positive test rate number matches my intuition.

I notice I am confused by the continuing decline in test counts. It’s faster than the rate of new vaccinations, and the number of cases isn’t declining. I haven’t seen a good explanation. Seems like an important piece of the puzzle.

Prediction for next week: Positivity rate of 5.9% (up 0.4%) and deaths decline by 8%.

Deaths

Oklahoma reported 1,716 deaths due to finding previously uncounted ones. I’m calling that 100 deaths from this week based on their previous weeks, and assuming the rest were in the past.

Date	WEST	MID

...
Beliefs as emotional strategies
by Kaj_Sotala — 2021-04-09T14:28:16.590Z

In “

Crony Beliefs

”, Kevin Simler suggests the analogy of beliefs as employees in a company, who may be hired to carry out different tasks. “Meritorious” ones, he suggests, are in the business of accurately modeling the world, whereas “crony” employees are there just to win favor from others.

I like the analogy (“employees within a brain” sounds a lot like “

subagents

”) and the idea of beliefs that have been "hired" for the purpose of currying favor from others, but I think the essay could go further still.

I think that many beliefs that do not appear at all crony are nonetheless rooted in complicated social strategies and that one’s thinking can drastically change by altering the assumptions behind those strategies. The strategies in question can be much subtler than just “believing what others in your current social group believe”. Rather, such socially motivated beliefs can include deeply-held lifelong conceptions of metaphysical topics such as the nature of free will, morality, and talent.

Let’s look at this in the light of three real-world case studies.

Believing in free will to earn merit

(The person in question reviewed this section before publication and let me know which parts were fine to share.)

Some time ago I happened to talk with someone who mentioned that they had a strong need to believe in

non-deterministic free will

. They knew that this made no intellectual sense, but they still needed to maintain this as a compartmentalized belief. At a time when they were younger, they'd stopped believing in free will and had gotten into a suicidal depression. The only way they got through was by coming to believe in free will again.

They explained that they didn’t know why they needed to believe in it, and figured that their motivational system was just intrinsically wired up that way. It felt like a hardwired, unquestionable innate need.

In response, I said that in my experience, these kinds of things are learned emotional strategies of some sort. They were skeptical of that claim… but thought about it for a while anyway. After a while, they told me that they had been raised to believe that only things you got with hard work mattered. Most of their life, they had specifically chosen to do hard things because doing easy things had made them feel unbearably empty.

The way they thought about it, for something to be hard work, it has to be your own choice. And s...
Highlights from The Autobiography of Andrew Carnegie
by jasoncrawford — 2021-04-08T22:03:19.231Z
Link: https://rootsofprogress.org/andrew-carnegie-autobiography

I’ve been reading Andrew Carnegie’s autobiography, published late in his life, in the early 1900s. Here are some interesting themes and quotes. (Emphasis added in all block quotes below.)

Science and steel

One key to Carnegie‘s success in the iron business is that he was one of the first to seriously apply chemistry:

Looking back to-day it seems incredible that only forty years ago (1870) chemistry in the United States was an almost unknown agent in connection with the manufacture of pig iron. It was the agency, above all others, most needful in the manufacture of iron and steel. The blast-furnace manager of that day was usually a rude bully… who in addition to his other acquirements was able to knock down a man now and then as a lesson to the other unruly spirits under him. He was supposed to diagnose the condition of the furnace by instinct, to possess some almost supernatural power of divination, like his congener in the country districts who was reputed to be able to locate an oil well or water supply by means of a hazel rod. He was a veritable quack doctor who applied whatever remedies occurred to him for the troubles of his patient.

Part of the problem was that the ores and other inputs to smelting were inconsistent in composition:

The Lucy Furnace was out of one trouble and into another, owing to the great variety of ores, limestone, and coke which were then supplied with little or no regard to their component parts. This state of affairs became intolerable to us.

This is where chemistry was able to help:

We finally decided to dispense with the rule-of-thumb-and-intuition manager, and to place [Henry Curry] in charge of the furnace….

The next step taken was to find a chemist as Mr. Curry’s assistant and guide. We found the man in a learned German, Dr. Fricke, and great secrets did the doctor open up to us. Ironstone from mines that had a high reputation was now found to contain ten, fifteen, and even twenty per cent less iron than it had been credited with. Mines that hitherto had a poor reputation we found to be now yielding superior ore. The good was bad and the bad was good, and everything was topsyturvy. Nine tenths of all the uncertainties of pig-iron making were dispelled under the burning sun of chemical knowledge.

It wasn’t just that some materials were of low quality, but that the right mix of materials was needed, no matter the purity of the inputs:

A

...