trentmkelly/LessWrong-43k huggingface dataset:
Safe Scrambling?
by Hoagy — 2020-08-29T14:31:27.169Z

Status: half-formed thought on a potential piece of an alignment strategy that I've not heard discussed but probably exists somewhere, might just be missing a concept name.

Idea

Any alignment scheme that plans to allow potentially unsafe systems to be trained but then tests their safety before deploying runs the risk of being uncompetitive by requiring the system to be trained many times before a safe system is found.

One solution to this problem is to find ways of 'scrambling' an unsafe system to quickly create many variants of the system. It would require that in some of the new systems, the subsystems that have learned to perform to a high level are retained, or remain sufficiently intact to be retrained far more quickly, without retaining the elements that made the original unsafe.

Has there been much work in this area? I'm not really sure where to start looking, but curious to pursue since it suggests a practical research direction that could be worked on with current systems and plausibly assist alignment proposals.

Thoughts on scrambling as a safety component

Retaining some of the previous structures would be likely to result in the same incentive gradients that led to the development of an unsafe system in the first place, reducing the safety of the system.

More worryingly, it could result in being able to quickly train a variety of unsafe systems, thereby acting as an adversarial attack on whatever safety-checker is in place, so you'd be wary of using such a system unless you were very confident in the robustness of the safety system.

On the other hand, we might expect that if the computation can loosely be decomposed into that which allows good performance in training and that which provides potentially unsafe goals then scrambling might prove an easy way to retrain the offending subsection.

The crux of this would potentially be the extent to which the nature of the non-alignment is bound up with the computation which makes it successful. For example, a reasoner which deduces correct answers indirectly as an emergent property of maximising some non-aligned goal would be difficult to decompose in this way, whereas a system which directly solves the given problem, but only under certain conditions which always hold in the training distribution might be decomposed more easily.

If we were picking functions according to NNs with gradien...
Spoiler-Free Review: Death and Taxes
by Zvi — 2020-08-29T13:10:01.216Z

Death and Taxes is a short Tier 3 game, meaning it is a Good game but not universally Worth It or a Must Play. It is a series of discrete choices and has some interesting questions to ask, and I likely will use it as a reference point in the future once I’ve given people a chance to play. People who don’t generally play games should still be good to go here.

The game is on sale for $8.44 on Steam through the 31st.

This is one of those experiences that is best when you know as little information as possible going in, so that’s all I am going to say at this time. Enjoy!
Please take a survey on the quality/impact of things I've written
by MichaelA — 2020-08-29T10:39:33.033Z

If you’ve read anything I’ve written on the EA Forum or LessWrong, I’d really appreciate you taking this brief, anonymous survey. Your feedback is useful whether your opinion of my work is positive, mixed, lukewarm, meh, or negative. And remember what mama always said: If you’ve got nothing nice to say, self-selecting out of the sample for that reason will just totally bias Michael’s impact survey.

Don’t Panic![1]

I plan to use people’s responses as inputs - rather than definitive answers - in my ongoing efforts to plan my career and improve in various ways. And I’ll combine these inputs with a lot of other inputs.

Thus, you shouldn’t feel that this is uncomfortably high-stakes, nor that you should only take the survey if you’re really confident in what you’d say. You can just provide any tentative thoughts you have, and I can be responsible for working out how much weight I should give them, whether and how they should affect my decisions, etc.

(This is a good division of labour, as I know more about myself and the context of my work than you do, but you have the advantage of existing outside of my swirling vortex of alternating imposter syndrome and overconfidence.)

I’ve also added a comment below with more info on why I’m running this survey, but you’re very welcome to answer the survey without reading that comment.

Update: See also my reflections on the survey responses I've gotten so far, and Should surveys about the quality/impact of research outputs be more common?

The Douglas Adams reference, not the Coldplay song. ↩︎
Updates and additions to "Embedded Agency"
by Rob Bensinger — 2020-08-29T04:22:25.556Z

Abram Demski and Scott Garrabrant's "Embedded Agency" has been updated with quite a bit of new content from Abram. All the changes are live today, and can be found at any of these links:

as a hand-drawn sequence (LW link, AIAF link);
as blog posts (MIRI link, LW link, AIAF link);
and as an arXiv paper (link).

Abram says, "I'm excited about this new version because I feel like in a lot of cases, the old version gestured at an idea but didn't go far enough to really explain. The new version feels to me like it gives the real version of the problem in cases where the previous version didn't quite make it, and explains things more thoroughly."

This diff shows all the changes to the blog version. Changes include (in addition to many added or tweaked illustrations)...

Changes to "Decision Theory":

"Observation counterfactuals" (discussed in the counterfactual mugging section at the end) are distinguished from "action counterfactuals" (discussed in the earlier sections). Action counterfactuals are introduced before the five-and-ten problem.
The introduction to the five-and-ten problem is now slower and more focused (less jumping between topics), and makes the motivation clearer.
Instead of highlighting "Perhaps the agent is trying to plan ahead, or reason about a game-theoretic situation in which its action has an intricate role to play." as reasons an agent might know its own action, the text now highlights points from "Embedded World-Models": a sufficiently smart agent with access to its own source code can always deduce its own conditional behaviors.
ε-exploration and Newcomblike problems now get full sections, rather than a few sentences each.
Added discussion of "Do humans make this kind of mistake?" (Text versions only.)

Changes to "Embedded World-Models":

"This is fine if the world 'holds still' for us; but because the map is in the world, it may implement some function." changed to "... because the map is in the world, different maps create different worlds."
Discussion of reflective oracles now gives more context (e.g., says what "oracle machines" are).
Spend more time introducing the problem of logical uncertainty: emphasize that humans handle logical uncertainty fine (text versions only); say a bit more about how logic and probability theory differ; note that the two "may seem superficially compatible, since probability theory is an extension of Boolean logic"; and des

...
Objective Dog Ratings: The Irish Wolfhound
by Callmesalticidae — 2020-08-29T04:01:08.810Z

Like other sighthounds, Irish wolfhounds are thin, wiry, and not a little bit tall. Indeed, wolfhounds are the tallest of the Tall Dogs. In personality, I have no complaints: they are intelligent, respond well to training, and get along with people so well that they often make terrible guard dogs. They are very unlikely to eat a small child, and that is a definite improvement over the Original Wolf™.

Unfortunately, wolfhounds look unkempt and scraggly, like they just got out of a week-long bender. This is not a dog which cares about maintaining a professional appearance. They mostly die of bone cancer, which is less the fault of bad genes than of size (other large dogs are also prone to bone cancer), but they also have a predisposition to heart problems which does seem genetic in nature.

Irish wolfhounds are perfectly adequate dogs, but they need to dress for the job they want, not the job they have, and their numerous genetic bottlenecks (with subsequent inbreeding) are also a cause for concern.

Rating: ★★ out of ★★★ (Fine)

Original post (w/ more pictures) here.
Objective Dog Ratings: An Introduction & Explanation
by Callmesalticidae — 2020-08-29T03:54:55.898Z

Because they’re not all good dogs, Brent.

Obviously, not everyone wants the same thing from dogs, but that’s not necessarily the same as saying that any dog is as good as the next, or even that any dog rating system must be subjective. If we’re going to entertain the idea that some dogs are simply better than other dogs, though, then we have to specify how that is.

What we’re looking for is a dog with a certain amount of wolfishness, a dog which is as close to being a wolf as one can get without sacrificing any of those essential characteristics which define a dog as such. Basically, a dog which a politically progressive, forward-thinking wolf would not be ashamed to know. They must be loyal, intelligent, and hardworking, they must have a sense of dignity, they must like humans, and above all they must be healthy. A dog which is perfect in every other way, but is unhealthy, is a bad dog, because it would not be good to be that dog.

Unfortunately, I’m incapable of doing anything without taking it at least a little bit seriously, so ratings will be on a three-star scale, from ★ (Mediocre) to ★★★ (Good). It is also possible that some dogs will not get any stars at all. Those are bad dogs, Bront.

I do not know which dog breed will turn out to be the dog breed, the dog of the gods, but I do have my suspicions (some breed of spitz-type, probably), and it’s important to note that this has nothing to do with how much I personally like a dog. Some of my favorite dogs will get no more than two stars, and some may even get just one star. This isn’t “Dogs which are the best at sitting in my lap and being petted,” or “Most Instagrammable dogs.”

Let us begin.
The Case for Human Genetic Engineering
by GeneSmith — 2020-08-28T22:21:35.782Z

This is the first post in what I hope will be a series of posts arguing that genetically engineering humans may provide a huge benefit to individuals and society as a whole. In the interest of creating something readable, this post will largely ignore the controversies and unintended consequences of such a project, but I plan to address those in later posts. It will also ignore perhaps the most impactful genetic change of all: increased intelligence. Such a change deserves a post of its own.

I'm also going to be avoiding the 800 pound elephant in the room: the long history of eugenics and the terrible ideologies that embraced it. Ensuring that whatever programs are implemented do not promote some terrible genocidal agenda and do not create some permanent two-tiered society is obviously at the top of the priority list for anyone looking to do real genetic engineering. But like with genetic engineering for intelligence, there is too much to address in one post so I will be saving that for a later one.

A quick disclaimer before I begin: when I first sat down to write a post about genetic engineering, I planned to thoroughly research everything I wrote about and give links to most of the claims I made. While I will do so for what I judge to be the less commonly understood facts presented in this piece, this will not be as thoroughly researched and comprehensive as I originally planned.

As a result, many of the conclusions I draw in this piece will be based on my own incomplete knowledge and are therefore liable to be wrong. If you spot any particularly glaring errors, or if the pacing is off, or if you get too bored and don't finish reading please let me know in the comments. That being said I think I have read enough about this topic to have something worth reading.

Part 1: A Changing World

Human history is a story of accelerating change. The rapid growth in brain size and general intelligence that took place between 3 million and 50,000 years ago enabled the explosion of human populations and power that culminated in our modern globe-spanning civilization. There is still some debate in the field of anthropology about WHY exactly evolution favored larger brain sizes and increased intelligence so consistently for so long. Whatever the reasons were, they must have been very compelling. Relative to resting metabolic rate -- the total amount of calories an an...
My guide to lifelogging
by Matthew Barnett — 2020-08-28T21:34:40.397Z

I've defended the practice of lifelogging as a means of life extension here. In this post I'll provide a fairly comprehensive guide on how to lifelog. Since lifelogging exists on a spectrum from "taking a picture every so often" to "recording every single detail of your life, in uncompressed HD video along with continuous MRI scans and storage in a nuclear-safe vault" this guide will present two categories for lifelogging, the first for lower cost options and the second higher-cost options. "Cost" here refers not only to the monetary price of buying the equipment, but also the convenience costs of setting up the equipment, storing the data, and perhaps social embarrassment.

Over the last several months I have spent many hours of research to determine the best setups in terms of time and energy required to record my life. I also recommend viewing Mati Roy's setup.

I intend to update this guide as I learn more, so keep in mind that this post is a work-in-progress.

Lower-cost lifelogging

Archiving social media

The lowest hanging fruit of lifelogging is probably creating a long-term archive of your social media data. The method of archiving your social media data will necessarily depend on the websites you visit, but here are some guides for common websites:

Facebook
Discord
Instagram
Reddit

Keylogging

I do not keylog myself, but Mati Roy has informed me that Spyrix works well.

Taking pictures

These days, smartphones generally have high quality cameras, and are much less of a hassle than buying professional equipment.

Storage

Since everything in the lower-cost section here takes up a small amount of space, cloud storage is an appropriate way of storing data for the long-term. Google Drive offers 15 GB of free storage, though I would also suggest storing a local copy of every file along with checking out the subheader on audio and video compression, and the subheader on long-term storage in the section on higher cost lifelogging.

[ETA: Mati Roy comments below that Google Photos offers unlimited free storage for pictures and video.]

Higher-cost lifelogging

Screen recording

The most salient way that I lifelog is by recording everything that happens on my computer screen, along with a full video of my face and room. I achieve this setup by using Open Broadcaster Software (OBS) and record a continuous split-screen between my screen and my USB camera.

I have heard that OBS can be quite ...
How to teach things well
by Neel Nanda — 2020-08-28T16:44:27.817Z
Link: https://www.neelnanda.io/blog/mini-blog-post-18-how-to-teach-things-well

(This is a post on my thoughts on good teaching techniques from a daily blogging project, that I thought might be of interest to LessWrong readers)

Introduction

This is a blog post on how to teach things well. I’ll mostly be focusing on forms of teaching that involve preparation and structure, like talks and tutoring, but these ideas transfer pretty broadly. I think teaching and explaining ideas is an incredibly important skill, and one that most people aren’t great at. I’ve spent a lot of time practicing teaching ideas, and I think I’ve found a bunch of important ideas and approaches that work well. I’m giving a talk next week, so I’ll initially focus on how to give good talks, but try to outline the underlying concepts and high-level ideas of teaching. And then talk about how these can transfer to contexts like tutoring, and to teaching specifically maths or applied rationality - the main areas I have actual teaching experience with.

Note: I mostly care about teaching concepts and ideas, and teaching things to people who genuinely want to learn and be there, so my advice will focus accordingly.

I think it’s useful to think about good teaching even if you don’t intend to spend much time teaching - learning and teaching are flip sides of the same process. I’ve found that even when in the role of a student, understanding what good teaching looks like can often fix a lot of the shortcomings of a bad teacher!

Framing

The key insight of this post is that good teaching requires you to be deliberate, and keep the purpose in mind: learning is a process of information compression. When you’re learning something new, you essentially receive a stream of new information. But human cognition doesn’t work by just storing a flood of information as is. The student takes in the information stream, extracts out the important ideas, converts it to concepts, and stores those in their mind. This is a key distinction, because it shows that the job of a teacher is not to give the student information, it’s to get the student to understand the right concepts. Conveying information is only useful as a means to an end to this goal.

In practice, it often works to just give a stream of information! Good students have learned the skill of taking streams of information and converting it to concepts. Often this happens implicitly, they student will absorb and memorise a lot of data, and over t...
Zibbaldone With It All
by Randomini — 2020-08-28T11:37:22.263Z

Less Zettelkasten! More Zibbaldone!

A Zettelkasten requires you to intricately interconnect and crosslink your thoughts, figuring out exactly how each concept relates to every other concept. A Zibbaldone is writing down whatever random thing comes to mind - an omnisubject diary of sorts. That's what people are throwing into their Zettelkasten, with their careful annotations and interrelations. What if everyone could access everyone else's Zettelkasten, and interrlate them into a glorious omniscient noospheric substrate? Wow! Whatta thought!

Buuuuuuullshit. This is semantic web stuff all over again - and right at the cusp of it having been clearly and entirely outmoded by automatic natural language processing. We don't need to manually mark up pages when Google will index them all for us anyway! Humans might think in that chaotic, random Zettelkastian way, but explicit sentence composition and task completion is temporally linear, and so we need to output our thoughts in much the same way. Not that we wouldn't love to manually annotate our every passing thought, but god, who has time for that?

What we're really missing is a tool that will take whatever random trash I throw out of my brain, figure out how it relates to historical junk I threw out of my brain, and point out the connections for me. Hell, ideally mid-composition: a modern ersatz Clippy pops up - "Hey, it looks like you're talking about Wittgenstein again! Do you want me to autofill what you already think about it, or have you changed your mind about the topic?"

Not only that: once you get your thoughts out of your head bereft of order, an AI could rephrase your thoughts back to you more clearly, more concisely, help interrogate what you mean, point out logical contradictions, challenge your ideas. We need only Zibbaldone out our stream of consciousness and it can be autonomously interrlated into a glorious omniscient noospheric substrate! Wow! Whatta thought!

...just ignore the risks of mentally coupling with an opaque box, whose biases you don't understand, whose intentions may be short term and not long term, who may network you with exactly the wrong people. Zibbaldone to Zettelkasten mapping seems like a middling-difficulty problem with existing tools, and whoever solves it is going to make a lot of money. When we can auto-generate a Zettelkasten from a...
"On Bullshit" and "On Truth," by Harry Frankfurt
by Callmesalticidae — 2020-08-28T00:44:49.126Z

Salticidae Philosophiae is a series of abstracts, commentaries, and reviews on philosophical articles and books.

Harry Frankfurt asks, “What is bullshit, anyway?” Also, “What is truth?” but we all know that book proposal wouldn’t have flown except as a companion to the first one.

Highlights

Something can be true, and still be bullshit.
Something can be a lie, and yet not be bullshit.
Bullshit is that which is (1) unconcerned with truth and (2) intended to change attitudes rather than beliefs.
Truth is useful to us as individuals and as societies
Truth-seeking and truth-telling must be rewarded and their inverse must be punished.
Truth is truth whether or not anyone believes it or even knows it.

New or uncommon terminology

On Bullshit is described as a prolegomenon to On Truth, or an extended introduction that serves to discuss and interpret the work in a manner that is more exhaustive than the typical introduction.

Book-by-book

On Bullshit

There is not much literature on bullshit, and no "theory of bullshit" or rigorous analysis thereof. This is in large part because we all assume that we recognize and evade bullshit pretty well.

According to Max Black, humbug is essentially a (false) statement made, not to convince you about that thing, but to convince you of something else. For example, one might make blatantly and obviously exaggerated or otherwise false statements about U.S. history not to convince another of these things, but to convince another of one's patriotic fervor.

Starting from this definition of humbug, Frankfurt makes a number of comparisons and caveats that might be useful:

Bullshit may be made carelessly, and we could easily compare bullshit to shoddy goods.
Shit is excreted, not crafted. However, advertising can be carefully-crafted bullshit.
Similes are not lies, but they can be made too thoughtlessly. In their own way, they can be bullshit.

Frankfurt argues that bullshit is, to start with, deliberate misrepresentation. Some say that lying requires intent; others, that any false statement is a lie. Bullshitting, however, is not exactly the same as lying. Indeed, bullshit can be true. Frankfurt's position is that bullshit is distinguished not by its truth or falsity, but by a disregard for the truth; as he puts it, honest folk and liars are playing the same game, to convey the facts or to obsc...