trentmkelly/LessWrong-43k huggingface dataset:
Alignment Newsletter #26
by Rohin Shah — 2018-10-02T16:10:02.638Z

Highlights

Building safe artificial intelligence: specification, robustness, and assurance (Pedro A. Ortega, Vishal Maini et al) (summarized by Richard): In this blog post, the DeepMind safety team divides AI safety into the problems of specification, robustness and assurance. Specification ensures that an AI system's behaviour aligns with the operator's true intentions, i.e that our "ideal specification" of a fully alligned AI system is consistent with the AI's actual behavior. Problems in this category include avoiding side-effects and the creation of dangerous sub-agents. Robustness ensures that an AI system continues to operate within safe limits upon facing peturbations like adversarial inputs or distributional shift, both by preventing these problems arising and by being able to recover from them. It also includes criteria like safe exploration and cautious generalisation. Assurance ensures that we can understand and control AI systems during operation, by monitoring them and enforcing restrictions. Interpretability and interruptability are examples of monitoring and enforcement respectively. I'd encourage you to look at the table in the original post, since it also categorises many more AI safety problems.

Richard's opinion: I like this framing - I think it's an improvement on the categorisation into specification and robustness problems from the AI safety gridworlds paper. In particular, it's useful to separate properties that we want an AI to have from mechanisms by which we might control or limit the damage from an AI that doesn't have those properties.

I do worry that this approach doesn't contain scalability as a core concern in the same way that IDA and agent foundations do. Solutions to AI safety problems which work for sub-human-level AI and even human-level AI may not translate to superintelligent AI. Relatedly, I think that maintaining the same goals under distributional shift should be considered a specification problem, because as AIs get smarter they'll be able to handle tasks increasingly different from the ones they trained on, and also because scaling up a system is itself a form of distributional shift.

Rohin's opinion: I also like this framing. Unlike Richard, I am not worried about the absence of scalability as a core concern -- scalability seems to be a desideratum about ...
Separate comments feeds for different post listings views?
by Said Achmiz — 2018-10-02T16:07:22.942Z

(Inspired by discussions like this one, but also a thing that I’ve thought for a while would be a good idea.)

Currently, there’s a single “recent comments” feed, which includes comments posted to any post—frontpage, Meta, personal blogs, Alignment Forum… even drafts! (The last of which, I assume, is merely a bug.)

It would be cool if there were multiple comments feeds, one for each view, viz.:

Recent comments on all posts

Recent comments on frontpage posts only

Recent comments on Alignment Forum posts only

Recent comments on Meta posts

It’s probably a lot of work for you guys to implement a UI for this, so I can totally understand it not being priority, but would it be possible to implement this feature in the backend and expose it in the API?
Feedback from emotions
by Elo — 2018-10-02T03:53:09.614Z

Bearlamp: Previous, First
Greaterwrong: Previous, First
Lesswrong: Previous, First

Mental health can often feel like the inability to get clarity around if it’s, “just me” or if it’s “the world” that’s crazy. There’s an open question in any interpersonal problem “is it me or is it them”. Basic game theory might have you look at the general strategies and take a precommitment, like Tit for tat, with forgiveness. Something like, “It’s always me” or “it’s always them” - as the opinion that is formed in response to the stimuli being presented. These strategies tend to look like mental health problems when applied far too liberally. Some examples of these are in the List of Maladaptive Schemas.

If you play fixed mindset belief games, you will be bested by people who can see your fixed mindset and predict it. And beat it.

Unfortunately for basic game theory, advanced game theory comes along and sees all the other people playing with Tit-for-tat, with forgiveness strategies and generates a one-up strategy whereby advanced game theoretic players can beat basic game theoretic players, Just by playing one move ahead of the basic players.

(movie: The Princess Bride)

Unfortunately for advanced game theory, there exists expert game theory players who have seen that strategy and devised advanced strategies for solving the “how do I beat basic, and advanced game players”.

And unfortunately for expert game theory players there exists the halting problem. Where there will always be another level of play strategy. And there will always be another strategy taking into account all previous strategies. And this is an infinite loop.

how do I get feedback on an infinitely recursive system with the halting problem?

This question strikes at the core of the interface between self and the external world. We are each a chinese room brain. This is the problem of other minds. When we design an experimental apparatus and attempt to glean feedback information from reality as if we are not in it, we don’t really answer the question here.

I only have one answer. And it’s an unfortunately frustrating one. I hint at the answer in the emotional training model but that’s not ultimately obvious enough.

Feedback has to come from within.

How do I know what to do? How do I gauge what is right and wrong where all I have to go on is the intention to gage right and wrong, and a collection of informat...
LW Update 2018-10-01 – Private Messaging Works
by Raemon — 2018-10-01T21:28:47.017Z

Along with various performance updates to improve site-speed, we've finally got a private messaging system that looks reasonable (and now works properly on mobile)

Noteworthy updates:

When you send someone a private message (by navigating to their profile page and clicking "send a message") it automatically creates a new conversation with that user.
You can click "conversation options" to add new people to the conversation, or give it a custom title.

There are still some obvious improvements to make (such as making it easier to message a post author from a post rather than having to navigate to their profile), but the system should be basically usable now.
What the Haters Hate
by Jacob Falkovich — 2018-10-01T20:29:42.489Z

Cross-posted from Putanumonit. A bit more culture-warlike and snarky than the LW standard. Won't make as much sense to people outside the US. Previously on the subject of the IDW and outgroups.

Ezra Klein’s latest essay on the YouTube right / Intellectual Dark Web is perhaps the voxiest Vox article ever. The Vox specialty is walking 90% of the way towards truth in careful, reasoned steps, and then taking a left turn off a steep cliff.

Klein’s article responds to a challenge by Dave Rubin, wondering how he (Rubin) suddenly became a conservative reactionary:

Group Definitions

Klein claims that Dave Rubin is part of some group, and they both agree that it’s a bad group to be part of. The question is: who is Dave Rubin in a group with? And what makes them part of the same group?

You could ask Dave Rubin which groups he identifies with, and he’ll name examples like liberals, gays, Jews, and the Intellectual Dark Web. But Klein notes that even the IDW will “draw a circle around the most respectable subgroup of this world, use their preferred term for themselves, and worry over the unsavory characters they were associating with”. If you trust Rubin to define his own groups, there will be suspiciously few Nazis in them.

Another way to identify this group is via a network graph, a network whose nodes are people Ezra Klein is suspicious of and whose edges are podcast appearances and YouTube interviews. Lucky for Klein, Rebecca Lewis of the Data & Society research institute created a chart of this network using an algorithm, and we all know that algorithms are objective and trustworthy.

(Of course, after the algorithm had run they still had to manually remove all the inconvenient data points, like Noam Chomsky appearing on Stefan Molyneux’ show).

Here’s how it works: Nazis have a Nazi number of 0. If you interact with someone whose Nazi number is k, your number is at most k+1. I interviewed Geoffrey Miller, who did an event with Sam Harris, who recorded a podcast with Ezra Klein, who interviewed W. Kamau Bell, who interviewed Richard Spencer, who’s a known Nazi. So my Nazi number is 5, and yours is at most 6 by virtue of reading Putanumonit. Congratulations! You are within 6 degrees of separation from a Nazi, and so you’re practically one yourself.

Klein noticed that his own Nazi number is an unflattering 2, and reminds us that “it’...
The tails coming apart as a strategy for success
by Bucky — 2018-10-01T15:18:50.228Z

[Epistemic status: A proposed causal mechanism for something I’ve been using for a while which seems to work empirically (n=1!)]

So, the tails come apart. I really like the elegance of this theory and how it seems to touch every part of life. It seems highly applicable.

For example, say you wanted to become the world’s most successful author. There is no single talent for you to learn which makes you the most successful. You must be a good writer but also a good publicist and business manager. Your writing skill can be broken down into various skills such as humour, prose, research ability etc. You need to get good at hiring the right people.

You don’t necessarily need to be the best at any individual skill. The tails come apart so the person who writes the world’s most beautiful prose might not be able to find themselves an audience if they’re lacking in other skills.

This is very similar to Scott Adams’ concept of a talent stack. He claims that having many good but unspectacular skills, which support each other, is a more efficient strategy than working on a single amazing talent.

The tails coming apart is a good causal mechanism to explain how the talent stack might work and significantly increases my estimate of how useful the concept is. I’ve been thinking in talent stack terms for a while and it seems to work for me.

***

I work in mechanical engineering. The talent which sets me apart the most is maths (I studied maths and physics, not engineering). Relatively few people I work with can follow in detail the maths I do, and probably none could generate the work themselves. Within the less wrong community I’m sure I would stand out a lot less but, in my day-to-day work, I could improve my maths overnight +1σ and no-one would notice the difference.

So in order to improve my overall engineering value it is a better use of my time to focus on some adjacent skills which either:

1. Fill in gaps in my knowledge from not studying engineering (low time commitment for big relative gains)

2. Move me into the top 10% within the company in a skill which is useful alongside maths

Sometimes these are completely new skills, sometimes they may be things I’m already good at which can be modified to be more useful at work.

***

Part of me thinks this should be obvious. I suspect some people have a vague model for something similar in their head.

What surprises me is how few people act ...
GreaterWrong—new theme and many enhancements
by Said Achmiz — 2018-10-01T07:22:01.788Z

(Previous posts: [1], [2], [3], [4])

GreaterWrong.com has just added several new features and UI enhancements:

Modern Less Wrong theme

There is now a new theme (bringing the total to nine themes to choose from), called “Less”. (This theme is inspired by the design of the new, i.e. current, Less Wrong site.)

Here’s how it looks on a desktop:

And on a phone:

(See the About page for how to switch themes.)

Mobile theme tweaker

The theme tweaker feature (which lets you do things like invert colors—instantly creating a “dark mode” version of any theme—as well as adjust brightness, contrast, saturation, and hue) is now available on mobile devices.

Open the theme selector (gear button in the lower-left of the screen), and then tap this button:

And you’ll see the theme tweaker screen:

(See the About page for more on the theme tweaker.)

Strong vote display

Strong upvotes and downvotes now display properly on GreaterWrong, in all themes.

(Themes, starting from top left and going across by row: default, grey, ultramodern, zero, brutalist, rts, classic, less.)

Alignment Forum view

Click the “AF” icon next to any Alignment Forum post, and you’ll be taken to a listing of all the Alignment Forum posts.
it's hard to look at reality when you've already decided what reality looks like
by brown — 2018-10-01T02:53:58.278Z

So here I am, posting a thing.

I have a soapbox, which may have unintentionally put myself up as a source of authority, at least to myself.

There's an SMBC comic of Zach Weinersmith comics where he's said words to the effect of “my brain is weird, I get thoughts like “why are people still using this argument I already satirised it in a comic”” [1]

After I wrote my evidence not theory post I noticed some slight irritation while chatting to my father. "Hey you're doing this dynamic even though I already explained to some people on the internet why it's bad".

Let's try and steel man this subagent of mine that is worried about being authoritative,

Giving a speech to a large number of people, future allies and so on is a task I believe I wouldn't take lightly.

You're trying to explain to this crowd of people what your worldview is, what kind of person you are. This allows for greater coordination & cooperation. Folks in the audience will offer you different opportunities depending on who you say you are.

It doesn't seem too unreasonable that committing to be a certain way gets you a dump of utility. As you're "fixed" [2]. Yet it can also constrain you. Pretty sure this is going to happen regardless. [3]

This youtube artist marydoodles has talked about getting stuck in a style. In her case it's watercolour time-lapse with often a funny twist in the painting. I wouldn't be surprised if singers feel similarly, The audience’s “We liked your old stuff better" might be a bit rough. Bo Burnham a comedian has a wonderful if saddening (to me) song on it. [4]

let's imagine in a blog post: I say " X " . Later I find in particular cases turns out " ¬X ", turns out reality is vast and complicated and can't be compressed into little english predicates, what do ya know.

However maybe: I've just announced to the tribe that I'm going to stand by the norm of " X ". I'm kinda motivated not to notice " ¬X " because hey, that would be kinda awkward.

So what is this dynamic and how can I solve it or avoid it:

bias toward reporting experience flavoured things "This saddens me" as opposed to "this is sad"

acknowledging the limit of my current perspective, not constraint my future self through slogans of the form "X is true"

some form of declaimer about the kinds of linguistic mess that I'm about and is happening here.

knowledge of the kinds of people who enjoy reading these things.

If a hypot

...
Leto among the Machines 
by Virgil Kurkjian — 2018-09-30T21:17:11.223Z

I've always been surprised that there's not more discussion of Dune in rationalist circles, especially considering that:

1. It's a book all about people improving their minds to the point where they become superhuman.

2. It's set in a world where AI Goal Alignment issues are not only widely understood, but are integrated into the foundation of every society.

3. It's ecological science fiction — dedicated to "the dry-land ecologists, wherever they may be" — but what that secretly means is that it's a series of novels about existential risk, and considers the problem on a timescale of tens of thousands of years.

For those of you who are not familiar, Dune is set about 20,000 years in the future. About 10,000 years before the events of the first book, Strong Artificial Intelligence was developed. As one might expect, humanity nearly went extinct. But we pulled together and waged a 100-year war against the machines, a struggle known as the Butlerian Jihad (This is why a butler is “one who destroys intelligent machines”). We succeeded, but only barely, and the memory of the struggle was embedded deep within the human psyche. Every religion and every culture set up prohibitions against "thinking machines". This was so successful that the next ten millennia saw absolutely no advances in computing, and despite the huge potential benefits of defection, coordination was strong enough to prevent any resurgence of computing technology.

Surprisingly, the prohibition against "thinking machines" appears to extend not only to what we would consider to be Strong AI, but also to computers of all sorts. There is evidence that devices for recording journals (via voice recording?) and doing basic arithmetic were outlawed as well. The suggestion is that there is not a single mechanical calculator or electronic memory-storage device in the entire Imperium. There are advanced technologies, but nothing remotely like computers — the Orange Catholic Bible is printed on "filament paper", not stored on a Kindle.

While I appreciate the existential threat posed by Strong AI, I've always been confused about the proscription against more basic forms of automation. The TI-81 is pretty helpful and not at all threatening. Storing records on paper or filament paper has serious downsides. Why does thi...
The Unreasonable Effectiveness of Deep Learning
by Richard_Ngo — 2018-09-30T15:48:46.861Z
Link: http://thinkingcomplete.blogspot.com/2018/02/the-unreasonable-effectiveness-of-deep.html

This essay consists of summaries, explanations, and discussions of several papers which provide high-level arguments and intuitions about why, conceptually, deep learning works. Particular areas of investigation are "Which classes of functions can deep neural nets approximate well in principle?"; "Why can they quickly learn functions which have very small training loss?"; and "Why do the functions they learn generalise so well?".

Understanding deep learning requires rethinking generalisation - Zhang, Bengio, Hardt, Recht and Vinyals, 2017

Machine learning in general is about identifying functions in high-dimensional spaces based on finitely many samples from them. In doing so, we navigate between two potential errors: learning a function which is too simple to capture most of the variation in our data (underfitting) and learning a function which matches the data points well, but doesn't generalise (overfitting). Underfitting implies higher training error; overfitting implies higher test error. Attempting to avoid underfitting is called optimisation, and attempting to avoid overfitting is called regularisation. We can represent a particular neural network with w weights as a w-dimensional space; when training it, we are trying to find the point in that space which most closely corresponds to the true underlying function. A loss function is a heuristic based on our data which tells us approximately how close that correspondence is; one reason that training is difficult is because good loss functions are generally not convex, but rather have some local minima, which a training method like gradient descent has difficulty escaping. However, we can get better results with stochastic gradient descent (SGD): the randomness provided by only updating some variables allows more chance of escaping local minima, in a way comparable to simulated annealing. [2] SGD also allows proofs of convergence similar to those for total gradient descent, and is much easier to compute.

Another problem is the fact that because our data are finite, there are many models which have very low loss but are very far from the truth. In such extreme cases of overfitting, a learner could effectively "memorise" every piece of input data without their model capturing any of the underlying patterns. An easy way to avoid this is to use a model which doesn&#x...
An Invitation to Measure Meditation
by justinpombrio — 2018-09-30T15:10:30.055Z

Update: Thank you all for the suggestions, and all of the links to resources!

Here is my plan, which I hope will balance measuring useful things with being easy enough that I'll actually stick with it:

Every Monday, I will complete a meditation test on Quantified Mind. On alternating Mondays, I will alternate whether I do the test immediately after meditating or not. These don't look like a gold standard for attention tests, but they're better than what I would code by hand and they're more of a standard.

Every Monday, I will play a game(s?) of Dual 2-back and record my score. (I'll switch to a higher N if 2-back ever gets easy.)

I will use TagTime to randomly ping me for an experience sample as a set of free-form tags, about twice per day, every day.

I'll report back in a couple months with some plots and commentary.

When I wrote this post, my mindset was embarrassingly close to "I'm a Bayesian scientist meditating! It's going to be like Harry and Hermione in HPMOR!". Except of course, if there really were magic, a ton of people would already have been studying it, and the best way to start would be by asking about it. Thanks for all the advice.

An Invitation to Measure Meditation

I've begun to meditate regularly, so it seems like a good opportunity
to experiment and search for measurable effects of meditation. This is
an open invitation to propose easy measurements for me to
intermittently perform. I don't have any brilliant ideas for what to
measure (beyond looking at Wikipedia's list of
research on meditation),
but it seems silly to waste the opportunity.

How am I meditating?

For the past ~20 days, I've been meditating while walking for an hour
or two, though I intend to switch to primarily breath-following
meditation. At least five days a week, one hour a day. I expect to
keep this up for a couple months, but it's a big commitment so no
promises.

I am following a ~500 page textbook on meditation called The Mind

Illuminated, by John Yates (a.k.a., Culadasa) and others. It is, to
quote the cover, "A complete meditation guide", which "integrat[es]
Buddhist wisdom and brain science for greater mindfulness". Most of it
is organized into instructions and techniques for how to meditate in
10 stages (corresponding roughly to the 9 stages described by
Asanga, which I can't find a
reference for), but there are also five "interludes" that describe
five increasingly

...