trentmkelly/LessWrong-43k huggingface dataset:
Problems in AI Alignment that philosophers could potentially contribute to
by Wei Dai — 2019-08-17T17:38:31.757Z

(This was originally a comment that I wrote as a follow up to my question for William MacAskill's AMA. I'm moving it since it's perhaps more on-topic here.)

It occurs to me that another reason for the lack of engagement by people with philosophy backgrounds may be that philosophers aren't aware of the many philosophical problems in AI alignment that they could potentially contribute to. So here's a list of philosophical problems that have come up just in my own thinking about AI alignment.

Decision theory for AI / AI designers

How to resolve standard debates in decision theory?

Logical counterfactuals

Open source game theory

Acausal game theory / reasoning about distant superintelligences

Infinite/multiversal/astronomical ethics

Should we (or our AI) care much more about a universe that is capable of doing a lot more computations?

What kinds of (e.g. spatial-temporal) discounting is necessary and/or desirable?

Fair distribution of benefits

How should benefits from AGI be distributed?

For example, would it be fair to distribute it equally over all humans who currently exist, or according to how much AI services they can afford to buy?

What about people who existed or will exist at other times and in other places or universes?

Need for "metaphilosophical paternalism"?

However we distribute the benefits, if we let the beneficiaries decide what to do with their windfall using their own philosophical faculties, is that likely to lead to a good outcome?

Metaphilosophy

What is the nature of philosophy?

What constitutes correct philosophical reasoning?

How to specify this into an AI design?

Philosophical forecasting

How are various AI technologies and AI safety proposals likely to affect future philosophical progress (relative to other kinds of progress)?

Preference aggregation between AIs and between users

How should two AIs that want to merge with each other aggregate their preferences?

How should an AI aggregate preferences between its users?

Normativity for AI / AI designers

What is the nature of normativity? Do we need to make sure an AGI has a sufficient understanding of this?

Metaethical policing

What are the implicit metaethical assumptions in a given AI alignment proposal (in case the authors didn't spell them out)?

What are the implications of an AI design or alignment proposal under different metaethical assumptions?

Encouraging d

...
How can you use music to boost learning?
by Matthew Barnett — 2019-08-17T06:59:32.582Z

I often find that I am able to appreciate the beauty of a subject more while listening to music (especially instrumental music). Hearing the notes while I think about the topic helps creates a lot of subconscious connections with the material, solidifying what I am learning as a distinct set of memories. In general I think that associations with powerful sensory experience is just a good way to remember things and learn.

However, I have also heard that listening to music can distract you when you are trying to do deep work. Apparently, there is some research on this, but I have barely scratched the surface of the literature, and I wouldn't know where to start.

Is there an optimal way to use music to learn? Should I employ certain strategies, like putting on the music only after I've read something, so that I can think about what I just read while music floats through my consciousness?
A Primer on Matrix Calculus, Part 3: The Chain Rule
by Matthew Barnett — 2019-08-17T01:50:29.439Z

This post concludes the subsequence on matrix calculus. Here, I will focus on an exploration of the chain rule as it's used for training neural networks. I initially planned to include Hessians, but perhaps for that we will have to wait.

Deep learning has two parts: deep and learning. The deep part refers to the fact that we are composing simple functions to form a complex function. In other words, in order to perform a task, we are mapping some input x to an output y using some long nested expression, like y=f1(f2(f3(x))). The learning part refers to the fact that we are allowing the properties of the function to be set automatically via an iterative process like gradient descent.

Conceptually, combining these two parts is easy. What's hard is making the whole thing efficient so that we can get our neural networks to actually train on real world data. That's where the backpropagation enters the picture.

Backpropagation is simply a technique to train neural networks by efficiently using the chain rule to calculate the partial derivatives of each parameter. However, backpropagation is notoriously a pain to deal with. These days, modern deep learning libraries provide tools for automatic differentiation, which allow the computer to automatically perform this calculus in the background. However, while this might be great for practitioners of deep learning, here we primarily want to understand the notation as it would be written on paper.1 Plus, if we were writing our own library, we'd want to know what's happening in the background.

What I have discovered is that, despite my initial fear of backpropagation, it is actually pretty simple to follow if you just understand the notation. Unfortunately, the notation can get a bit difficult to deal with (and was a pain to write out in Latex).

We start by describing the single variable chain rule. This is simply ddxf(g(x))=f′(g(x))g′(x). But if we write it this way, then it's in an opaque notation and hides which variables we are taking the derivative with respect to. Alternatively we can write the rule in a way that makes it more obvious what we are doing: ddxf(g(x))=dfdgdgdx, where g is meant as shorthand for g(x). This way it is intuitively clear that we can cancel the fractions on the bottom, and this reduces to dfdx, as desired.

It turns out, that for a function f:Rn→Rm and g:Rk→Rn, the chain rule can be written as ∂...
Beliefs Are For True Things
by Davis_Kingsley — 2019-08-15T23:23:33.175Z

One of the core principles -- maybe the most core principle -- of the art of rationality is that beliefs are for true things. In other words, you should believe things because they are true. You should not believe things that are not true.

Holding that beliefs are for true things means that you do not believe things because they are useful, believe things because they sound nice, or believe things because you prefer them to be true. You believe things that are true (or at least that you believe to be true, which is often the best we can get!).

Eliezer referred to this principle as "the void", writing in his "The Twelve Virtues of Rationality":

Before these eleven virtues is a virtue which is nameless.

Miyamoto Musashi wrote, in The Book of Five Rings:

“The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means. Whenever you parry, hit, spring, strike or touch the enemy’s cutting sword, you must cut the enemy in the same movement. It is essential to attain this. If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him. More than anything, you must be thinking of carrying your movement through to cutting him.”

Every step of your reasoning must cut through to the correct answer in the same movement. More than anything, you must think of carrying your map through to reflecting the territory.

Musashi wrote that you must always think of carrying your motion through to cutting; I write, with Eliezer, that every belief and every step in your belief must cut through to knowing the truth.

Beliefs, after all, are for true things, and if you lose sight of that you will lose your epistemics. If you think only of what gives you an advantage in a debate, of what sounds nice, of what wins you the admiration of your peers, of what is politically correct, or of what you would prefer to be true, you will not be able to actually believe true things.

I would like to take the perhaps unusual step of closing with a poem by Rudyard Kipling, which addresses this point (among others) rather well:

As I pass through my incarnations in every age and race,
I make my proper prostrations to the Gods of the Market-Place.
Peering through reverent fingers I watch them flourish and fall.
And the Gods of the Copybook Headings, I notice, outlast them all.

We were living in trees whe

...
What experiments would demonstrate "upper limits of augmented working memory?"
by Raemon — 2019-08-15T22:09:14.492Z

Wikipedia has this discussion of working-memory-as-ability-to-discern-relationships-simultaneously:

Other have argued that working memory capacity is better characterized as "the ability to mentally form relations between elements, or to grasp relations in given information. This idea has been advanced by Halford, who illustrated it by our limited ability to understand statistical interactions between variables.[34]"

These authors asked people to compare written statements about the relations between several variables to graphs illustrating the same or a different relation, as in the following sentence:

"If the cake is from France, then it has more sugar if it is made with chocolate than if it is made with cream, but if the cake is from Italy, then it has more sugar if it is made with cream than if it is made of chocolate".

This statement describes a relation between three variables (country, ingredient, and amount of sugar), which is the maximum most individuals can understand. The capacity limit apparent here is obviously not a memory limit (all relevant information can be seen continuously) but a limit to how many relationships are discerned simultaneously.

A common argument I've heard is that large monitors, notebooks, whiteboards, etc, are important tools to expand working memory.

I notice I'm not 100% sure what this means – in particular in the context of "discerning relationships simultaneously."

In this blogpost on distributed teams , Elizabeth plots out her model of worker productivity, which looks like this:

I look at any chunk of that, and it makes sense.

If I were to try to summarize the whole thing without looking at the reference drawing, I would definitely not be able to (without a lot of memorization and/or thinking about the model to get it deeply entangled within myself)

If I have the model right in front of me, I still can't really explain it, it's too complicated.

Diagrams help – I'm pretty sure I could track more moving parts with a diagram than without a diagram. But how much do they help? And what does that mean?

I'm interested in this as part of a general hypothesis that working-memory might be a key bottleneck on intellectual progress. It seems like you should be able to formalize the limit of how many relationships people can reason about at once, and how much visual aids and othe...
Clarifying some key hypotheses in AI alignment
by Ben Cottier — 2019-08-15T21:29:06.564Z

We've created a diagram mapping out important and controversial hypotheses for AI alignment. We hope that this will help researchers identify and more productively discuss their disagreements.

Diagram

A part of the diagram. Click through to see the full version.

Caveats

This does not decompose arguments exhaustively. It does not include every reason to favour or disfavour ideas. Rather, it is a set of key hypotheses and relationships with other hypotheses, problems, solutions, models, etc. Some examples of important but apparently uncontroversial premises within the AI safety community: orthogonality, complexity of value, Goodhart's Curse, AI being deployed in a catastrophe-sensitive context.

This is not a comprehensive collection of key hypotheses across the whole space of AI alignment. It focuses on a subspace that we find interesting and is relevant to more recent discussions we have encountered, but where key hypotheses seem relatively less illuminated. This includes rational agency and goal-directedness, CAIS, corrigibility, and the rationale of foundational and practical research. In hindsight, the selection criteria was something like:

The idea is closely connected to the problem of artificial systems optimizing adversarially against humans.

The idea must be explained sufficiently well that we believe it is plausible.

Arrows in the diagram indicate flows of evidence or soft relations, not absolute logical implications — please read the "interpretation" box in the diagram. Also pay attention to any reasoning written next to a Yes/No/Defer arrow — you may disagree with it, so don't blindly follow the arrow!

Background

Much has been written in the way of arguments for AI risk. Recently there have been some talks and posts that clarify different arguments, point to open questions, and highlight the need for further clarification and analysis. We largely share their assessments and echo their recommendations.

One aspect of the discourse that seems to be lacking clarification and analysis is the reasons to favour one argument over another — in particular, the key hypotheses or cruxes that underlie the different arguments. Understanding this better will make discourse more productive and help people reason about their beliefs.

This work aims to collate and clarify hypotheses that seem key to AI alignment in particular (by "alignment" we mean the problem o

...
Welcome to Humboldt SSC [Edit With Your Details]
by benjaminikuta — 2019-08-15T16:35:20.904Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Tessercube — OpenPGP Made Mobile
by Suji Yan — 2019-08-15T09:34:43.355Z

I cited several words from my colleague Neruthes. He's not at LessWrong but he also joined the most recent Shanghai LessWrong meetup: https://www.lesswrong.com/events/zR4atrRmiaqGLvjYj/shanghai-lesswrong-meetup#NZHcXphJXcsTFwpmf

Recently we have been working on this project in the sense that I feel there is no good OpenPGP utility on Mobile (especially iOS). By good, I mean the UX should be good, and the license should be AGPL or GPL.

In the process, we got the idea of App Penetration — by making OpenPGP into keyboards (input methods), we can literally be end-to-end encrypted when using any channel of communication, as long as the other side can decrypt — on Facebook Messenger, on Telegram, on iMessage, whatever.

For now, we have been releasing Android beta test versions on

Google Play

. The iOS version on

App Store

. It might be a bit early to announce because there are plenty of bugs and a big shortage of tutorials, but I believe hardcore users can go through it.

There can be a lot of bugs and UX flaws. If you find any bug, just go to

GitHub

and open an issue. And I will appreciate!

In larger perspective, building

Tessercube

is just a humble beginning. We would like to give general public proper tools of encryption and make them possible to protect their privacy and enable the people to really own their data. That's why we also made

Maskbook

, an encryption and programmable layer on top of all existing giants, such as Facebook, Twitter, etc. I will write a separate post for our story and our approach.

([I:b])
A Primer on Matrix Calculus, Part 2: Jacobians and other fun
by Matthew Barnett — 2019-08-15T01:13:16.070Z

I started this post thinking that I would write all the rules for evaluating Jacobians of neural network parameters in specific cases. But while this would certainly be useful for grokking deep learning papers, frankly it's difficult to write that in Latex and the people who have written The Matrix Calculus You Need For Deep Learning paper have already done it much better than I can do.

Rather, I consider my comparative advantage here to provide some expansion on why we should use Jacobians in the first place. If you were to just read the paper above, you might start to think that Jacobians are just notational perks. I hope to convince you that they are much more than that. In at least one setting, Jacobians provide a mathematical framework for analyzing the input-output behavior of deep neural networks, which can help us see things which we might have missed without this framework. A specific case of this phenomenon is a recently discovered technique which was even more recently put into a practical implementation: Jacobian regularization. Here we will see some fruits of our matrix calculus labor.

Deep learning techniques require us to train a neural network by slowly modifying parameters of some function until the function begins returning something close to the intended output. These parameters are often represented in the form of matrices. There are a few reasons for this representation: the matrix form is compact, and it allows us to use the tools of linear algebra directly. Matrix computations can also be processed in parallel, and this standardization allows programmers to build efficient libraries for the training of deep neural networks.

One quite important matrix in deep learning is the Jacobian.

In one sense, the Jacobian matrix is just a way of organizing gradient vectors. Gradient vectors, in turn, are just ways of organizing partial derivatives of an expression. Therefore, the Jacobian matrix is just a big matrix which allows us to organize a lot of partial derivatives. Formally, the Jacobian of f:Rn→Rm is defined by the following matrix. We denote fi(x) as the mapping from Rn→Ri, where Ri is the real number line in the ith coordinate of the output vector Rm. Then the Jacobian is simply

⎡⎢
⎢
⎢
⎢
⎢⎣∇f⊺1∇f⊺2⋮∇f⊺m⎤⎥
⎥
⎥
⎥
⎥⎦

But describing the Jacobian as just some big rectangular box of partial derivatives of vector-valued functions hides the intu... ,
Partial summary of debate with Benquo and Jessicata [pt 1]
by Raemon — 2019-08-14T20:02:04.314Z

Note: I'll be trying not to engage too much with the object level discussion here – I think my marginal time on this topic is better spent thinking and writing longform thoughts. See this comment.

Over the past couple months there was some extended discussion including myself, Habryka, Ruby, Vaniver, Jim Babcock, Zvi, Ben Hoffman, Jessicata and Zack Davis. The discussion has covered many topics, including "what is reasonable to call 'lying'", and "what are the best ways to discuss and/or deal with deceptive patterns in public discourse", "what norms and/or principles should LessWrong aspire to" and others.

This included comments on LessWrong, email, google-docs and in-person communication. This post is intended as an easier-to-read collection of what seemed (to me) like key points, as well as including my current takeaways.

Part of the challenge here was that it seemed like Benquo and I had mostly similar models, but many critiques I made seemed to Ben to be in the wrong abstraction, and vice-versa. Sometimes I would notice particular differences like "In my model, it's important that accusations be held to a high standard", whereas Ben felt "it's important that criticism not be held to higher standards than praise." But knowing this didn't seem to help much.

This post mostly summarizes existing online conversation. I'm hoping to do a followup where I make good on my promise to think more seriously through my cruxes-on-ontology, but it's slow going.

Comment Highlights

This begins with some comment highlights from LessWrong which seemed useful to gather in one place, followed by my takeaways after the fact.

I attempt to pass Ben's ITT

In the comments of "Rationalization" and "Sitting Bolt Upright in Alarm", a few things eventually clicked and I attempted to pass Ben's Ideological Turing Test:

Let me know how this sounds as an ITT:

Thinking and building a life for yourself

Much of civilization (and the rationalsphere as a subset of it and/or memeplex that's influenced and constrained by it) is generally pointed in the wrong direction. This has many facets, many of which reinforce each other. Society tends to:

Schools systematically teach people to associate reason with listening-to/pleasing-teachers, or moving-words-around unconnected from reality. [Order of the Soul]

Society systematically pushing people to live apart from each other, to work until they nee

...
Subagents, trauma and rationality
by Kaj_Sotala — 2019-08-14T13:14:46.838Z

[Content note: discussion of trauma, child and sexual abuse, sexual violence, lack of self-worth, dissociation, PTSD, flashbacks, DID, personality disorders; some mildly graphic examples of abuse and trauma mentioned in text form]

I have spent over two years doing emotional support for people who had survived long-term childhood trauma, and in these cases spawning agents to deal with unbearable suffering while having no escape from it is basically a standard reaction that the brain/mind takes. The relevant psychiatric diagnosis is DID (formerly MPD, multiple personality disorder). In these cases the multiple agents often manifest very clearly and distinctly. It is tempting to write it off as a special case that does not apply in the mainstream, yet I have seen more than once the progression from someone suffering from CPTSD to a full-blown DID. The last thing that happens is that the person recognizes that they "switch" between personalities. Often way later than when others notice it, if they know what to look for. After gaining some experience chatting with those who survived severe prolonged trauma, I started recognizing subtler signs of "switching" in myself and others. This switching between agents (I would not call them sub-agents, as they are not necessarily less than the "main", and different "mains" often take over during different parts of the person's life) while a normal way to operate, as far as I can tell, almost never rises to the level of conscious awareness, as the brain carefully constructs the lie of single identity for as long as it can.

--

shminux

As the above comment suggests, the appearance of something like distinct subagents is particularly noticeable in people with heavy trauma, DID being the most extreme example.

This post will interpret the appearance of subagents as emerging from unintegrated memory networks, and argue that - as shminux suggests - the presence of these is a matter of degree. There’s a continuous progression of fragmented (dissociated) memory networks giving arise to increasingly worse symptoms as the degree of fragmentation grows. The continuum goes from everyday procrastination and akrasia on the “normal” end, to disrupted and dysfunctional beliefs on the middle, and conditions like clinical PTSD, borderline personality disorder, and dissociative identity disorder on th...