trentmkelly/LessWrong-43k huggingface dataset:
LDL 3: Deep Learning Pedagogy is Hard
by magfrump — 2017-10-24T18:15:27.233Z

One thing that I’ve commented on a few times so far and that I repeatedly bemoan to myself as I work my way through various courses, is that none of the experiences I’m having are really tailored to me. All of these approaches to teaching deep learning are tied up in first: rebuilding neural networks from the ground up, and second: using very specific techniques on toy problem sets.

After quite some time of finding this frustrating (and about a year after I wrote my own exercises for a different course where… I did basically the exact same thing) I took a minute to think about why this would be and what the challenges are in teaching this material.

Before I get into this too much I want to point out another anecdote: when working on an earlier exercise I ran some code that didn’t match what the instructions inspected. They had a loss value for a specific network and problem, and I got a different loss value. My code still ran, and my code ended up getting significantly better performance than the expectation. Something like 80% vs 73%.

Probably the difference was that some of my hyperparameters somehow were off by a little bit, or maybe I computed a different number of things between setting the random seed and doing the training. But it turns out that if you have enough data, and your hyperparameters aren’t too far off, then the network does most of the work of convergence for you. Especially if you’re using the right kind of learning rate decay and adam optimization.

This is one of the big reasons that I don’t really care about hyperparameter tuning—given a specific application it can improve performance, but it isn’t going to bridge any theoretical gaps.

What I do care about is being able to set up a new dataset on my own computer, and turn the ins and outs of that dataset into a reinforcement learning setup that has the appropriate features, like the ability to record some memories about its past environment.

If I wanted to design an automatically graded online coding exercise to teach this skill, what would I need?

First, I would need a dataset or simulation environment large enough to be used for a deep reinforcement learning problem.

Second, I would need a specific reinforcement learning framework (whether Q-learning or policy gradient or something else).

Third, I would need a way to map the dataset or environment to the framework that can be easily e

...
Humans can be assigned any values whatsoever...
by Stuart_Armstrong — 2017-10-24T12:03:02.000Z

A putative new idea for AI control; index here. Crossposted at LessWrong 2.0. This post has nothing really new for this message board, but I'm posting it here because of the subsequent posts I'm intending to write.

Humans have no values... nor do any agent. Unless you make strong assumptions about their rationality. And depending on those assumptions, you get humans to have any values.

An agent with no clear preferences

There are three buttons in this world, B(0), B(1), and X, and one agent H.

B(0) and B(1) can be operated by H, while X can be operated by an outside observer. H will initially press button B(0); if ever X is pressed, the agent will switch to pressing B(1). If X is pressed again, the agent will switch back to pressing B(0), and so on. After a large number of turns N, H will shut off. That's the full algorithm for H.

So the question is, what are the values/preferences/rewards of H? There are three natural reward functions that are plausible:

R(0), which is linear in the number of times B(0) is pressed.

R(1), which is linear in the number of times B(1) is pressed.

R(2) = I(E,X)R(0) + I(O,X)R(1), where I(E,X) is the indicator function for X being pressed an even number of times,I(O,X)=1-I(E,X) being the indicator function for X being pressed an odd number of times.

For R(0), we can interpret H as an R(0) maximising agent which X overrides. For R(1), we can interpret H as an R(1) maximising agent which X releases from constraints. And R(2) is the "H is always fully rational" reward. Semantically, these make sense for the various R(i)'s being a true and natural reward, with X="coercive brain surgery" in the first case, X="release H from annoying social obligations" in the second, and X="switch which of R(0) and R(1) gives you pleasure".

But note that there is no semantic implications here, all that we know is H, with its full algorithm. If we wanted to deduce its true reward for the purpose of something like Inverse Reinforcement Learning (IRL), what would it be?

Modelling human (ir)rationality and reward

Now let's talk about the preferences of an actual human. We all know that humans are not always rational (how exactly we know this is a very interesting question that I will be digging into). But even if humans were fully rational, the fact remains that we are physical, and vulnerable to things like coercive brain surgery (and in practice, to a who

...
Being Correct as Attire
by Unreal — 2017-10-24T10:04:10.703Z

[ Related to: Belief as Attire, Lost Purposes, Something to Protect ]

Having correct beliefs is Important, with a capital I.

And yet, this too, feels susceptible to Goodhart's Law.

Imagine being in a middle of a group conversation. They seem to be having an intellectual discussion. Maybe on Bitcoin or AI or EA. Whatever it is.

All seems to be going as expected. But at the corner of your attention, you sense something is off.

People are being a little too eager to correct each other's factual errors. When it's unclear which fact is correct, people keep talking about it for another 5 or 10 minutes, before someone suggests looking it up. It takes another 5 minutes for anyone to actually look it up.

At other times, as soon as there is a hint of a factual ambiguity, someone leaps to the internet, phone appearing magically in hand. You fail to question this action, thinking it virtuous.

People seem to be talking about important, scientific-sounding things. But you also have a vague sense of disconnectedness, as though this conversation will have no impact on tomorrow or the next day. You walk away without any new updates.

You hear a very similar conversation on the topic at the next party, with largely the same facts. Same opinions. Same disagreements.

Having correct beliefs is Important, so as soon as I notice an error, I correct it, right?

This is not totally unlike saying, Getting the right answer is Good, so I should try to get right answers on my tests, right?

I'm not saying truth-seeking is a purely instrumental process.

Truth for its own sake, yes.
Correctness for its own sake, no.

Getting correct beliefs is only a subset of truth-seeking. If the only thing you try for is correct beliefs, it is likely to fall to Lost Purposes.

But how can you tell whether it's connected to the larger goal or not?

One way I can try to tell is to check for model-based reasons the facts are being brought up. Is there a model under consideration?

Maybe it's not explicitly being discussed but is lurking close-by underneath. I can try to directly ask, "Why is this relevant?" or I can poke at it, "Is the reason you brought this up because you expect the price of Bitcoin to rise? Or maybe you're considering buying Ethereum instead?"

One reason it's sometimes recommended to Double Crux on actions rather than beliefs, is that a disagreement o

...
Typical Minding Guilt/Shame
by Unreal — 2017-10-24T09:39:35.498Z

Guilt/shame can be thought to have two potential functions. One is mechanistic, and the other is for signaling.

These functions don't have to go together.
[ For the purposes of this post, I am clustering guilt/shame into one category. I do think they have different functions from each other, but that's not the focus of this post. ]

In some cases, a person feeling guilty will be motivated to change their behavior and then do so.

For example, shame recently got me to learn how to stop slamming car doors. I got multiple external signals that people don't like their car doors being slammed, and that I tend to slam them. So, feeling kind of bad, I installed a TAP to "summon sapience" whenever I close car doors and started practicing another way to close car doors. (I make my hand follow the car door all the way to a close.)

But I didn't really signal outwardly to anyone else that I was feeling shame, in particular not to any of my Lyft drivers. The shame came somewhat gradually after learning that slamming doors was bad.

~

Other cases, guilt/shame will function both mechanistically and as signaling.

I don't have ready examples of this, so I leave it as an exercise to the reader.

~

And then there are cases where guilt/shame sort of backfires, mechanistically.

Two classes of responses I've experienced:

1) I avoid the bad thing or the source of the guilt.

2) I deny that anything bad was done (or that even if it was done, it wasn't 'on purpose' or it wasn't 'my fault').

In the past, 1) has been my most common response to guilt/shame. I definitely felt lots of guilt/shame, so it showed up in my face and my body language (the signaling part). But it didn't result in a direct fix. It resulted in avoidance.

When I realized this—that feeling guilt/shame wasn't actually fixing the thing—my approach to guilt/shame changed.

Now I kind of "do guilt/shame less." I do something else. (Because, as previously learned, it doesn't DO the thing. So what's the point? It's anti-helpful.)

That said, guilt/shame probably is still mechanistically useful for plenty of people. (I doubt it would have gotten this far as an emotion if its only function was signaling. I suspect it is, in fact, useful for getting people to change their behavior in reasonable and healthy ways.)

But now we have some people wh

...
Continuing the discussion thread from the MTG post
by habryka — 2017-10-24T02:20:22.980Z

I moved and copies this discussion out of the latest MTG color-wheel post, since I would prefer the discussion on the post to stay on the object-level.

Commentary by Conor Moreton:

[Meta/cultural note: as of this writing, the parent comment I made in reply to CoolShirtMcPants' elaboration on horoscopes is at -2, which I think is a bad sign re: LW culture in general. CSMcP was making a broad claim of the form "categorical psych tools are bad," which is both a) reasonable and b) in context the sort of claim Scott sighs about in his excellent post Yes We Have Noticed The Skulls. It was a knee-jerk, base-rate, cached objection to an entire category of Thing based on that category being generally bad/useless/misleading, when the post in question was about a specific instance, started out with a link to Fake Frameworks, made explicit bids to be treated fresh/in good faith, and was written as the 29th entry in a series of posts that have been generally agreed to contain non-zero value and rationalist virtue.

The comment above implicitly (and maybe clumsily) made the claim "I suspect you're only using your generally sensible prior, and I think it's better in this case to construct a posterior that combines your generally sensible prior in a rational way with awareness of the source and context."

Regardless of whether the posterior ends up being "yeah, still bullshit" or "maybe I'll give this more charity than I otherwise would have," the requested operation "instead of just commenting using your base rate, combine your base rate with your sense of whether a given person has demonstrated 'worth listening to' nature" is one that LessWrongers should absolutely engage in, on the regular.

i.e. I strongly believe that the thrust of the comment ("You're leaving out context that a LWer ought not leave out, and this is somewhat undermining the point you're trying to make") is correct, defensible, and prosocial given the community's goals. I think it was innocuous at worst, and definitely not the sort of thing that ought to be in negative territory. I'm taking it as an instance of a class, and making an impassioned defense of other comments in its class, and saying "Don't downvote these."

If LessWrongers in general are inclined to downvote one member making

...
React and Respond
by Gordon Seidoh Worley — 2017-10-24T01:48:16.405Z

Some may find it annoying, but one of my favorite things about the English language is that it's full of synonyms that point to roughly the same place in concept space but with each word being a distinct vector carrying nuanced differences in meaning. We see this with "react" and "respond", which both roughly have a meaning of "do something because something else happened", but each of which carries its own implications about the how the doing is done.

"Respond" is the older word so we'll start with it. It comes from Latin through French and is constructed from the roots "re-" and "spondere". "Re-" serves like "back" in its prepositional sense in English as in "go back there" or "do it back to him" and is synonymous with "in return". "Spondere" means "to pledge", but over time "respond" morphed to mean something less like "make a pledge back" and more like "answer back" or "answer in return". The result is our modern word "respond" that connotes a deliberate action made in return to another one, carrying some but not all of the original weight of "pledge" with it.

"React" is newer. It first appears in the 1640s in the sense of a physical reaction from "re-" plus "act" from Latin "actus" which itself draws from Proto-Indo-European "*ag-" which covers the notions of driving, drawing out or forth, and moving. So we can say "react" is to "move in return" to another movement, and it carries that sense through today in a generalized way as both the word we use to talk about what chemicals do when they are "moved" to react and the immediate return actions people make when acted upon.

Together "react" and "respond" create a dimension along we can describe what is done in return. Specifically, we more call those things "reactions" that are done quickly, automatically, or without deliberation and more call those things "responses" that are done carefully, voluntarily, or with deliberation. To give an example, suppose you go to see a movie about factory farming. While watching it you react with disgust, horror, indifference, etc. and may even post about yo

...
The Strengths of the Two Systems of Cognition
by Rossin — 2017-10-24T00:02:56.859Z

The two systems of cognition a la Kahneman each seem to each have roles that they fulfill better than the other and in which, for optimal performance, the other system ought not to interfere. Until recently, I had only really thought about when System 2 ought to override System 1, but I believe other cases are worth considering as well.

It is important to be able to get System 2 to shut up and let System 1 work in certain circumstances. Physical movement is an excellent example of this. When you shoot a basketball, throw a football, or use a video game controller you may notice that when you are “on fire” you are not focusing at all on the mechanics of what you are doing. In fact, if you start thinking about what you are doing better than usual and begin analyzing the mechanics of the movements, you will likely begin performing worse. Similarly, when you focus on some specific motion that is supposed to contribute to a correct movement, applying the conscious effort of System 2 to force yourself to do it, you will generally exhibit worse performance than if you just hold an image in your mind of what the movement should generally look like.

The override of System 2 causes you to neglect other important factors that go into a movement: and there are too many for system 2 to think about at once. The strength of System 1 is that it can handle all those factors without trouble.

Thus, we can see where it is important to learn the skill of not trying too hard or focusing conscious attention too forcefully. We often cannot consciously keep in mind all the necessary factors, though we are capable subconsciously, as long as System 2 does not interfere with the process by introducing an override in one specific area. While learning this skill, it is important that one not try too hard to not try too hard. Thus, it can be a very difficult skill to learn, I think particularly for analytical people. See The Inner Game of Tennis for a more thorough exposition of this idea.

The skill of effortless effort, is a very useful one, though, I think, not quite as important as learning how to get your mind to actually try, with System 2 reevaluating the lazy heuristics of System 1, which is one of the necessary skills for long term rational decision making. System 2 overriding System 1 is necessary for dealing with many cognitive biases. To avoid allowing representativeness to alt

...
Four Scopes Of Advice
by namespace — 2017-10-23T09:01:25.063Z

There's a very common failure mode people fall into where they'll ask for 'advice on doing something', receive excellent advice, and fail to follow it. For a long time this was mysterious to me. Then a friend provided a possible explanation that completely changed how I look at it. As I explained to my friend, the problem isn't that the person giving advice isn't trustworthy, quite often the person asking wants the advice and trusts the opinion of the person giving it. So why don't they follow it? My friend hypothesized that people let their identity get mixed up in how they wanted to do the thing, and then can't bring themselves to do it another way. Essentially his hypothesis is that they're being asked to change too much. This seems plausible enough, but it got me thinking more broadly about what scope of advice people are looking for when they ask.

To start we can imagine four modes of planning, divided thusly:

Mission - What you are trying to accomplish.

Strategic Planning - What broad goals you intend to satisfy to get there.

Tactical Planning - Concrete near term objectives which will let you satisfy the strategic goals.

Operational Planning - The absolute lowest levels of getting the work done, who will do what, what needs to be done to make tactics work, etc.

The hierarchy of willingness to take advice then is basically a mirror image of this.

Unwilling - The zeroth level. The one receiving advice is willing to change nothing based on what counsel they are given.

Operational Advice ("Use this kind of bolt.") - The one receiving advice would like to hear suggestions on how to accomplish the task they've already set for themselves, but aren't particularly interested in hearing what tasks make sense in the service of what goals. This is probably what most people asking for advice actually want.

Tactical Advice ("The car should have four wheels.") - Given a set of broad goals, the one receiving advice is open for suggestions on how they should go about trying to accomplish them. This might mean for example that significant deviations from the original plan are allowed as long as they better serve the goals which the action is going towards. Most advice I give is on this level whether it's asked for or not.

Strategic Advice ("You should build a car

...
UnTAPed Learning
by PDV — 2017-10-23T05:52:21.184Z

[Epistemic Status: Request for Proposal]

For anyone who is familiar with CFAR's teaching program, the concept of the TAP will be very familiar. It's an acronym that expands either to Trigger-Action Pattern (for naturally-occurring instances) or Trigger-Action Plan (for purposefully-implemented instances); the latter is known academically as "implementation intentions". While there is room for contest for the title of "Most Important CFAR Skill" in terms of which one has the most impact of regular use, in terms of driving adoption of techniques, TAPs are the foundation of the entire curriculum. From simple skills to complex ones, the default flow of practicing a skill is "install a TAP to practice this", and the default call to action of most CFAR classes is "spend five minutes brainstorming TAPs to install".

They're extremely useful...if they work at all.

For me, they don't. I struggled to find any example at all of a trigger-action patterns in my own life, and even following the extreme repetition, hard-mode path to installing a TAP, found that it vanished as soon as I stopped consciously thinking about keeping it up.

I liked most of the techniques I learned at my workshop. Many seemed quite useful to apply. I remember approximately none of them, because, deprived of the core tool to maintain them, every piece of practice looked like

It's a strong foundation, for most people. But it's still a foundation; if it doesn't hold, nothing else is any use. So:

What other things have people devised to turn a technique from a sketch of ideas to something used often enough that it's a proper skill?

As a starting point, things that I use in my particular case:

Anki.

Beeminder.

Calendar reminders.

These are all reasonably good at their particular, time-based niches. But none of them work well for irregularly-occurring opportunities or for fuzzier, less specific types of practice.

An alternate question, which seems to me to be similar in applicability, is:
How would you build a daily or weekly routine from scratch, with literally no existing example to piggyback on?
Outline of an approach to AGI Estimation
by  — 2017-10-22T16:12:11.417Z

We are worried about what will happens when we make a system that can do the important things that human can do, like programming and science. Will it explode off into infinity as it finds better ways to improve itself or will it be a slower more manageable process? There are a number of productive ways we can react to this, not limited to:

Attempt to make AI systems controllable by focusing on the control problem

Attempt to predict when we might get agi by looking at progress, e.g. ai impacts

Attempt to predict what happens when we get artificial general intelligence by looking at current artificial intelligences and current general intelligences and making predictions about the intersection.

Figure out how to make intelligence augmentation, so we can improve humans abilities to compete with full AGI agents.

This blog post is looking at the third option. It is important, because the only reason we think AGIs might exist is the existence proof of humans. Computer AGI might be vastly different in capabilities but we can't get any direct information about them, our only possible approach to predicting their capability is to adjust our estimate of what humans can do based on the difference between humans and narrow AI on different tasks and those tasks importance for economically important generality.

There are three core questions to AGI Estimation that I have identified so far.

What do we mean by 'general intelligence'

How important is 'general intelligence' to different economic activities

How good are narrow AIs at parts of the processes involved in general intelligence vs humans. Can we estimate how good an AGI would be at another task based on the comparison?

Generality

We often speak and act as if there is something to one person being better than another person and mental acts in general. We try and measure it with IQ, we think there is something there. However there are a number of different things "general intelligence" might actually be each of which might lead to better performance, while still being general in some way.

Better at tasks in general

Better at absorbing information by experimentation, so that per bit of information the task performance improves quicker

Better at holding more complex behaviours, so that the limit of task performance is higher

Better at understanding verbal and cultural information, so that they

...