trentmkelly/LessWrong-43k huggingface dataset:
Interactive model knob-turning
by Gust — 2017-10-28T19:42:46.824Z

(Please discuss on LessWrong 2.0)

(Cross-posted from my medium channel)

When you are trying to understand something by yourself, a useful skill to check your grasp on the subject is to try out the moving parts of your model and see if you can simulate the resulting changes.

Suppose you want to learn how a rocket works. At the bare minimum, you should be able to calculate the speed of the rocket given the time past launch. But can you tell what happens if Earth gravity was stronger? Weaker? What if the atmosphere had no oxygen? What if we replaced the fuel with Diet Coke and Mentos?

To really understand something, it's not enough to be able to predict the future in a normal, expected, ceteris paribus scenario. You should also be able to predict what happens when several variables are changed is several ways, or, at least, point to which calculations need to be run to arrived at such a prediction.

Douglas Hofstadter and Daniel Dennett call that "turning the knobs". Imagine your model as a box with several knobs, where each knob controls one aspect of the modeled system. You don't have to be able to turn all the possible knobs to all possible values and still get a sensible, testable and correct answer, but the more, the better.

Doug and Dan apply this approach to thought experiments and intuition pumps, as a way to explore possible answers to philosophical questions. In my experience, this skill is also effective when applied to real world problems, notably when trying to understand something that is being explained by someone else.

In this case, you can run this knob-turning check interactively with the other person, which makes it way more powerful. If someone says “X+Y = Z” and “X+W = Z+A”, it’s not enough to mentally turn the knobs and calculate “X+Y+W = Z+A+B”. You should do that, then actually ask the explainer “Hey, let me see if I get what you mean: for example, X+Y+W would be Z+A+B”?

This interactive model knob-turning has been useful to me in many walks of life, but the most common and mundane application is helping out people at work. In that context, I identify six effects which make it helpful:

1) Communication check: maybe you misunderstood and actually X+W = Z-A

This is useful overall, but very important if someone uses metaphor. Some metaphors are clearly vague and people will know that and avoid th...
10/28/17 Development Update: New editor framework (markdown support!), speed improvements, style improvements
by habryka — 2017-10-28T09:59:07.667Z

Here is a quick summary of the changes I just pushed:

We now have a new editor framework (we moved from ory-editor to draft-js-plugins). The primary reason for this was performance, though we also had a few other problems with the editor. Please let me know in case anything breaks, but overall I expect things to be a lot more stable now.

With the move to the new framework comes markdown support. So feel free to use markdown in any of the text fields from now, it should be automatically translated into its rich-text equivalent (you can insert block-level images via the Toolbar and inline-images via markdown)

We significantly decreased our bundle size with this update, though I still have some more cleaning up to do. The editor was the biggest part of the rework that I was blocked on, so wrapping up the performance improvements should now be a lot faster
We also made some style improvements and navigation improvements to the sequences and collection navigation. Though we are still wrapping up some stuff in that domain, so expect more in the coming days.

Edit: I also made some changes to the font and typography, as you might have noticed. I actually like this font (Garamond Premiere Pro) less, but the other one had rendering issues on Windows machines, so here we are. I will adjust the page piece by piece to work better with the new font, and in general add a bunch more font and typography polish soon.

On that note, I also slightly restructured the frontpage to have a bit of a less chaotic structure and be easier to skim. This cost us some vertical space, but I think is worth the cost in structure and getting rid of that weird card-hover interaction.
Logical Updatelessness as a Robust Delegation Problem
by Scott Garrabrant — 2017-10-27T21:16:18.076Z

(Cross-posted on AgentFoundations.org)

There is a cluster of problems in understanding naturalized agency which I call Robust Delegation. It refers to the way in which a limited agent might direct a more powerful agent to do its bidding. The limit agent starts out in control because it gets to design the more powerful agent, but it is not easy to retain that power. In most applications the limited agent is a human and the powerful agent is an AI.

Example of problems in this subfield include corrigibility, value learning, and informed oversight.

Logical Updatelessness is one of the central open problems in decision theory. An updateless agent is one that does not update on its observations, but instead chooses what action it wants itself to output upon being input those observations. It can use this to get counterfactually mugged (which is a good thing). This is basically the only way we know how to make a reflectively stable agent.

The problem is that we do not know how to translate updatelessness into a logical setting. In the toy models in which we can define updatelessness, the agent is logically omniscient and just makes empirical observations. Thus, starts out just as powerful as it will ever be. In fact, we can view it as though the only agent is the agent existing at the beginning of time. That agent chooses a policy, a function from inputs to outputs, and all the future agents can just blindly follow that policy without thinking.

In the logical setting, the future agents are smarter than the past agents. They learn more logic in addition to making empirical observations. The agent at the beginning time does not have the power to contain a policy from all the logical observations to actions. This agent does not choose a simple policy to follow in the future, but instead chooses a complicated program to run in the future.

I claim that logical updatelessness is hard while empirical updatelessness is easy because an empirically updateless agent is deferring control to a simple policy that it understands, while a logically updateless agent has to defer control to a more powerful agent that it does not understand. Thus, logical updatelessness can be viewed as a robust delegation problem.

If we think about this just in terms of reflective stability, a reflectively stable agent has to never have a conflict between what its future self wants and what its past self w...
Guided Mental Change Requires High Trust
by namespace — 2017-10-27T19:15:10.530Z

There is a general theme to a lot of the flashpoints I’ve witnessed so far on LW 2.0. We're having the usual disagreements and debates like “what is and isn’t allowed here” or “is it worse to be infantile or exclusive?”. I think underlying a lot of these arguments are mistaken notions about how ideas get made. There is also a lack of self awareness about how the open nature of LessWrong fundamentally caps how much trust can be expected among participants. These are systemic issues, they're not something Oliver and Ben can easily fix.

Let's start with the bit everyone knows. Changing your mind, really changing your mind is hard. It&#x27;s stupid hard. A large part of why LW 2.0 might be useful is that it has the promise of helping people think better. In addition to its native difficulty, people are usually in a state of active defense against others getting them to do it. Even on LessWrong people are generally more interested in reading things that come with the pleasant feeling of insight rather than homework. In one of his recent posts Eliot expressed nothing less than outright contempt for such people. The problem is that this behavior makes sense.

See, not only is changing your mind hard, it's dangerous. There are people who make their living off getting others to change their mind in dangerous ways, and a lot of our reluctance to perform weird mental gymnastics is totally reasonable in the face of that. Strange formal systems of interaction, weird fake frameworks that are supposed to work well anyway, bizarre ideas about what the real threats facing humanity are, look I'm not saying this stuff is bad but you have to be acutely aware of what you're really asking from people. If you dress in rags and look vaguely threatening, people won't want to follow you into the dark alley of the human psyche.

Combine these two things together, and it's fairly obvious why CFAR finds that their methods translate well in person but are hard to teach over the internet. They seem convinced that the problem is in the explanation, the operative explanation. They're giving advice at the level of &#x27;use this bolt&#x27;, which is entirely missing the point. What's missing isn't instructions but trust. This is a high effort, weird thing being pushed by secretive private sector psychologists. Double Crux righ...
LDL 6: Deep learning isn't about neural networks
by magfrump — 2017-10-27T17:15:20.115Z

Yesterday I ran into an “interesting” problem.

The problem is, that sometimes when I try to train my network to pick hyperparameters, it just completely fails to learn.

This is a pretty frustrating problem!

I don’t really know what could be causing it. I could come up with guesses—too many negative initial weights that ReLUs prevent me from training out of, for example—but these don’t really give me useful predictions or interventions so I don’t want to pretend like I understand it.

What concerns me about this is that it may result in a bad choice of hyperparameters for my model going forward.

I’d really like to get reasonably answers out of my model as I zoom in to more specific values but if my model is going to behave in completely wacky ways when I start a new loop to do the exact same thing, it really brings down my level of trust.

In theory I could run several trials, and set random seeds, and check my weight initializations by digging into the keras documentation, but while some of those things might help none of them address the more philosophical problem of why I should expect sometimes that I’ll initialize my model and it will just consistently fail to learn anything for a dozen sets of parameters in a row, and then when I reset the model ONCE (which I thought I was already doing between every choice of parameters) it starts working for every set of parameters—those same sets of parameters that weren’t working before.

Anyway WTF.

Having thought about other things for five minutes, I’m realizing one of the most mysterious things to me in my code base is, somewhat ironically, the generator function I wrote to pull mini-batches of data from the dataset on hard disk. The function opens the file and runs through it and loops around, and it could be that (despite my shuffling the data) there’s a section with a bunch of really difficult examples and when it gets to that section and that’s what stops the algorithm from learning. Then it will just work its way out of this section on its own, and I don’t have the diagnostic tools available to figure out what to do about it.

That’s sort of depressing but I think it offers its own somewhat silly and obvious lesson:

Don’t assume someone else’s code is giving you problems.

So I worked things out enough to have a model that was definitely learning, and I left it running overnight so that it could train a reasonab

...
NYC Solstice and Megameetup Funding Reminder
by Rachel Shu — 2017-10-27T14:14:01.164Z

Hey all, we're coming up on the final weekend of our Kickstarter. Details in previous post, and a couple updates here:

Megameetup has 16 confirmed attendees. This is shaping up to be a really good chance to form productive conversations and friendships with other rationalists.

Solstice is currently at $2,740 (54% funded), with 65% of funding window elapsed. Please contribute - even a little helps.

To clarify for people buying multiple tickets, sponsors at $70+ automatically receive two tickets.

There will be additional tickets for purchase post-Kickstarter, conditional on meeting our goal, of course.

We're offering these incredibly cute stickers above certain backer levels!

Both are only open until Monday, Oct 30th - please give if you can to the Kickstarter, and we're excited to see you at the Megameetup!
Solstice Kickstarter page
Megameetup registration and details
NYC Solstice and Megameetup Funding Reminder
by clarkey — 2017-10-27T14:12:35.879Z

Hey all, we're coming up on the final weekend of our Kickstarter. Details in previous post, and a couple updates here:

Megameetup has 16 confirmed attendees. This is shaping up to be a really good chance to form productive conversations with other rationalists.

Solstice is currently at $2,740 (54% funded), with 65% of funding window elapsed. Please contribute - even a little helps.

To clarify for people buying multiple tickets, sponsors at $70+ automatically receive two tickets.

There will be additional tickets for purchase post-Kickstarter, conditional on meeting our goal, of course.

We're offering these incredibly cute stickers above certain backer levels!

Both are only open until Monday, Oct 30th - please give if you can to the Kickstarter, and we're excited to see you at the Megameetup!
Solstice Kickstarter page
Megameetup registration and details
Maths requires less magical ability than advertised. Also, messy details are messy (and normal).
by Ben Pace — 2017-10-27T03:11:50.773Z
Link: https://wirehead-wannabe.tumblr.com/post/166830719366/the-secret-tedious-math-behind-famous-ppls

This is one of the most insightful things I've read about the field of mathematics in a long time. (And I read it on tumblr!)

Content includes:

Quotes from top mathematicians/physicists explaining how other top mathematicians systematically try to make their work seem more like a work of genius (as opposed to something *you* could do)

Positive argument for mathematical insight being hard yet doable through hard work as opposed to only doable via magic/genius

Then a separate person follows up with arguments that totally surprised my assumptions. I find that the pendulum in math lectures is swung way too far towards 'formalism only, no intuition', but the person (a math tutor) explains how if you swing the pendulum too far the other way and a lecture is all intuition and no messy details, this causes the students also to lose understanding.
Bringing In Some Outside Ideas
by mothlight — 2017-10-26T21:19:04.459Z

I remember reading various musings on the fact that lesswrong keeps reinventing the wheel or not bringing in enough outside ideas, and I happen to think I have some that are worth at least considering/looking into.

I have done cursory searches for them using this site's search, so as not to duplicate someone else while being unaware of it. If there is in fact coverage of these that I didn't find, please let me know in the comments.

BetterExplained - a site that explains math and some other stuff, and seeks to produce really good explanations of things, and has an interesting model for how to approach making explanations. (This would be so great if combined with a hypothetical functioning Arbital!)

Master of Memory - mnemonics, the kind Brienne already talked about, but with practice problems and specifics, and a full Spanish course, (apparently.) I do not know the actual quality of this course, but the fact that I already memorised three phone numbers (in a minute each), and also a few poems and a paragraph of text, suggests that mnemonics is at least somewhat effective as an approach to learning. Things to explore on that website - the startup guide and then the other methods (some are not covered in the startup guide) and then possibly the podcast.

There's a Russian methodology called TRIZ - the acronym might be changed up to be TRIP, expanding to Theory of the Resolution of Inventor Problems (wikipedia suggests Theory of Inventive Problem Solving instead). It's a very systematic way of approaching problem solving. I'm learning it at the moment, and I might be able to do a post describing the basics of it soon. Meanwhile, there's a wikipedia page. It doesn't seem to be entirely consistent with the things I'm learning, but I am well aware that I just began.

Thank You For Arguing - a book on persuasion/rhetoric, somewhat more systematic than other persuasion things I've encountered, advocates redefining words your opponent uses in one chapter (which led to amusing mental images of consternated my-model-of-rationalist-adjacents), but this chapter can be skipped and the book otherwise contains very interesting advice, some of which I've installed TAPs to apply. I think they work but success rates on social stuff are really hard to measure.

I do not commit to anything because I've seriously tried committing and

...
LDL 5: What to do first
by magfrump — 2017-10-26T18:11:18.412Z

I spent most of the day yesterday building and debugging my minibatch generator for notMNIST. It was a day well spent, in that I expect it to make future such tasks much easier since I’m more familiar with the outline, and now I can run experiments on the notMNIST dataset.

But that said, I have a lot of uncertainty about this process.

For example, training for a couple of epochs (once I got it working so that I was reducing loss AT ALL) got me up to about 60% accuracy. I’m continuing training and seeing some decent results but every epoch takes 10 minutes or so and it failed to keep running overnight when my computer went to sleep. I don’t expect to hit the limit of returns on retraining until at least a dozen epochs, which makes this whole process a pain in the ass.

And while I literally just took three courses on what to do when you’re first training a deep learning model, I actually DON’T have a clear answer as to what the best set of first actions here are.

I do have a space of actions to sort through, so I guess I’ll write this up and maybe today I can experiment with the results of different courses of action, and maybe some day someone will be in the same situation and be able to work more efficiently. Or maybe you’ll all stop reading my posts because of the decline in quality. But writing has been helping me stay working on my end so I’m going to the end of the Umeshism.

Some possible ways to improve performance on a model that is progressing, but slowly :

Train the model

Easy, fire and forget, high chance of success

Takes a long time (10 min per update), hard to tell how long it will keep working

Change learning parameters

Can often see differences within first several mini-batches, somewhat amenable to grid search

Unclear how short term advantages in learning will translate into final model performance, may result in worse performance or breaking the model completely

Add model complexity

Can increase overall cap on performance even if other methods are unable to do so

Requires a lot of smaller decisions, likely to make initial learning slower, may not pay off in terms of performance until later in the process

Once I’ve written this out, it becomes pretty clear to me that training my model more instead of tuning the hyperparameters is probably just a way for me to feel like I’m doing something while being lazy.

Rationality success, I guess!
Slack for your belief system
by Unreal — 2017-10-26T08:19:27.502Z

Follow-up to Zvi’s post on Slack

You can have Slack in your life. But you can also have Slack in your belief system.

Initially, this seems like it might be bad.

Won't Slack result in a lack of precision? If I give myself Slack to believe in whatever, won't I just end up with a lot of wrong beliefs? Shouldn't I always be trying to decrease the amount of Slack in my beliefs, always striving to walk the narrow, true path?

Claims:

For some things, the only way to stumble upon the Truth is to have some Slack. In other words, having no Slack in your belief system can result in getting stuck at local optima.
Having Slack allows you to use fake frameworks in a way that isn't epistemically harmful.
If you are, in fact, just correct, I guess you should have zero Slack. But—just checking—are you ALSO correct about how you come to Know Things? If your way of coming to conclusions is even a little off, giving yourself zero Slack might be dangerous. (Having zero Slack in your meta process multiplies the problem of no-Slack to all downstream beliefs.)
I'm willing to make the more unbacked, harder-to-define claim that there exists no individual human alive who should have zero Slack in their beliefs, on the meta level. (In other words, no human has a truth-seeking process that will reliably get all the right answers.)

[ I want to note that I fully believe I could be wrong about all four claims here, or thinking about this in the entirely wrong way. So fight me. ]

Now, I'm going to specifically discuss Slack in one's meta process.

So, while I can apply the concept of Slack to individual beliefs themselves (aka "holding beliefs lightly"), I am applying the concept more to the question of "How do I come to know/understand anything or or call a thing true?"

So, I'm not discussing examples of "I believe X, with more or less Slack." I'm discussing the difference between, "Doing a bunch of studies is the only way to know things" (less Slack) vs. "Doing a bunch of studies is how I currently come to know things, but I'm open to other ways" (more Slack).

The less Slack there is in your process for forming beliefs, the more constraints you have to abide before being able to claim you've come to understand something.

Examples of such constraints include:

I only buy it if it has had at leas

...