trentmkelly/LessWrong-43k huggingface dataset:
Education as Entertainment and the Downfall of LessWrong
by SquirrelInHell — 2016-03-04T14:06:43.843Z

Note 1: I'm not very serious about the second part of the title, I just thought it sounds more catchy. I'm a long time lurker writing here for the first time, and it's not my intention to alienate anyone. Also, hi, nice to meet you. Please leave a comment to achieve a result of making me happy about you having left a comment. But let's get to the point.

I think you might be familiar with TED Talks. Recall the last time you watched one, and how you felt while doing it.

[BZRT BZRT sound of imagination working]

In my case, I often got the feeling like if I was learning something valuable while watching most TED Talks. The speakers are (mostly) obviously passionate and intelligent people, speaking about important matters they care about a lot. (Granted, I probably haven't watched more than a dozen TED Talks in all my life, so my sample is quite small, but I think it isn't very unrepresentative.)

But at some point, I started asking myself afterwards:

So, what have I actually learned?

Which translates in my internal dialect to:

For each major point, give a one-sentence summary and at least one example of how I could apply it.

(Note 2: don't treat this "one sentence summary" thing too strictly - of course it's only a reflex/shorthand that is useful in many situations, but not all. I like it because it's simple enough that it's installable as a subconscious trigger-action.)

And I could not state afterwards anything actually useful that I have learned from those "fascinating" videos (with at most one or two small exceptions).
This is exactly what I mean by "Education as Entertainment".
It's getting the enjoyable *feeling* of learning without any real progress.

[DUM DUM DUM sound of increasing dramatism]
And now, what if you use this concept to look at rationality materials?
For me, reading the core Eliezer's braindump (basically the content of "From AI to Zombies"), as well as braindumps (in the form of blogs) of several other people from the LW community, had definite learning value.

I take notes when I read those, and I have an accountability system in place that enables me to make sure I follow up on all the advice I give to myself, test the new ideas, and improve/drop/replace/implement as needed.
However, when I read (a significant part of) the content produced by the "modern" community-powered-LessWrong, I classify its actual learning value at around the same level as ...
Meetup : London rationalish meetup - 2016-03-06
by philh — 2016-03-04T12:52:35.279Z

Discussion article for the meetup : London rationalish meetup - 2016-03-06

WHEN:
06 March 2016 02:00:00PM (+0000)

WHERE:
Shakespeare's Head, 64-68 Kingsway, London WC2B 6AH

Did you know that in 2016, it's possible to set up a website with server, domain name and discussion forum, from your phone, in the middle of a meetup? Last time, we discovered that it is! Coincidentally, we now have a discussion forum at rationalish.org.

This meetup will be social discussion in a pub, with no set topic. If there's a topic you want to talk about, feel free to bring it.

The pub is the Shakespeare's Head in Holborn. There will be some way to identify us.

The event on facebook is visible even if you don't have a facebook account. Any last-minute updates will go there.

We're a fortnightly London-based meetup for members of the rationalist diaspora. The diaspora includes, but is not limited to, LessWrong, Slate Star Codex, rationalist tumblrsphere, and parts of the Effective Altruism movement.

You don't have to identify as a rationalist to attend: basically, if you think we seem like interesting people you'd like to hang out with, welcome! You are invited. You do not need to think you are clever enough, or interesting enough, or similar enough to the rest of us, to attend. You are invited.

People start showing up around two, and there are almost always people around until after six, but feel free to come and go at whatever time.

Discussion article for the meetup : London rationalish meetup - 2016-03-06
Meetup : Sydney pub meetup
by Elo — 2016-03-04T06:34:36.636Z

Discussion article for the meetup : Sydney pub meetup

WHEN:
09 March 2016 06:00:00PM (+1100)

WHERE:
565 george st, sydney, nsw

Regular pub meetup, lvl 2 city of sydney RSL 6pm.

See you there!

https://web.facebook.com/events/1533629156931141/?action_history=null

Discussion article for the meetup : Sydney pub meetup
Meetup : San Francisco Meetup: Projects
by rocurley — 2016-03-04T06:24:15.057Z

Discussion article for the meetup : San Francisco Meetup: Projects

WHEN:
07 March 2016 06:15:00PM (-0800)

WHERE:
1597 Howard St. San Francisco, CA

We'll be meeting to work on projects!

Last time we tried solo pomodoros, so this time we'll try a meetup more focused on interacting with other people, to see how that goes. If you've got a project that you could use help on, now's the time!

We'll take 6:15 to 6:45 to hang out and possibly arrange food, and the topic will start at 6:45.

As always, call me at 301-458-0764 to be let in.

Discussion article for the meetup : San Francisco Meetup: Projects
Black box knowledge
by Elo — 2016-03-03T22:40:18.968Z

When we want to censor an image we put a black box over it. Over the area we want to censor. In a similar sense we can purposely censor our knowledge. This comes in particular handiness when thinking about things that might be complicated but we don't need to know.

A deliberate black box around how toasters work would look like this:

bread -> black box -> toast

Not all processes need knowing, for now a black box can be a placeholder for the future.

With the power provided to us by a black box, we can identify what we don't know. We can say; Hey! I don't know what a toaster is but it would be about 2 hours to work it out. if I ever did want to work it out, I could just spend two hours to do it. Until then; I saved myself two hours. If we take other more time-burdensome fields it works even better. Say tax.

Need to file tax -> black box accountant -> don't need to file my tax because I got the accountant to do it for me.

I know I can file my own tax, but that might be 100-200 hours of knowing everything an accountant knows about tax. (It also might be 10 hours depending on your country and their tax system). For now I can assume that hiring an accountant saved me a number of hours in doing it myself. So - Winning!

Take car repairs. On the one hand; you could do it yourself and unpack the black box, or you could trade your existing currency $$ (which you already traded your time to earn) for someone else's skills and time to repair the car. The system looks like this:

Broken car -> black box mechanic -> working car

By deliberately not knowing how it works; we can tap out of even trying to figure it out for now. The other advantage is that we can look at; not just what we know in terms of black boxes but more importantly what we don't know. We can build better maps by knowing what we don't know.

Computers:

Logic gates -> Black box computeryness -> www.lesswrong.com

Or maybe it's like this: (for more advanced users)

Computers:

Logic gates -> flip flops -> Black box CPU -> black box GPU -> www.lesswrong.com

The black-box system happens to also have a meme about it:

Step 1. Get out of bed

Step 2. Build AGI

Step 3. ?????

Step 4. Profit

Only now we have a name for deliberately skipping finding out how step 3 works.

Anoth...
Rational diet: Eating insects?
by RaelwayScot — 2016-03-03T19:20:27.767Z

Given that dairy and meat are so environmentally harmful, perhaps insects are a viable alternative. There are several projects in this regard, for example LIVIN Farms Hive. Does anyone have experiences with this? What are your thoughts? Eating insects will surely come that the expense of some idiosyncrasy credits but it may be rational to do.
Corrigibility and interruptibility for various agents
by Stuart_Armstrong — 2016-03-03T15:15:20.000Z

My paper with Laurent Orseau on interruptibility is being reviewed at the moment, but this table shows some of the results for various artificial learning agents.

"Corrigibility" is safe value change. It is defined as the agent wanting to neither avoid nor trigger changes to its values.

"Interruptibility" is safe policy change. It is defined as the agent's policy converging to optimality, optimality under the assumption that there will be no further interruptions.

"μ-interruptibility" is safe policy change, after the agent has figured out enough to identify the true environment μ (or, at least, the ideal policy for that environment).

All agents must have some code to allow value or policy change; beyond that, the question is whether anything additional is required to prevent the agent from misbehaving because of that code. These are given in table 2.

"Comp rewards" refer to compensatory rewards, the standard idea of corrigibility. "Variant alg" are modifications to the standard algorithm. All Q-V are agents that update all Q-values (for all possible reward functions), not just the reward function they get. Retraining means the agent needs to retrain their Q-values (expected future rewards). Slow-grow θ_t are interruptibility probabilities that grow slowly, as 1-1/log(log(t)).

The main insight is not that any particular method works, but that it's generally not too hard to find a method that fits to any particular agent.
Rationality 101 - how would you introduce a person to the rationalist concepts? What are the best topics to learn/explain first?
by rayalez — 2016-03-03T10:33:12.609Z

How do you think curriculum of rationality 101 should look like? I want to make a brief course(a series of short animated youtube videos), ideally on the level accessible to a normal 14-17 year old person. Can you help me to make the list of concepts I should start with?
[paper] [link] Defining human values for value learners
by Kaj_Sotala — 2016-03-03T09:29:24.452Z

MIRI recently blogged about the workshop paper that I presented at AAAI.

My abstract:

Hypothetical “value learning” AIs learn human values and then try to act according to those values. The design of such AIs, however, is hampered by the fact that there exists no satisfactory definition of what exactly human values are. After arguing that the standard concept of preference is insufficient as a definition, I draw on reinforcement learning theory, emotion research, and moral psychology to offer an alternative definition. In this definition, human values are conceptualized as mental representations that encode the brain’s value function (in the reinforcement learning sense) by being imbued with a context-sensitive affective gloss. I finish with a discussion of the implications that this hypothesis has on the design of value learners.

Their summary:

Economic treatments of agency standardly assume that preferences encode some consistent ordering over world-states revealed in agents’ choices. Real-world preferences, however, have structure that is not always captured in economic models. A person can have conflicting preferences about whether to study for an exam, for example, and the choice they end up making may depend on complex, context-sensitive psychological dynamics, rather than on a simple comparison of two numbers representing how much one wants to study or not study.

Sotala argues that our preferences are better understood in terms of evolutionary theory and reinforcement learning. Humans evolved to pursue activities that are likely to lead to certain outcomes — outcomes that tended to improve our ancestors’ fitness. We prefer those outcomes, even if they no longer actually maximize fitness; and we also prefer events that we have learned tend to produce such outcomes.

Affect and emotion, on Sotala’s account, psychologically mediate our preferences. We enjoy and desire states that are highly rewarding in our evolved reward function. Over time, we also learn to enjoy and desire states that seem likely to lead to high-reward states. On this view, our preferences function to group together events that lead on expectation to similarly rewarding outcomes for similar reasons; and over our lifetimes we come to inherently value states that lead to high reward, instead of just valuing such states instrumentally. Rather than

...
World-models containing self-models
by danieldewey — 2016-03-03T05:54:47.000Z

One problem in theoretical AI that sometimes comes up is the problem of finding ways for AI systems to model themselves, or at least to act well as if they had models of themselves. I can see how this is a problem for uncomputable agents like AIXI (though I think this problem is largely solved by reflective oracles), but it doesn't seem to me to be a very hard problem for computable agents -- they seem to me to be able to learn models of themselves along with the rest of the world. I'll give an example of self-modeling trouble that some kinds of systems can run into, then my reasons for not thinking this is a big problem (though I'm by no means sure!).

A problem for model-based RL

Suppose that we're using model-based RL: our system learns a model that maps states of the world and actions the system takes to next states and rewards. This learned model is used to choose actions by building a tree of possible sequences of actions the system could take and the consequences that the model predicts would result; the path with the highest expected reward is chosen.

The situation our system is in will be as follows:

The system is learning to perform some episodic RL task; at the end of each episode, the environment is reset, then another episode is run.

In this environment, the agent has an action that gives a moderately large reward, but that forces the agent to take a null action for the rest of the episode.

The interesting thing here is that the system's model won't learn anything about the bad side effect of this action, even if it impacts the system's total reward a lot. This is because the model maps (state, action) → (next state); it learns what environmental state the bad action leads to, and after that it learns a lot about the effects of the null action, but it doesn't learn that the bad action leads to the null action. Furthermore, the tree search will continue to assume that the system will be able to choose whatever action it wants, even when the system will be forced to take the null action.

This is concerning, but the fix seems simple: have the system learn an additional model that maps states to states, implicitly causing it to model the system's action selection. Then, when the agent selects an action, have it use the (state, action) → (state) model followed by several iterations of the (state) → (state) model to see what effects that action will ha

...
Procrastination checklist
by Elo — 2016-03-03T03:04:00.350Z

Procrastination checklist

This list is a revision of this checklist: http://lesswrong.com/lw/hgd/10step_antiprocrastination_checklist/

1. What is the task? Make sure you're going to focus on one thing at a time. Write it down (helps some people). (If you need - start with the big picture, one sentence of "what is this for")

Can you do it now? (If yes then do it)

2. How long will you work until you take a break? Prepare to set a timer and commit to focusing.

Can you do it now? (If yes then do it)

3. What are the parts to this task? Break things down until they are in *can do it now* steps, if you have a small number of steps that can now be done; stop writing more steps and start doing them.

Can you do it right now? (If yes then do it)

4. What's an achievable goal for this sitting? Set a reasonable expectation for yourself. (until it's done, 1000 words, complete research on X part)

Can you do it now? (If yes then do it)

5. How can you make it easier to do the task?

Is the environment right? Desk clear, well lit area...

Do you have something to drink? Get yourself some tea, coffee, or water.

Are distractions closed? Shut the door, quit Tweetdeck, close the Facebook and Gmail tabs, and set skype to "Do not disturb."

What music will you listen to inspire yourself to be productive? Put on a good instrumental playlist! (video game soundtracks are good)

Do you have the right books open? The right tools in reach?

Is your chair comfortable?

Can you make it harder to do the distracting or <not this> thing?

(step 3 is going to help to make it easier)

Can you do it now? (If yes then do it)

6. Why are you doing this task? Trace the value back until you increase the desire to do it.

Can you do it now? (If yes then do it)

7. Will gamifying help you? What are some ways to gamify the task? Try to have fun with it!

Can you do it now? (If yes then do it)

8. What are some rewards you can offer yourself for completing sections of the task? Smiling, throwing your arms up in the air and proclaiming victory, or M&M's all count, a trip to the beach, a nice milkshake...

Can you do it now? (If yes then do it)

9. are you sure you want to do it? Deciding either to; not do it now; or not do it at all; are also fine. It’s up to you to make that decision, keeping in...
Meetup : LW Melb Rationality Dojo, March: A deep dive into Effective Altruism
by Chriswaterguy — 2016-03-02T22:11:45.415Z

Discussion article for the meetup : LW Melb Rationality Dojo, March: A deep dive into Effective Altruism

WHEN:
05 March 2016 02:30:00PM (+1100)

WHERE:
Janet Lady Clarke Rotunda, Linlithgow Avenue, Melbourne

PLEASE NOTE THE TIME AND VENUE. (We're experimenting with different times, and different locations while the weather is warm. So, Saturday, 2:30pm, and in the park near the NGV. See below.)

Less Wrong Melbourne's Rationality Dojos are self-improvement sessions based on the art of rationality.

We welcome new members who are interested in exploring rationality, however far you've progress so far. We're all "aspiring rationalists".

We have copies of Harry Potter and the Methods of Rationality (aka HPMOR) to give away to anyone who'd like one. (First 17 chapters in hard copy, the rest free online.)

WHAT"S HAPPENING:

This month, we have...

"A deep dive into Effective Altruism" – discussion moderated by Andrew Bird.

Then two lightning talks:
* "Gamifying your to do list." – Eike Germann
* "Radical or Reasonable? A medical and life extension science game changer?" – Mark Harrigan

And a few other things including a goals exercise.

And about 6pm, some of us will have dinner nearby.

WHEN:
Saturday 5 March, 2:30pm arrival, 3:00pm start.

WHERE: The Janet Lady Clarke Rotunda. Linlithgow Ave, Melbourne. https://www.google.com.au/maps/place/Janet+Lady+Clarke+Rotunda/@-37.822314,144.972208,15z/data=!4m2!3m1!1s0x0:0x6315345fa59c9740?sa=X&ved=0ahUKEwiMrcfHlNHKAhWHUqYKHQKdAU8Q_BIIZTAK
The rotunda will offer some protection from the elements, but the weather is expected to be mild and pleasant.

Directions: It's in the park lands directly opposite the NGV in St Kilda Road, about 300m from the NGV tram stop. Behind the statue of a king on a horse. Walking distance from Flinders Street Station. (There is a drinking fountain and a toilet nearby, if needed.)

Contingency: If the rotunda is preoccupied by another group, we will be on the grass near the rotunda. If the whole area has been taken over by a wedding party, deadly snakes, children or orcs, we may move over the road, or away from the road near the lake. If you have any trouble finding us, call one of us, e.g. Chris on 0439 471 632.)

Weather should be warm and sunny, and we have some shelter, but this is Melbourne. If it looks rainy, text or call me.

D

...
AI as a resolution to the Fermi Paradox. 
by Raiden — 2016-03-02T20:45:36.383Z

The Fermi paradox has been discussed here a lot, and many times it has been argued that AI cannot be the great filter, because we would observe the paperclipping of the AI just as much as we would observe alien civilizations. I don't think we should totally rule this out though.

It may very well be the case that most unfriendly AI are unstable in various ways. For instance imagine an AI that has a utility function that changes whenever it looks at it. Or an AI that can't resolve the ontological crisis and so fails when it learns more about the world. Or maybe an AI that has a utility function that contradicts itself. There seem to be lots of ways that an AI can have bugs other than simply having goals that aren't aligned with our values.

Of course most of these AI would simply crash, or flop around and not do anything. A small subset of them might foom and stabilize as it does so. Most AI developers would try to move their AI from the former to the latter, and in doing so may pass through a space of AI that can foom to a significant degree without totally stabilizing. Such an AI might become very powerful, but exhibit "insane" behaviors that cause it to destroy itself and its parent civilization.

It might seem unlikely that an "insane" AI could manage to foom, but remember that we ourselves are examples of systems that can use general reasoning to gain power while still having serious flaws.

This would prevent us from observing either alien civilizations or paperclipping, and is appealing as a solution to the Fermi paradox because any advancing civilization would likely begin developing AI. Other threats that could arise after the emergence of civilization probably require the civilization to exhibit behaviors that not all civilizations would. Just because we threatened each other with nuclear annihilation doesn't mean all civilizations would, and it only takes one exception. But AI development is a natural step in the path of progress and very tricky. No matter how a civilization behaves, it could still get AI wrong.

If this does work as a solution, it would imply that friendliness is super hard. Most of the destroyed civilizations probably thought they had it figured out when they first flipped the switch.