trentmkelly/LessWrong-43k huggingface dataset:
LW Migration Announcement
by Vaniver — 2018-03-22T02:17:13.927Z

The votes are in, and of the eligible 376 voters, 102 voted to Migrate and 15 voted to archive. With 87% in favor, we’re going ahead with the transition, which will begin tomorrow (3/22) at 6pm Pacific time. Trike will take a snapshot of the database, transfer it to us, and we’ll begin the database import of new material since the last import. At 7pm, the DNS server will point to our server instead of Trike’s, and so people visiting lesswrong.com will see the new site. The import will run in parallel with the site operating, so there won’t be any downtime of the new site.

Some things to note:

Old links will continue to work, redirecting to the right new URL. We’ll be watching the analytics to notice any failures, and you can also alert us through Intercom (which we’ll shut off a few days into the transition, as problems get fixed).

Lesserwrong links will redirect to the same page at lesswrong.

We’ll add old karma scores to new karma scores, and may adjust the vote weight algorithm accordingly.

Please report issues here.

Also, for those of you near Berkeley, we’ll be throwing a launch party on April 7th at 7pm (LW, Facebook).
Hammertime Final Exam
by alkjash — 2018-03-22T01:10:00.662Z

This is part 30 of 30 in the Hammertime Sequence. Click here for the intro.

One of the overarching themes from CFAR, related to The Strategic Level, is that what you learn at CFAR is not a specific technique or set of techniques, but the cognitive strategy that produced those techniques. It follows that if I learned the right lessons from CFAR, then I would be able to produce qualitatively similar – if not as well empirically tested – new principles and approaches to instrumental rationality.

After CFAR, I wanted to design a test to see if I had learned the right lessons. Hammertime was that sort of test for me. Now here’s that same test for you.

The Final Exam

I will give three essay prompts and three difficulty levels. Original ideas would be great, but shining a new light on old hammers is also welcome!

Prompts

Design a instrumental rationality technique.
Introduce a rationality principle or framework.
Describe a cognitive defect, bias, or blindspot.

Difficulty Levels

Bronze Mace mode. Write one essay on one of the topics above.

Steel Cudgel of the Lion mode. Write two of three.

Vorpal Dragonscale Sledgehammer of the Whale mode. Write all three. For each essay, give yourself five minutes to brainstorm and five minutes to write.

Here are my answers.

1. Cooperate First

There’s an old story about a famous painter of the Realist school who spent a whole year of his training painting still lives of eggs. Each day, he would draw a single egg over and over. He must have produced thousands of sketches and paintings of eggs. His teacher knew exactly how important fundamentals are.

This same motif is deeply embedded in stories all over the world:

Return to fundamentals. Practice your fundamentals.

The iterated prisoner’s dilemma is one of the fundamental lessons of rationality. The world is more like a number of iterated prisoner’s dilemmas than you’d think. Human beings are more like tit-for-tat players than you’d think. It follows:

Cooperate First!

The first move you make in any interaction with a new acquaintance should be a cooperate, even if you expect them to defect. Perhaps even if you observe them defecting already.

Here’s a lesson I learned from meditating on the maxim Cooperate First:

Cooperating First feels like accepting an unfair game from the inside. There will be many situations in life where things are framed in a slightly but noticeably unfair way towards you initially....
Naming the Nameless
by sarahconstantin — 2018-03-22T00:35:55.634Z

Epistemic status: political, opinionated, personal, all the typical caveats for controversial posts.

I was talking with a libertarian friend of mine the other day about my growing discomfort with the political culture in the Bay Area, and he asked why I didn't just move.

It's a good question. Peter Thiel just moved to L.A., citing the left-wing San Francisco culture as his reason.

But I like living in the Bay, and I don't plan to go anywhere in the near future. I could have said that I'm here for the tech industry, or here because my friends are, or any number of superficially "practical" reasons, but they didn't feel like my real motivation.

What I actually gave as the reason I stay was... aesthetics.

Wait, what?

Let's Talk About Design

I'm not a designer, so I probably don't have the correct vocabulary to express what I see. Please bear with me, while I use simple and ignorant language; if any of my readers have a more sophisticated understanding, I'd love to hear about it in the comments.

Stuff that's marketed to Bay Area bourgeois bohemians has a coherent appearance. You see it in websites that are all smooth scrolling and gradients and minimalism -- see the sample websites on Squarespace, for instance. You see it in the product design on the labels and menus of cafes and juice bars and coffee shops -- The Plant Cafe is a good example. You see it in the almost-identical, smoothly minimalist layouts of every tech-startup office.

Professional designers may be getting bored of this "light-contrast, minimalist elegance" or "objectively beautiful, but mostly unremarkable, templates", and are trying out more deliberately jarring styles like Brutalism.

But for your typical consumer, the generic California/BoBo style works fine. It signals elegance, which means, more or less, that it's designed for educated, high-Openness, upper-middle-class, urban people. When I enter a space or a website with this aesthetic, or buy a product with this branding, it's shorthand for "Ahhhh, this place is run by competent professionals who know how to give me a pleasant experience. I will not feel harried or inconvenienced or confused here; I will be well taken care of. I will easily be able to slot my existing behavior patterns into the implicit "rules" of how to use and navigat...
The Math Learning Experiment
by Qiaochu_Yuan — 2018-03-21T21:59:04.682Z

Last Sunday I and a small group of volunteers ran an experiment. We got a bunch of CFAR alumni together and asked half of them to tutor the other half in math (in a broad sense, so including computer science) while paying close attention to what was happening in the minds of the tutees, focusing on issues like where the tutees were confused, where they were frustrated, where they were anxious or afraid, etc.

I had a few motivations in trying this. One was a sense that most rationalists could benefit from learning more math on the margin. Another was a sense of there being various obstacles standing in the way of that, for example an identity of "not being a math person" or various flavors of math anxiety (in addition to the fact that it's a hassle / expensive to find a math tutor, and that people have more urgent things to do), and wanting to see the extent to which those obstacles could be debugged at scale. A third was to explore running events at all.

The fourth and most important motivation for me came out of thoughts I've been having since reading Inadequate Equilibria. There's a cluster of people, including but not limited to Eliezer, Critch, and Nate, who (according to me) have what I internally call "trustworthy inside views," another name for which might be the ability to reliably generate useful gears models, and act based on them. This is the thing they do instead of using modest epistemology; it's the thing that allowed Eliezer to write HPMoR, among many other things. And what all of the people who seem to me to have this ability have in common is that they all have strong backgrounds in a technical subject like math, physics, or computer science (in addition to something else, this isn't sufficient). Probably there are other ways to acquire this ability, but acquiring it through learning technical subjects in a particular way seemed promising, and I wanted to see if I could help others make any progress in that direction at all, or at least understand the obstacles in the way of that.

I learned less about inside view than I was hoping. My plan was to run around observing all of the tutor-tutee pairs, but it was harder to do this productively than I expected: I kept feeling like I was missing important conversational context and not knowing how to model the tutee because of that. We tutored for 3 rounds ...
Rationality Feed: Last Month's Best Posts
by sapphire — 2018-03-21T14:12:26.432Z

I write a daily rational feed. I write up summaries/teasers for the previous day's article that I found interesting and/or enjoyable. I follow most rationalist blogs as well as LW2.0 and the EA Forum on RSS. The daily feed is posted in the

SSC Discord

and on my

Wordpress Blog

. However the rational feed includes quite alot of articles and many people have requested a more heavily pruned feed. Here we go:

=== AI Alignment:

https://www.lesserwrong.com/posts/QWyYcjrXASQuRHqC5/brains-and-backprop-a-key-timeline-crux - Modern deep learning is based on gradient descent with backpropogation of error. If the human brain implements this algorithm than to get to AGI we just need to improve existing algorithms. This implies short time horizons for AGI. Discussion of the relevant neuroscience and deep learning is included. - by jacobjacob

https://ai-alignment.com/iterated-distillation-and-amplification-157debfd1616 - "This is a guest post summarizing Paul Christiano’s proposed scheme for training machine learning systems that can be robustly aligned to complex and fuzzy values, which I call Iterated Distillation and Amplification (IDA) here. IDA is notably similar to AlphaGoZero and expert iteration."

=== Independence and Group Cohesion:

https://www.ribbonfarm.com/2018/03/01/luxuriating-in-privacy/ - "I have been somewhat of a cheerleader for group ritual and small group agency, lamenting the capacities and mental states lost in the transition away from a communal, close-knit society, toward an atomized, market-driven society. In reality, the thought of living in a communal, close-knit society, surrounded daily with family and friends, perhaps living in close quarters with many siblings or children, fills me with horror. Here I will allow my own heart its expression, and be a cheerleader for privacy. For something precious has been gained as well as lost in the transition to social modernity."

https://www.lesserwrong.com/posts/Mhaikukvt6N4YtwHF/dragon-army-retrospective - Dragon Army is a military inspired rationalist house. They recently completed a six month Phase One. The house did quote well relative to a normal rationalist house (B+ grade) but fell far short of its goals (D-). the biggest problems were caused by four problematic roles: Ghost who were not engaged enough, White Knights who held others to an unreasonable standard and felt betrayed, ...
Explicit and Implicit Communication
by lionhearted (Sebastian Marshall) — 2018-03-21T08:58:34.415Z

I write an essay every Thursday. Every so often, one seems to really resonate with people.

The piece I just wrote on the nature of explicit and implicit communication both got an enthusiastic reader response and seems directly relevant to a number of the projects and explorations people are doing here, so I'm bringing it over here.

Two points worth mentioning —

1. I take, I think, a relatively fair stance on the tradeoffs and benefits between implicit and explicit communication. But some people are heavily invested in explicit communications models, almost to the identity level, and might not like what they read. I just ask you to bring an open mind — I think the examples of implicit communication here are all clear and convincing cases where explicit can underperform.

2. It was written for a more general audience, hence a different mix of anecdote, different levels of rigor in definition, etc. I made some stylistic choices where I err on the side of persuasive writing, style, and poetics over more technical and higher-precision definitions and epistemology. Had I written this for LW to start, all the language and some of the reasoning chains would be shifted about 20 degrees or so — but nevertheless, I think the general points here are really, really important. Don't let style or pedantry get in the way of understanding for you here if you can help it.

I decided to post this once I got a lot of reader replies like this —

"Think this is one of the highest quality and most insightful pieces you've written - huge amount of original and actionable content that helps build some structure for a vague set of intuitions I've had for a while."

Ok, here we go —

Unity: Communication

SABOTAGE

“A second type of simple sabotage requires no destructive tools whatsoever and produces physical damage, if any, by highly indirect means. It is based on universal opportunities to make faulty decisions, to adopt a noncooperative attitude, and to induce others to follow suit. Making a faulty decision may be simply a matter of placing tools in one spot instead of another. A non-cooperative attitude may involve nothing more than creating an unpleasant situation among one’s fellow workers, engaging in bickerings, or displaying surliness and stupidity. […]

Acts of simple sabotage are occurring throughout Europe. An effort should be made to add to their efficiency, lessen their detectability, and...
Lightness and Unease
by TurnTrout — 2018-03-21T05:24:26.289Z

Light

Month 1

January 6th is the day I finished reading Superintelligence, and that’s the day my life felt like it entered protagonist mode. I tore through ten books over the next week and a half, from The Art of Strategy to Thinking: Fast and Slow. I proceeded to finish the considerable remainder of the Sequences I’d left unread. It wasn’t fear that fueled me – it was a sweet blend of curiosity and protectiveness, about and for this cruel and beautiful world in which we live. It was a sense of the possibilities afforded to those lucky enough to live in what Scott Alexander dubbed the carefree springtime of the universe. It was a rekindling of my youthful inquisitiveness, when I’d spend afternoons lost in math books, coding a C++ text RPG, or caring for a caterpillar over its evolution, just to set it free.

Month 2

I applied for the upcoming CFAR workshop and was later accepted (the interview was the first time where I heard another person say things like "updating"). I continued my non-technical reading, summarized a technical alignment result, and reviewed Naïve Set Theory. The concepts in that small set theory book expanded my mathematical horizons beyond measure (okay, I haven’t studied measure theory yet, so not quite beyond measure). This is also when things really clicked for me – I started generating novel insights regularly. Finally, I decided to try for 5 minutes before giving up on my CHAI application; this led to an idea of which I am truly proud.

I do not yet know whether I will be accepted for CHAI’s internship, but the idea merits two posts of its own: one for the proposal itself, and one for the wonderful emotional and psychological process of discovery I experienced.

Month 3

I find myself eagerly reading chapters and completing problem sets from AI: A Modern Approach, well beyond what is required for my class. I’m writing more often, learning more deeply, and generally my hedonic index is through the roof relative to last fall. I know I’ve given the impression of having let up over the last month, but I really haven’t – there are multiple projects of mine whose fruits I look forward to sharing.

Forwards

I’ve noticed that I go through phases of intense interest in activities, commitments, and games – this shift tended to occur multiple tim...
The Strategic Level
by alkjash — 2018-03-21T05:00:00.598Z

This is part 29 of 30 in the Hammertime Sequence. Click here for the intro.

I find myself dragging my feet on the last couple days of each Hammertime cycle. From this and several other data points, I think current my writing attention span is around a week, and drafts and outlines sitting for more than a week feel too stale to finish. Had I known this in advance, I would probably have structured Hammertime as six 5-day sprints.

Reinforcement Learning?

What happens when reinforcement learning isn’t enough?

You playing a game of Go against sensei. On move twenty-four, sensei invades your three-space extension with devastating precision, cutting a group you thought was safe into two scattered dragons. The left dragon tries to run away, but sensei cuts its escape route off with a delicate leaning attack on your corner enclosure. It dies with abandon.

The right dragon, now facing the massive wall sensei built up by attacking the left group, tries frantically to make life locally. Its second eye is poked out unceremoniously by a well-placed tesuji. Because of your struggle, sensei has fifty points of territory and thickness radiating across the entire board. You resign.

What is a novice supposed to learn from a game like this? If your teacher leaves you to your own devices to review the game, you might easily conclude any of the following, if not a dozen other things:

Don’t make three space extensions.
Never try to run away.
Do not respond to leaning moves.
Sacrifice early.
Study life and death.

Let’s say you learn lesson 1, don’t make three space extensions. The next week’s teaching game, you dutifully plod out two spaces from each approach. Sensei’s stones are balanced and efficient while yours are over-concentrated and unimaginative. You lose handily by points.

What happens now? Do you return to three-space extensions, frustrated with two-space ones?

Over-correction and Learning Stopsigns

The Strategic Level is a CFAR flash class about learning strategically: updating in such a way that will actually prevent the same failure modes in the future. The kind of learning above is definitely not strategic.

As I see it, there’s two common and overlapping kinds of failure modes in learning, where the lessons learned can be worse than nothing.

The first kind is over-correction:

Had an argument: “I should be more understanding.”

Had a panic attack: “I should just care less about everythin...
Computational Complexity of P-Zombies
by Gordon Seidoh Worley — 2018-03-21T00:51:31.103Z
Link: https://mapandterritory.org/computational-complexity-of-p-zombies-fc56909af96f

I’ve claimed that p-zombies require exponentially more resources than phenomenally conscious agents and used this to justify that AGI will necessarily be phenomenally conscious. I’ve previously supported this claim by pointing to a related result in integrated information theory showing that p-zombies constructed as feed-forward networks are exponentially larger than behaviorally equivalent conscious networks, but now I’d like to see if I can get a similar result in noematology.

NB: This account is provisional and not sufficiently well-founded to constitute a proof given my assumptions. That said, it should be precise enough to be falsifiable if wrong.

First, some notation. Given an agent A, let a behavior be a pair {x, A(x)} where x is the world state before the agent acts and A(x) is the world state after. For example, if when holding a teacup A takes a sip of tea, then x is “holding a cup of tea” and A(x) is “taking a sip of tea”. If we have a second agent, Z, we may wish to ask if, given the same initial world state x, Z(x)=A(x), viz. Z and A exhibit the same behavior when presented with x. In an obvious sense this question is meaningless because A and Z cannot be said to observe the exact same reality nor is there to any epistemically valid method within phenomenalism by which we may directly compare two world states, but given an observer agent O, we can consider if A(x)=Z(x) with respect to O’s experience of A(x) and Z(x) and this is what we will mean when we ask if A exhibits the same behavior as Z.

Now let A be a phenomenally conscious agent and Z a p-zombie of A, meaning that Z and A cannot be distinguished based on observed behavior and Z is not phenomenally conscious (although it is, of course, cybernetic). More formally, we are supposing that A and Z agree on the set of behaviors X such that for all {x, y} in X, y=A(x)=Z(x). I then claim that Z is exponentially larger than A with the cardinality of X, where by “larger” I mean physically made of more stuff.

The short explanation of my reasoning is that Z must do with only the ontic what A does with ontology. Unfortunately this hides a lot of my understanding about what phenomenal consciousness does and how it works, so instead I’ll explain by relating A and Z to computational models that can describe them and use those to show that Z must be exponentially larger than A.

Since Z is not phenomenally con...
Inference & Empiricism
by SatvikBeri — 2018-03-20T15:47:00.316Z

Speaking very roughly our best tools for figuring out the truth are inference and empiricism. By inference I mean using things like Math, Logic, and theory in general to conclude new facts from things we assume to be true. By empiricism I mean looking at the world, doing experiments, etc.

Inference tends to work particularly well when you're highly confident in your premises. Empiricism tends to work particularly well in domains of high uncertainty.

Nothing prevents you from combining the two – for example, my basic applied thought framework is to "run towards uncertainty" – that is, have a theory, identify the points of highest uncertainty in the theory, figure out the smallest experiment/action to resolve that uncertainty, do it. Basically the scientific method. This is what I call "Risk Driven Development" in the context of programming.

People from highly theoretical degrees tend to struggle with high-uncertainty domains after graduating from school because their go-to tool is pure inference, and inference without empiricism fails in the real world because your assumptions are never 100% true, not even close. (They generally learn empiricism with practice.)

The failure modes of high empiricism without theory are much more subtle. Pure empiricism pretty much always works decently well. Failures look more like "didn't invent general relativity" – theory tends to gather a small number of large victories. Less commonly, theory lets you avoid a mistake the first time you do something, or more generally learn from fewer examples.

One major point of contention among programmers is how much value you gain from using abstractions that are 95% true vs. 100% true. Programmers who are really good at inference gain a huge advantage from 100% true abstractions. Programmers who aren't gain a 5% advantage, and thus see it as a huge cost with little benefit. The vast majority of functional programming advocates you meet will be people whose preferred method is inference.

Someone who is strong at inference and weak at empiricism entering a high-uncertainty domain like flirting will often be given advice like "don't think too much." This doesn't usually work: the advice-giver has an "empiricism button", so to speak, that they can activate in that situation. The pure theorist does not, so simply turning off...