trentmkelly/LessWrong-43k huggingface dataset:
Interactive Bayes Theorem Visualization
by Allen Kim — 2018-03-03T04:00:24.771Z
Link: https://allenkim67.github.io/bayes-demo/

This visualization was inspired by the Julia Galef video that I linked to in the opening paragraph. One thing I liked about that video is that it represents probability space as a box which, for understanding Bayes' rule, I think is much more intuitive than the circular venn-diagram approaches. Some improvements that I think this visualization offers is showing the different probabilities updating in relation to each other, generalizing to more than two events for both priors and likelihoods, and explicitly showing the discarding of conditional probabilities of unobserved events and normalization of the posteriors.
Monthly Meta: Common Knowledge
by Chris_Leong — 2018-03-03T00:05:09.488Z

One of my favourite aspects of Less Wrong is that we've created a community where it is easier for us to communicate with each other because we have a shared understanding of certain concepts. A large amount of highly quality content has been posted to the site since it started, so I think it would be a useful exercise to reflect on which ideas should become part of this common knowledge for this edition of Monthly Meta.

I'd suggest that this should work as follows: each post should contain one concept, with a short definition and an explanation of why you think it is important. Please don't submit your own ideas. I'll go first.
Social Technology
by Samo Burja — 2018-03-02T19:54:23.415Z

This is an excerpt from the draft of

my upcoming book

on great founder theory. It was originally published on SamoBurja.com. You can

access the original here.

Although people are relatively aware of the material technology that powers their lives, they are less aware of the non-material technology that influences them — namely, social technology. Just as HTTP is the operating protocol for the web, politeness is the operating protocol for our social interactions. Following the protocol will lead to predictable and desirable outcomes. Breaking the protocol will lead to inaccessible websites, or, perhaps, unwanted social awkwardness. Politeness, just like HTTP, can be

documented

and taught.

When we talk about “social technology”, we are not referring to social media platforms like Facebook or Reddit, which are properly material technologies made possible by the right arrangements of consumer electronics, server farms, and computer networks. We mean something analogous to the idea of social engineering, a concept that came about at the end of the 19th century that refers to the intentional design of specific social arrangements and ways of operating. The Reddit platform itself is not social technology, but its use of moderators is. Facebook is not a social technology, but the expectation that you will regularly post nice pictures on Facebook for mom is. Similarly, airplanes are not social technology, but people generally agreeing to tie you to your seat if you keep trying to open the service doors during a flight is. Social and material technologies often act symbiotically, but they are functionally distinct.

It’s important to note that all except the simplest social technologies are designed. Though many of our crucial social technologies seem like natural parts of reality today, this was not always so. At some point they required intentional construction and adoption. Many social technologies we take for granted, including the very idea of having such critical systems as currency, law, and government, were born from concerted human agency. It is for this reason that we call it social technology, rather than social “norms”, or take a more broad anthropological or philosophical approach. Much like material technology, social technology is designed, adopted, and scaled. It is proceduralized and documentable.

Social technology...
Improved regret bound for DRL
by Vanessa Kosoy — 2018-03-02T12:49:27.000Z

% operators that are separated from the operand by a space

% autosize deliminaters

% operators that require brackets

% operators that require parentheses

% Paper specific

There is a simple way to somewhat improve the regret bound for DRL we derived before. Specifically, we have the following

#Theorem 1

There is some C∈(0,∞) s.t. the following holds.

Consider I an interface, α,ϵ∈(0,1), H={υk=(μk,rk)∈ΥI}k∈[N] for some N∈N. For each k∈[N], let σk be an ϵ-sane policy for υk. For each k∈[N], let Mk be the corresponding MDP. Denote

τ:=1NN−1∑k=0maxs∈Sksupγ∈[1−α,1)∣∣∣dVMk(s,γ)dγ∣∣∣+1

Assume that

i. For each k∈[N], α≤1−γMk.

ii. α≤1N2(lnN)4/3(1ϵ+|A|)−1τ8/3

Then, there is an ¯I-policy π∗ s.t.

1NN∑k=0(EU∗υk(1−α)−EUπ∗¯υk[σk](1−α))≤Cα1/4N1/2(lnN)1/3(1ϵ+|A|)1/4τ1/3

This gives gives us better dependence on N but worse dependence on τ compared to what we had before. However, we can also get the best of both bounds with a single algorithm, since the only difference between the algorithms is in the choice of the parameters η and T (which can be chosen to give the better of the two bounds for the given parameters).

To show Theorem 1, we use the following proposition which appears in Russo and Van Roy as Proposition 3 (see page 16):

#Proposition 1

Consider a probability space (Ω,P∈ΔΩ), N∈N, R⊆[0,1] a finite set and random variables U:Ω→R, K:Ω→[N] and J:Ω→[N]. Assume that K∗P=J∗P and I[K;J]=0. Then

I[K;J,U]≥2N(E[U∣J=K, K]−E[U])2

Theorem 1 is now proved exactly as before, with Proposition 1 replacing what was previously called Proposition B.3 and setting η and T to

η=α1/4N−1/2(lnN)1/3(1ϵ+|A|)1/4τ1/3

T=α−1/4N−1/2(lnN)−1/3(1ϵ+|A|)−1/4τ2/3
March 2018 Media Thread
by ArisKatsaris — 2018-03-02T01:06:54.316Z

This is the monthly thread for posting media of various types that you've found that you enjoy. Post what you're reading, listening to, watching, and your opinion of it. Post recommendations to blogs. Post whatever media you feel like discussing! To see previous recommendations, check out the older threads.

Rules:

Please avoid downvoting recommendations just because you don't personally like the recommended material; remember that liking is a two-place word. If you can point out a specific flaw in a person's recommendation, consider posting a comment to that effect.

If you want to post something that (you know) has been recommended before, but have another recommendation to add, please link to the original, so that the reader has both recommendations.

Please post only under one of the already created subthreads, and never directly under the parent media thread.

Use the "Other Media" thread if you believe the piece of media you want to discuss doesn't fit under any of the established categories.

Use the "Meta" thread if you want to discuss about the monthly media thread itself (e.g. to propose adding/removing/splitting/merging subthreads, or to discuss the type of content properly belonging to each subthread) or for any other question or issue you may have about the thread or the rules.
Quick Nate/Eliezer comments on discontinuity
by Rob Bensinger — 2018-03-01T22:03:27.094Z

This isn't a proper response to Paul Christiano or Katja Grace's recent writings about takeoff speed, but I wanted to cross-post Eliezer's first quick comments on Katja's piece someplace more linkable than Twitter:

There's a lot of steps in this argument that need to be spelled out in more detail. Hopefully I get a chance to write that up soon. But it already raises the level of debate by a lot, for which I am grateful.

E.g. it is not intuitive to me that "But evolution wasn't trying to optimize for STEM ability" is a rejoinder to "Gosh hominids sure got better at that quickly." I can imagine one detailed argument that this might be trying to gesture at, but I don't know if I'm imagining right.

Similarly it's hard to pin down which arguments say '"Average tech progress rates tell us something about an underlying step of inputs and returns with this type signature" and which say "I want to put the larger process in this reference class and demand big proof burdens."

I also wanted to caveat: Nate's experience is that the label "discontinuity" is usually assigned to misinterpretations of his position on AGI, so I don't want to endorse this particular framing of what the key question is. Quoting Nate from a conversation I recently had with him (not responding to these particular posts):

On my model, the key point is not "some AI systems will undergo discontinuous leaps in their intelligence as they learn," but rather, "different people will try to build AI systems in different ways, and each will have some path of construction and some path of learning that can be modeled relatively well by some curve, and some of those curves will be very, very steep early on (e.g., when the system is first coming online, in the same way that the curve 'how good is Google’s search engine' was super steep in the region between 'it doesn’t work' and 'it works at least a little'), and sometimes a new system will blow past the entire edifice of human knowledge in an afternoon shortly after it finishes coming online." Like, no one is saying that Alpha Zero had massive discontinuities in its learning curve, but it also wasn't just AlphaGo Lee Sedol but with marginally more training: the architecture was pulled apart, restruct

...
Hammertime Intermission #2
by alkjash — 2018-03-01T18:20:00.894Z

The final cycle of Hammertime will return on 3/12.

This is an open thread for feedback about Hammertime.

Other Resources

Owen Shen has been writing about instrumental rationality for a long time at MindLevelUp. This material is much more coherent and actually cites the science behind the techniques.

SquirrelInHell has taken on a similar project but at a much deeper level at BeWellTuned. These techniques are BLACK MAGIC powerful but may cost your soul. Consider Hammertime a gentle tutorial mission for BeWellTuned.

Epistemic Rationality

A piece of the third cycle may be devoted to applying instrumental rationality tools towards increasing epistemic standards in practice.

What are the immediate applications of Hammertime tools towards truth-seeking? Are there TAPs, Design decisions, CoZE experiments, or Focusing techniques that you would immediately recommend to increase epistemic standards? For example, I’ve seen people claim that they’ve installed a gut-level instinct for Bayes Theorem.

How does truth-seeking interface with instrumental rationality, especially on corrupted hardware? We pay a whole lot of lip service to “I desire to believe what is true,” and yet an enormous amount of our practical advice sounds like “Act as if [obviously false statement] is true.”

Meta-Cognitive Blind Spots

I have a friend who got so excited about conquering new fears that he’s been constantly flitting around, looking for aversions to squash with CoZE. When I asked him to try sticking to one thing for a while, he nearly refused, saying “It feels hard to look at.” My words:

when ppl get excited about CoZE

there’s something ridiculous about it

you’re literally looking for things that scare you

like actually scare you

and that’s not fun

One time this week, I was Focusing on the feeling of hating algebraic geometry homework. I found the feeling’s true name: “Working on problems in other fields makes me feel low-status.” The aversion completely vanished, but I was left with the sinking feeling that I’m status-seeking all the way down. In fact, this whole paragraph reporting this character flaw? It’s a self-deprecation status move too. Fuck.

[Commenting guidelines: At least one line of poetry per comment.]
Ms. Blue, meet Mr. Green
by alexei — 2018-03-01T13:43:39.821Z

Blue and Green are references to the

MTG Color Wheel personalities

. (Very fun reading, but not required to understand this post.)

Ms. Blue: Hi, Mr. Green! Heard you’ve been traveling recently. Where have you been?

Mr. Green: Oh, I’m so glad you asked! I just came back from Bali. I went to a three weeks long workshop, where they taught us to align our chakras and channel cosmic energy into our third eye. I’ve gone to different astral planes, became one with the universe, and communed with god.

Ms. Blue: *Slowly walks backwards, takes out her phone, and starts dialing the local psychiatric hospital.*

Mr. Green: Hey, I’m not crazy!

I’m sure most of you have met or heard someone who used this kind of language. I’ve heard it called woo-woo or woo-speak. I’m going to refer to it as “mystical language” here, but I’m open to a better name. For most you, this language is probably an instant cue to tune out the listener. They’re obviously out of touch with reality and what they are saying is such garbage, that it’s not even worth listening to. I certainly used to have that exact reaction. In this post, I want to show how to approach and understand this language, how to learn it, and how you might even derive some benefit from it.

Understanding the language

“Consciousness is a quantum leap from the physical to the non-physical dimension.” --Sadhguru

I think one particularly annoying thing to Ms. Blue and her tribe is how mystical language often borrows words and concepts from physics, e.g. quantum, energy, dimensions, reality, and so on. In physics, there is a clear separation of

the map and the territory

. The territory is the source of truth; it’s the thing that doesn’t go away when you close your eyes. Everyone can look at it and agree on what they’re seeing. The map is the model we have of the world. If this model is good, it can make accurate predictions about the territory.

The mystical language has something similar. The territory, however, is your own mind: all the sensations as they are occurring in this moment. If you’re looking at a candle, you will likely have multiple sensations associated with it: the visual sensation of the candle, the brightness sensation from the light, the smell of the burning wax, a warm sensation from the flame, perhaps a sense of an observer who is watching the candle. Our mind does a good job of taking related sensations and combining them...
Friendship
by alkjash — 2018-03-01T06:00:00.657Z

This is part 20 of 30 of Hammertime. Click here for the intro.

There’s a serious and scary phenomenon which Valentine’s recent posts have been touching on: much of who you are only exists (or is expressed) in the presence of other people. In the words of Bishop Berkeley, esse est percipi: To be is to be perceived. Hammertime will always be an incomplete endeavor unless it is applied to social settings – there are major chunks of the psyche only accessible in such settings.

Up to now, Hammertime has mostly been a set of tools for the individual rationalist in a social vacuum. Today I want to talk the problem of other human beings, and how to go about designing social interactions that are conducive to the practice of instrumental rationality.

Hammertime Day 20: Friendship

Background: The Intelligent Social Web

There’s good evidence in biology that the power of the human brain largely evolved to solve ever-complexifying social problems. Much of the heavy cognitive machinery in your head is primarily built for and responds best to social interaction. Brains are extremely good at detecting social threats and anomalies, at regulating implicit status ladders, at reading body language, and at simulating other brains.

This post is a start at the Design of optimal two-person interactions.

Iterated Games

Rationalists spend a lot of time railing against the failings of causal decision theory, and promoting alternatives that solve them. The uncomfortable truth, however, is that you will not make causal decision theorists cooperate on the prisoner’s dilemma by throwing tomes of philosophy at them, and many many people are causal decision theorists. Not all hope is lost though: there’s a known, albeit unglamorous, solution to coordination failures within the framework of causal decision theory: iterated games.

Iteration is the easiest path to building strong friendship: make interactions longer and more regular.

In the middle of January, I began contacting friends and setting up regularly weekly chats. Almost nobody refused. A handful of interactions fizzled out, but the ones that lasted have been unbelievably positive. I kept ramping up the number of conversations until it felt actively fatiguing. Today this habit alone allows me to talk to an average of one extra person per day for an hour and a half.

Human beings are unbelievably reciprocal crea...
Kidneys, trade, sacredness, and space travel
by Benquo — 2018-03-01T05:20:01.457Z

To the trader mindset, sacred values are nothing but a confusion; if you don’t like the deal, you just haven’t been offered a high enough price.

There’s something important the trader mindset can’t see. Its modus operandi is to take two different representations of value and profits from resolving discrepancies. It is agnostic as to the validity of those representations. Thus, the trade orientation tends to collapse the map-territory distinction, and in particular confuse exchange rates (i.e. prices) and stores of value.

Consider this music video.

The protagonist is fixated on an image that's been marketed to her by someone wealthy enough to control a planet. The image isn't very detailed, and she's willing to undertake a dangerous and arduous journey, which implies that things aren't very good back home.

She's in a world where travel is expensive. Somehow, improbably, in outer space, she has to pay a toll. This should clue us in that something sketchy is going on.

Tolls are one of the classic modes of rent extraction, second only to land rents in their centrality as an image. There's a plausible excuse for tolls on improvements like bridges, but you don't need bridges in space - you can only collect the toll by preventing people from going around you. This should inform how we interpret the subsequent interactions where she pays for fuel, repair to her spaceship, and repair to her body; it's not obvious how much of the price is needed to pay for the cost of the service, and how much is a rent extracted by a predatory monopolist.

At each stage, the protagonist sacrifices capacity (in the form of mobility affordances, maybe the most concrete and central instance of capacity, from the Latin capere, meaning to take hold of something - she trades away her hand, then her leg, then her remaining limbs, then her spaceship (albeit getting a fully functioning body back as far as we know)) for a some progress towards her destination. Then, once she gets there, she finds that she's traded away her ability to move, for relocation to a place that's no longer providing the service it advertised. It's true at each point that you wouldn't be helping her by preventing her from making the trade, but focusing on that aspect of the situation makes one a price-taker in a case where that attitude doesn't actually unlock a...
Ambiguity Detection
by TurnTrout — 2018-03-01T04:23:13.682Z

Note: the most up-to-date version of this proposal can be found here.

Introduction

If I present you with five examples of burritos, I don’t want you to pursue the simplest way of classifying burritos versus non-burritos. I want you to come up with a way of classifying the five burritos and none of the non-burritos that covers as little area as possible in the positive examples, while still having enough space around the positive examples that the AI can make a new burrito that’s not molecularly identical to the previous ones.

- AI Alignment: Why It's Hard and Where to Start

Consider the problem of designing classifiers that are able to reliably say "I don't know" for inputs well outside of their training set. This problem is studied as open-category classification in the literature [6]; a closely-related problem in AI safety is inductive ambiguity detection [4].

Solution of generalized open-category classification could allow for agents who robustly know when to ask for clarification; in this post, we discuss the problem in the context of classification. Although narrower in scope, the solution of the classification subproblem would herald the arrival of robust classifiers which extrapolate conservatively and intuitively from their training data.

It's obvious that we can't just teach classifiers the unknown class. One, this isn't compact, and two, it doesn't make sense - it's a map/territory confusion (unknown is a feature of our current world model, not a meaningful ontological class). Let's decompose the concept a bit further.

Weight regularization helps us find a mathematically-simple volume in the input space which encapsulates the data we have seen so far. In fact, a binary classifier enforces a hyperplane which cleaves the entire space into two volumes in a way which happens to nearly-optimally classify the training data. This hyperplane may be relatively simple to describe mathematically, but the problem is that the two volumes created are far too expansive.

In short, we'd like our machine learning algorithms to learn explanations which fit the facts seen to date, are simple, and don't generalize too far afield.

Prior Work

Taylor et al. provide an overview of relevant research [4]. Taylor also introduced an approach for rejecting examples from distributions too far from the training distribution [5].... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Extended Quote on the Institution of Academia
by Ben Pace — 2018-03-01T02:58:11.159Z

From the top-notch 80,000 Hours podcast, and their recent interview with Holden Karnofsky (Executive Director of the Open Philanthropy Project).

What follows is an short analysis of what academia does and doesn't do, followed by a few discussion points by me at the end. I really like this frame, I'll likely use it in conversation in the future.

Robert Wiblin: What things do you think you’ve learned, over the last 11 years of doing this kind of research, about in what situations you can trust expert consensus and in what cases you should think there’s a substantial chance that it’s quite mistaken?

Holden Karnofsky: Sure. I mean I think it’s hard to generalize about this. Sometimes I wish I would write down my model more explicitly. I thought it was cool that Eliezer Yudkowsky did that in his book, Inadequate Equilibria. I think one thing that I especially look for, in terms of when we’re doing philanthropy, is I’m especially interested in the role of academia and what academia is able to do. You could look at corporations, you can understand their incentives. You can look at Governments, you can sort of understand their incentives. You can look at think-tanks, and a lot of them are just like … They’re aimed directly at Governments, in a sense. You can sort of understand what’s going on there.

Academia is the default home for people who really spend all their time thinking about things that are intellectual, that could be important to the world, but that there’s no client who is like, “I need this now for this reason. I’m making you do it.” A lot of the times, when someone says, “Someone should, let’s say, work on AI alignment or work on AI strategy or, for example, evaluate the evidence base for bed nets and deworming, which is what GiveWell does … ” A lot of the time, my first question, when it’s not obvious where else it fits, is would this fit into academia?

This is something where my opinions and my views have evolved a lot, where I used to have this very simplified, “Academia. That’s like this giant set of universities. There’s a whole ton of very smart intellectuals who knows they can do everything. There’s a zillion fields. There’s a literature on everything, as has been written on Marginal Revolution, all that sort of thing.” I really never know when to expect that something was going to be neglected and when it wasn’t, and it takes a giant lite...