trentmkelly/LessWrong-43k huggingface dataset:
Voting Phase of 2018 LW Review 
by Ben Pace — 2020-01-08T03:35:27.204Z

For the past 1.5 months on LessWrong, we've been doing a major review of 2018 — looking back at old posts and asking which of them have stood the test of time.

The LessWrong 2018 Review has three major goals.

First, it is an experiment in improving the LessWrong community's longterm feedback and reward cycle.
Second, it is an attempt to build common knowledge about the best ideas we've discovered on LessWrong.
Third, after the vote, the LessWrong Team will compile the top posts into a physical book.

We spent about 2 weeks nominating posts, and 75 posts received the the 2 required nominations to pass through that round. (See all nominations on the nominations page.) Then we spent a month reviewing them, and users wrote 72 reviews, a number of them by the post-authors themselves. (See all reviews on the reviews page.)

And finally, as the conclusion of all of this work, we are now voting on all the nominated posts. Voting is open for 12 days, and will close on Sunday, January 19th. (We'll turn it off on Monday during the day, ensuring all timezones get it throughout Sunday.)

The vote has a simple first section, and a detailed-yet-optional second section based on quadratic voting. If you are one of the 430 users with 1000+ karma, you are eligible to vote, then now is the time for you to participate in the vote by following this link.

For all users and lurkers, regardless of karma, the next 12 days are your last opportunity to write reviews for any nominated posts in 2018, which I expect will have a significant impact on how people vote. As you can see, all reviews are highlighted when a user is voting on a post. (To review a post, go to the post and click "Write A Review" at the top of the post.)

This is the end of this post. If you'd like to read more detailed instructions about how to vote, the rest of the text below contains instructions for how to use the voting system.

How To Vote

Sorting Posts Into Buckets

The first part of voting is sorting the nominated posts into buckets.

The five buckets are: No, Neutral, Good, Important, Crucial. Sort the posts however you think is best.

The key part is the relative weighting of different posts. For example, it won't make a difference to your final vote if you put every post in 'crucial' or every post in 'good'.

Fine-Tuning Your Votes

The system we're using is quadratic voting (as I discussed a few weeks ago).

Once you're happy

...
How to Throw Away Information in Causal DAGs
by johnswentworth — 2020-01-08T02:40:05.489Z

When constructing a high-level abstract causal DAG from a low-level DAG, one operation which comes up quite often is throwing away information from a node. This post is about how to do that.

First, how do we throw away information from random variables in general? Sparknotes:

Given a random variable X, we can throw away information by replacing X with f(X) for some function f.
Given some other random variable Y, f(X) “contains all information in X relevant to Y” if and only if P[Y|f(X)]=P[Y|X]. In particular, the full distribution function (y→P[Y=y|X]) is a minimal representation of the information about Y contained in X.

For more explanation of this, see Probability as Minimal Map.

For our purposes, starting from a low-level causal DAG, we want to:

Pick a node Xi
Pick a set of nodes XS (with i∈S)
Replace Xi by f(Xi) for some function f, such that
P[X¯S|f(Xi)]=P[X¯S|Xi]

Here ¯S denotes all the node indices outside S. (Specifying S rather than ¯S directly will usually be easier in practice, since XS is usually a small neighborhood of nodes around Xi.) In English: we want to throw away information from Xi, while retaining all information relevant to nodes outside the set S.

Two prototypical examples:

In a digital circuit, we pick the voltage in a particular wire at a particular time as Xi. Assuming the circuit is well designed, we will find that only the binary value bin(Xi) is relevant to voltages far away in the circuit or in time. So, with all “nearby” voltages as XS, we can replace Xi by bin(Xi).
In a fluid, we pick the (microscopic) positions and momenta of all the particles in a little cell of spacetime as Xi. Assuming uniform temperature, identical particles, and some source of external noise - even just a little bit - we expect that only the total number and momentum of particles in the cell will be relevant to the positions and momenta of particles far away in space and time. So, with all “nearby” cells/particles as XS, we can replace the microscopic positions and momenta of all particles in the cell with the total number and momentum of particles in the cell.

In both examples, we’re throwing out “local” information, while maintaining any information which is relevant “globally”. This will mean that local queries - e.g. the voltage in one wire given the...
Morality vs related concepts
by MichaelA — 2020-01-07T10:47:30.240Z

Cross-posted to the EA Forum.

How can you know I’m talking about morality (aka ethics), rather than something else, when I say that I “should” do something, that humanity “ought” to take certain actions, or that something is “good”? What are the borderlines and distinctions between morality and the various potential “something else”s? How do they overlap and interrelate?

In this post, I try to collect together and summarise philosophical concepts that are relevant to the above questions.[1] I hope this will benefit readers by introducing them to some thought-clarifying conceptual distinctions they may not have been aware of, as well as terms and links they can use to find more relevant info. In another post, I similarly discuss how moral uncertainty differs from and overlaps with related concepts.

Epistemic status: The concepts covered here are broad, fuzzy, and overlap in various ways, making definitions and distinctions between them almost inevitably debatable. Additionally, I’m not an expert in these topics; indeed, I expect many readers to know more than me about at least some of them, and one reason I wrote this was to help me clarify my own understandings. I’d appreciate feedback or comments in relation to any mistakes, poor phrasings, etc. (and just in general!).

Also note that my intention here is mostly to summarise existing ideas, rather than to provide original ideas or analysis.

Normativity

A normative statement is any statement related to what one should do, what one ought to do, which of two things are better, or similar. “Something is said by philosophers to have ‘normativity’ when it entails that some action, attitude or mental state of some other kind is justified, an action one ought to do or a state one ought to be in” (Darwall). Normativity is thus the overarching category (superset) of which things like morality, prudence (in the sense explained below), and arguably rationality are just subsets.

This matches the usage of “normative” in economics, where normative claims relate to “what ought to be” (e.g., “The government should increase its spending”), while positive claims relate to “what is” (including predictions, such as what effects an increase in government spending may have). In linguistics, the equivalent distinction is between prescriptive approaches (involving normative claims about “better” or “correct” uses of language) and descrip

...
AIRCS Workshop: How I failed to be recruited at MIRI.
by Arthur Milchior — 2020-01-07T01:03:53.650Z

This blog post is cross-posted from my personal blog. It will touch on two related topics:

Why and how I applied to MIRI and failed to secure an internship.

My experience at the AI Risk for Computer Scientists workshop (AIRCS).

If you're only interested in the AIRCS workshop, you may skip to the second section directly. Ideally, I'd have liked to make two separate entries, as they may pertain to different points of interest. However, both topics are extremely intertwined to me, and I could not make a clear cut in this experience. I should also note that me failing to secure an internship at MIRI probably have had a drastic impact in how I write about it, if only because I'd have been more constrained in what I describe had I got the internship.

With respect to people's name, I'll adhere to the following rule: Only mention names if what they said was done so publicly. That means that for books, facebook public pages or lectures, I will assume I can use the name of the author or teacher, and I will keep the name to myself for private discussions. You should probably also read Buck's comment, answering to this post.

Miri and me.

I discovered MIRI through Eliezer Yudkowsky, as I first began reading HPMOR and then Rationality, from A.I. to Zombie. Like almost everyone, I'm not sure what it is MIRI exactly do. I know at least that MIRI's intended goal is to save the world from unaligned AGI. But whatever it is they concretely do, it seems quite fun - I mean, it seems to involve type theory and category theory. I also read some articles they wrote, and skimmed through many other. While interesting, I've never caught enough details to actually imagine how to even start implementing anything they speak of. Reading some of their writings reminded me of several epistemology books I read years ago, but written more precisely, with clear code in mind, and to a computer-science-savy audience.

As I said, fun stuff, fun work!

In February 2019, Eliezer Yudkowsky shared on facebook a post by Buck Schlegeris stating that "I think that every EA who is a software engineer should apply to work at MIRI, if you can imagine wanting to work at MIRI." (and some more stuff). When I read that, I thought I should give it at least a try. I really didn't know how I could have helped them given my professional background - mostly logic and automata theory - but if they say that we should giv

...
Definitions of Causal Abstraction: Reviewing Beckers & Halpern
by johnswentworth — 2020-01-07T00:03:42.902Z

Author's Notes: This post is fairly technical, with little background and minimal examples; it is not recommended for general consumption. A general understanding of causal models is assumed. This post is probably most useful when read alongside the paper. If your last name is "Beckers" or "Halpern", you might want to skip to the last section.

There’s been a handful of papers in the last few years on abstracting causal models. Beckers and Halpern (B&H) wrote an

entire paper

on definitions of abstraction on causal models. This post will outline the general framework in which these definitions live, discuss the main two definitions which B&H favor, and wrap up with some discussion of a conjecture from the paper. I'll generally use notation and explanations which I find intuitive, rather than matching the paper on everything.

In general, we’ll follow B&H in progressing from more general to more specific definitions.

General Framework

We have two causal models: one “low-level”, and one “high-level”. There’s a few choices about what sort of “causal model” to use here; the main options are:

Structural equations
Structural equations with a DAG structure (i.e. no feedback loops)
Bayes nets

B&H use the first, presumably because it is the most general. That means that everything here will also apply to the latter two options.

Notation for the causal models:

We’ll write XH for the variables in the high-level model and XL for variables in the low-level model.
We’ll use capital-letter indices to indicate choosing multiple indices at once. For instance, with S= (1,2,3), XS would be (X1,X2,X3).
We’ll write interventions as XS←X∗S. For instance, with S=(1,2,3), XS←X∗S would be equivalent to the three simultaneous interventions (X1←X∗1,X2←X∗2,X3←X∗3). Usually both S and X∗ will be unspecified, to indicate a generic intervention.

Next, we need some connection between the high-level and low-level model, to capture the intuitive notion of “abstraction”. At its most general, this connection has two pieces:

A mapping τ between values of the variables in the models: XH=τ(XL)
A mapping ω between interventions: (XHSH←XH∗SH)=ω(XLSL←XL&#

...
Exploring safe exploration
by evhub — 2020-01-06T21:07:37.761Z

This post is an attempt at reformulating some of the points I wanted to make in “Safe exploration and corrigibility” in a clearer way. This post is standalone and does not assume that post as background.

In a previous comment thread, Rohin argued that safe exploration is currently defined as being about the agent not making “an accidental mistake.” I think that definition is wrong, at least to the extent that I think it both doesn't make much sense and doesn't describe how I actually expect current safe exploration work to be useful.

First, what does it mean for a failure to be an “accident?” This question is simple from the perspective of an engineer outside the whole system—any unintended failure is an accident, encapsulating the majority of AI safety concerns (i.e. “accident risk”). But that's clearly not what the term “accidental mistake” is pointing at in this context—rather, the question here is what is an accident from the perspective of the model? Intuitively, an accident from the perspective of the model should be some failure that the model didn't intend or wouldn't retroactively endorse. But that sort of a definition only makes sense for highly coherent mesa-optimizers that actually have some notion of intent. Maybe instead we should be thinking of this from the perspective of the base optimizer/loss function? That is, maybe a failure is an accidental failure if the loss function wouldn't retroactively endorse it (e.g. the model got a very low reward for making the mistake). By this definition, however, every generalization failure is an accidental failure such that safe exploration would just be the problem of generalization.

Of all of these definitions, the definition defining an accidental failure from the perspective of the model as a failure that the model didn't intend or wouldn't endorse seems the most sensical to me. Even assuming that your model is a highly coherent mesa-optimizer such that this definition makes sense, however, I still don't think it describes current safe exploration work, and in fact I don't think it's even really a safety problem. The problem of producing models which don't make mistakes from the perspective of their own internal goals is precisely the problem of making powerful, capable models—that is, it's precisely the problem of capability generalization. Thus, to the extent that it's reasonable to say this for any ML

...
Open & Welcome Thread - January 2020
by habryka — 2020-01-06T19:42:36.499Z

If it’s worth saying, but not worth its own post, here's a place to put it. (You can also make a shortform post)

And, if you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ.

The Open Thread sequence is here.
Looking at Mixers
by jefftk — 2020-01-06T16:10:01.773Z

This is not an official post by the BIDA Board, and the Board has
not reviewed it.

In Spring 2015 BIDA bought a Mackie 1604-VLZ Pro mixer used for $275.
It wasn't the ideal mixer, but it was reasonably close and a good
price. It's been starting to wear out for a while, however, and last
night one of the two monitor mixes stopped working. While this one
may be fixable, I think we should probably get a new mixer that's
closer to what we want and sell this one. Here's what I think would
be good in a mixer for a dance like BIDA:

At least 10 XLR inputs, ideally 16.

At least 2 pre-fader monitor outputs, ideally 3+

EQ with sweepable mids.

XLR outputs for the mains, and ideally the monitors.

Ideally, the monitors are covered by the mute.

Ideally, the monitors are covered by the EQ.

Ideally, few extra controls we're not going to use that,
when set wrong, keep things from working in hard-to-diagnose ways
(this is an issue with our current board).

I'm thinking that an analog board probably still makes sense, given
that we have a range of people who run sound at the dance, but I could
be convinced otherwise. Looking around, here's are the options that
look best to me:

Mackie
ProFX22v3: analog board with pretty much everything we need,
though on the expensive end. It has 17 xlr inputs, two of which can
be used as DIs, and 3 pre-fader monitor outputs. The EQ includes the
monitors, but the mute unfortunately doesn't. About $700 new.

Mackie
ProFX22v2: by going back a generation we can get something almost
as good for less. This brings us down to 16 xlr inputs, none of which
can be a DI, and two pre-fader monitor outputs. About $500 new.

Mackie
ProFX16v3: another direction we could go is a smaller board.
Dropping down a size from the ProFX22v3 brings us down to 11 xlr
inputs, but doesn't change anything else. The big question is how
often we would find ourselves needing more than 11 inputs. About $500
new.

Behringer
XR18: this is a tablet-controlled mixer, which probably won't
work for us. But it has sixteen XLR/TRS combo inputs, six monitor
outputs, and lets you mix from anywhere in the hall. It includes its
own WIFI, and I know a lot of sound people like it a lot. About
$600 new.

KitchenAid
KSM105: this is a very popular analog mixer, but it doesn't meet
any of our requirements. I don't und

...
What were the biggest discoveries / innovations in AI and ML?
by Eli Tyre — 2020-01-06T07:42:11.048Z

These could be theoretical breakthroughs (like "the idea of a perceptron" or [something Judea Pearl did]), or they could be watershed developments / new applications that don't necessarily involve much new theory (like AlexNet or AlphaGo). Anything that seems like an important development in AI, is fair game.

I want an independently-generated list of all the interesting developments of AI, over the whole life of the fields, for a research project that I'm working on.

Feel free to include ones that you, personally, think were a big deal in some way, even if most people don't think so.

Thanks!
The Universe Doesn't Have to Play Nice
by Chris_Leong — 2020-01-06T02:08:54.406Z

It's often helpful to think about the root cause of your disagreements with other people and it seems to me that one such cause is that I believe the universe doesn't have to play nice in terms of our ability to know anything, while other people do.

Here are some examples of where this assumption can be used:

The problem of skepticism - How do we know that any of the objects that we seem to perceive are real? Countless philosophers have tried to answer this question, but I'm happy to bite the bullet as I don't see any reason for believing that the universe has to give us this ability
Godel's theorem/incomputability: It may seem obvious that every mathematical truth has a proof or that we can write a program to compute everything, but we now know this is false
Heisenberg's uncertainty principle: We might imagine that if we were clever enough we could find a scheme for gaining perfect information about a particle, but this isn't the case
Problem of induction - Induction may have worked in the past, but we can't conclude that it'll work in the future without using induction itself which would be a circular argument
Boltzmann Brains - Universes where the majority of consciousness is Boltzman Brains may seem absurd, but we could construct a simulation where the majority of agents would be Boltzmann brains so it is theoretically coherent. I haven't checked the maths, but maybe we just can't know if we are a Boltzmann Brain or not.
Bayesianism - How do we form a prior given that the prior is before any observation that could help us determine a reasonable prior? Some people see this as an argument against Bayesianism, but I see it as just another way the universe isn't nice
Theories that can explain anything - Evolutionary psychology has often been criticised as being able to provide just-so stories for whatever we observe. I don't believe that the universe is always kind enough to provide us with a nice clean empirical test to determine if a theory is true or not as opposed to subtly impacting our expectations over a wide range of results. This becomes worse once you take into account varying positions within a theory, as there could be dozens of incompatible schemes for constraining expectations based on a theory with almost nothing in common
Non-empirically testable facts - Some people believe if there isn

...
What we Know vs. How we Know it?
by Elizabeth — 2020-01-06T00:30:01.261Z

Two weeks ago I said:

The other concept I’m playing with is that “what we know” is inextricable from “how we know it”. This is dangerously close to logical positivism, which I disagree with my limited understanding of. And yet it’s really improved my thinking when doing historical research.

I have some more clarify on what I meant now. Let’s say you’re considering my ex-roommate, person P, as a roommate, and ask me for information. I have a couple of options.

Scenario 1: I turn over chat logs and video recordings of my interactions with the P.

E.g., recordings of P playing music loudly and chat logs showing I’d asked them to stop.

Trust required: that the evidence is representative and not an elaborate deep fake.

Scenario 2: I report representative examples of my interactions with P.

E.g., “On these dates P played music really loudly even when I asked them to stop.”

Trust required: that from scenario 1, plus that I’m not making up the examples.

Scenario 3: I report summaries of patterns with P

E.g., “P often played loud music, even when I asked them to stop”

Trust required: that from scenario 2, plus my ability to accurately infer and report patterns from data.

Scenario 4: I report what a third party told me

E.g. “Mark told me they played loud music a lot”

Trust required: that from scenario 3, plus my ability to evaluate other people’s evidence

Scenario 5: I give a flat “yes good” or “no bad” answer.

E.g., “P was a bad roommate.”

Trust required: that from scenario 3 and perhaps 4, plus that I have the same heuristics for roommate goodness that you do.

The earlier the scenario, the more you can draw your own conclusions and the less trust you need to have in me. Maybe you don’t care about loud music, and a flat yes/no would drive you away from a roommate that would be fine for you. Maybe I thought I was clear about asking for music to stop but my chat logs reveal I was merely hinting, and you are confident you’ll be able to ask more directly. The more specifics I give you, the better an assessment you’ll be able to make.

Here’s what this looks like applied to recent reading:

Scenario 5: Rome fell in the 500s AD.

Even if I trust your judgement, I have no idea why you think this or what it means to you.

Scenario 4: In Rome: The Book, Bob Loblaw says Rome Fell in the 500s AD.

At least I can look up why Bob thinks this.

Scenario 3: Pottery says Rome fell between 300 and 500 AD.

Usef...