trentmkelly/LessWrong-43k huggingface dataset:
The Zettelkasten Method
by abramdemski — 2019-09-20T13:15:10.131Z

[Epistemic Status: Scroll to the bottom for my follow-up thoughts on this from months/years later.]

Early this year, Conor White-Sullivan introduced me to the Zettelkasten method of note-taking. I would say that this significantly increased my research productivity. I’ve been saying “at least 2x”. Naturally, this sort of thing is difficult to quantify. The truth is, I think it may be more like 3x, especially along the dimension of “producing ideas” and also “early-stage development of ideas”. (What I mean by this will become clearer as I describe how I think about research productivity more generally.) However, it is also very possible that the method produces serious biases in the types of ideas produced/developed, which should be considered. (This would be difficult to quantify at the best of times, but also, it should be noted that other factors have dramatically decreased my overall research productivity. So, unfortunately, someone looking in from outside would not see an overall boost. Still, my impression is that it's been very useful.)

I think there are some specific reasons why Zettelkasten has worked so well for me. I’ll try to make those clear, to help readers decide whether it would work for them. However, I honestly didn’t think Zettelkasten sounded like a good idea before I tried it. It only took me about 30 minutes of working with the cards to decide that it was really good. So, if you’re like me, this is a cheap experiment. I think a lot of people should actually try it to see how they like it, even if it sounds terrible.

My plan for this document is to first give a short summary and then an overview of Zettelkasten, so that readers know roughly what I’m talking about, and can possibly experiment with it without reading any further. I’ll then launch into a longer discussion of why it worked well for me, explaining the specific habits which I think contributed, including some descriptions of my previous approaches to keeping research notes. I expect some of this may be useful even if you don’t use Zettelkasten -- if Zettelkasten isn’t for you, maybe these ideas will nonetheless help you to think about optimizing your notes. However, I put it here primarily because I think it will boost the chances of Zettelkasten working for you. It will give you...
Relabelings vs. External References
by Chris_Leong — 2019-09-20T02:20:34.529Z

The following distinction is crucial to my understanding of consciousness and epistemology. It is also extremely useful for seeing through some of the language games people play in terms of framing. There are fundamentally two ways in which we can define entities and much confusion is created by failing to distinguish between the two.

The first way is relabelling where we define something in terms of what we've already defined. If we've already defined the meaning of hydrogen atoms, oxygen atoms and molecules, then we can simply define water as a molecule consisting of one hydrogen atom and two oxygen atoms, if that's how we want to define it. Defining water in this way postulate a new entity in our fundamental ontology, but instead reuses existing elements.

The second way is to define an external reference; that is, a reference that doesn't purely build on top of already defined language or meaning. It is a very common belief that we can only define something in terms of things that have already been defined, but if that were the case, then we wouldn't even be able to get our first definition of the ground. All our definitions would either be circular or involve an infinite regress.

External references provide a not completely satisfactory, but pragmatically necessary workaround. Instead of being impractically purist when it comes to definitions, we allow ourselves to point to (or vaguely gesture at) things without being able to fully say what they are.

Let's imagine that quarks are the foundational element of physics as far as we can tell. Well, we could define quarks by how they interact with each other, but that is a surface level characteristic, which doesn't seem to explain what exactly they are at their fundamental core And if these quarks have a nature beyond their behaviour, then it makes sense to think about them as external references.

Some might say that the observable behaviour of these quarks is all that there is to their nature. If this is the case, then I would like to ask what is the difference between an atom and a simulation of an atom? Or even if I concede these quarks can be defined in terms of behaviour, what exactly is behaviour at its core? It doesn't seem like we can define behaviour in terms of behaviour and saying "It is what it is" seems like a way of obscuring that we don't know what it is. At the end of the day, it seems there has to...
Novum Organum: Preface
by Francis Bacon — 2019-09-19T22:51:41.622Z

Previously: Ruby's introduction to the Novum Organum sequence

We have used Francis Bacon's Novum Organum in the version presented at www.earlymoderntexts.com. Translated by and copyright to Jonathan Bennett. Prepared for LessWrong by Ruby.

Ruby's Reading Guide

Novum Organum is organized as two books each containing numbered "aphorisms." These vary in length from three lines to sixteen pages. Titles of posts in this sequence, e.g. Idols of the Mind Pt. 1, are my own and do not appear in the original.

While the translator, Bennett, encloses his editorial remarks in a single pair of [brackets], I have enclosed mine in a [[double pair of brackets]].

Bennett's Reading Guide

[Brackets] enclose editorial explanations. Small ·dots· enclose material that has been added, but can be read as though it were part of the original text. Occasional •bullets, and also indenting of passages that are not quotations, are meant as aids to grasping the structure of a sentence or a thought. Every four-point ellipsis . . . . indicates the omission of a brief passage that seems to present more difficulty than it is worth. Longer omissions are reported between brackets in normal-sized type.

‘Organon’ is the conventional title for the collection of logical works by Aristotle, a body of doctrine that Bacon aimed to replace. His title Novum Organum could mean ‘The New Organon’ or more modestly ‘A New Organon’; the tone of the writing in this work points to the definite article.

Aphorism Concerning the Interpretation of Nature: Preface

by Francis Bacon

Those who have taken it on themselves to lay down the law of nature as something that has already been discovered and understood, whether they have spoken in simple confidence or in a spirit of professional posturing, have done great harm to philosophy and the sciences. As well as succeeding in •producing beliefs in people, they have been effective in •squashing and stopping inquiry; and the harm they have done by spoiling and putting an end to other men’s efforts outweighs any good their own efforts have brought. Some people on the other hand have gone the opposite way, asserting that absolutely nothing can be known—having reached this opinion through dislike of the ancient sophists, or through uncertainty and fluctuation of mind, or...
Novum Organum: Introduction
by Ruby — 2019-09-19T22:34:21.223Z

In light of its value as a rationalist text, its historical influence on the progress of science, and its general expression of the philosophy and vision which guides LessWrong 2.0, the moderation team has seen fit to publish Novum Organum as a LessWrong sequence. (Image: the engraved title page.)

Quotes in this post are from Francis Bacon's Novum Organum in the version by Jonathan Bennett presented at www.earlymoderntexts.com

In 1620, Francis Bacon’s Novum Organum was published. Though the work might be succinctly described as Bacon’s views on empiricism and inductivism, it is far more than a list of experimental steps to be followed. It is an entire epistemology and philosophy—possibly the epistemology and philosophy which underlay the Scientific Revolution.

Bacon was damning of the science of his time and preceding centuries. He saw the pseudo-empirical syllogistic paradigm as deeply flawed and incapable of making progress.

If those doctrines ·of the ancient Greeks· hadn’t been so utterly like a plant torn up by its roots, and had remained attached to and nourished by the womb of nature, the state of affairs that we have seen to obtain for two thousand years—namely the sciences stayed in the place where they began, hardly changing, not getting any additions worth mentioning, thriving best in the hands of their first founders and declining from then on—would never have come about. (74) [1]

He also believed that the unaided human mind was incapable of getting far on its own.

Nearly all the things that go wrong in the sciences have a single cause and root, namely: while wrongly admiring and praising the powers of the human mind, we don’t look for true helps for it. (9)

Not much can be achieved by the naked hand or by the unaided intellect. Tasks are carried through by tools and helps, and the intellect needs them as much as the hand does. (2)

When the intellect of a sober, patient, and grave mind is left to itself (especially in a mind that isn’t held back by accepted doctrines), it ventures a little way along the right path; but it doesn’t get far, because without guidance and help it isn’t up to the task, and is quite unfit to overcome the obscurity of things. (21)

Nonetheless, he was optimistic that if the old doctrines were abandoned, idols of the mind (i.e., biases, fallacies, and confusions) were cleared out, and his precise, careful empirical method was foll...
Interview with Aella, Part I
by Jacob Falkovich — 2019-09-19T14:05:18.523Z

Cross-posted from Putanumonit.

Things that happen offline notice: if you’re reading this I strongly predict that you will enjoy attending a Slate Star Codex meetup. I will be at the New York meetup on Saturday 9/21, come say hi!

This is part 1 of the interview, talking about Aella’s background, Twitter, Jews, the homeless, LSD ranches, Inuit mythology, insecurity, and sex. Part 2 is about psychedelics and philosophizing and will post next week.

I’m here in a secret location in the financial district of Manhattan with the most interesting woman in the world.

And we are drinking whiskey, which is appropriate.

You’re doing all the homework beforehand. It’s really cool. I haven’t experienced this before.

In these interviews, I try to channel Tyler Cowen. Before he interviews people he reads everything they’ve produced so he’s not introducing listeners to a person, but diving really deep into how they think. Because we set this interview up on short notice, I spent 8 hours yesterday consuming your entire blog and podcast archive.

Have you heard of Nardwuar? He’s an interviewer who also does this. Last night when you sent me the questions I felt that this would be like a Nardwuar interview.

I want to start with an Askholey question. Do people know the real you? Would people who follow you online be surprised if they got to spend a lot of time with you?

I don’t know what that means, “the real me”. Most likely people would be surprised by things that aren’t that interesting, like small habits I have. Things that are interesting, I tend to talk about because stories are a commodity. Learning to identify the aspects of my life that can serve as a commodity is really useful because you can put it out there and then get attention and start a dialogue in ways that are new and creative. And so the things that aren’t novel are things that might surprise you, like how I take my coffee or something.

Do you identify with the story of you that’s out there?

I don’t know what that means. There’s a strong way in which I feel like the stuff I write about myself in the past is a completely different person. The sense that I have when writing about myself in the past, is like a story that’s not me. It’s just a dream that I woke up with in my head one day an...
Gears vs Behavior
by johnswentworth — 2019-09-19T06:50:42.379Z

Thankyou to Sisi Cheng (of the Working as Intended comic) for the excellent drawings.

Suppose we have a gearbox. On one side is a crank, on the other side is a wheel which spins when the crank is turned. We want to predict the rotation of the wheel given the rotation of the crank, so we run a

Kaggle competition

.

We collect hundreds of thousands of data points on crank rotation and wheel rotation. 70% are used as training data, the other 30% set aside as test data and kept under lock and key in an old nuclear bunker. Hundreds of teams submit algorithms to predict wheel rotation from crank rotation. Several top teams combine their models into one gradient-boosted deep random neural support vector forest. The model achieves stunning precision and accuracy in predicting wheel rotation.

On the other hand, in a very literal sense, the model contains no

gears

. Is that a problem? If so, when and why would it be a problem?

What is Missing?

When we say the model “

contains no gears

”, what does that mean, in a less literal and more generalizable sense?

Simplest answer: the deep random neural support vector forest model does not tell us what we expect to see if we open up the physical gearbox.

For instance, consider these two gearboxes:

Both produce the same input-output behavior. Our model above, which treats the gearbox as a literal black box, does not tell us anything at all which would distinguish between these two cases. It only talks about input-output behavior, without making any predictions about what’s inside the gearbox (other than that the gearbox must be consistent with the input/output behavior).

That’s the key feature of gears-level models: they make falsifiable predictions about the internals of a system, separate from the externally-visible behavior. If a model could correctly predict all of a system’s externally-visible behavior, but still be falsified by looking inside the box, then that’s a gears-level model. Conversely, we cannot fully learn gears-level models by looking only at externally-visible input-output behavior - external behavior cannot, for example, distinguish between the 3- and 5-gear models above. A model which can be fully learned from system behavior, without any side information, is not a full gears-level model.

Why would this be useful, if what we really care about is the externally-visible behavior? Several reasons:

First and foremost, if we a

...
Don't depend on others to ask for explanations
by Wei Dai — 2019-09-18T19:12:56.145Z

I'm often reluctant to ask for explanations on LW, and to typical-mind a bit I think this may be true of others as well. This suggests that when you're writing something for public consumption, it's better to err on the side of too much rather than too little explanation. If there's too much explanation, people can just skip over it (and you can make it easier by putting explanations that may be "too much" in parentheses or footnotes), but if there's too little explanation people may never ask for it. So in the future if you ever think something like, "I'll just write down what I think, and if people don't understand why, they can ask" I hope this post will cause you to have a second thought about that.

To make it clearer that this problem can't be solved by just asking or training people to be less reluctant to ask for explanations, I think there are often "good" reasons for such reluctance. Here's a list that I came up with during a previous discussion with Raymond Arnold (Raemon):

I already spent quite some time trying to puzzle out the explanation, and asking is like admitting defeat.

If there is a simple explanation that I reasonably could have figured out without asking, I look bad by asking.

It's forcing me to publicly signal interest, and maybe I don't want to do that.

Related to 3, it's forcing me to raise the status of the person I'm asking, by showing that I'm interested in what they're saying. (Relatedly, I worry this might cause people to withhold explanations more often than they should.)

If my request is ignored or denied, I would feel bad, perhaps in part because it seems to lower my status.

I feel annoyed that the commenter didn't value my time enough to preemptively include an explanation, and therefore don't want to interact further with them.

My comment requesting an explanation is going to read by lots of people for whom it has no value, and I don't want to impose that cost on them, or make them subconsciously annoyed at me, etc.

By the time the answer comes, the topic may have left my short term memory, or I may not be that interested anymore.
What technical prereqs would I need in order to understand Stuart Armstrong's research agenda?
by Vaughn Papenhausen — 2019-09-18T17:17:05.602Z

I've looked at a few of Stuart Armstrong's posts that he put up related to his research agenda (though only ones before he posted the full agenda), and felt like I was missing some prereqs. My background is in philosophy. What subjects or particular resources should I study to be able to read his work?
What are the studies and literature on the traditional medicine theory of humorism?
by Rudi C — 2019-09-18T16:06:36.437Z

I live in Iran, and here people strongly believe in Avicenna’s humorism (or what is thought of it in popular culture anyways.). It is believed on the level of it being “common sense.” For example, if you eat fish, milk, broccoli, and tomato sauce, all of which are “cold”, you’re supposed to balance that out by eating walnuts and dates.
My personal impression is that there is probably some truth to this simplistic model of nutrition, as I see a lot of anecdotal evidence for it, but well, I like to see what science is on the subject.

Note that the humorism believed in here (Iran) is not a strawman; People don’t believe that humor imbalance is the root cause of all diseases. It is mostly believed that if you eat very imbalanced foods, you will have a significant chance of getting “unwell.” E.g., you can get stomachache, vomit or get a sore sensation in the mouth. (I am not actually very knowledgeable on the traditional lores here, and the different imbalances are known for different symptoms.)

I might add that both my parents are experienced specialized medical doctors, and they, too, believe that there is something to all this.

My personal “wishful thinking bias” in this matter is that I like the whole thing to be false. I generally dislike nutritional restrictions, and I dislike traditions and alternative medicine. :))

Cursory Internet searches did not lead me to good meta analyses on this subject. I just found unempirical denials that these beliefs are now considered pseudoscience. I personally suspect premature theoretical disbelief instead of carefully studying subtle effects.
The unexpected difficulty of comparing AlphaStar to humans
by Richard Korzekwa  — 2019-09-18T02:20:01.292Z

This is crossposted from the AI Impacts blog.

Artificial intelligence defeated a pair of professional Starcraft II players for the first time in December 2018. Although this was generally regarded as an impressive achievement, it quickly became clear that not everybody was satisfied with how the AI agent, called AlphaStar, interacted with the game, or how its creator, DeepMind, presented it. Many observers complained that, in spite of DeepMind’s claims that it performed at similar speeds to humans, AlphaStar was able to control the game with greater speed and accuracy than any human, and that this was the reason why it prevailed.

Although I think this story is mostly correct, I think it is harder than it looks to compare AlphaStar’s interaction with the game to that of humans, and to determine to what extent this mattered for the outcome of the matches. Merely comparing raw numbers for actions taken per minute (the usual metric for a player’s speed) does not tell the whole story, and appropriately taking into account mouse accuracy, the differences between combat actions and non-combat actions, and the control of the game’s “camera” turns out to be quite difficult.

Here, I begin with an overview of Starcraft II as a platform for AI research, a timeline of events leading up to AlphaStar’s success, and a brief description of how AlphaStar works. Next, I explain why measuring performance in Starcraft II is hard, show some analysis on the speed of both human and AI players, and offer some preliminary conclusions on how AlphaStar’s speed compares to humans. After this, I discuss the differences in how humans and AlphaStar “see” the game and the impact this has on performance. Finally, I give an update on DeepMind’s current experiments with Starcraft II and explain why I expect we will encounter similar difficulties when comparing human and AI performance in the future.

Why Starcraft is a Target for AI Research

Starcraft II has been a target for AI for several years, and some readers will recall that Starcraft II appeared on our 2016 expert survey. But there are many games and many AIs that play them, so it may not be obvious why Starcraft II is a target for research or why it is of interest to those of us that are trying to understand what is happening with AI.

For the most part, Starcraft II was chosen because it is popular, and it is difficult for AI. S...
Three Stories for How AGI Comes Before FAI
by John_Maxwell — 2019-09-17T23:26:44.150Z

Epistemic status: fake framework

To do effective differential technological development for AI safety, we'd like to know which combinations of AI insights are more likely to lead to FAI vs UFAI. This is an overarching strategic consideration which feeds into questions like how to think about the value of AI capabilities research.

As far as I can tell, there are actually several different stories for how we may end up with a set of AI insights which makes UFAI more likely than FAI, and these stories aren't entirely compatible with one another.

Note: In this document, when I say "FAI", I mean any superintelligent system which does a good job of helping humans (so an "aligned Task AGI" also counts).

Story #1: The Roadblock Story

Nate Soares describes the roadblock story in this comment:

...if a safety-conscious AGI team asked how we’d expect their project to fail, the two likeliest scenarios we’d point to are "your team runs into a capabilities roadblock and can't achieve AGI" or "your team runs into an alignment roadblock and can easily tell that the system is currently misaligned, but can’t figure out how to achieve alignment in any reasonable amount of time."

(emphasis mine)

The roadblock story happens if there are key safety insights that FAI needs but AGI doesn't need. In this story, the knowledge needed for FAI is a superset of the knowledge needed for AGI. If the safety insights are difficult to obtain, or no one is working to obtain them, we could find ourselves in a situation where we have all the AGI insights without having all the FAI insights.

There is subtlety here. In order to make a strong argument for the existence of insights like this, it's not enough to point to failures of existing systems, or describe hypothetical failures of future systems. You also need to explain why the insights necessary to create AGI wouldn't be sufficient to fix the problems.

Some possible ways the roadblock story could come about:

Maybe safety insights are more or less agnostic to the chosen AGI technology and can be discovered in parallel. (Stuart Russell has pushed against this, saying that in the same way making sure bridges don't fall down is part of civil engineering, safety should be part of mainstream AI research.)

Maybe safety insights require AGI insights as a prerequisite, leaving us in a precarious position where we will have acquired the capability

...