trentmkelly/LessWrong-43k huggingface dataset:
Karma fluctuations?
by  — 2020-06-11T00:33:26.604Z

I see karma on posts fluctuating (in particular going down) more than I would expect coming from other vote-based websites. Is downvoting really used here for posts that are not spam or trolling? Or do people just change their minds a lot?

The FAQ has: We encourage people to vote such that upvote means “I want to see more of this” and downvote means “I want to see less of this.” But I guess I’m surprised if people actually behave that way? And that some posts are controversial enough to receive active downvotes vs passive ignoring.
Keep: Move checked items to bottom
by jefftk — 2020-06-11T00:30:02.566Z

I've been using Keep as my main note-taking app for years, and our
house recently switched after Wunderlist was

discontinued.
It turns out they added a list feature, where checked items are
sorted below unchecked ones:

This is on by default for new users, but I've been using Keep long
enough (disclosure: I work for
Google) that I seem to have been grandfathered into the old default:

I didn't know about this new display option, and would grumble to
myself about how my housemates wouldn't delete things from the shared list
when they were done with them, and how we were building up so much
clutter. When I asked them, though, I learned that it wasn't a
problem for them because they were all on the new layout.

Turns out there's a setting:

I'm much happier with it now, and having old items stick around at the
bottom feels useful (what do we usually get...) instead of feeling
like clutter.
Self-Predicting Markets
by  — 2020-06-10T22:39:25.933Z

[Confident but not novel. Cross-posted from Grand, Unified, Crazy.]

The story of Hertz has fundamentally changed how I view the stock market. This isn’t a novel revelation – now that I understand it, I’ve seen the core insight mentioned elsewhere – but it took a concrete example to really drive the point home.

The short version of the Hertz story is that the company went bankrupt. They have nearly $20 billion in debt, and as far as anybody can tell, no path to recovery; they’re bankrupt because their business model has been losing money for years despite several attempts to turn things around. The twist? Their stock is still trading, and at time of writing they have a market cap of $900 million.

I notice I am confused.

On any intuitive understanding of the market this shouldn’t be possible. The company is literally worthless. Or really, worse – it’s less than worthless given its debt load. People are paying positive money to own negative money. On a naive view this is another nail in the coffin of the Efficient Market Hypothesis.

After noticing that I was confused, I tried to generate hypotheses to explain this story:

Maybe the EMH really is wrong and the markets are nonsense.
Maybe bankruptcy laws are so complex and tangled that the expected value of the company really is positive after all is said and done.
Maybe the markets expect Hertz to get a government bailout for some reason.

Some of these are plausible (in particular the second), but none of them were particularly satisfying, so I tried asking myself why I, in a hypothetical world, would buy Hertz stock in this situation. I gave myself the standard answer: because I expected the stock go up in value in the future. Then I realized that this answer has nothing to do with the value of the company.

I had been making the mistake of viewing the stock market as a predictor of company value over the short-to-medium term, but this isn’t true. The stock market is a predictor of itself over the short-to-medium term. If people think the stock will go up tomorrow, then the stock will go up today – it doesn’t matter what the value of the company does at all. The company can be literally worthless, and as the Hertz story proves, people will still buy as long as they think the stock will go up tomorrow.

Now in practice, there are a bunch of traders i...
Can Covid-19 spread by surface transmission?
by Raemon — 2020-06-10T22:09:24.983Z

I've seen a lot of claims lately that coronavirus doesn't spread on surfaces, and therefore we can stop worrying about packages (and probably takeout food, even uncooked?).

My understanding of the claim is something like "we just haven't actually seen any evidence of surface transmission, despite the entire world paying attention, therefore it probably doesn't happen."

It's a bit trickier to prove a negative, but can anyone who either strongly believes that covid can't spread via surfaces, or who believes that it can (at least sometimes), write up the reasons for their beliefs?
[AN #103]: ARCHES: an agenda for existential safety, and combining natural language with deep RL
by Rohin Shah — 2020-06-10T17:20:02.171Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

HIGHLIGHTS

AI Research Considerations for Human Existential Safety (Andrew Critch et al) (summarized by Rohin): This research agenda out of CHAI directly attacks the problem longtermists care about: how to prevent AI-related existential catastrophe. This is distinctly different from the notion of being "provably beneficial": a key challenge for provable beneficence is defining what we even mean by "beneficial". In contrast, there are avenues for preventing AI-caused human extinction that do not require an understanding of "beneficial": most trivially, we could coordinate to never build AI systems that could cause human extinction.

Since the focus is on the impact of the AI system, the authors need a new phrase for this kind of AI system. They define a prepotent AI system to be one that cannot be controlled by humanity and has the potential to transform the world in a way that is at least as impactful as humanity as a whole. Such an AI system need not be superintelligent, or even an AGI; it may have powerful capabilities in a narrow domain such as technological autonomy, replication speed, or social acumen that enable prepotence.

By definition, a prepotent AI system is capable of transforming the world drastically. However, there are a lot of conditions that are necessary for continued human existence, and most transformations of the world will not preserve these conditions. (For example, consider the temperature of the Earth or the composition of the atmosphere.) As a result, human extinction is the default outcome from deploying a prepotent AI system, and can only be prevented if the system is designed to preserve human existence with very high precision relative to the significance of its actions. They define a misaligned prepotent AI system (MPAI) as one whose deployment leads to human extinction, and so the main objective is to avert the deployment of MPAI.

The authors break down the risk of deployment of MPAI into five subcategories, depending on the beliefs, actions and goals of the developers. The AI developers could fail t...
Maximizing impact at startup with high EA potential?
by knite — 2020-06-10T06:25:00.521Z

An early stage startup (SF Bay) has brought me on in a position of technical leadership. The product has the potential to save high-billions of human hours, and eliminate low-billions of tons of carbon emissions. Given that all startups are wildly optimistic, moving over two zeroes still lands at ~X00m hours and ~X0m tons of carbon saved.

It took me three weeks to realize I should run the numbers. After doing that, it took me three minutes to realize I've suddenly found myself in a potentially high impact and high leverage role. And that is precisely why I'm writing this post.

Beyond doing my job to the best of my abilities, what the heck else should I do? Are there EA/LW/rationalist resources and connections I should be contacting? Leadership coaches? Should I be advertising our job openings here? Talking to 80K hours? Playing NDA chicken and sharing more details about what the company is actually doing?

Advice greatly appreciated!
What past highly-upvoted posts are overrated today?
by Thomas Kwa — 2020-06-09T21:25:56.152Z

I've noticed the quality of LW posts varies greatly, even among top posts. Some of this is subjective or comes from what I want out of LW, but I suspect some posts are not nearly as useful or fun to readers in 2020 as their karma makes them appear.

Answers should be links to particular posts with more than ~75 karma that the answerer has a relatively unknown strong criticism of, or a general pattern (e.g. posts from LW 1.0 often have a higher score than they deserve because [reason]). Please give a reason why, and don't automatically downvote a post someone else links.
BBE W1: HMCM and Notetaking Systems
by Spiracular — 2020-06-09T19:20:31.975Z

HMCM and Notetaking Systems

BBE W1C1

(Build a Better Exobrain Week 1, Commentary 1)

Back in the old days, when the internet was bad for notetaking, some obsessive notetaker named Lion Kimbro wrote a Document.

A long, long time after that, the document came up on some LessWrong thread I was reading.*

This week, I decided to flip through it.

HMCMETYT

HMCMETYT is an abbreviation of How to Make a Complete Map of Every Thought You Think.

The title definitely wouldn't have gotten past a publishing editor. For sanity's sake, I'm going to shorten the abbreviation to HMCM henceforth.

I'm going to give an overall review of it here. If you don't care about that, skip ahead to Categorizing Notetaking Systems.

Reviewy Review

Its title makes a big claim, and that claim doesn't quite match with what I got out of reading it. That being said, I still think I got a lot out of it.

His abstractions were often fun to engage with, and I'm currently using a modified version of his tagging system (tags will probably be my next post). Try as I might, though, I never quite felt like I "grokked" the benefits of his mindmapping practice.

HMCM is very stream-of-consciousness at times, and describes a system of physical notetaking with laborious indexing. The abstractions aged pretty well, the physical instructions did not.

I would say that overall, I found HMCM to be an interesting book.** However, I think I was an exceptionally good audience for it, and I'm not sure that I'd recommend it to anyone who isn't notetaking-obsessed.

Glad That's Over

Turns out I don't like writing reviews! With the "review" out of the way, I will henceforth be jumping around and expanding on topics in a very non-linear fashion, according to my own whims and interests.

Lets talk about...

Categorizing Notetaking Systems

I am astonished that there isn't a field of study of notebooks. I have searched on the net, and while I have found a page here and there on some type of notebook method, it is almost ALWAYS one of the following two things:

The Diary: A bunch of entrees, chronologically based, maybe with a TOC, in which a person keeps a record of their thoughts. AKA "The Journal".

The Category Bins: A bunch of notes, stuffed into category bins, maybe 2 or 3 levels deep.

That's IT. In all the world, people have only been putting their notes in the above two ways.

-- Lion Kimbro, Introduction***

To the world's cred

...
BBE W1: Personal Notetaking Desiderata Walkthrough
by Spiracular — 2020-06-09T19:19:05.860Z

Personal Notetaking Desiderata Walkthrough

(Build a Better Exobrain Week 1, Project 1)

A key step I took in becoming a better notetaker was listing out what I was looking for. That's what I'm going to try to walk readers through today. So take out your current favorite bullet-point software, and let's get started.

Brief Explanation of Desiderata

For people who haven't run into the term before, desiderata (singular desideratum) translates pretty literally to "something desired." It usually comes up in a mathematical context*, where desiderata are typically presented as a list of propositions feeding into a search for the system that best satisfies them.

A good solution or system would keep as many of these propositions true as it can.

(Possibly with different weightings; the truth of some propositions may be more important than others.)

As a simplified illustrative example, take this SMBC comic on theodicy. A theologian is searching for a system that satisfies 3 propositions regarding God:

God is Omnipotent

God is Omniscient

God is Omnibenevolent

These propositions would be his desiderata.

The recognition that you might not find a single system that completely satisfies all desiderata (see again, the SMBC comic) is inherent in the framing. It's part of why you might call something "desiderata" instead of "requirements."

Prompts

Here are some optional prompts, as a starting point.

What Situations are you Trying to Solve?

If it's mostly for tracking due-dates, maybe pick a calendar.

If it's memorization, consider a flashcard system.

If it's for taking sequential notes on a lot of reading, consider an annotator.

Would you benefit from being able to reference other notes in a non-hierarchical fashion? Or is a strict tree more than good enough? Are you going to sort by topical tags? Do you need a search bar for words?

etc.

What Type of Note Taking?

Software

If this question stumps you, you might want to check out the partner post's description of categories of notetaking software.

Some categories:

File System

Outliner

Timeline

Gantt Chart (& other multi-threaded)

Calendar

Tags

MindMap

Wiki

Personal wiki

Flashcards

Annotator

Level of standardization?

Templates?

Built into the software, or personalized?

What specialized features would you like?

What formats does it need to accept?

Length of Input?

Physical

If you're open to it being purely offline, you migh

...
Build a Better Exobrain: Week 0 Overview
by Spiracular — 2020-06-09T19:09:14.177Z

BBE Overview

One Sentence Idea

The spirit of Hammertime, applied to the subject of Obsessive Notetaking.

(On further consideration, Gwern catches the intention I'm going for better than this does.*)

Quick Disclaimer

This is not an Intro to Notetaking course. Perhaps I will write one someday, but this is not it.

This is the write-along of one notetaking superuser, trying to work out how to be an even better notetaking superuser.

This is aired in public for a few reasons. First, it is part of the nature of writing to want some of it to be read. Perhaps it will be of use to other notetaking superusers. Also, I wanted to summon the blunt force of visible accountability.

It is my dream that this will come to resemble -in equal parts- a love-letter to notetaking, and a grueling practice-regimen.

With that said, let's see how I do.

Core Goal

My core goal is to make at least 4 reference-docs over this next month that I pull up at least 100x afterwards. Ideally, more like 1000x.

(For it to count, it must be brought up for the purpose of reading it, not writing to it. Constant updating is a cost, not a benefit.)

This seems achievable to me. I already have a few notes that probably pass this bar. My cheatsheet for BASH is top among them, but my personal list of Hamming Questions Variants gets pulled up a lot as well.

Secondary Goals

There are many secondary goals that would go well with this. Here are just a few of them.

Post on LessWrong 1/week

Could be things from the notes

Could be meta about the notetaking process

Could be double-meta about the notetaking learning process

There is No Triple-Meta

Learn things I expect my future self would endorse

Train to Retain, not Learn to Lose

Long Description: Make lasting seeds that my best future-self would endorsedly chose to repopulate her brainspace with, via looking them back up.

Keep a reference system that pays dividends

Have a laconic lookup for my Hammers Nails and Models

Move the best of my old notes to a good lookup system

Better Structuring of Thought

Better Knowledge of the Structuring of my Thought

(Complete list in personal notes)

Materials

My core notetaking software is the personal wiki software Tiddlywiki, with a side of StackEdit for quick uncategorized thoughts.

I have been extremely happy with Tiddlywiki. I don't expect this to change. I might go over why some other time; it would likely be an entir

...
Public Static: What is Abstraction?
by johnswentworth — 2020-06-09T18:36:49.838Z

Author’s Note: Most of the posts in

this sequence

are essentially a log of work-in-progress. This post is intended as a more presentable (“public”) and higher-confidence (“static”) write-up of some formalizations of abstraction. Much of the material has appeared in other posts; the first two sections in particular are drawn almost verbatim from the opening “What is Abstraction?” post.

Let's start with a few examples (borrowed from

here

) to illustrate what we're talking about:

We have a gas consisting of some huge number of particles. We throw away information about the particles themselves, instead keeping just a few summary statistics: average energy, number of particles, etc. We can then make highly precise predictions about things like e.g. pressure just based on the reduced information we've kept, without having to think about each individual particle. That reduced information is the "abstract layer" - the gas and its properties.
We have a bunch of transistors and wires on a chip. We arrange them to perform some logical operation, like maybe a NAND gate. Then, we throw away information about the underlying details, and just treat it as an abstract logical NAND gate. Using just the abstract layer, we can make predictions about what outputs will result from what inputs. Note that there’s some fuzziness - 0.01 V and 0.02 V are both treated as logical zero, and in rare cases there will be enough noise in the wires to get an incorrect output.
I tell my friend that I'm going to play tennis. I have ignored a huge amount of information about the details of the activity - where, when, what racket, what ball, with whom, all the distributions of every microscopic particle involved - yet my friend can still make some reliable predictions based on the abstract information I've provided.
When we abstract formulas like "1+1=2*1" and "2+2=2*2" into "n+n=2*n", we're obviously throwing out information about the value of n, while still making whatever predictions we can given the information we kept. This is what abstraction is all about in math and programming: throw out as much information as you can, while still maintaining the core "prediction" - i.e. the theorem or algorithm.
I have a street map of New York City. The map throws out lots of info about the physical streets: street width, potholes, power lines and water mains, building facades, signs and stoplights, etc. But f

...
More on disambiguating "discontinuity"
by Aryeh Englander — 2020-06-09T15:16:34.432Z

There have already been numerous posts and discussions related to disambiguating the term "discontinuity". Here is my attempt.

For the purposes of the following discussion I’m going to distinguish between (a) continuous vs. discontinuous progress in AI research, where discontinuity refers specifically to a sharp jump or change in the AI research progress curve relative to the previous curve; (b) slow vs. fast rate of progress, referring to the steepness of the progress curve slope, regardless of whether or not it’s discontinuous; and (c) long vs. short clock time – i.e., whether progress takes a long or short time relative to absolute time and not relative to previous trend lines. What exactly counts as discontinuous / fast / short will depend on what purpose we are using them for, as below.

There seem to be three or four primary AI-risk-related issues that depend on whether or not there will be a discontinuity / fast takeoff speed:

Will we see AGI (or CAIS or TAI or whatever you want to call it) coming far enough ahead of time such that we will be able to respond appropriately at that point? This question in turn breaks down into two sub-questions: (a) Will we see AGI coming before it arrives? (I.e., will there be a “fire alarm for AGI” as Eliezer calls it.) (b) If we do see it coming, will we have enough time to react before it’s too late?
Will the feedback loops during the development of AGI be long enough that we will be able to correct course as we go?
Is it likely that one company / government / other entity could gain enough first-mover advantage such that it will not be controllable or stoppable by other entities?

Let’s deal with each of these individually:

Question 1/a: Will we see AGI coming before it arrives? This seems to depend on all three types of discontinuity:

If there’s discontinuous progress relative to the previous curve, then presumably that jump will act as a fire alarm (although it might be too late to do anything about it by then).
If there’s continuous but sufficiently fast rate of progress and/or a sufficiently short clock time in the lead up to AGI, then that might act as a fire alarm in the sense that people will see the world start going crazy due to sufficiently advanced AI and that will be a wake-up call.
If progress is continuous AND sufficiently slow AND takes a s

...
Are there good ways to find expert reviews of popular science books?
by Kaj_Sotala — 2020-06-09T14:54:23.102Z

As the title says. Non-expert reviews only evaluate how good the writing was, or how impressed the reviewer was by the ideas; but I typically want something that evaluates the actual scientific claims.

My one trick has been to look at Google Scholar for papers that cite the book and search for "review" within those, which sometimes finds reviews published at academic journals, but this doesn't seem very effective overall.
A Practical Guide to Conflict Resolution: Communication
by  — 2020-06-09T01:04:50.332Z

[As described in the introduction, this post is about the different ways we communicate, the impact that can have on conflict, and how to choose the best communication tool for the job.]

If you’re anything like me, you crave specific advice for conflict resolution. Say this. Say it in this way. Don’t say that. Don’t use words that are longer than 10 letters when translated into Brazilian Portuguese. That sort of thing. But instead of focusing on what or what not to say, the most useful specific advice I can give is actually to focus on where you communicate.

Marshall McLuhan coined the famous phrase “the medium is the message”, and oh boy was he ever right. Every medium has different characteristics which impact how we communicate, and how conflict will spread or resolve. Here are just a few of the characteristics that matter:

Speed of communication, aka bandwidth. Most people can speak much faster than they can type.
Speed of response, aka latency. This can be anywhere from snail mail, which takes days per message, to instant messaging which is usually real-time.
Ability to absorb cross-talk. Laggy video-conferencing is particularly bad at this.
Audience size. Compare an in-person conversation to an email list with hundreds of subscribers.
Participant size. Are those hundred subscribers just reading, or can they add their own opinions to the mix?
Available side-channels. In-person communication gives you a whole bunch of important extra communication channels like tone of voice, facial expression and posture.
Norms. Most media are bound to specific codes of behaviour, either explicitly or implicitly.

With all of these variables, it’s no surprise that picking the right venue for your conflict is hugely valuable. Of course, you often don’t have a choice of where the conflict starts; they just do. But you always have the opportunity to move it, and it’s usually pretty easy when everyone involved is operating in good faith. Just go “hey, this would be easier to talk about in-person, do you mind if I swing by your desk?” and you’d be amazed at how easy it gets. Changing to a better venue is often both the easiest and the most effective thing you can do to resolve a conflict.

That said, with so many possible characteristics to consider it can be pretty daunting to figure out which one to suggest. F...