trentmkelly/LessWrong-43k huggingface dataset:
An agile approach to pre-research
by DirectedEvolution — 2020-06-25T18:29:47.645Z

My composition teacher in college told me that in some pottery schools, the teacher holds up your pot, examines it, comments on it, and then smashes it on the floor. They do this for your first 100 pots.

In that spirit, this post's epistemic status is SMASH THIS POT.

When I'm doing research, I often start with a vague question based on an unsettled emotion, such as excitement, confusion, frustration, or need. Lately, I've been investigating visual imagery. I start off with a question that's as hazy as "should I try and train my visual imagination so that I can be a better scientist?"

All too often, when I leap into searching Google Scholar with just a hazy question in mind, I get lost in a rabbit hole. My mind substitutes alternative metrics of success. How many papers have I read, and to what depth? What do they say, in general? How many papers can I find? How should I group them? Where to start research? Elizabeth think's it's with a question, not a book. Letting my chance findings on Google Scholar define my research agenda seems wrong, an example of lost purposes.

After taking detailed in-the-moment notes on how I investigated this question, I extricated some structure for my agile research process.

During this process, I'm glancing at titles, looking for perhaps 1-2 minutes at an abstract, or spending maybe 5 minutes looking at the text of an article.

1) Note the hazy idea that sparks your curiosity. Eg. "should I try and train my visual imagination?"

2) Try to decompose that into some sub-questions, which might require a paragraph or so to specify. Here are some examples:

I know we have different visual faculties for processing surface details vs. spatial relationships. People may have more developed skills or stronger innate potential for one or the other. Do most people tend to be average in both, or is it common to be much stronger in one than the other? Does one or the other correspond to feeling like you have a more vivid imagination?
Can visual imagination be improved with practice? If so, how?
What's more helpful to a scientist/mathematician/engineer - a strong visual imagination, or skill in using visualization tools such as drawings, physical models, and software?
What's more helpful to a scientist - a strong internalized knowledge of various skills, or knowing how to navigate a wide range of reference texts? I imagine investigating this question by te

...
A Personal (Interim) COVID-19 Postmortem
by Davidmanheim — 2020-06-25T18:10:40.885Z

I think it's important to clearly and publicly admit when we were wrong. It's even better to diagnose why, and take steps to prevent doing so again. COVID-19 is far from over, but given my early stance on a number of questions regarding COVID-19, this is my attempt at a public personal review to see where I was wrong.

I have been pushing for better forecasting and preparation for pandemics for years, but I wasn't forecasting on the various specific questions about Pandemics on most platforms until at least mid-March, and I failed in several ways.

Mea Culpa

I was late to update about a number of things, and simply wrong in some cases even on the basis of known information. The failures include initially being slow to recognize the extent of the threat, starting out dismissive about masks, being more concerned about hospital-based transmission than ended up being justified, being overconfident in the response of the US government, and in early March, over-confidently getting a key fact wrong about transmission being at least largely via aerosol droplet versus physical contact. I have a number of excuses, of course. Most other experts agreed with my views, my grandfather passed away in January, followed by his wife in early March, I was under a lot of stress, I was very busy with my personal life, I was trying to do a number of other high-priority projects, I was not paying attention to the details, and so on. But predictive accuracy doesn't care about WHY you were wrong, especially since there are always such excuses. And the impact of my poor judgement was also likely misleading to others in the community.

At the same time, I feel the perhaps egotistical need to note where I was correct early, and what I got right - followed by a clearer description of my failures. I started saying there would be PPE shortages due to COVID-19 by January, and was writing about the supply chain issues well before COVID. I submitted this paper November last year with Dave Denkenberger, which was largely finished last summer, and it was accepted in February, which then took 3 months to get published. The delay was in part due to other demands on my time, but in retrospect, if it had been available 3 months earlier, it would have been far, far more impactful.

I also understood the failure mode we ended up seeing, and in my 2018 paper, discussing overconfidence in c...
There's an Awesome AI Ethics List and it's a little thin
by AABoyles — 2020-06-25T13:43:20.858Z
Link: https://github.com/marikyu7/awesome-artificial-intelligence-ethics

I recently stumbled upon the Awesome Artificial Intelligence Ethics list, and was somewhat underwhelmed with its quality. While it has enough links to look like a good and useful Awesome List, I think I'd come away from it mostly confused and unsure where to go next. Some obvious oversights include:

The books section is quite short, given the amount of material that's been published in this space since SuperIntelligence.

LessWrong is in the Blogs section, but the Effective Altruism and AI Alignment Fora are not.

There are a ton of great introductory-to-in-the-weeds podcasts on AI Ethics topics, but there's no podcasts section (or other section in which podcasts would clearly be appropriate).

There are number of technical research agenda that would be to list on the off chance anyone happens upon this list and wants to work their way up to the actual state of research.

I'm going to make some pull requests, but I can only suggest what I know about. Perhaps you can help?
DontDoxScottAlexander.com - A Petition
by Ben Pace — 2020-06-25T05:44:50.050Z
Link: https://DontDoxScottAlexander.com

Here's the petition Scott asked us to make.

This is a request for a specific action by the New York Times editors:

We, the undersigned, urge the New York Times to respect Scott Alexander's request to not reveal his real name in a planned piece discussing the Slate Star Codex blog and community.

[...]

That's all. It seems to me really important for public discourse on the internet for journalists to respect this norm in this situation.

Please share it in the places you share things, and email it to the prominent people who you know that the New York Times respects and care about.

Please sign.

Acknowledgements

Thanks to Rob Bensinger for making the petition with me.

Thanks to Paul Graham, Steven Pinker and many others for their early signatures.

Thanks to Sarah Haider and Tanner Greer for independently organising a petition and then joining forces with ours.

Thanks to so many other people who are still unsubscribing from the NYT, giving them respectful-but-firm feedback, and otherwise supporting Scott in this situation. It's been great to see so much love and support for SSC these past 48 hours.
Preview On Hover
by jefftk — 2020-06-24T22:20:02.980Z

A couple years ago, Wikipedia

added
a feature where if you hover over an internal link you'll see a
preview of the target page:

Other sites with similar features include gwern.net:

And LessWrong:

In general, I like these features a lot. They dramatically lower the
barrier the following internal links, letting you quickly figure out
whether you're interested. On the other hand, they do get in the
way. They pop up, overlapping the text you're reading, and mean you
need to be paying more attention to where the mouse goes.

I decided I wanted to add a feature like this to my website, but
without any overlap. The right margin seemed good, and if you're
reading this on jefftk.com with a window at least 1000px wide then
hovering over any link from one of my blog posts to another should
show a preview:

Here are my five most recent posts if you'd like to give it a try:

Coronavirus and Rents

Second Wave?

Keep: Move checked
items to bottom

Growing Independence

Pantry Staples for DIY

There are a lot of options for how to implement this, but I decided to
use an iframe that just loads the relevant page. Feel free to look at
the page source and see exactly how it works, but the general idea is:

It loads the page directly, not a stripped down version like
the examples above. My
pages are simple enough that this is fine.

It's a preview, not a full page, so set
scrolling=no.

It's a same-origin ("friendly") iframe, so I can reach into it and
add a click listener, so clicking on the iframe takes you
through to the previewed post.

I don't want comments, ads, analytics, or anything else
potentially slow to
run, and I don't use JS for rendering, so I use sandbox
to turn off JS.

Once it's open it stays open until you hover a different preview.

It appears vertically aligned with the hovered link, and moves
with the page out of view as you scroll down.

If you hover over a second link close to the first one, it
reuses the same vertical position to avoid looking jumpy.

If you hover many links in quick succession it starts loading
the first one immediately, and then discards any links that have been
overtaken by events.

I'm pretty happy with it, but if you find any bugs let me know!

Comment via: facebook
Don't punish yourself for bad luck
by Sherrinford — 2020-06-24T21:52:37.045Z

The following text first summarizes the standard moral-hazard model. Afterwards, I point out that it implies that you always get punished for bad luck. The third part is completely speculative: I speculate on how you should behave towards yourself.
A brief summary of a moral-hazard setting
A Moral Hazard situation occurs when someone takes too much risk, or does not reduce it enough because someone else bears the cost.
The following situation is a typical textbook example. A worker works for a firm, and her effort influences the probability that the firm has high revenue. The worker can exert high or low effort, the firm's revenue can be high or low, and low revenue is more likely when effort is low, but can also occur when effort is high. Moreover, the worker has to get a wage that compensates her for forgoing whatever else she would do with her time.
Suppose the firm would, in principle, be willing to compensate the worker for high effort (which means that we assume that the additional expected revenue gained from high effort ist at least as high as the additional wage needed to make the worker willing to exert high effort). Because workers are usually assumed to be risk-averse, the firm would take the risk of low revenue and the worker gets a wage that is constant in all states of the world.
However, now also suppose the firm cannot directly observe the effort - this constitutes a situation of asymmetric information, because the worker can observe her own effort and the firm cannot. Then the firm cannot condition the payment on the worker's effort. It also cannot just conclude that the worker exerted low effort by observing low revenue, because we assumed that low revenue can also occur when the worker exerted high effort.
The second-best optimal solution (that is, the best solution given this information problem) is to condition payments on the revenue - and thus, on the result instead of the effort to get it. The worker gets a higher wage when the firm has high revenue. Thereby, the firm can design the contract such that the worker will choose to exert high effort.
In this setting of asymmetric information, the worker gets the same expected utility as in the setting with symmetric information (in which effort could be observed), because the firm still has to compensate her for not doing something else. But because the worker now faces an uncertain in...
Betting with Mandatory Post-Mortem
by abramdemski — 2020-06-24T20:04:34.177Z

Betting money is a useful way to

ensure you have some skin in the game when making assertions;

get a painful reminder of when you're wrong, so that you'll update;

make money off of people, if you're right.

However, I recently made a bet with both a monetary component and the stipulation that the loser write at least 500 words to a group chat about why they were wrong. I like this idea because:

It enforces that some cognitive labor be devoted to the update, rather than relying on the pain of lost cash. Even if you do think it through privately, the work of writing it up will help you remember the mistake next time. (If you don't want to spend that amount of time thinking about why you were wrong, then perhaps you aren't very interested in really updating on this bet.)

People usually make small-cash bets anyway, so there's not that much skin in the game. Being forced to write publically, or to a select group of peers such as a slack/discord server, makes it feel real for me in a way that losing a small sum of money doesn't.

Where normal bets may benefit the participants, these kinds of public bets have more benefit for the whole audience. Observers get a lot more information about the structure of the disagreement, and the update which the loser takes from it.

Often, by the time a bet is decided, a lot of other relevant information has come in as well. A public post-mortem gives the loser a chance to convey this information.

This kind of bet will often be positive-sum in reputational terms: the winner gets a public endorsement from the loser, but the loser may gain respect from the audience for their gracious defeat and judicious update.

Furthermore, if the loser's write-up is anything short of honest praise for the winner's views, the write-up may provide hints at a continuing disagreement between the loser and winner which can lead to another bet.

This idea feels similar to Ben's Share Models, Not Beliefs. Bets focus only on disagreement with probabilities, not the underlying reasons for those disagreements. Declaring a winner/loser conveys binary information about who was more correct, but this is very little information. Post-mortems give a place for the models to be brought to light.

A group of people who engaged in betting-with-post-mortems together would generally be getting a lot more feedback on practical reasoning and where it can go wrong.
Quick Look #1 Diophantus of Alexandria
by SebastianG  — 2020-06-24T19:12:14.672Z

https://www.storyofmathematics.com/hellenistic_diophantus.html

Diophantus of Alexandria, a 2nd Century Greek mathematician, had a lot of the concepts needed to develop an Algebra. However, he was unable to fully generalize his methods of problem solving, even if he invented some interesting methods.

Ancient math was written in paragraphs, using words for the most part, thus making reading it very, very painful compared to the compact elegance of modern mathematical notation. However, I was surprised to see Diophantus (or his very early editors at least) develop some interesting and helpful notation in his algebra.

Final sigma ‘ς’ represented the unknown variable, but there were different symbols for variables of every power so for x^2… x^6 each had a unique variable. In fact, this situation persisted into the 17th century, even Fermat used N for unknown and S for the unknown-squared and C for the unknown cubed!

The problem with this is that it meant Diophantus couldn’t devise general methods to solve algebraic problems which had multiple unknowns, and it wasn’t obvious to him that one CAN frequently relate x^2 to x.

The cool thing about this notation from the past though, is how it makes obvious something that Algebra I – Algebra II students mess up frequently. You can’t just combine x^2 + x^3, these are different variables whose relation concerns the base. And almost everyone has made this mistake in their early math career. Some never recover.

Although the editor of my copy, Sir Thomas L. Heath, claims that Diophantus experienced limited options as a mathematician because all the letters of the Greek alphabet were in use as letters except for the final sigma, which Diophantus used to represent the unknown variable, I think D could have invented more variables quite easily. We see this in his invention of the subtraction sign as an inverted psi, and his use of a different variable with superscript for an unknown to the nth power up to the sixth. There was also the extinct digamma and all the Egyptian symbols which at least could have cribbed off of. Surely, the problem was not a lack of imagination, but merely satisfaction with the method then in use. Besides, one person can only invent so much, unless that person is Leonard Euler or Von Neumann, neither of whom had any limits. D. merely didn't see the limits of his notation.

Although D’s problems are surprisingly...
What's the name for that plausible deniability thing?
by  — 2020-06-24T18:42:54.705Z

There's a concept I remember reading about here, of the idea that you can't just suddenly refuse to answer dangerous questions. You have to consistently refuse to answer some random sample of totally normal questions, so that "refusing to answer a question" doesn't itself become a source of information.

Unfortunately I can't remember what it's usually called, and haven't been able to turn it up via search ("plausible deniability" is way too broad, and I can't find the right narrowing criteria). What is this concept/process called?
Can I archive content from lesswrong.com on the wayback machine (internet archive, archive.org) ?
by nick lacombe — 2020-06-24T18:15:21.866Z

There are some great information on lesswrong.com (LW) that seems to be available publicly (I can access it in an incognito chrome window) and I would like to increase the chances of this information surviving for a long time.

When I try saving a LW page it looks like it does not render correctly on the wayback machine. Ex: https://web.archive.org/web/20200624170623/https://www.lesswrong.com/s/FrqfoG3LJeCZs96Ym/p/8qccXytpkEhEAkjjM

I opened a github issue on LW's repo since I assume it is an issue with the source code of LW.
The EA forum seems to have the same issue and it looks like the EA forum's repo is a fork of lesswrong's repo. I am also writing here since it might have more visibility for non tech people.
Abstraction, Evolution and Gears
by johnswentworth — 2020-06-24T17:39:42.563Z

Meta: this project is wrapping up for now. This is the second of probably several posts dumping my thought-state as of this week.

It is an empirical fact that we can predict the day-to-day behavior of the world around us - positions of trees or buildings, trajectories of birds or cars, color of the sky and ground, etc - without worrying about the details of plasmas roiling in any particular far-away star. We can predict the behavior of a dog without having to worry about positions of individual molecules in its cells. We can predict the behavior of reinforced concrete without having to check it under a microscope or account for the flaps of butterfly wings a thousand kilometers away.

Our universe abstracts well: it

decomposes into high-level objects whose internal details are approximately independent of far-away objects, given all of their high-level summary information

.

It didn’t have to be this way. We could imagine a universe which looks like a cryptographic hash function, where most bits are tightly entangled with most other bits and any prediction of anything requires near-perfect knowledge of the whole system state. But empirically, our universe does not look like that.

Given that we live in a universe amenable to abstraction, what sorts of agents should we expect to evolve? What can we say about agency structure and behavior in such a universe? This post comes at the question from a few different angles, looking at different properties I expect evolved agents to display in abstraction-friendly universes.

Convergent Instrumental Goals

The basic idea of abstraction is that any variable X is surrounded by lots of noisy unobserved variables, which mediate its interactions with the rest of the universe. Anything “far away” from X - i.e. anything outside of those noisy intermediates - can only “see” some abstract summary information f(X). Anything more than a few microns from a transistor on a CPU will only be sensitive to the transistor’s on/off state, not its exact voltage; the gravitational forces on far-apart stars depend only on their total mass, momentum and position, not on the roiling of plasmas.

One consequence: if an agent’s goals do not explicitly involve things close to X, then the agent cares only about controlling f(X). If an agent does not explicitly care about exact voltages on a CPU, then it will care only about controlling the binary states (and...
[AN #105]: The economic trajectory of humanity, and what we might mean by optimization
by Rohin Shah — 2020-06-24T17:30:02.977Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

HIGHLIGHTS

Modeling the Human Trajectory (David Roodman) (summarized by Nicholas): This post analyzes the human trajectory from 10,000 BCE to the present and considers its implications for the future. The metric used for this is Gross World Product (GWP), the sum total of goods and services produced in the world over the course of a year.

Looking at GWP over this long stretch leads to a few interesting conclusions. First, until 1800, most people lived near subsistence levels. This means that growth in GWP was primarily driven by growth in population. Since then population growth has slowed and GWP per capita has increased, leading to our vastly improved quality of life today. Second, an exponential function does not fit the data well at all. In an exponential function, the time for GWP to double would be constant. Instead, GWP seems to be doubling faster, which is better fit by a power law. However, the conclusion of extrapolating this relationship forward is extremely rapid economic growth, approaching infinite GWP as we near the year 2047.

Next, Roodman creates a stochastic model in order to analyze not just the modal prediction, but also get the full distribution over how likely particular outcomes are. By fitting this to only past data, he analyzes how surprising each period of GWP was. This finds that the industrial revolution and the period after it was above the 90th percentile of the model’s distribution, corresponding to surprisingly fast economic growth. Analogously, the past 30 years have seen anomalously lower growth, around the 25th percentile. This suggests that the model's stochasticity does not appropriately capture the real world -- while a good model can certainly be "surprised" by high or low growth during one period, it should probably not be consistently surprised in the same direction, as happens here.

In addition to looking at the data empirically, he provides a theoretical model for how this accelerating growth can occur by generalizing a standard economic model. Typically, the economic model assumes technology i...