trentmkelly/LessWrong-43k huggingface dataset:
How Pet Owners Can Help Wild Animals and The Environment
by ozymandias — 2018-06-01T21:57:36.466Z

Many people talk about wild-animal suffering as if it is completely intractable, or as if it is inherently opposed to environmental conservation, or as if it’s impossible to know what effects our actions could possibly have on wild animals. But, in fact, there is one simple action any cat or dog owner can do to help wildlife and the environment at the same time: keep your cat inside, and keep your dog inside unless you're supervising them. (My statistics here come from this review. See also Utility Farm’s excellent blog post on a similar topic.)

Both cats and dogs are predators, and their largest effect on wildlife is through predation. Most studies suggest cats primarily hunt mammals and birds, taxa which are generally considered to be moral patients. Cats kill between 1.3 and 4 billion birds annually in the United States and 100 and 300 million in Canada. In some areas, cats are the largest source of anthropogenic mortality for birds. Cats also kill between 6.3 and 22.3 billion mammals annually in the United States; they kill between 2.8 and 4.9 million native animals annually in Australia and a million per month in Finland. While the effects of dogs are less studied, dogs also hunt wildlife and no doubt kill millions or billions of animals a year.

Some people might wonder if this is part of the natural cycle of life: after all, predators are part of nature, and eliminating them may cause harm to ecosystems. Perhaps cats and dogs are preventing worse deaths through starvation. Others might wonder whether some other predator would eat the prey if the cats and dogs did not.

However, the evidence suggests that this is not the case. Cats have contributed to 14% of modern mammal, reptile, and bird extinctions. While dogs are understudied, they likely also contribute to extinctions. Since cats and dogs contribute to extinctions, one would expect the mortality rate to decrease significantly in the absence of cat and dog predation; the wildlife cats and dogs kill would have lived much longer otherwise. In addition, extinctions can make ecosystems less stable and result in an incalculable loss of ecosystem services.

Thinking just about deaths from predation underestimates the scope of the problem. It’s widely known that cats tend to play with prey, and while playing is fun for the cat, it’s horrifying for the prey. (Imagine how you would feel if a giant repeatedly...
Strong Votes [Update: Deployed]
by Raemon — 2018-06-01T21:03:23.457Z

Update: This has now been pushed to the live site. For now, on Desktop, strong upvotes require click-and-hold. Mobile users just tap multiple times. This is most likely a temporary solution as we get some feedback about how the respective modes work under realistic conditions.

Over on our dev-site, lessestwrong.com, we have a fairly major feature-branch: Recalibrated voting power, and Strong Upvotes/Downvotes.

tl;dr – Normal votes on the test server range in power from 1 to 3 (depending on your karma). You have the option of holding down the up/downvote button for a strong vote, which ranges in power from 1 to 15.

We're looking for feedback on the UI, and some implementation details.

Flow vs Stock

This post by Jameson Quinn notes that there's two common reasons to upvote or downvote things. This is similar to my own schema:

Conversational Flow – When you like (or dislike) the effect a comment has on a conversation, and you want to give the author a metaphorical smile of appreciation, or awkward silence/stare.

"Ah, good point" (+)
"Hmm, this gives me something to think about" (+)
"This comment cited sources, which is rare. I want to reward that." (+)
"This was clever/funny." (+)
"I think this post contains an error." (–)
"This comment is technically fine but annoying to read" (–)
"I don't think the author is being very charitable here" (–)
Some combination of the above (upvote or downvote, depending)

Signifying Importance – When you think other people should go out of their way to read something (or, definitely should not). Ideally, posts and comments that contribute to the longterm stock of value that LessWrong is accumulating.

"I learned something new and useful" (++)
"The argumentation or thought process illustrated by this post helped me learn to think better." (++)
"This post contains many factual errors" (––)
"This comment is literal spam" (––)
"The reasoning here is deeply bad." (––)

People instinctively use upvoting to cover both Flow and Importance, and this often results in people upvoting things because they were a good thing to say in a conversation. But then later, if you want to find the most useful comments in a discussion, you end up sifting through a bunch not-actually-useful stuff.

People also often unreflectively upvote things they like, without paying much attention to whether the arguments are good, or whether it's good for the longterm health...
Amplification Discussion Notes
by William_S — 2018-06-01T19:03:35.294Z

Paul Christiano, Wei Dai, Andreas Stuhlmüller and I had an online chat discussion recently,

the transcript of the discussion is available here

. (Disclaimer that it’s a nonstandard format and we weren't optimizing for ease of understanding the transcript). This discussion was primarily focused on amplification of humans (not later amplification steps in IDA). Below are some highlights from the discussion, and I’ll include some questions that were raised that might merit further discussion in the comments.

Highlights

Strategies for sampling from a human distribution of solutions:

Paul: For example you can use "Use random human example," or "find an analogy to another example you know and use it to generate an example," or whatever.

There is some subtlety there, where you want to train the model that sample from the real human distribution rather than from the empirical distribution of 10 proposals you happen to have collected so far. If samples are cheap that's fine. Otherwise you may need to go further to "Given that [X1, X2, ...] are successful designs, what is a procedure that can produce additional successful designs?" or something like that. Not sure.

Dealing with unknown concepts

Andreas: Suppose you get a top-level command that contains words that H doesn't understand (or just doesn't look at), say something like "Gyre a farbled bleg.". You have access to some data source that is in principle enough to learn the meanings of those words. What might the first few levels of questions + answers look like?

Paul: possible questions: "What's the meaning of the command", which goes to "What's the meaning of word X" for the words X in the sentence, "What idiomatic constructions are involved in this sentence?", "What grammatical constructions are involved in the sentence"

Answers to those questions are big trees representing meanings, e.g. a list of properties of "gyre" (what properties the subject and object typically have, under what conditions it is said to have occurred, why someone might want you to do it, tons of stuff most of which will be irrelevant for the query)

Which come from looking up definitions, proposing definitions and seeing how well they match with usage in the cases you can look at, etc.

Limits on what amplification can accomplish

Pau

...
Into the Kiln: Insights from Tao's 'Analysis I'
by TurnTrout — 2018-06-01T18:16:32.616Z

Note: real analysis is not on the MIRI reading list (although I think it should be).

Foreword

As a young boy, mathematics captivated me.

In elementary school, I'd happily while away entire weekends working through the next grade's math book. I was impatient.

In middle school, I'd lazily estimate angles of incidence that would result if I shot lasers from my eyes, tracing their trajectories within the classroom and out down the hallway. I was restless.

In high school, I'd daydream about what would happen to integrals as I twisted functions in my mind. I was curious.

And now, I get to see how it's all put together. Imagine being fascinated by some thing, continually glimpsing beautiful new facets and sampling exotic flavors, yet being resigned to not truly pursuing this passion. After all, I chose to earn a computer science degree.

Wait.

Analysis I

As in Linear Algebra Done Right, I completed every single exercise in the book - this time, without looking up any solutions (although I did occasionally ask questions on Discord). Instead, I came back to problems if I couldn't solve them after half an hour of effort.

A sampling of my proofs can be found here.

1: Introduction

2: The Natural Numbers

In which the Peano axioms are introduced, allowing us to define addition and multiplication on the natural numbers {0,1,2,…}.

3: Set Theory

In which functions and Cartesian products are defined, among other concepts.

Recursive Nesting

How can you apply the axiom of foundation if sets are nested in each other? That is, how can the axiom of foundation "reach into" sets like A={B,…} and B={A,…}?

Show that if A and B are two sets, then either A∉B or B∉A (or both).

Proof. Suppose A∈B and B∈A. By the pairwise axiom, we know that there exists a set S={A,B}. We see that there does not exist an S′∈S such that S′∩S=∅. That is, if we choose A, one of its elements is B, which is also an element of S - this violates the axiom of foundation. The same reasoning applies if we choose B. Then ¬(A∈B∧B∈A), so either A or B (or both) is not an element of the other.

4: Integers and Rationals

In which the titular sets are constructed, allowing the exploration of absolute value, exponentiation, and the incompleteness (colloquially, the "gaps") of the rationals.

Readers newer to mathematics may find it interesting that even though there are (countably) infinitel... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Open Thread June 2018
by Elo — 2018-05-31T22:34:56.656Z

If it’s worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

Check if there is an active Open Thread before posting a new one (use search for Open Thread ).

Monthly open threads seem to get lost and maybe we should switch to fortnightly.

What accomplishments are you celebrating from the last month?

What are you reading?
Toolbox-thinking and Law-thinking
by Eliezer Yudkowsky — 2018-05-31T21:28:19.354Z

Tl;dr:

I've noticed a dichotomy between "thinking in toolboxes" and "thinking in laws".

The toolbox style of thinking says it's important to have a big bag of tools that you can adapt to context and circumstance; people who think very toolboxly tend to suspect that anyone who goes talking of a single optimal way is just ignorant of the uses of the other tools.

The lawful style of thinking, done correctly, distinguishes between descriptive truths, normative ideals, and prescriptive ideals. It may talk about certain paths being optimal, even if there's no executable-in-practice algorithm that yields the optimal path. It considers truths that are not tools.

Within nearly-Euclidean mazes, the triangle inequality - that the path AC is never spatially longer than the path ABC - is always true but only sometimes useful. The triangle inequality has the prescriptive implication that if you know that one path choice will travel ABC and one path will travel AC, and if the only pragmatic path-merit you care about is going the minimum spatial distance (rather than say avoiding stairs because somebody in the party is in a wheelchair), then you should pick the route AC. But the triangle inequality goes on governing Euclidean mazes whether or not you know which path is which, and whether or not you need to avoid stairs.

Toolbox thinkers may be extremely suspicious of this claim of universal lawfulness if it is explained less than perfectly, because it sounds to them like "Throw away all the other tools in your toolbox! All you need to know is Euclidean geometry, and you can always find the shortest path through any maze, which in turn is always the best path."

If you think that's an unrealistic depiction of a misunderstanding that would never happen in reality, keep reading.

Here's a recent conversation from Twitter which I'd consider a nearly perfect illustration of the toolbox-vs.-laws dichotomy:

David Chapman: "By rationalism, I mean any claim that there is an ultimate criterion according to which thinking and acting could be judged to be correct or optimal... Under this definition, 'rationalism' must go beyond 'systematic methods are often useful, hooray!'... A rationalism claims there is one weird trick to correct thinking, which guarantees an optimal result. (Some rationalisms specify

...
Monopoly: A Manifesto and Fact Post
by sarahconstantin — 2018-05-31T18:40:00.479Z

Epistemic Status: exploratory. I am REALLY not an economist, I don’t even play one on TV.

[Edit: After some discussion in the comments I've updated that that GDP-to-gold, or GDP-to-oil, are bad proxy measures for economic growth. Further thoughts on this in Oops on Commodity Prices]

You can call it by a lot of names. You can call it crony capitalism, the mixed economy, or corporatism. Cost disease is an aspect of the problem, as are rent-seeking, regulatory capture, and oligopoly.

If Scrooge McDuck’s downtown Duckburg apartment rises in price, and Scrooge’s net worth rises equally, but nothing else changes, the distribution of purchasing power is now more unequal — fewer people can afford that apartment. But nobody is richer in terms of actual material wealth, not even Scrooge. Scrooge is only “richer” on paper. The total material wealth of Duckburg hasn’t gone up at all.

I’m concerned that something very like this is happening to developed countries in real life. When many goods become more expensive without materially improving, the result is increased wealth inequality without increased material abundance.

The original robber barons (Raubritter) were medieval German landowners who charged illegal private tolls to anyone who crossed their stretch of the Rhine. Essentially, they profited by restricting access to goods, holding trade hostage, rather than producing anything. The claim is that people in developed countries today are getting sucked dry by this kind of artificial access-restriction behavior. A clear-cut example is closed-access academic journals, which many scientists have begun to boycott; the value in a journal is produced by the scholars who author, edit, and referee papers, while the online journal’s only contribution is its ability to restrict access to those papers.

Scott Alexander said it right:

LOOK, REALLY OUR MAIN PROBLEM IS THAT ALL THE MOST IMPORTANT THINGS COST TEN TIMES AS MUCH AS THEY USED TO FOR NO REASON, PLUS THEY SEEM TO BE GOING DOWN IN QUALITY, AND NOBODY KNOWS WHY, AND WE’RE MOSTLY JUST DESPERATELY FLAILING AROUND LOOKING FOR SOLUTIONS HERE.

Except that it’s pretty easy to see why. We have a lot of trolls sitting under bridges charging tolls to people who want to cross. Modern Raubritter can easily maintain a hard-to-refute image that they’re providing value, and so make it hard for anyone to coordinate to avoid them...
Announcement: Legacy karma imported
by habryka — 2018-05-31T02:53:01.779Z

Hey Everyone,

I finally got around to properly importing the old karma from LessWrong 1.0. Your vote-weight and the karma displayed on your profile should now reflect all the contributions you made to LessWrong over the years.

We might change this around again at some future point in time, but we just stuck with the same karma formula as the old LessWrong had, with old main posts being worth 10 karma per vote, and everything else being worth 1.

Sorry for this taking so long. Comment or ping us on Intercom if you notice anything karma-related being broken, or your karma not being properly imported.
Epistemological Weight Class
by musicmage4114 — 2018-05-30T23:43:40.519Z

On a certain level, most people have a general idea of who they should and should not debate, with regards to any given subject with which they are familiar. This sense is, essentially, an assessment of how hard it would be to win a debate with the given person on the given subject. This assessment can be made on two different levels: how hard it would be for me personally, or how hard it would be for any given person (for a more "absolute" assessment). The latter formulation, when approached with a certain level of objectivity, allows us to evaluate - what I will term as - a person's epistemological weight class. Asking how hard it would be for us, then, is simply comparing our own epistemological weight class to someone else's.

But how useful a concept is this, really? It's awfully vague, and basically impossible to reliably quantify. However, it is something that we seem to innately know about a person, so long as we have some amount of knowledge about their amount of knowledge (or at least a means of predicting that amount). Epistemological weight class, then, is a form of meta-knowledge.

Because epistemological weight class is purely a function of the amount of knowledge an individual has about a given subject, my initial presentation of the concept - "how hard it would be to win a debate" - is actually missing a crucial addendum: the assumption of all other factors being equal. These factors can otherwise distort our initial assessment of the difficulty of victory, and are often the reason someone is able to prevail in debate against someone with a comparatively higher weight class.

So what do we do with this meta-knowledge? Here, the metaphor is fairly apt: awareness of someone's weight class allows us to better assess whether we are "out of our league". For example, I would never be so foolish as to engage Patrick Stewart in debate about the most effective methods of learning to act, nor would I dream of engaging Amal Clooney on the subject of Middle East politics. On the flip side, I would be perfectly comfortable discussing the performance and theory of vocal music with anyone below the level of, say, Diana Damrau, and Judith Butler is perhaps the only person I'd avoid if I wanted to discuss the applications of queer theory to video games, though her presumed lack of familiarity with that particular medium might theoretic...
Logical Inductors Converge to Correlated Equilibria (Kinda)
by Diffractor — 2018-05-30T20:23:54.000Z

Logical inductors of "similar strength", playing against each other in a repeated game, will converge to correlated equilibria of the one-shot game, for the same reason that players that react to the past plays of their opponent converge to correlated equilibria. In fact, this proof is essentially just the proof from Calibrated Learning and Correlated Equilibrium by Forster (1997), adapted to a logical inductor setting.

However, there are two complications. The first complication is large differences in power between the logical inductors. To be specific, imagine a logical inductor playing matching pennies against another logical inductor of great strength that can simulate what the highest-valued action will be (though not the exploration steps). Also, the outcome {heads,heads}is assigned ϵ utility, while {tails,tails}is 0 utility. This is essentially a Death-in-Damascus scenario. The unique Nash equilibria is randomizing at approximately 50/50 odds, but in repeated plays of this game with density-zero exploration, the best move is to just play heads to get the ϵ utility, because the inductor can't fool its opponent except on days where it explores, which happen with asymptotic frequency 0. So sufficiently large differences in power between the inductors can lead to results that aren't Nash or correlated equilibria.

As such, in order to guarantee correlated equilibria, we will need to make a "no subjective dependencies" assumption. More precisely, in the limit, conditioning on your own move can't shift your probabilities of your opponent's move. The ϵ-matching pennies against a powerful predictor game fails this, because the expected utility of the agent (conditional on it playing "tails") must be ϵ (if it was less than ϵ, the utility would go up because of exploration steps where it fools the opponent, so the limiting behavior is playing "tails" with probability limiting to 0, and ϵ of those plays of "tails" are ones where it succesfully fools the opponent due to exploration). So, conditional on playing "tails", the probability of the opponent playing "tails" is about 1−ϵ. Conditional on playing "heads", the probability of the opponent playing "tails" is 0.

The second complication is that all Nash equilibria are correlated equilibria, so maybe logical inductors converge to Nash equilibria in most cases. In general, I don't expect this, because it's possible

...
Safety in Machine Learning
by Gordon Seidoh Worley — 2018-05-29T18:54:26.596Z

In this week's AI alignment newsletter was a link to a paper titled "Safe Policy Learning from Observations". The paper uses the word "safe" or "safety" to describe ML systems 34 times, but reading through I didn't get a very clear sense of what the authors mean by "safe" or why "safe" was an appropriate word to describe the property they care about. The main clues came in these lines of the introduction:

Safe policy learning is an ambiguous term [Garcıa and Fernández, 2015]. Safety can be an absolute measure, e.g. worst-case criterion [Tamar et al., 2013], or relative to a baseline policy [Ghavamzadeh et al., 2016].

From the first referenced paper, "A Comprehensive Survey on Safe Reinforcement Learning", it becomes clear that here "safety" is taken to mean the kind of safety people are interested in at factories, offices, on streets, and in the air: avoiding doing things that lead to injury or death. The concerns raised related to safety are all cases of what we around these parts call Goodharting, but generally in the context of AI being used in places where over-optimization on the wrong thing could lead to somebody getting hurt. There is not, however, any reference to concerns about existential risks.

This reminds me of something I often forget: many more people are working on AI safety in the context of prosaic safety than are working in the context of existential safety. Nonetheless these folks may do work that is of value to those concerned about existential risk, even if only incidentally. For myself their work seems not that relevant since I'm not directly engaged with machine learning, but I can imagine how for others it may be.

However for whatever work may be done towards existential AI safety by work on commonplace safety, I can't help by wonder how much it really helps. Yes, prosaic safety gives better respectability to AI safety as a whole so that concern about existential risks from AI seem less unusual, but how much does it occlude work on existential safety or make it appear that more is being done than really is? I have no real answers to this, and I'm sure others have considered the idea already in more detail (although I can only recall discussing it in person and so have nothing to link), but I was surprised enough to realize I live in a bubble of...