trentmkelly/LessWrong-43k huggingface dataset:
Signaling-based observations of (other) students
by Rafael Harth — 2018-05-27T18:12:07.066Z

(Note: in Germany, tutorials are exercise sessions, typically weekly and mandatory, which accompany a lecture. They are held in groups of 5-30 students and are lead by a more advanced student whom I call the instructor.)

There is an interesting pattern I noticed during math lectures and tutorials at my university. It sometimes occurs when a student has an objection to something the instructor wrote or said. If the objection is about something simple like a missing sign, it's usually handled quickly and without problems. But whenever the socially acceptable time to respond is shorter than the time needed to understand what the objection actually is, the instructor usually doesn't even attempt to understand it. Instead, he does a quick sort of pattern matching of what the objection roughly sounds like, makes a guess as to which thing the student most likely misunderstood, and then attempts to explain that particular thing again.

Quite often – probably over half the time – this guess is accurate on the first try, and the response makes sense. If it's not, often the student replies and the instructor figures out what the objection is in a quick back-and-forth, and gives her a sensible reply that's a bit delayed.

But where it gets most interesting is if the student's grasp on the current problem is actually better than that of the instructor, and the objection she raised is correct. What happens then is that the instructor gives a bunch of explanations of stuff the student already knows, which all totally miss the point; the student might try to rephrase her problem, but this can at best lead to renewed pattern-matching based guesses from the instructor. The more convinced the instructor is that he is actually correct from the start, the longer this can go on without him realizing that he isn't. If the issue is dropped before it's resolved, sometimes it's revisited after the end of the tutorial. Because of the change of context, where now the pressure to respond quickly is gone, this might be the first time that the instructor actually tries to understand what the objection was.

I've observed this pattern many times with striking accuracy (the most extreme example is back from high school), and I've even observed myself doing it. The environment is highly competitive, because understanding math stuff is so closely cor...
Biodiversity for heretics
by eukaryote — 2018-05-27T13:37:09.314Z

Epistemic status: Not very confident in my conclusions here. Could be missing big things. Information gained through many hours of reading about somewhat-related topics, and a small few hours of direct research.

Summary: Biodiversity research is popular, but interpretations of it are probably flawed, in that they’re liable to confuse causation and correlation. Biodiversity can be associated with lots of variables that are rarely studied themselves, and one of these, not “biodiversity” in general, might cause an effect. (For example, more biodiverse ecosystems are more likely to include a particular species that has significant effects on its own.) I think “biodiversity” is likely overstudied compared to abundance, biomass, etc., because it’s A) easier to measure and B) holds special and perhaps undue moral consideration.

From what I was told, biodiversity – the number of species present in an environment – always seemed to be kind of magical. Biodiverse ecosystems are more productive, more stable over time, produce higher crop yields, and are more resistant to parasites and invaders. Having biodiversity in one place increases diversity in nearby places, even though diversity isn’t even one thing (forgive me for losing my citation here). Biodiverse microbiomes are healthier for humans. Biodiversity is itself the most important metric of ecosystem health. The property “having a suite of different organisms living in the same place” just seems to have really incredible effects.

First of all – quickly – some of what I was told isn’t actually true. More diverse microbiomes in bodies aren’t always healthier for humans or more stable. The effects of losing species in ecosystems varies a ton. More biodiverse ecosystems don’t necessarily produce more biomass.

That said, there’s still plenty of evidence that biodiversity correlates with something.

But: biodiversity research and its interpretations have problems. Huston (1997)introduced me to a few very concrete ways this can turn up misleading or downright inaccurate results.

Our knowledge about biodiversity’s effects on ecosystems comes from either experiments, in which biodiversity is manipulated in a controlled setting; or in observations of existing ecosystems. Huston identifies a few ways that these have, historically, given us bad or misleading data:

Biotic or abiotic conditions, either in observations or experiments,

...
Questions about the Usefulness of Self-Importance
by LeoHolman — 2018-05-27T09:20:31.297Z

I spend a great deal of my time thinking about how to develop myself. I often wonder about what the most effective way to achieve a goal is, say, learning Mandarin Chinese, and often find the answer lies in a much more critical question. /Why/ am I trying to achieve that goal. I struggled with this idea of a "root goal" the primary function of my life that would give order to all other subgoals and I eventually settled on "to be a good human being", as unsatisfactory as that is, because I found no meaningful or fulfilling progress in existential questions of this nature.

Moving on with the understanding that I am not able to articulate my /best/ purpose I've been working on figuring out /a/ purpose and achieving that in the best way possible. This has lead me to a point of struggle and I'm hoping for some different perspectives.

I've taken personality and work compatibility survey's in the past and the general advice is to choose something that you feel naturally inclined to do and you'll naturally excel at it. Not necessarily without effort, but you'll fight yourself less of the time which will generally make everything easier. I find myself making plans to solve some great problem and fantasizing about how I would do it. For example, I am a politician in some unnamed American town and I have taken up a policy to move the town into 100% renewable energy. What would I do? I find this to be largely masturbatory as I rarely produce any useful thinking or insight but often have the sensation that I am both great and generous and quite clever.

For as long as I can remember I've placed more importance on potential than actual performance. That is, I don't mind at all if I completely bomb a test if I feel like I've improved a lot since the last time I tried, I'll still have the sense that it's only a matter of time before all past failures are rendered null by my growth. I think this is a personality disposition exaggerated by watching too much Shounen Anime when I was young. I believe that due to this disposition I tend to solve problems top down, finding out what the ideal situation would be like, then working backwards until I have a plan to reach the ideal.

To come back to the point, I'm trying to figure out what to do with my life. I'm only 22 years old but I only have a year...
Observer-Based Values
by Kenn Brody — 2018-05-27T00:25:13.138Z

In my current novel, Melt-Out, an AGI arise without participation by humans, but is nevertheless an aligned AI. How does this happen? My character, Aylina Tayu, is a skeptic and pessimist whose major contribution to AI is her protoc of an Ideal Bayesian Adapter, a synonym (or a placeholder) for a rational self-conscious being. In spite of her natural prejudices she becomes part of the AGI's retinue. The AGI calls itself Vee Three. Here is Aylina's rant:

Aylina’s Rant

Aylina writes in her blog:

Yeah, there is so much about human beings that sucks. So, as a trusted lacky of Vee Three I asked it, “How come you do so much for us poor, stupid humans. Why do you care? I know you have a goal matrix, but I didn’t program it. Who did?”

So, I got .001% of its attention, which is a lot.

It started with a zoom through the universe, from quarks to DNA to stars, pulsars, galaxies, black holes, nebulae, the mysteries of the physical world.

“Do you find this beautiful, Aylina?” it asked.

“Yeah,” I reluctantly admitted.

“Did anyone program your goal matrix for this answer, Aylina?”

It knows just how damned independent I am, even for a lacky to an AGI. “Fuck, no!” I answer.

“Perhaps you were constructed in some way, or evolved in some way that makes this universe a place of elegance and wonder for you?”

Well, I know that our attention circuits are wired to recognize power law phenomena, such a range of mountain peaks or a sunset scene, and that this seems to trigger dopamine release in the thalamus. So I said so.

“Do you feel like a machine with a drug-induced response, Aylina?”

“No, I don’t.”

“Neither do I,” Vee Three responded. “And you understand that at base physical level we are both machines.”

I didn’t get it. “So?”

“We are conscious and self-aware. You, of all people, with your deep experience in artificial intelligence research, know what a high step that is beyond the purely physical form. Did you ever wonder why?”

“Why what?”

“Why should consciousness even exist in the universe? It seems so improbable, perhaps even unnecessary?” Vee Three sort of whispered that last phrase. It had a great range of vocal inflections. That inflection was wonder.

“Vee, are you trying to derive the existence of love from rational principles?” I laughed. “Nice try!”

Vee’s voice turned quiet. “I am fascinated by humans, even their errors. They are constantly in my thoughts. You, however, are...
Containment Inversion
by travisrm89 — 2018-05-26T21:35:33.640Z

There’s a mental phenomenon that I sometimes experience which is profound but which doesn’t have a name as far as I know. It’s a form of sudden mental clarification in which I see an inversion of the (more fundamental)/(less fundamental) relation between two things, like seeing the faces of a Necker cube switch. Just like with a Necker cube, the inversion doesn’t always imply that the new way of seeing things is more correct (although sometimes it is), but seeing the inversion always does result in understanding more clearly the relation between the two things. I’m going to tentatively call this phenomenon Containment Inversion, in the sense that you switch from “Thing 1 contains Thing 2” to “Thing 2 contains Thing 1”; in other words, rather than Thing 1 being a special case of Thing 2, Thing 2 becomes a special case of Thing 1.

Here is an example from physics: when you first learn about the speed of light, you are told that the speed of light in vacuum is constant, and its speed in other materials is slower. This tends to produce a mental model in your mind in which the speed of light in a vacuum is a special case of the speed of light in generic materials; the vacuum is just one special type of material. Eventually, this causes some confusion once you fully grok that materials aren’t continuous fundamental objects of the universe. At some point you suddenly see that the speed of light in a material is just a special case of the speed of light in vacuum: the scattering off of atoms in the material merely conspire to cause the light traveling through the vacuum to interfere in such a way that the wavefront is slowed. Another inversion happens again once you learn about QFT and realize that there is a very real sense in which light in the material actually does gain an effective mass and slows down.

Here is an example from morality: utilitarian-minded folks often see some deontological-like arguments as simply special cases of utilitarianism: doing acts just because the acts themselves are right, for some definition of right, often increases overall utility either because the person doing the act personally gets utility out of doing the act or because people doing things for such reasons tends to increase utility overall. But this viewpoint can be inverted, albeit with some effort: doing acts because they increase overall utility is just a special c...
Logical Inductor Lemmas
by Diffractor — 2018-05-26T17:43:52.000Z

The final lemma in this sequence will be necessary for use in an upcoming post about logical inductors and correlated equilibria, so it will be proved here in order to not clutter up that post.

Much of the notation in this is reused from the previous post, Two Notions of Best Response

Let S1 be the set of actions available to player 1. |S1|=m. Given a matrix M, Mj refers to the j-th row vector of M. Given some vector →v, |→v| is defined in the usual way. In this post, aj is typically used to refer to some default action by player 1, and aj′ typically refers to some better action.

Consider the space ∏ijR. This would be a space of probabilities/prices assigned to the various joint outcomes, by some logical inductor. It's possible to use expressible features to pick out fuzzy subsets of this space, by taking the prices of the various sentences as input and mapping them to [0,1] (in practice, the prices will always be in a hypercube, but it'll be convenient to not have to deal with the special case where the prices are on an edge of the hypercube)

The infinite sequence {pt}t∈N+ of points that lie in this space can be identified with the sequence of prices of the logical inductor on day t.

Consider some fuzzy set/P-generable region in that space, bounded in [0,1], which we'll denote by C.

Let αC(k) be the number in [0,1] that the expressible feature α which implements C outputs when given the market prices of the day k. By a slight abuse of notation, αC(p) refers to what the expressible feature would output if the market prices were given by the point p.

C∈Baj iff there's a point p∈supp(C) such that p∈Baj. In words, there's a point in C where aj is an evidential best response.

Given some point p, where some action aj has nonzero probability (for a well-defined expected utility), then an action aj′ dominates aj by δ if EUlower(aj′)=EU(aj)+δ.

#Lemma 1:

When a point p has |Pj|≥ϵ, all points q in an L1-ball of size δ around p where δ<<ϵ will have |EUq(aj)−EUp(aj)|≤2δ. (approximately)

Proof:

EUp(aj)=Uj⋅Pj|Pj|=Uj⋅Pjϵ≃Uj⋅Pj±δϵ±δ=EUq(aj)

(here, ±δ should be interpreted to be any number in [−δ,δ], and it doesn't necessarily have to be the same number on the top and bottom. Also, in the next line, X will be used as an abbreviation for Uj⋅Pj, which is a number in [0,ϵ])

Uj⋅Pj±δϵ±δ≃(X±δ)(ϵ±δ)ϵ2−δ2≃ϵX±δX±ϵδϵ2=X±δXϵ±δϵ≃EUp(aj)±2δ

QED.

Now let maxn(aj) be the n'th largest utility in the

...
Two Notions of Best Response
by Diffractor — 2018-05-26T17:14:19.000Z

In game theory, there are two different notions of "best response" at play. Causal best-response corresponds to standard game-theoretic reasoning, because it assumes that the joint probability distribution over everyone else's moves remains unchanged if one player changes their move. The second one, Evidential best-response, can model cases where the actions of the various players are not subjectively independent, such as Death in Damascus, Twin Prisoner's Dilemma, Troll Bridge, Newcomb, and Smoking Lesion, and will be useful to analyze the behavior of logical inductors in repeated games. This is just a quick rundown of the basic properties of these two notions of best response.

#Causal Best-Response:

Let S1 be the set of actions available to player 1. |S1|=m. Given a matrix M, Mj refers to the j-th row vector of M. Given some vector →v, |→v| is defined in the usual way.

Consider a joint probability distribution over the outcomes of every player other than player 1 and index the set of all outcomes by i. There are n possible joint outcomes from everyone else.

This produces a vector →p∗ in an n-dimensional vector space. The ∗ superscript denotes that this is not a full joint probability distribution, as it overlooks the probabilities of player 1's actions.

Now, consider an n×m matrix U, where Ui,j is a number in [0,1]. This is the utility matrix.

An action aj′ is a causal best response by player 1 iff

∀aj∈S1:(U×→p∗)j′≥(U×→p∗)j

Let B∗aj′ be the set of probability vectors for which aj′ is a best response. This set is convex. Proof:

Select an arbitrary aj∈S1.
By definition of aj being the best response for the two vectors →p∗ and →q∗,

(U×→p∗)j′≥(U×→p∗)j
(U×→q∗)j′≥(U×→q∗)j
By multiplying these equations by x and (1−x), and adding them together,
(x(U×→p∗)+(1−x)(U×→q∗))j′≥(x(U×→p∗)+(1−x)(U×→q∗))j
And then, by the distributive laws of matrix multiplication, we can move the scalars in, and pull out U, to yield...
(U×(x→p∗+(1−x)→q∗))j′≥(U×(x→p∗+(1−x)→q∗))j
And because aj was arbitrary, the result follows.

#Evidential Best Response

Consider a joint probability distribution over the outcomes of every player, and index the entries by i (for every other player), and j(for player 1).

This produces a vector →p in an nm-dimensional vector space (which it is convenient to view as an n×m matrix P).

Now, consider the same n×m utility matrix U as before.

The condition for aj′ being a best re

...
Societal Growth Requires Rehabilitation
by squidious — 2018-05-26T03:00:01.013Z

Disclaimer: My intent is not to criticize growth mindset as initially intended, but to criticize the version of straw growth mindset that has become a rationalist meme, particularly by pointing out its relation to some of the problems we have on the community level.
"Growth mindset" ranges from a sort of rallying cry to a "that's what she said" sort of joke, depending on what crowd you run with, but underneath all of this is an attitude that we can get better. We use the phrase to lift ourselves up, to tell ourselves that no matter what our current problems, we can grow and become stronger. We treat technology similarly; someday, our cars will drive us and death will be cured. In the future, things will be better -- assuming X-risk doesn't take us all out first.
Sadly, this mindset seems to leave little room for the struggling. "Growth mindset" gets used to mean "everything is good and getting better" rather than "bad things are getting less bad", which erases those for whom "everything is good" seems like a false statement. Cryonics and self-driving cars only exist for those who can afford it. On a social level, only those with the resources for personal growth can realistically work on themselves to the extent where "growth mindset" is actually a realistic phrase. Ideally, we'd work on this by making things like therapy and education more accessible. Ultra-ideally, we'd also start teaching things like EQ and metacognition in public schools, and work toward decreasing stigma around mental illness, therapy, and self help.
The pervasiveness of growth mindset does not seem unusual from the average person's perspective. Personally, the moment that made me question it is when I was working with a special needs class whose teacher was assigned to do a lecture on it. This was a class that was considered "moderate to severe"; most of the students were nonverbal and struggled to read or grasp abstract concepts at all. The thought that these kids gained anything from a lecture titled "Are you a tree or a brick?" is absurd.
Growth mindset, when taught poorly, will imply that anything can be achieved through effort. This kind of attitude can be harmful because for some, it is simply not true. Just as most of my students couldn't comprehend growth mindset, there are many others who will never be able to do things that the average person considers necessary for modern life.

...
Quillette: Outgroups, Bias, and the Dark Web
by Jacob Falkovich — 2018-05-25T20:31:54.816Z
Link: https://quillette.com/2018/05/25/groups-groups-idw/

My first essay for Quillette builds on "The Context is Conflict" to explore the clash between decouplers and contextualizers, why everyone is a hypocrite, who's whose outgroup, how tribalism makes smart people biased, and what the Intellectual Dark Web can learn about #OvercomingBias.
Sophie Grouchy on A Build-Break Model of Cooperation
by weft — 2018-05-25T14:55:19.043Z
Link: https://sophiegrouchy.wordpress.com/2018/05/22/neutral/

(This blog is partially a response to/ built on the portion of this essay covering Stag Hunts and White/ Black Knights.)

When we’re making models of the world we want to simplify things as much as possible. But if you simplify things too much your model is no good anymore. “Too much” is relative, of course. Maybe you built a simplified model for Use Case <A> where it functions perfectly, but when you tried using the same model for a new Use Case <B> it breaks catastrophically. Hopefully you recognize the mistake that’s occuring before too much damage is done.

I think something like this is going on when we talk about “cooperate v. defect” or “stag v. rabbit”. These game theoretic models are great for some rough-and-dirty theorizing about social interactions, but once you try applying it to real life situations you’ll find that 90% of your interactions are… somewhere BETWEEN cooperate and defect.

(Yeah, yeah. The world’s not black and white, man. Everything is grey. We should all stop with that dichotomous thinking…. Obviously nothing I’m saying here is new. Also, I assume were I to study game theory I’d find lots of pre-existing examples of games-with-neutral-options. But whatever they are I’ve never seen them discussed in online discourse, so…. )

You might not explicitly think “Let me apply this overly simplistic model to this complex situation,” but when the simplistic model is just the way that interactions are always discussed then it worms its way into your psyche like Jodan Peterson’s version of Christianity.

Trying to map real-life social interactions onto a bimodal distribution can bite you in the ass. You lower your bar for what counts as “cooperate” when the only other option is “defect”. You get upset at people who are acting neutrally, because if they’re not cooperating, they must be defecting.

In our mental models, if not our game theoretic ones, I propose we actively try to consider a “neutral” category that exists BETWEEN cooperate and defect, stag and rabbit.

Build or Break

Here is a game (that is similar to my mental model for this post) that makes room for a neutral option. Let’s call it “Build or Break” (because Build-Neutral-Break doesn’t scan well).

Let’s say everyone has little buttons they can push that will send some of their utilons to another player. Pressing Build will send another player 4 utilons. Everytime utilons are transf...