trentmkelly/LessWrong-43k huggingface dataset:
The value of low-conscientiousness people on teams 
by CraigMichael — 2021-06-14T02:06:25.504Z

[Apologies in advance if I sound like I'm over-generalizing high-conscientiousness or low-conscientiousness people. This is mostly from my own experience, so I'm sure I'm wrong on some counts and may, in fact, be over-generalizing at times. Ohh, and also apologies to Mick Jaggar.]

Please allow me to introduce myself. I'm a man of mess and wile. I've been scoring around 20% (in trait conscientiousness on Big Five tests) for a long, long year, cut by many a sharp wire’s height.
At first glance, conscientiousness as a construct seems a bit like intelligence, in the sense that it would seem everyone would be better off with more of it. So, why would natural selection produce people like me who are very low in conscientiousness? Have I only escaped a Darwin award by the grace of the almighty simulator?

I have a hypothesis that low-conscientiousness people may function a bit like dichromats (color blind people) on teams of hunter-gathers. While dichromats can't see some colors, they can detect color-camouflaged objects better than non-color blind people (trichromats). So, teams with mixtures of dichromats and trichromats may have out-competed teams with only trichromats (or only dichromats, for that matter).

Perhaps some diversity of conscientiousness in groups could produce a competitive advantage? There is some evidence in this direction. Just from skimming Wikipedia:

The world's most conscientious nations are also some of the poorest.
Groups with only conscientious members have difficulty solving open-ended problems.
Those scoring low on conscientiousness make better decisions after unanticipated changes in the context of a task.
Conscientiousness has been found to be positively correlated with business and white-collar crime.

But, specifically, I'd like to surface patterns I've observed in my own experience that I haven't seen discussed elsewhere. I work with, and am related to, many high-conscientiousness people. I've come to appreciate their strengths relative to mine, but I also have first-hand experience with some failure modes of conscientiousness taken too far. Any virtue taken to an extreme can become a vice, as Dieter said. (No, not that Dieter, Dieter Uchtdorf).

Allow me to present four of my observed failure modes of high-consciousness.

#1 Not all messes are worth the risk of cleaning them up

This is the most frequent failure mode ...
Five Suggestions For Rationality Research and Development

by LoganStrohl — 2021-06-14T01:09:26.532Z

Make field work the bulk of what you do. For every hour you spend engaging with other people's thoughts or reflecting on your own thoughts, spend at least two hours, and ideally two days, trying to directly observe whatever it is you're studying.

Add other people to your sensorium. Make conversation with others a big part of your work, and approach those interactions with the intent to study the experience of others.

Corollary: If you're developing a technique or method, then by all means teach it to people. That's crucial. But don't teach them so they will know your technique, or even so you'll be better at teaching it. Instead, teach them so you can find out what happens for them when they try to use it.

Focus on topics in the intersection of "things you're personally interested in", "things you think might actually matter to someone in particular", and "places where the existing art seems deficient".

If it seems worthwhile to you, go after it. This is not a field with people in the position to tell you what you're allowed to study, or when, or how. This is a frontier. No bus will ever arrive, so you'll have to use your feet. Stop waiting for permission. Just get to work.
Precognition
by jasoncrawford — 2021-06-14T00:38:29.791Z
Link: https://jasoncrawford.org/precognition

It’s almost impossible to predict the future. But it’s also unnecessary, because most people are living in the past. All you have to do is see the present before everyone else does.

To be less pithy, but more clear: Most people are slow to notice and accept change. If you can just be faster than most people at seeing what’s going on, updating your model of the world, and reacting accordingly, it’s almost as good as seeing the future.

We see this in the US with covid: The same people who didn’t realize that we all should be wearing masks, when they were life-saving, are now slow to realize/admit that we can stop wearing them.

For a dramatic historical example (from The Making of the Atomic Bomb), take Leo Szilard’s observations of 1930s Germany:

Adolf Hitler was appointed Chancellor of Germany on January 30, 1933. … In late March, Jewish judges and lawyers in Prussia and Bavaria were dismissed from practice. On the weekend of April 1, Julius Streicher directed a national boycott of Jewish businesses and Jews were beaten in the streets. “I took a train from Berlin to Vienna on a certain date, close to the first of April, 1933,” Szilard writes. “The train was empty. The same train the next day was overcrowded, was stopped at the frontier, the people had to get out, and everybody was interrogated by the Nazis. This just goes to show that if you want to succeed in this world you don’t have to be much cleverer than other people, you just have to be one day earlier.”

How to be earlier

1. Independent thinking. If you only believe things that are accepted by the majority of people, then by definition you’ll always be behind the curve in a changing world.

2. Listen to other independent thinkers. You can’t pay attention to everything at once or evaluate every area. You can only be the first to realize something in a narrow domain in which you are an expert. But if you tune your intellectual radar to other independent thinkers, you can be in the first ~1% of people to realize a new fact. Seek them out, find them, and follow them.

I was taking covid precautions in late February 2020, about three weeks ahead of official “lockdown” measures—but only because I was tuned in to the people who were six weeks ahead.

But:

3. Distinguish independent thinkers from crackpots. Both are “contrarian”; only one has any hope of being right. This is an art, honed over decades. Pay attention to bot...
Shall we count the living or the dead?
by Anders_H — 2021-06-14T00:38:22.968Z

This is a linkpost for an entry in my Substack newsletter at https://countheliving.substack.com/p/shall-we-count-the-living-or-the . In this entry, we announce a new arXiv preprint, "Shall we count the living or the dead". This preprint builds on my earlier work that has previously been discussed on Less Wrong, and which led to a response first on Less Wrong and later in the European Journal of Epidemiology by Carlos Cinelli and Judea Pearl.

The Substack entry contains a link to an animated video on YouTube, which explains a simplified version of the argument
Looking Deeper at Deconfusion
by adamShimi — 2021-06-13T21:29:07.811Z

Introduction

My PhD thesis probably wins the prize of weirdest ever defended at my research lab. Not only was it a work of theory of distributed computing in a formal methods lab, but it didn’t even conform to what theory of distributed computing is supposed to look like. With the exception of one paper (interestingly, the only one accepted at the most prestigious conference in the field), none of my published works proposed new algorithms, impossibility results, complexity lower bounds, or even the most popular paper material, a brand new model of distributed computing to crowd even more the literature.

Instead, I looked at a specific formalism introduced years before, and how it abstracted the more familiar models used by most researchers. It had been introduced as such an abstraction, but the actual formal connection was missing. What I did was put it on a more formal footing, and also explain what was lost in abstraction.

It’s not that it wasn’t interesting. My advisors were curious if not impressed. I published multiple papers. But it was quite clear that no one really knew what to do with my work, be it reviewers, colleagues, advisors, or even me.

We didn’t know I was doing deconfusion.

You might have heard of it: Nate Soares, MIRI’s executive director,

coined

the term to describe the sort of work MIRI was and is doing; things like

Functional Decision Theory

and

Logical Induction

. He described it as:

something like “making it so that you can think about a given topic without continuously accidentally spouting nonsense.”

In hindsight, this quote captures both my graduate work and my current research in AI Alignment. Yet I fail to realize this despite reading the post multiple times in the last few years. Deconfusion sounded either like fundamental research or like distillation; two subjects I was interested in, but which didn’t quite fit my interests and my intuitions for the thing I was aiming at.

Similarly, I see many people use deconfusion to reference those two topics. Fundamental research because deconfusion is used to point at what MIRI is doing, and they have been traditionally focused on deconfusion related to fundamental questions of agency of rationality.

Distillation

because it involves dissolving one form of confusion -- the confusion coming from inferential distance.

So anyone who wants to do actual new research and doesn’t subscribe to ...
What other problems would a successful AI safety algorithm solve?
by DirectedEvolution — 2021-06-13T21:07:29.244Z

Corporations and governments are in some ways like superintelligences, and in others ways not. Much of economics, political science, and sociology seem to tackle the problem of why institutions fail to align with human interests. Yet the difference in architecture and capabilities between brains and computer programs suggests to me that aligning collective bio-superintelligence is a quite different problem from aligning AI superintelligence. It might be that because we can see and control the building blocks of AI, and have no hard ethical limits on shaping it as we please, aligning AI with human values is an easier problem to solve than aligning human institutions with human values. A technical solution to AI safety might even be a necessary precursor to understanding the brain and human relationships well enough to provide a technical solution to aligning human institutions.

If we had a solid technical solution to AI safety, would it also give us technical solutions to the problem of human collective organization and governance? Would it give us solutions to other age-old problems? If so, is that reason to doubt that a technical solution to AI safety is feasible? If not, is that reason for some optimism? Finally, is our lack of a technical account of what human value is a hindrance to developing safe AI?
Avoiding the instrumental policy by hiding information about humans
by paulfchristiano — 2021-06-13T20:00:51.597Z

I've been thinking about situations where alignment fails because "predict what a human would say" (or more generally "game the loss function," what I call the instrumental policy) is easier to learn than "answer questions honestly" (overview).

One way to avoid this situation is to avoid telling our agents too much about what humans are like, or hiding some details of the training process, so that they can't easily predict humans and so are encouraged to fall back to "answer questions honestly." (This feels closely related to the general phenomena discussed in Thoughts on Human Models.)

Setting aside other reservations with this approach, could it resolve our problem?

One way to get the instrumental policy is to "reuse" a human model to answer questions (discussed here). If our AI has no information about humans at all, then it totally addresses this concern. But in practice it seems inevitable for the environment to leak some information about how humans answer questions (e.g observing human artifacts tells you something about how humans reason about the world and what concepts would be natural for them). So the model will have some latent knowledge that it can reuse to help predict how to answer questions. The intended policy may not able to leverage that knowledge, and so it seems like we may get something (perhaps somewhere in between the intended and instrumental policies) which is able to leverage it effectively. Moderate amounts of leakage might be fine, but the situation would make me quite uncomfortable.
Another way to get something similar to the instrumental policy is to use observations to translate from the AI's world-model to humans' world-model (discussed here). I don't think that hiding information about humans can avoid this problem, because in this case training to answer questions already provides enough information to infer the humans' world-model.
I have a strong background concern about "security through obscurity" when the alignment of our methods depends on keeping a fixed set of facts hidden from an increasingly-sophisticated ML system. This is a general concern with approaches that try to benefit from avoiding human models, but I think it bites particularly hard in this case.

Overall I think that hiding information probably isn't a good way to avoid the instrumental policy, and for now I'd strongly prefer to pursue approaches to this prob...
Answering questions honestly given world-model mismatches
by paulfchristiano — 2021-06-13T18:00:08.396Z

(This post is superseded by our writeup on Eliciting Latent Knowledge.)

In a recent post I discussed one reason that a naive alignment strategy might go wrong, by learning to “predict what humans would say” rather than “answer honestly.” In this post I want to describe another problem that feels very similar but may require new ideas to solve.

In brief, I’m interested in the case where:

The simplest way for an AI to answer a question is to first translate from its internal model of the world into the human’s model of the world (so that it can talk about concepts like “tree” that may not exist in its native model of the world).
The simplest way to translate between the AI world-model and the human world-model is to use the AI world-model to generate some observations (e.g. video) and then figure out what states in the human world-model could have generated those observations.
This leads to bad predictions when the observations are misleading.

This is distinct from the failure mode discussed in my recent post — in both cases the AI makes errors because it’s copying “what a human would do,” but in this case we’re worried that “what a human would do” may be simpler than the intended policy of answering questions honestly, even if you didn’t need a predictive model of humans for any other reason. Moreover, I’ll argue below that the algorithm from that post doesn’t appear to handle this case.

I want to stress that this post describes an example of a situation that poses a challenge for existing techniques. I don’t actually think that human cognition works the way described in this post, but I believe it highlights a difficulty that would exist in more realistic settings.

Formal setup

Human world-model

I’ll imagine a human who has a simple world model W = (S, P: Δ(S), Ω, O: S → Ω) where:

S is a space of trajectories, each describing a sequence of events in the world. For example, a trajectory s ∈ S may specify a set of rigid objects and then specify how they move around over time.
P is a probability distribution over trajectories. It includes both a prior over initial states (cars are probably on the road and fish are probably in the ocean) and a dynamics model that tells us how likely a trajectory is under the laws of physics (most trajectories approximately satisfy Newton’s laws).
Ω is a space of observations, for example videos.
O tells

...
Experiments with a random clock
by Malmesbury — 2021-06-13T17:08:03.200Z
Link: https://www.telescopic-turnip.net/experiments/a-random-clock/

I may have found a solution to one of my biggest, longest-standing, most irredeemable problems. For most of my life, I have been consistently late. Whether it’s appointments, attending events, taking trains or joining a zoom call, I’m typically 10 minutes late for everything and it’s ruining my life – not because I actually miss the train (though that happens too) but because I’m constantly rushing and panicking. Whatever I do, I start it in a state of maximum stress and guilt. Obviously, I tried pretty much everything to address the problem, including various artificial rewards and punishments, telling a therapist about it, having people call me to remind me to get ready, taking nootropics, and many more ridiculous ideas. So I thought, “how do all these well-adjusted adults manage to be perfectly on time all the time?” and I did what any well-adjusted normie would do: I tried to formally frame the problem in terms of expected utility theory.

Tricking myself: single-player game theory

Imagine I have to attend a very important scientific conference on the effect of dubstep on mosquitos. The figure below plots how much I enjoy the event depending on the time I arrive.

Arriving early by ten minutes or one hour does not make any difference (or so I presume – this never happened to me). Being just a few minutes late is not a big deal either, since it’s just going to be the speaker testing her microphone or other formalities of no importance. Beyond that, it starts becoming really rude (with some variation depending on which culture you live in) and I risk missing some crucial information, like the definition of a crucial concept central to understanding the equations of mosquitos’ taste for Skrillex.

The second aspect of the problem is how much time I can save by arriving later, which is just a straight line:

Why would I arrive ten minutes early to the Skrillex-as-a-cure-for-dengue talk, when I could spend ten more minutes reading about exorcism under fMRI? Summing both aspects of the problem, the grand unified utility curve looks something like this:

There you have it: the utility peak, the most rational outcome, is obtained by being just a few minutes late. I suppose for most people, this basically means you should arrive on time, since the peak is not that far from the start of event. But chronically-late people like myself have a distorted vision of the utility cur...
The Cult Deficit: Analysis and Speculation
by rogersbacon — 2021-06-13T05:33:06.602Z

(originally posted at Secretum Secretorum)

Abstract

Using a dataset derived from the long-running “Cults” podcast by Parcast, I find that the number of new cults began to increase in the 50s, peaked in the 70s/80s, and has been in steady decline in recent decades. I discuss various factors (historical, technological, cultural, psychedelic drugs) that may have played a role in the rise and fall of cults since the 1950s and speculate on the future of cults.

Introduction

There does not seem to be as many cults as there were in the past. Ross Douthat notes this trend in his 2014 New York Times article “

The Cult Deficit

” and discusses two thinkers who have suggested that this lack of cults is, contrary to popular opinion, a bad sign for society.

“Philip Jenkins, a prolific religious historian, who argues that the decline in “the number and scale of controversial fringe sects” is both “genuine and epochal,” and something that should worry more mainstream religious believers rather than comfort them. A wild fringe, he suggests, is often a sign of a healthy, vital center, and a religious culture that lacks for charismatic weirdos may lack “a solid core of spiritual activism and inquiry” as well.”

“Peter Thiel’s argument is broader: Not only religious vitality but the entirety of human innovation, he argues, depends on the belief that there are major secrets left to be uncovered, insights that existing institutions have failed to unlock (or perhaps forgotten), better ways of living that a small group might successfully embrace.

This means that every transformative business enterprise, every radical political movement, every truly innovative project contains some cultish elements and impulses — and the decline of those impulses may be a sign that the innovative spirit itself is on the wane. When “people were more open to the idea that not all knowledge was widely known,” Thiel writes, there was more interest in groups that claimed access to some secret knowledge, or offered some revolutionary vision. But today, many fewer Americans “take unorthodox ideas seriously,” and while this has clear upsides — “fewer crazy cults” — it may also be a sign that “we have given up our sense of wonder at secrets left to be discovered.”1

Both Thiel and Jenkins take it as self-evident that cults are in decline, and to my knowledge there has been no quantitative analysis of cult for...
Covid vaccine safety: how correct are these allegations?
by DPiepgrass — 2021-06-13T03:08:23.858Z

One month ago I clumsily tried to persuade my 74-year-old father that Tucker Carlson is probably wrong about Covid vaccines killing 3000 people, and if not, my father should get the vaccine anyway because he is in a high-risk group.

Well, move over lab-leak discussion, because this video is a tad more explosive: it alleges an ongoing and almost systematic censorship of information about vaccine side-effects, and it manages to do so in a way that fails to trip my BS detectors. While the LessWrong community isn't known for its expertise in vaccine science and epidemiology, it's usually pretty good about separating the true from the false, so here I am to ask for your comments. While the video is extremely long, the most controversial claims come near the beginning.

It's hosted by Bret Weinstein who (says Wikipedia) 'came to national attention during the 2017 Evergreen State College protests' and 'is among the people referred to collectively as the "intellectual dark web"'. The video description says Robert Malone is "the inventor of mRNA Vaccine technology"; he doesn't have a Wikipedia page, but he is mentioned on the page about mRNA vaccines as having 'developed a high-efficiency in-vitro and in-vivo RNA transfection system using cationic liposomes, which were used "to directly introduce RNA into whole tissues and embryos", as well as various cells types' in 1989. Finally there's Steve Kirsch, a red-tribey-sounding serial entrepreneur who mostly does tech firms but has been "researching adverse reactions to COVID vaccines". Steve frequently interrupts the other two, but at least seems very knowledgeable and well-connected (not to mention wealthy).

At first I was going to make this a 'question', but there's a lot to unpack and I think it could generate a lot of discussion, so I made it a 'post' instead.

Summary of the video's main points & discussions:

3:40 Prophylactic ivermectin is about "100% effective at preventing people from contracting Covid when taken properly" (an "Argentina study" is mentioned in support of this)
8:00 Bret gushes in the beginning about Steve's "article in Trial Site News". Here is a list of Steve's (seven) articles in Trial Site News. (edit: the original article was removed from there, albeit still accessible via direct link, and was later put behind a paywall with Kirsch's name removed.)
11:12 Steve became interested in the issue

...