trentmkelly/LessWrong-43k huggingface dataset:
Blink: The Power of Thinking Without Thinking (book review)
by Andrew Vlahos — 2021-10-03T23:21:33.313Z

Summary

Blink is a book about people making unconscious emotional judgments, and how this isn't always a bad thing. In certain circumstances people subconsciously figure something out, and change their behavior based on this, far sooner than they gain conscious awareness. People here talk all the time about the need to de-bias themselves and think things through logically, which is a commendable goal, but working on biased hunches is much faster - and sometimes the biases lead to more accurate judgements.

The book is about when unconscious emotional judgements should be listened to, and when they shouldn't. Blink also says how to get better results.

When Emotional Judgements are Good.

Feelings of Unease

Imagine that I ask you to play a simple gambling game. In front of you are four decks of cards: two are red and two are blue. Each card gives you money or costs you money, and you should turn over 100 cards. one at a time, and try to maximize your winnings.

The experimenters measured the activity of the sweat glands in the palms of the hands. When someone got a big penalty card from one of the decks, the sweat increased, and their behavior started to change as well. They started taking more cards from decks of the other color, making adjustments in response to data without keeping track of exact results.

The decks were not quite equal. The red decks had much higher penalties and only slightly higher rewards than the blue decks. The gamblers started saying that they had a hunch the blue decks were better (correctly) around the 50th card, and could say what the exact difference was by the 80th. But when did they start having noticeably more sweat turning over red cards instead of blue cards?

The tenth. They gained a correct emotional bias, usually in the right direction, in just a fifth of the time needed to realize they had a hunch. This is called the "Bechara gambling task" and gained quite a lot of follow-up studies.

This isn't 15,200 studies, a lot of them are books about it, but there still are a lot.

(The replication studies show a weaker effect: subjects are able to have a hunch at the same time as the sweat and behavior change, but this still happens significantly before they can give a reason why):

Another example Gladwell gave in the first chapter was a marble statue. It was a very rare 2600 year old kouroi statue, found almost perfectly preserved, matching the ...
Vax passports - theory and practice
by tkpwaeub — 2021-10-03T22:35:11.396Z

It’s been about three weeks since New York City began - at least, nominally - enforcing its new Key to NYC program. That’s our new mandate requiring proof of vaccination for indoor dining. I’ve eaten out several times since it started, and I think that’s given me some new insight into how this ordinance is going to work. I’m not nearly as skeptical of it as I was, and I hope that what I’m about to share helps convince some of the businesses that are hostile to the idea that it’s worth their effort to comply cheerfully.

I work for state government, and I’ve been doing a lot of field work lately in different parts of the city - Queens and Staten Island, to be specific. Staten Island, as I’m sure you’re aware, is somewhat more Republican than the rest of the city, so not surprisingly there’s a bit more resistance to the mandate. A few places I saw had desperate handwritten signs saying “Take Out Only”. One pizza parlor in Brooklyn turned its tables upside down. A Dunkin Donuts in Queens had a sign that apologized for being closed for indoor dining for the foreseeable future. I encountered outright defiance at a deli in Manhattan’s financial district when I showed my CDC card. I can’t remember the exact words but it was something like “I don’t care if you’re not vaccinated, I don’t believe in all that.” It’s not clear what the cashier hoped to gain by saying that. A few doors down, at a similar place, they were keeping their trays behind the counter, and when I asked for a tray, they asked to see my CDC card and took down my name and number “in case the health department came around.” A diner in Hell’s Kitchen had the dining section cordoned off, and checked my ID and CDC card (I have both on a lanyard) at a distance of about four feet. I very much doubt they were able to read either. More on that below.

In the month between when the order was announced and the beginning of enforcement, it seems there wasn’t a lot done to educate the general public. Whether that’s because of a lack of resources or because they genuinely didn’t know exactly how to educate people on it is anyone’s guess. There’s very little daylight between education and regulation. The best auditors I know are the ones who are able to teach regulated entities the how’s and why’s of compliance. I don’t know the extent to which venues were provided with education and guidance.

This w...
How to build a mind - neuroscience edition
by Jan — 2021-10-03T21:45:59.629Z
Link: https://universalprior.substack.com/p/how-to-build-a-mind-neuroscience

TL;DR: The early development of the brain, from molecular biology via dynamical systems to the theory of computation. Some speculation about how everything fits together.

Apologia

There is a story about John Nash that I like a lot. John Nash is a famous mathematician who got a (

fake

) Nobel prize for his work in game theory as well as a

Hollywood adaptation

starring Russell Crowe. But (if you trust his biography) his contribution to mathematics is at least as great. He had a very curious way of working:

Nash's main mode of picking up information he deemed necessary consisted of quizzing various faculty members and fellow students. [...] Some of his best ideas came "from things learned only halfway, sometimes even wrongly, and trying to reconstruct them - even if he could not do so completely. (A Beautiful Mind, Sylvia Nasar, pg 68)

He started out with something very wrong, then went with his first (wrong) attempt at a solution to one of his colleagues, who helped make it slightly less wrong. Nash then repeated this procedure until, in the end, he had a working proof.

Apart from some hard questions about ownership (can Nash really claim that he produced the proof?) and the obvious issue that Nash turned very schizophrenic in his later years, I admire this way of working (at least for doing very hard research) and aspire to do the same. I really really want to get answers - and I don't care if I have to recruit help and produce a lot of nonsense on the way. This is, of course, just a poor fig leave of a defense for all the nonsense I am producing on this platform. Mea culpa.

1

Several perspectives on development

After

building a digital person

, writing

love letters to albatrosses

, philosophizing about

scale in academia

, and asking large language models

pointed questions

, it is time to write about something where I actually have institutional credentials, i.e. neuroscience and the development of the brain. Why is this an interesting topic?

2

While traditionally brain development is a very biology-heavy topic, there are some beautiful results and ideas that generalize far beyond biology and become relevant for people working on artificial intelligence or computations in general. And those results are not widely known, especially not outside a small community in (computational) neuroscience. This essay is written with this in mind and I have tried to reduce jargon to a min...
What LCC call numbers would you give "Rationality: AI to Zombie"?
by Stirling_Westrup2 — 2021-10-03T21:01:47.203Z

I am currently writing a book and have realized I want to cite Rationality as one of my sources and I'm not sure (a) how to cite the work (in the Chicago author-date style, or (b) how to classify it under the Library of Congress system, which is the system I use for sorting my books (even electronic ones). I'm pretty sure I can work out (a) on my own but checking for how books on rationality have previously been classified, they seem all over the map, so some help would be appreciated.

Thanks in advance for any suggestions.
The Dark Side of Cognition Hypothesis
by Cameron Berg — 2021-10-03T20:10:57.204Z

It is sometimes claimed that the ultimate, unifying goal of artificial intelligence research is to instantiate human-level cognition in a computational system (e.g., Minsky, 1961; Lake et al, 2017). If artificial general intelligence (AGI) of this sort is ever successfully developed, the consequences would be unimaginable in scope—surely, it would be the most impressive invention of our tool-making species to date.

In what follows, I’ll argue that current AI systems almost entirely lack a critical facet of human-level cognition. I’ll discuss the reasons why it is particularly hard for us to recognize—let alone instantiate—this aspect of our cognition, and I'll investigate the predictions that emerge from this sort of account. After sketching this general picture, I'll argue that the framing put forward here ultimately has the potential to unify AI engineering and AI safety as one single project.

Introduction: How We Come to Understand the Mind

At the outset, it is worth asking the extent to which cognitive science bears relevance to AI. It seems clear, however, that if the overarching goal of AI research is to capture the computations that comprise human-level cognition, then a sufficiently comprehensive understanding of human cognition seems a necessary precondition for bringing about this outcome. In other words, if we want to build mind-like computational systems, it follows that we must first understand the mind to some sufficient degree.

What, then, are the epistemological resources we have at our disposal for understanding the mind? Philosophers and cognitive scientists generally answer along the following lines: to the degree that “the mind is what the brain does,” as Minsky put it, the investigations of neuroscience and psychology allow us to better understand the mind as a standard, third-person, external, empirical object of scientific inquiry (Minsky, 1988).

But the mind is also unlike other objects of scientific inquiry in one particular way. In addition to—and in some sense prior to—objective inquiry, we can also come to understand the mind through first-person, subjective experience of having (or being) minds ourselves. For instance, our use of a standard cognitive vocabulary (i.e., speaking about beliefs, values, goals, and thoughts as such) both in scientific research and in everyday conversation does not happen because we have c...
[Book Review] Evolution of Sex
by Alex Hollow — 2021-10-03T18:54:32.258Z
Link: https://alexhollow.wordpress.com/2021/10/03/book-review-evolution-of-sex/

I was prompted to write this book review after reading Paul Christiano’s post on the evolution of sex and George’s response to Paul Christiano’s post on the evolution of sex. Those two posts are quite interesting! Paul Christiano points out that “sex slightly lowers the average [fitness] but increases the variance”, and argues that increased variance is beneficial over time. George points out that there are lots of ways to increase variance without imposing the same costs as sex. So why sex? George says “I don’t know, nobody really does, that’s why evolution is complex”. Is it really the case that no one knows why sex could evolve?

I recently got a copy of W. D. Hamilton’s book, Narrow Roads of Gene Land Volume 2: The Evolution of Sex, where he attempts to answer the question. I’ll let him introduce the topic: “The main topic of this book is why sexual ways of reproduction are so abundant, when, for natural selection to work and for progressive evolution to occur, much simpler and more efficient ways would seem to suffice”. This seems like exactly what George was asking for, so let’s practice the virtue of scholarship by reading a bunch of papers on the evolution of sex!

The Book

Just like Volume 1 (which I may write a separate book review on), this book alternates between autobiographical accounts and published papers. Each chapter first has Hamilton describing what was happening in his life and research while he was working on the paper, and then the paper. I focused on the three papers most relevant to the parasite theory of sex, out of 18 total papers.

Paper 1: Environmental fluctuation, polymorphism, and the maintenance of sex

This first paper we'll talk about begins by introducing us to some puzzles.

First puzzle: why do all the social species (humans, bees, termites) reproduce sexually? Our best theories of selection for social cooperation relies on the individuals who are cooperating being related to each other. But in clonal populations (like in aphids, armadillo polyembryony, and human monozygotic twins), we don’t observe more social cooperation than in non-clonal populations. This is counter to what we would expect from our kin selection/inclusive fitness theory.

Second puzzle: If relatedness is so important, why would someone choose to have children that are only 50% related to them when they could be having children that are 100% related to them?

Third ...
Dissolving the Experience Machine Objection
by leogao — 2021-10-03T16:56:28.312Z

The experience machine objection is often levied against utilitarian theories that depend on utility being a function of observations or brain states. The version of the argument I'm considering here is a more cruxified version that strips out a bunch of confounding factors and goes something like this: imagine you had a machine you could step into that would perfectly simulate your experience in the real world. The objection goes that since most people would feel at least slightly more willing to stay in reality than go in the machine, there's at least some value to being in the "real" world, therefore we can't accept any branch of utilitarianism that assumes utility is soley a function of observations or brain states.

I think if you accept the premise that the machine somehow magically truly simulates perfectly and indistinguishably from actual reality, in such a way that there is absolutely no way of knowing the difference between the simulation and the outside universe, then the simulated universe is essentially isomorphic to reality, and we should be fully indifferent. I'm not sure it even makes sense to say either universe is more "real", since they're literally identical in every way that matters (for the differences we can't observe even in theory, I appeal to Newton's flaming laser sword). Our intuitions here should be closer to stepping into an identical parallel universe, rather than entering a simulation.

However, I think it's not actually possible to have such a perfect experience machine, which would explain our intuition for not wanting to step inside. First, if this machine simulates reality using our knowledge of physics at the time, it's entirely possible that there are huge parts of physics you would never be able to find out about inside the machine, since you can never be 100% sure whether you really know the Theory of Everything. Second, this machine would have to be smaller than the universe in some sense, since it's part of the universe. As a result, the simulation would probably have to cut corners or reduce the size of the simulated universe substantially to compensate.

These things both impact the possible observations you can have inside the machine, which allows you to distinguish between simulation and reality, which means it's totally valid to penalize the utility of living inside a simulation by some amount depending ...
Porchfest 2021
by jefftk — 2021-10-03T14:20:03.703Z

Yesterday, I played at Somerville Porchfest with several contradance
friends. This is an annual festival that has been running since 2011,

modeled
after Ithaca's event of the same name. Bands sign up to play music
on their porches and people wander around the city listening. It was
canceled in 2020, and postponed to the Fall in 2021, so this was the
first one in two and a half years. (And I had missed the previous event
because I was

travelling.)

I had initially been
hoping to play with Kingfisher and then Whirlwind,
But my bandmates weren't available and I ended up organizing a pickup
group the day before. This ended up being super fun: unlike my
regular bandmates, these were musicians I hadn't seen since before
the pandemic (aside from my dad), and it was nice to be reminded of
how much I like them all.

I've made a lot of changes to my rhythm stage
setup over the last few years, and it was really helpful to have a
chance to run it live. You learn so much more than practicing at
home: what does the crowd like? What sounds do you prefer in the
moment? How can the interface better accommodate the demands of live
performance? Afterwards I had a nice list of tweaks and new features
to work on.

Partway through the set I noticed contradance folks informally dancing
in our neighbors' empty driveway, and offered them a mic in
case they wanted to call. A group of ~12 ended up dancing a few:

(youtube)

(youtube)

I feel somewhat conflicted on this. On one hand, I love playing for
dancing and it was great to get the opportunity to do that for the
first time in almost 2 years. I also heard from several of the dancers
that they were really excited to get to dance. And it's probably not
(much?) higher risk than what people are already doing.

On the other hand, it seems to me like this is a higher level of risk
than our local contradance community has been comfortable with. The
only organized dancing in Massachusetts I know of since before the
pandemic was at Pinewoods this summer, with outdoor pavilions, and
they closed
in mid-August, when cases were 2/3 of what they are now. Similarly,
had been booked to play an outdoor contra in early September, which
was cancelled in late August. Outdoor transmission remains very rare,
however, and I'm not sure I would have cancelled an outdoor event.

(I used to organize outdoor dances i...
A Framework of Prediction Technologies
by isaduan — 2021-10-03T10:26:34.928Z

Summary

This is the first piece of a blog post series that explores how AI prediction services affect the risks of war. It is based on my 10-week summer research project at Stanford Existential Risk Institute.

In this post, I will discuss different prediction technologies - how they might be relevant to nation-state governments and what their development trajectories might look like. The second post borrows from international relations literature to examine what implications follow for the risk of war. The third post concludes with three possible world scenarios and a few final remarks.

Some warnings of my epistemic status:

I have tried to survey different prediction technologies, but I am still by no means intimately familiar with them. I probably miss out on things and misjudge others.

I have focused on rationalist models of wars and analyzed two pathways in which prediction services could change the general probability of wars:

The amount of private information, which affects the probability of reaching peace agreements;

The cost of wars, including offense-defense balance, international institutions, and the influence of domestic audiences.

However, wars break out for various reasons and for no reason. Further research that distinguishes different types of war (e.g. great power fringe war, nuclear war, high causality war) will likely be more useful, but that is beyond the scope of this analysis.

Overall, I hope to provoke discussions, not to come to any conclusion. I make no rigorous quantifying model. I mostly just read, think, and talk to people. When I give my guess, it should be taken with the highest degree of reservation.

So, here’s an overarching guess: AI prediction services could gradually reduce the generic risk of war, not by reducing private information in peace bargaining, but by increasing the cost of war relative to peace.

A Framework of Prediction Technologies

Prediction is about generating new information from known information.[1] I understand prediction technologies as modeling tools. They structure known information in ways that generate useful new information. I outline a framework to understand different prediction technologies involving AI.[2]

[1] Data Analysis Pipelines

1.1. Classifiers and regressions

Four use-cases

Early warnings of social unrest, terrorist activities, or foreign adversaries’ military action. Governments can

...
AI Prediction Services and Risks of War
by isaduan — 2021-10-03T10:26:34.882Z

This is the second piece of a blog post series that explores how AI prediction services affect the risks of war. It is based on my 10-week summer research project at Stanford Existential Risk Institute. See my first post for a summary of my project and my epistemic status.

In the first post, I have surveyed different prediction technologies, how they might be relevant to nation-state governments, and what their development trajectories might look like. This post applies international relations literature to examine what implications follow for the risk of war. The next post will describe three possible “world order” scenarios after prediction capability takes off.

Risks of War

I choose to focus on the rationalist explanations of war within international relations literature. They are not perfect explanations, but they seem to me plausible enough to be a starting point.

Private Information

If we assume that most wars are wasteful, and there is usually a range of peaceful ways to resolve the conflicts that both sides would prefer than going to war, why do people still go to war?

One explanation is that, when pondering whether to go to war, leaders have private information about their military capabilities and the costs of war. They thus have an incentive to bluff for better deals. This makes it harder for them to reach peaceful agreements. Costly wars can be means by which nation-states screen each other’s military capability and reservation prices. Empirically, nation-states with reconnaissance satellites, which reduce private information about military capabilities and the exploitability of surprise attacks, are significantly less likely to be involved in high-casualty militarized disputes (MIDs).[1] This seems to weakly support the “private information” thesis, suggesting a possible pacifying effect of advanced prediction capability.

However, I have come to be skeptical of this mechanism. Nation-states still have mixed incentives: they want to avoid costly wars while striking better deals. Therefore, they have an incentive to create uncertainty and defy the other’s prediction capability.[2] It seems like military capabilities and costs of war are hard to predict but easy to misrepresent. Therefore, an increase in prediction capability does not necessarily lead to less private information - just as nation-states may seek new forms of secrecy and espionage...