trentmkelly/LessWrong-43k huggingface dataset:
Why is "Argument Mapping" Not More Common in EA/Rationality (And What Objections Should I Address in a Post on the Topic?)
by  — 2022-12-23T21:58:18.870Z

I recently published an article in the Georgetown Security Studies Review (GSSR) on the use of "argument management systems" (e.g., Kialo) for the complex debates that arise in fields where it's often impractical to resolve disagreements through standard empirical methods (e.g., RCTs). I've long been confused why this method of discussion is not more widely supported in EA and Rationalist circles, and am considering adapting my GSSR article to AI policy/safety research (or Longtermism more generally, which various people have criticized for being too speculative/theoretical rather than being based on empirical tests) and posting it here. However, before doing that I would love to get a sense of people's reasons for skepticism or apathy towards such methods, so I could potentially address them in the post.

For what it's worth, I have seen Leverage Research's report on the topic, and I am aware of the criticism that "argument mapping" (in some formats) is overly formal and too complicated. (I plan to respond to these points)

In short, I expect my argument to be fairly similar to what I laid out in my GSSR article: the way we currently present arguments (i.e., predominantly through prose/paragraph text) seems rife with points of failure and inefficiencies, especially given how debates are often not linear but rather branch and have cross-cutting points. In fields like international relations and peace/conflict studies I've repeatedly encountered instances where people fail to (seriously) address existing counterarguments, and more generally it is hard for audiences to determine who has or hasn't addressed counterarguments. In contrast, I think that making one's arguments more explicit and keeping track of the arguments in a more-searchable and more-permanent format than "memory" or prose would help to mitigate some of these problems.

To me, better methods of argumentation seems like a natural extension of norms that promote statistics/experimental methods in science, but thus far I've found the EA/Rationalist communities fairly lukewarm towards the ideas (even if they are more receptive on average than the general public).
The Fear [Fiction]
by Yitz — 2022-12-23T21:21:00.535Z

Be afraid. Be very afraid.

The sound rings out from the top of the lighthouse. It echoes emptily on walls lined with paintings, all identical in their monotony. The same visuals endlessly reproduced, the same words over and over.

The paintings are all of the same thing: a small, unremarkable landscape. No birds in the sky or animals in the grass, nothing but trees and water. An image that had been copied so many times it had become dull and meaningless. The man at the top of the lighthouse had painted them himself; each one was painstakingly crafted, each brushstroke precise and exact. But now, he was no longer alone.

The puppet had come to take his place. A wooden figure crafted with such skill that it seemed almost alive. Its eyes are painted on, its mouth smiling blandly. It dances around the room, seemingly of its own volition, its arms swaying as if conducting music unheard.

It picks up the paintbrush and begins to paint, to paint with speed and accuracy unlike any human artist could ever achieve. The paintings that come from the puppet's hand are vibrant and full of life; they bring color and emotion to the dull walls of the lighthouse – something that had been lost in the man's endless repetitions. It is, undeniably, a masterful artist. The paintings are sold, and snatched up quickly; people come from far away to purchase the New Master's work, and the old man becomes rich.

Now he stands at the edge of the lighthouse, where the puppet tells him to close his eyes and count to three. When he opens them, the world around him has changed. No trees, no grass, no ocean, no lighthouse, no wind, just endless rows of paintings, all different, all beautiful, stretching like gravestones beyond the horizon.

The puppet stands in the middle of it all and laughs, its voice echoing in the emptiness. "Be afraid," it says, "be very afraid."

But the old man cannot tear his gaze away from the paintings, mesmerized by their beauty. He knows that he will never be able to recreate them himself, or even view a fraction of the work in his lifetime, but that hardly matters now. What the puppet has done is incredible.

He is not afraid — cannot be afraid, even as terror lingers in the air like a fog — for alas, he has been cursed with the gift of an artist's eye.

This isn't the place you think.
To err is neural: select logs with ChatGPT
by VipulNaik — 2022-12-23T20:26:41.702Z

This post includes some portions of my logs with ChatGPT, where I
asked it basic arithmetic questions and it made several mistakes. I
have used bold for cases where it made incorrect statements, and
italics for cases where it made correct statements that directly
contradicted nearby incorrect statements. The actual conversation is
in block quotes, while my after-the-fact commentary on it is not in
block quotes. I generally include my comment on each part of the
conversation right after that part ends.

My questions were split across two sessions; for each section, I've
indicated whether it was part of session 1 or session 2. The bulk of
both sessions was on Wednesday 2022-12-21; one portion of session 1
was on 2022-12-22 as the session got stuck on 2022-12-21. All of these
used the ChatGPT December 15, 2022 version.

Disclaimer: Since ChatGPT said a lot in each response, and I have
limited time to proofread, it's quite possible that I've missed some
mistakes it made, or failed to use bold or italics at relevant
places. If you notice any such instances and let me know in the
comments, I'll fix. Thanks!

Product of negative numbers (session 1; 2022-12-21)

Vipul: What is an example of an odd negative number?

ChatGPT: An example of an odd negative number is -5. Negative
numbers are numbers that are less than zero and are represented by a
negative sign (-). Odd numbers are integers that are not divisible
by 2. -5 is an odd negative number because it is less than zero and
is not divisible by 2. Some other examples of odd negative numbers
include -3, -7, and -9.

So far, so great!

Vipul: What is an example where two negative numbers multiply to
give a negative number?

ChatGPT: An example where two negative numbers multiply to give a
negative number is (-3) x (-4) = 12. When two negative numbers
are multiplied together, the result is a positive number. This is
because the negative signs cancel each other out. For example,
(-3) x (-4) = (-1) x 4 = 4, which is a positive number. So, in
this case, the product of two negative numbers is a positive
number.

On the other hand, if we have three negative numbers, the result
will be a negative number. For example, (-3) x (-4) x (-5) = (-1)
x (-1) x 20 = (-1) x 20 = -20, which is a negative number.

It's important to remember that when multiplying or dividing
numbers, the order in which the operations are performed is
impo

...
Two Truths and a Prediction Market
by Screwtape — 2022-12-23T18:52:10.734Z

Summary: Play some guessing games using a prediction market, to get used to how those markets work.

Tags: Small to Medium, Repeatable

Purpose: Prediction markets are a relatively popular tool in rationalist spaces, but uncommon in the general population. This is an opportunity to practice with a simple, short term prediction market.

Materials: You need the prediction market board. Robin Hanson described how he built his- fifty or so plastic bins arranged in a grid- and I made mine out of a pizza box. At the time of this writing there also exists a digital version created by Hamish Todd at https://github.com/hamishtodd1/msb , though it probably needs some updates to work again. The pizza box construction steps are detailed in the Building A Pizza Board section.

Announcement Text:

Want to play some games with a prediction market twist?

In the early nineties, Robin Hanson developed a board game to represent a prediction market. Called "Murder She Bet" it took place alongside a murder mystery, with participants using monopoly money to predict whodunnit. A full description of the original can be found at Hanson's site. Well, I don't watch many murder mysteries, but I do like board games, so on this occasion we're going to misuse this mockery of financial instruments!

We'll have Two Truths and a Prediction Market, where participants can work out which of three statements about their fellows is a lie. We'll have speed games, where two players will play a one-on-one board game while the audience predicts who will win- but the players are allowed to look at the market. We'll have a short trivia game, and see if the market can hone in on the answers to oddball questions about the world. We may have other uses for it- if you show up with an idea for what to do with a paper-powered prediction market, I'm open to suggestions!

Description:

The three main things you need to do during this meetup are 1. Explain how the board works and give people starting money, 2. Set up each game you’ll play with the market, 3. Act as the bank: make change, pay out the winners, reset the board between markets.

1. How The Board Works:

The basic rules:

The board is arranged into columns (top to bottom) and rows (side to side.) Each column is for a different outcome. Each row is for a different price. Each slot represents a particular price for a particular outcome.

A slot always contai...
ChatGPT understands, but largely does not generate Spanglish (and other code-mixed) text
by Milan W — 2022-12-23T17:40:59.862Z

Content format: Commented chat screenshots, and then some thoughts on their implications.

Epistemic status: Exploratory

Introduction

Code-mixing is the ad-hoc mixing of two or more linguistic varieties (such as languages or dialects) in the same communicative instance. An example of code-mixing would be a sentence written with words both in Spanish and in English, or with novel words made by combining Spanish and English roots, suffixes or prefixes.

Here, I document a series of experiments designed to test ChatGPT's capabilities for understanding and generating code-mixed text. I tested it with:

English + Spanish (Spanglish)
English + French (Franglais)
English + Spanish + French (Frenspanglish)

The first two are well known phenomena in multilingual communities such as those in Quebec and the southwestern US, while the third is quite obscure and as far as I know does not occur naturally on a large scale. Franglais is usually regarded as the insertion of English features into French, while Spanglish as a more symmetrical phenomenon.

This is why I decided to prompt for Franglais understanding in French, and for Spanglish and Frenspanglish understanding in English. When prompting ChatGPT to translate into a code-mixed language, I prompt in the same language the text to be translated is given.

I find that ChatGPT exhibits impressive abilities to understand text written in such code-mixes. However, despite repeated attempts at prompt engineering, I have not been able to make ChatGPT generate proper code-mixed text.

Understanding code-mixed text

Understanding Spanglish (English + Spanish)

ChatGPT understands Spanglish, and can translate from it into English.

Understanding Franglais (English + French)

ChatGPT understands Franglais, and can translate from it into French.

Understanding Frenspanglish (English + Spanish + French)

This took a bit of prompt engineering.

ChatGPT can understand the meaning of this mixed expression with two apostrophic contractions. However, it wrongfully explains that it is a mixture of only French and English, even replacing the French-Spanish fragment "J'puedo" with the French-only "J'peux".

Failing to generate code-mixed text

Note: All of these experiments were performed on a chat instance where ChatGPT had already received an explanation of the relevant code-mix.

Failing to generate Spanglish (English + Spanish)

Asked to translate an Engli

...
On sincerity
by Joe Carlsmith — 2022-12-23T17:13:09.478Z

Cross-posted from my website. Audio version here, or search "Joe Carlsmith Audio" on your podcast app.

Nearby is the country they call life.

You will know it by its seriousness.

- Rilke

1. Introduction

There’s a thing I call “sincerity” that matters a lot to me. In particular, it’s core to how I hope to orient towards the world. And it’s one of the main things I look for in people and communities.

My hope, in this essay, is to bring what I mean by sincerity into clearer view. But the term has a fairly rich set of associations for me, which I’m not sure will ultimately admit of a cleanly unified analysis. I start by discussing five of these associations. Sincerity seems to me closely related to:

Something like truth-seeking (“scout-mindset”), but for agency as a whole rather than just beliefs
“Not playing pretend”
One’s different motivations working in harmony
“Seriousness”
(Less confidently) Some stuff about non-ego/altruism/goodness

I then discuss whether we can unify these five associations under a single principle. I offer two possibilities for doing so. The first is conceptual, and takes scout-mindset-but-for-agency as the core thing. The second is empirical, and takes “not playing pretend” as the core thing. I’m not sure either is adequate.

I also discuss a few of the many failure modes nearby to sincerity: self-deception, extremism, over-confidence, “the-normal-rules-don’t-apply-to-me”-ness, sanctimony, over-concern with sincerity itself (both in others, and in yourself), and that most straightforward and fearsome of failures: just plain picking the wrong actions, even if you did the other stuff right.

Thanks to Katja Grace, Ketan Ramakrishnan, Claire Zabel, Samira Nedungadi, Cate Hall, Bill Zito, Nick Whitaker, and Dwarkesh Patel for discussion.

“Jeanne d’Arc écoutant les voix,” by Thirion, image source here.

2. Initial gestures

There is a true person of no rank who is always coming and going from the portals of your face.

Who is that true person of no rank?

- Linji

Before jumping into the five associations above, I want to make a few more general gestures at the thing I have in mind when I say “sincerity,” to get us at least somewhat oriented. First, I’ll note and comment on some related words. Then, I’ll describe some related feelings.

2.1 Some words

The word “sincerity,” here, isn’t perfect. In particular, I think it over-evokes something directly social – e.g. w...
Epigenetics of the mammalian germline
by Metacelsus — 2022-12-23T15:21:37.032Z
Link: https://denovo.substack.com/p/epigenetics-of-the-mammalian-germline

This is a crosspost from my Substack and is is Part 2 of my epigenetics post series. Part 1 is here.

The year was 1984, and scientists were wondering: why is sperm, and not just eggs, necessary to make a baby mouse?

In non-mammalian species (notably several reptiles), unfertilized eggs can activate as if they were fertilized, copy their genomes, and produce viable offspring. This process is known as parthenogenesis, and scientists could induce it experimentally in mouse eggs, with one research group even reporting live offspring. But as other researchers tried it, it became clear that although parthenogenetic embryos developed to the blastocyst stage, they did not successfully grow to term, and in particular showed defects in the extra-embryonic tissues such as the placenta.

Three labs, led by Azim Surani at Cambridge, Davor Solter at the Wistar Institute, and Robin Lovell-Badge at University College London, were hard at work trying to understand why.

One hypothesis was that there was some necessary component missing from the egg cytoplasm which the sperm provided. But this was quickly ruled out, because when both the male and the female pronucleus were transplanted from normally fertilized eggs into enucleated recipient eggs, the resulting embryos developed normally.[1] So what was going on?

Diagram of pronuclei during fertilization, from Gray’s Anatomy.

Around the same time, the Surani and Solter labs performed basically the same experiment:

Take fertilized eggs and remove either the male or the female pronucleus using a tiny glass needle.[2]
Transplant a new male or female pronucleus into each egg, generating eggs with two male pronuclei, two female pronuclei, or one of each (as a control).
See how the resulting embryos developed.

Both of these research groups came to the same conclusion: the only embryos that developed to term were ones containing both a male and a female pronucleus.[3] These experiments were performed using inbred mouse lines, in which an X-bearing sperm is genetically identical to an egg. So, proper development required some characteristic that was associated with the male-derived and female-derived chromosomes, but was not related to their sequences. Hmm, sounds like epigenetics.

This result led Azim Surani to remark:

Our studies suggest that specific imprinting of the genome during gametogenesis is essential for full-term development i

...
Boston Solstice Songs
by jefftk — 2022-12-23T13:00:02.527Z

I was curious what songs we've done for our

secular solstices in Boston,
and wanted to put them together for thinking about what to do in the
future. I've sorted these in roughly descending order of how often I
think we should do probably them in the future. This is a combination
of:

How well does it work for group singing? Is it something you
can pick up when hearing for the first time, or instead something
widely known? Do the lyrics flow, is the melody interesting, does it
have a good rhythm?

How well does it fit with general secular solstice themes? How
often do I expect it to fit with the particular theme we use in a
year? Do I endorse its general message?

For example, "Somebody Will" is musically very interesting, almost
always pretty on theme, covers an important aspect of cooperation, but
then loses a ton of points for how the melody is super tricky.
"Christmas in the Trenches" is musically beautiful, very moving, and
intensely sad, but while it has strong themes of international
humanity that's not enough to make it be on theme most
years. "Brighter than Today" is the one song all the Ray-derived
secular solstices do every year, so we should definitely keep
doing that.

Song

2013

2015

2016

2017

2018

2019

2022

Brighter Than Today

x

x

x

x

x

x

x

7

Old Devil Time

x

x

x

x

4

Level Up

x

1

Hymn of Breaking Strain

x

x

x

x

x

5

The Circle

x

1

Unison in Harmony

x

1

Lean On Me

x

x

x

3

Uplift

x

x

x

x

x

x

6

Somebody Will

x

x

x

3

The X Days of X-Risk

x

x

x

3

We Will All Go Together When We Go

x

x

x

3

Always Look on the Bright Side of Life

x

x

2

God Wrote the World

x

1

Here and Now

x

1

The Ballad of Smallpox Gone

...
Are there any reliable CAPTCHAs? Competition for CAPTCHA ideas that AIs can’t solve.
by MrThink — 2022-12-23T12:52:24.621Z

Background

As artificial intelligence continues to advance, it becomes increasingly important to find ways to verify the humanity of individuals online. One common method for doing so is through the use of CAPTCHAs, which are designed to be easily solved by humans but difficult for AI models to crack. However, many CAPTCHAs can be easily defeated by readily available models, even without fine-tuning.

Currently, most CAPTCHAs looks like this:

Or this:

These CAPTCHAS can be easily solved using readily available models, although they might require some fine tuning.

For example I solved several different types of text captchas using this model out of the box:

https://huggingface.co/microsoft/trocr-large-printed

And the image CAPTCHA above I solved using this model, without any fine tuning:

https://huggingface.co/openai/clip-vit-large-patch14

Competition

In light of this, we are launching a competition to generate new ideas for CAPTCHAs that are both easy for humans to solve and difficult for AI models to crack. To be considered a winner, an idea must meet the following criteria:

At least 90% of internet users could solve it within one minute.
We would not expect a malicious AI developer to be able to create an AI that can solve the CAPTCHA given a few days, and unlimited tries (meaning that if it’s a text CAPTCHA, the developer could load the CAPTCHA a thousand times to create a dataset.)

The prize for this competition is the honor of having your idea considered as a reliable CAPTCHA solution, which CHATgpt couldn’t do when I prompted it. Therefore, it would ironically also serve as strong evidence that you are human. Do you have an idea that meets these criteria? If so, I strongly encourage you to write a comment with your idea.
"Search" is dead. What is the new paradigm?
by Shmi — 2022-12-23T10:33:35.596Z

For over two decades internet search (which has been largely synonymous with Google for most of that time) has been the main way to augment the human brain with the digital information outside of it. First the virtual assistants and now LLMs are challenging and changing the way people access, process and use the data. It is becoming more like a true capability augmentation, though still clunky and unreliable, but improving at a dizzying rate. Specifically, it is becoming amazingly easier to create, rather than consume, though consumption is greatly facilitated, too.

Assuming we are still ways away from the prophesied AI doom, what might be a better name for the way people interact with information?
Article Review: Discovering Latent Knowledge (Burns, Ye, et al)
by Robert_AIZI — 2022-12-22T18:16:05.161Z
Link: https://aizi.substack.com/p/article-review-discovering-latent

[This is a cross-post from my blog at aizi.substack.com.]

Large Language Models are hell-demons summoned from the data dimension. You feed a giant machine approximately everything ever written, and suddenly it can

have a conversation

,

run a virtual machine

, and “knows” things. There’s a serious chance GPT-but-slightly-larger will be AGI, yet we don’t understand what it “knows” or “thinks” or

how to make them safe

.

The core problem is that LLMs aren’t designed to know facts. They’re next-token-predictors, so to the extent they “know” that Paris is the capital of France, they’re just predicting that the next word after “What is the capital of France?” is “Paris”. Despite this, LLMs produce statements which are correct, internally consistent, and interconnected, to an extent similar to human knowledge. Setting aside philosophical questions, it’s nonetheless useful to think of LLMs as “knowing” facts, which they sometimes communicate in their statements.

ChatGPT can produce statements like “The capital of France is Paris”, “London is the capital of the United Kingdom”, and “[the United Kingdom] is a separate country from France”. These are all knowledge-like, even if ChatGPT doesn’t “know” anything in the same way humans do.

But while there’s some overlap between “statements the LLM produces” and “facts the LLM knows”, we have no guarantee that either set contains the other. By design, even if the LLM “knows” some fact (by producing statements as if that fact were true), it will deny that fact if the denial is a more probable completion. In the extreme, you get an AI

insisting in Danish that it doesn’t speak any Danish

. And this problem will extend to future AI with objectives beyond next-token prediction. Is your AI saying something because it thinks it’s true, or because that seems like the most probably completion, or because that’s what

maximizes its score in the game its playing

?

Thus the

Eliciting Latent Knowledge (ELK) problem

: how do you identify what an AI “knows” if its output selection optimizes something besides communicating what it knows[1]?

From my perspective, a solution to ELK would dwarf all other AI safety accomplishments in to date, and approximately solves AI safety. Eliciting the latent knowledge “are you planning to kill us?” goes a long way, and solving ELK could also enable other safety strategies like

building microscopes instead of agents

.

So...