trentmkelly/LessWrong-43k huggingface dataset:
Picasso in the Gallery of Babel
by samhealy — 2023-12-26T16:25:14.607Z

tl;dr

When (not if) generative models get so good at faking human artefacts that no human can distinguish between human and synthetic artefacts, Strange Things will happen to the way humans consume and think about and make art.

Claim 1: Models get better

Generative AI is well on the way to patching its 'tells'. Hallucination in LLMs, garbled text and polydactyly in image generators, outlying misclassifications in classifiers, etc., are being fixed in new versions. Even allowing for higher-fidelity models provoking more rigorous criticism and therefore discovery of new, subtler tells, GenAI will (perhaps quite soon) reach a point where no naked-eyed human can do better than guess at whether a given artefact is human-made or synthetic.

Claim 2: The library

In 'The Library of Babel', Jorge Luis Borges describes an unimaginably large, but still finite, library containing every possible 410-page book, which is to say, all possible permutations of 25 characters of length sufficient to cover 410 pages. The library has some interesting features:

Interesting library feature 1

It contains every book that was ever written (by person or algorithm), every book that will ever be written, and every book that will never be written.

Interesting library feature 2

It contains mostly garbage, because there are far more nonsensical than sensible ways of stringing characters together. Though it is not infinite, it is so large that the probability of discovering real or meaningful or even grammatical books by wandering through it at random is effectively zero. You could write the book you're looking for in less time than it would take to find it in the library.

Claim 3: The gallery

The conceit works just as well — maybe even better — with images[1]. The Gallery of Babel has dynamics identical to that of the Library, with, say, 8-bit RGB pixels instead of letters and 1024 x 1024-pixel images instead of books. And you could draw the image you're looking for in less time than it would take to find it in the gallery...

Claim 4: Directions

... until now.

GenAI models are directions to rooms in the library and the gallery that are not filled with garbage. This is new and important. By replacing the flat, featureless topology of the library/gallery's naïve indexing system with structured, navigable, semantic latent spaces[2], GenAI provides humans with an alternative to laboriously c...
Flagging Potentially Unfair Parenting
by jefftk — 2023-12-26T12:40:05.099Z

Recently I made a post in our

kids group
about something Lily (9y) had done that was funny, mischievous, and
also potentially embarrassing. A friend asked whether Lily knew I was
writing about her antics and said they would have felt mortified and a
bit betrayed if this had happened to them at this age.

I think it was really good they asked! While Lily knows I post this
sort of thing in the group, and this time already knew I'd posted this
one (and thought it was funny), the friend didn't know this.

Kids are in an awkward and vulnerable position, raised by people with
so much easily abused authority, and I'm happy to talk with friends
who think I might be being unfair to my kids.

I also wish raising this kind of concern were more acceptable in
general. The friend phrased their question in a softened and guarded
way and apologized in case it seemed prying, which I do think that was
a reasonable choice given the chances it would be poorly received.
This raises the cost of communicating anything, since it's more work
to phrase acceptably, and even if ideally phrased some people will
still take offense. Part of my motivation for this post is to make it
clear that I'm open to this kind of feedback, and perhaps encourage
others who are to let their friends know that.

Note that I'm not saying that society's bar for unsolicited parenting
advice is too low: I think people are often too free to offer advice
without some signal that it's wanted, and while receiving unsolicited
advice rarely bothers me, many people really don't like it. Instead,
it's specifically around noticing that someone may be being unfair to
their child where I'd love to see society move a bit in the direction
of friends speaking up, and parents taking it well when they do.
Link Collection: Impact Markets
by Saul Munn — 2023-12-26T09:01:48.815Z
Link: https://www.brasstacks.blog/lc-im/

0. Readme (or don't)

This is not a literature review. I'll vouch for links with an associated archived link, author name, and summary, but not the others.
Last updated: Dec 2023

1. Overviews

Impact Markets: The Annoying Details

(

a

), Scott Alexander

Comprehensive description of impact markets, reasoning from first principles. Extremely well-written, and a great introduction to the details of impact markets. Gets technical, but in an easy-to-follow way.

Toward Impact Markets

(

a

), Dawn Drescher

Comprehensive description of the benefits, risks (& proposed solutions), and current work on impact markets. Fairly technical.

Impact certificates and Impact Markets - Owen Cotton-Barratt

Impact Markets: A Funding Mechanism for Speculative Work

A Fresh FAQ on Impact Markets

Impact Certificates on a Blockchain

Will "impact certificates" value only impact?

Hypercerts: A new primitive for public goods funding

Crypto loves impact markets: Notes from Schelling Point Bogotá

Impact Certificates | Evan Miyazono, Head of Research at Protocol Labs | Green Pill #21

2. Subtopics

Altruistic equity allocation

(

a

), Paul Christiano

Original proposal of allocating altruistic equity. Somewhat technical.

Certificates of impact

(

a

), Paul Christiano

Original proposal of impact certificates. Somewhat out-of-date with current work. Somewhat technical.

Impact markets may incentivize predictably net-negative projects

(

a

), Ofer and Owen Cotton-Barratt

Describes how impact markets can incentivize funding some types of projects that have clear negative expected impact. Argues that impact markets may exhibit the behavior of those types of projects, and therefore that impact markets should never be funded on impact markets.

3. Implementations

Last updated: December 2023

Manifund

*, run by

Rachel

&

Austin

all of

Manifund's internal docs

are publicly available
includes the

ACX Forecasting Impact Mini-Grants

round, the

Open Philanthropy AI Worldviews Contest

, and the leftovers of

ACX Grants 2024

AI Safety GiveWiki

, previously Impact Markets, [?possibly previously something else]

Hypercerts

Gitcoin

(not an impact market, but they do retroactive quadratic public goods funding, which is pretty damn close)

NPX Advisors

(also in the "close-to-an-impact-market-but-not-quite" category. NPX recently

shut down

.)

The Impact Purchase

Experiment in Retroactive Funding: An EA Forum Prize Contest

Plan for Impact Certificate MVP

4. Related

...
How Emergency Medicine Solves the Alignment Problem
by StrivingForLegibility — 2023-12-26T05:24:35.579Z

Emergency medical technicians (EMTs) are not licensed to practice medicine. An EMT license instead grants the authority to perform a specific set of interventions, in specific situations, on behalf of a medical director. The field of emergency medical services (EMS) faces many principal-agent problems that are analogous to the principal-agent problem of designing an intelligent system to act autonomously on your behalf. And many of the solutions EMS uses can be adapted for AI alignment.

Separate Policy Search From Policy Implementation

If you were to look inside an agent, you would find one piece responsible for considering which policy to implement, and another piece responsible for carrying it out. In EMS, these concerns are separated to different systems. There are several enormous bureaucracies dedicated to defining the statutes, regulations, certification requirements, licensing requirements, and protocols which EMTs must follow. An EMT isn't responsible for gathering data, evaluating the effectiveness of different interventions, and deciding what intervention is appropriate for a given situation. An EMT is responsible for learning the rules they must follow, and following them.

A medical protocol is basically an if-then set of rules for deciding what intervention to perform, if any. If you happen to live in Berkeley California, here are the EMS documents for Alameda County. If you click through to the 2024 Alameda County EMS Field Manual, under Field Assessment & Treatment Protocols, you'll find a 186 page book describing what actions EMS providers are to take in different situations.

As a programmer, seeing all these flowcharts is extremely satisfying. A flowchart is the first step towards automation. And in fact many aspects of emergency medicine have already been automated. An automated external defibrillator (AED) measures a patient's heart rhythm and automatically evaluates whether they meet the indications for defibrillation. A typical AED has two buttons on it: "On/Off" and "Everyone is clear, go ahead and shock." A ventilator ventilates a patient that isn't breathing adequately, according to parameters set by an EMS provider.

A network router isn't a consequentialist agent. It isn't handed a criteria for evaluating the consequences of different ways it could route each packet, and then empowered to choose a policy which optimizes the consequence...
Rationality outreach vs. rationality teaching
by Lenmar — 2023-12-26T00:37:39.240Z

Epistemic status: Exploratory and in testing

I'm reasonably confident there are a lot of smart/curious people who would like to learn rationality, that is, how to think better and correlate the contents of your mind better to reality.

Framing rationalist outreach as establishing branches of the LW-community, Rationalist Clubs, Effective Altruist Meetups, etc. may be effective in growing the community to some extent, but anyone who doesn't already think of themselves as a rationalist will come only from whatever tribe the local branch seems to mainly consist of, whether that's weird engineers, animal-welfare vegans, crypto nuts, secular Buddhists, etc.

And then of course there are the cases where the assumption already exists that Effective Altruism is the thing Sam Bankman-Fried pretended he did before he stole all that money, and LessWrong is that place where they talk about how AI will become Evil Vaguely Judeo-Christian God who Tortures Us in the Future.

However, I am moving towards the conclusion that if detached from the tribal baggage, the majority of general-purpose debiasing tech/utilitarianism is not that inherently difficult to teach to smart/curious/motivated people, even from non-LW-median tribes. It is not outrageously Deep Magic to consider that students learn arbitrary parroting instead of knowledge, and continue to think like that after they graduate, or people use their moral philosophies to feel like they agree and associate with their tribes, and then go from there.

An old success case I found on my first search. I have had a pretty decent success rate with leftist-tribe friends and acquaintances as well and plan to continue testing. Of course, you need to have the norm of having genuine/abstract conversations first, but that's fun and useful to establish anyway.

(And any large-scale societal rise in the sanity waterline will presumably involve normalizing concepts outside the community, not expanding the community to that scale, so it's a good time to start.)
Exploring the Residual Stream of Transformers for Mechanistic Interpretability — Explained
by Zeping Yu — 2023-12-26T00:36:50.326Z

— by Zeping Yu, Dec 24, 2023

In this post, I present the paper “Exploring the Residual Stream of Transformers”. We aim to locate important parameters containing knowledge for next word prediction, and find the mechanism of the model to merge the knowledge into the final embedding.

For sentence “The capital of France is” -> “Paris”, we find the knowledge is stored in both attention layers and FFN layers. Some attention subvalues provide knowledge “Paris is related to France”. Some FFN subvalues provide knowledge “Paris is a capital”, which are mainly activated by attention subvalues related to “capitals/cities”.

Overall, our contributions are as following. First, we explore the distribution change of residual connections in vocabulary space, and find the distribution change is caused by a direct addition function on before-softmax values. Second, we prove using log probability increase as contribution score could help locate important subvalues. Third, we find attention/FFN subvalues on previous layers are direct “queries” to activate upper FFN subvalues by inner products. Last, we utilize our proposed method on experiments and a case study, which can verify our findings.

Background: Residual Stream Studies on Attention Layers

The residual stream of transformers has been studied in many works and have been proved useful for mechanistic interpretability. The main characteristic of residual stream is: both the attention and MLP layers each “read” their input from the residual stream (by performing a linear projection), and then “write” their result to the residual stream by adding a linear projection back in. (“A Mathematical Framework for Transformer Circuits”)

Residual stream of transformer. (Figure from: A Mathematical Framework for Transformer Circuits)

The “A Mathematical Framework for Transformer Circuits” paper focus on attention layers. They build a one-layer attention-only transformer, which can learn skip-trigram relationships like [b] … [a] -> [b] (e.g. [perfect] … [are] -> [perfect]). They also find a two-layer attention-only transformer can learn induction heads like [a][b] … [a] -> [b] (e.g. [Pot] [ters] … [Pot] -> [ters]).

Based on this finding, the “In-context Learning and Induction Heads” paper explore the induction heads. They provide evidence to support the hypothesis: induction heads might constitute the mechanism ...
Anki setup best practices?
by Sinclair Chen — 2023-12-25T22:34:34.639Z

I'm currently trying to use Anki to crash course japanese vocab before a trip in a few weeks. I feel like the defaults set a card too far into the future after I press Easy even if I had already gotten it wrong several times ahead of that.
Are there like a set of extensions that you're supposed or a settings list or something? I remember Anki being better than this when I was using it in high school years ago...
Why does expected utility matter?
by Marco Discendenti — 2023-12-25T14:47:46.656Z

In the context of decision making under uncertainty we consider the strategy of maximizing the expected monetary revenue and expected utility; we provide an argument to show that under certain hypothesis it is rational to maximize expected monetary revenue, then we show that the argument doesn't apply to expected utility. We are left with the question about how do we justify the rationality of the strategy of maximizing expected utility.

Expected monetary revenue

Suppose you have to choose one of two games A and B with an expected economic return of 1$ and 2$ respectively, which have a certain probability distribution.

If you play many times, say N, the law of large numbers and the central limit theorem might become relevant, and your probability distribution for the repetition of A and B will have their masses more and more sharply separated around N and 2N respectively.

At this point, it is clear that it is better for you to play B many times than to play A many times. You can predict this in advance by calculating the expected winnings of A and B. So assuming you can make "lots" of choices between A and B, you must prefer the one with the higher expected profit. But what if you can only make one choice? If the distributions of A and B overlap, does the expected profit still matter?

Even if you only have to make this particular choice once, it could be one of a long sequence of many different choices, not always between A and B, but between other sets of options. Even if the choices are different, we can still manage to take advantage of the LLN and the central limit theorem. Suppose we have a large number of choices between two games:

time 0: choose between games A0 and B0
time 1: choose between A1 and B1
time 2: ...
...
time N: choose between AN and BN

We can hope that again if you always choose the game with the higher expected return the statistical randomness will be increasingly irrelevant over time for large N and you will store a larger amount of money compared to a person with a different strategy.

In order for this hope to actually happen we need that the incomes of the games Ak and Bk are uniformly bounded and "small enough" compared to N. This could not happen if for example:

a single game has infinite expectation like in the St. Petersburg game;
a single game has such a big expected

...
Freeze Dried Raspberry Truffles
by jefftk — 2023-12-25T14:10:06.336Z

I've historically found a lot of "raspberry flavored" things pretty
disappointing. Raspberry lollypops? Raspberry slush? Anything with
raspberry extract? Even raspberry jam is a lot worse than just

squishing up some frozen
raspberries because a lot of the best raspberry flavors are lost
to in heating. But freeze-dried raspberries totally change this!

While I normally find truffles too much trouble and instead just eat ganache with a spoon, the holidays
are a good time for food that requires a bit more investment. The
kids and I made some chocolate raspberry truffles. For the filling we
did:

300g dark chocolate

250g heavy cream

50g freeze-dried raspberry powder

Mix the chocolate and cream and microwave, stirring until it's melted.
Then add the raspberry powder, mix, and put it in the fridge to
harden. Once it's fully cooled down it will have the consistency of
play-dough and you can roll it into balls or (better!) wrap it around
chunks of frozen raspberries. You can't get it to stick to the
raspberry, since chocolate is hydrophobic, but it will stick to
itself. Then it's back to the fridge or freezer until it's hard
again, and later you dip it into melted dark chocolate.

The raspberry powder is key here because it gives you good raspberry
flavor without adding any water, where water or juice in your filling
will keep the outer layer from coating it properly.

Refrigerate until you're ~ready to eat them, but they'll taste better
if you take them out of the fridge at least an hour before so they can
come back up to room temperature.
Pornographic and semi-pornographic ads on mainstream websites as an instance of the AI alignment problem?
by greenrd — 2023-12-25T13:19:57.026Z

In recent days, I have been served ads on Twitter that appeared to be advertising some kind of dating site. While technically I wouldn't say that they crossed the line into being actually pornographic, they did skirt very close to the line. The advertiser might argue that if the ads were actually advertising lingerie, then it would have been acceptable, but my judgement would be that the combination of the text of the ads, the clothing in the images, the model or AI-generated avatar's poses, and the service being advertised, collectively made them unacceptable. But that's just my opinion.

Then yesterday I witnessed an escalation - a similar Twitter ad which, shockingly, actually appeared to be extremely lascivious and pornographic, including what appeared to be partially-visible female genitalia. As a reminder, Twitter ads are served randomly and without warning, in amidst organic tweets and replies. In theory, anyone could have received these ads - 13-year old users, grandmothers, even people using Twitter live while giving a TED talk. While explicit pornographic ads are routinely shown on pornographic websites, Twitter users do not expect to be served them on Twitter.[1]

Update (26 Dec): I have just been served a new version of this explicit ad, with the same picture or a very similar picture. I have uploaded a censored version of the ad here, and corrected a probable minor error in this post below.

Now, as an ordinary Twitter user, I obviously can't prove that this particular phenomenon is due to AI - or that, even if AI was in some way involved in the production of this ad, that the AI was causally responsible for it crossing the line into actual porn. However, I would make two points:

Even if this was in fact a case of a human accidentally(!) or deliberately producing and submitting an ad that crossed the line into actual pornography, AI technology has developed to such a level that I believe that within a few months or years, we could see imperfectly-aligned AIs - especially if owned and operated by cybercriminals without many moral scruples - making similar errors in the real world. That is, even if this wasn't caused by an AI, it might as well have been.
In the case of this particular problem - pornographic and semi-pornographic ads being submitted through "self-service" ad platforms like Twitter's - I don't believe that the relevant actors need to waste

...
Defense Against The Dark Arts: An Introduction
by Lyrongolem — 2023-12-25T06:36:06.278Z

Epistemic status: I’m a non expert analyzing personal anecdotes and lower level political issues, take with a grain of salt.

Edit: The post is specifically about the Dark Arts as we see them in politics, not in intra or interpersonal relations. Thank you AnniePosting for recommending this specification.

Context

(for those who haven’t read, I highly recommend checking out my dialogue with lsusr linked above, it’s a relatively short read and introduces some important context to what I’m about to go over. But I will provide a shorter summary in this post just in case.)

Previously in the Dark Arts dialogue, I introduced some basic information on my encounters with the Dark Arts in public forum debate, how I used it to my advantage, and my analysis on what makes it work. However, after interacting with some commenters, I realized there was still a ton that people might be interested in. One of which being how to properly defend against the Dark Arts in real world situations.

(By the way, thank you Richard Horvath for your valuable comments, they are what inspired me to write up this post

I think this is an important addition to the site. There had been articles before about the "dark" side/arts, but I think this is the first one where the examples are not thought experiments and abstractions, but actual real world experience from an actual user.

It is helpful for understanding politics.

I think something along the lines of "Defense Against the Dark Arts" with actionable steps on recognizing and defusing them (and how to practice these) would be great. If you feel like you have the energy and time, more articles on offensive usage (practice) and on theoretical background (how to connect your practical experience to existing LW concepts/memes) would be also nice. But I think the first one (defense) would be the most useful for LW readers.

On that note, I think it may be helpful for me to expand a bit on ‘The Dark Arts’ as I see them.

For me, the Dark Arts are more than just ‘thought experiments and abstractions’, as Horvath says. They are something that I come into contact with more or less every day, not just in debate. I see it everywhere, in politics, in everyday interactions, and in patterns of thinking I see from people around me. This may seem incongruous to many people, and I think part of the reason fo...