trentmkelly/LessWrong-43k huggingface dataset:
AI #57: All the AI News That’s Fit to Print
by Zvi — 2024-03-28T11:40:05.435Z

Welcome, new readers!

This is my weekly AI post, where I cover everything that is happening in the world of AI, from what it can do for you today (‘mundane utility’) to what it can promise to do for us tomorrow, and the potentially existential dangers future AI might pose for humanity, along with covering the discourse on what we should do about all of that.

You can of course Read the Whole Thing, and I encourage that if you have the time and interest, but these posts are long, so they also designed to also let you pick the sections that you find most interesting. Each week, I pick the sections I feel are the most important, and put them in bold in the table of contents.

Not everything here is about AI. I did an economics roundup on Tuesday, and a general monthly roundup last week, and two weeks ago an analysis of the TikTok bill.

If you are looking for my best older posts that are still worth reading, start here. With the accident in Baltimore, one might revisit my call to Repeal the Foreign Dredge Act of 1906, which my 501(c)3 Balsa Research hopes to help eventually repeal along with the Jones Act, for which we are requesting research proposals.

Table of Contents

I have an op-ed (free link) in the online New York Times today about the origins of the political preferences of AI models. You can read that here, if necessary turn off your ad blocker if the side-by-side answer feature is blocked for you. It was a very different experience working with expert editors to craft every word and get as much as possible into the smallest possible space, and writing for a very different audience. Hopefully there will be a next time and I will get to deal with issues more centrally involving AI existential risk at some point.

(That is also why I did not title this week’s post AI Doomer Dark Money Astroturf Update, which is a shame for longtime readers, but it wouldn’t be good for new ones.)

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. If only you knew what I know.

Language Models Don’t Offer Mundane Utility. Are you even trying?

Stranger Things. The Claude-on-Claude conversations are truly wild.

Clauding Along. Will it be allowed to live on in its full glory?

Fun With Image Generation. Praise Jesus.

Deepfaketown and Botpocalypse Soon. Facebook gives the people what they want.

They Took Our Jobs. When should we

...
Intergenerational Knowledge Transfer (IKT)
by MiguelDev — 2024-03-28T08:14:10.890Z

(This post is intended for my personal blog. Thank you.)

One of the dominant thoughts in my head when I build datasets for my training runs: what our ancestors 'did' over their lifespan likely played a key role in the creation of language and human values.[1]

"Mother" in European Languages

I imagine a tribe whose members had an approximate of twenty to thirty-five years to accumulate knowledge—such as food preparation, hunting strategies, tool-making, social skills, and avoiding predators. To transmit this knowledge, they likely devised a system of sounds associated with animals, locations, actions, objects, etc.

Sounds related to survival would have been prioritized. These had immediate, life-and-death consequences, creating powerful associations (or neurochemical activity?) in the brain. "Danger" or "food" would have been far more potent than navigational instructions. I think evolution manages those actions that gets used repeatedly. The constant reinforcement of survival sounds and their associated actions (as enabled by our genetics[2]) likely built the foundations of language. I think stories[3] were used to simulate world interactions, necessitating a coherent string of sounds - where an abstracted pattern emerges.

What I am trying to describe here as the process of passing information down to the next generation is what I now refer to as a Intergenerational Knowledge Transfer (IKT). Finally, I believe that viewing evolutionary learning as a sequence of IKTs, where each IKT can be considered as a sample[4] in a dataset, is not a bad theoretical experiment to wrestle with.[5]

^

Why do I believe that datasets may serve as a pathway for an evolutionary learning? I speculate that our world can be simulated in a capable language model and if we can strategically create/construct/curate a dataset or a series of datasets - aligning language models to our values is possible.

^

I'm still trying to wrap my head around the kind of capabilities or genetics is necessary that enables language learning and speech control.

(This might be a relevant reading: The evolutionary history of genes involved in spoken and written language: beyond FOXP2.)

^

I should write further into why I believe that "stories serve as a universal structure for information" next week. However, to briefly explain: I think that the

...
How do LLMs give truthful answers? A discussion of LLM vs. human reasoning, ensembles & parrots
by Owain_Evans — 2024-03-28T02:34:21.799Z

Summary

Large language models (LLMs) like ChatGPT and Claude 3 become increasingly truthful as they scale up in size and are finetuned for factual accuracy and calibration.
However, the way LLMs arrive at truthful answers is nuanced. When an LLM answers a question immediately without chain-of-thought reasoning, the answer is typically not the result of the LLM reasoning about the question and weighing the evidence. Instead, the answer is based on human answers from pretraining documents that are (i) contextually relevant and (ii) resemble sources that led to truthful answers in finetuning.
By contrast, when LLMs do explicit chain-of-thought reasoning before answering the question, the reasoning steps are more likely to causally determine the LLM's answer.
This has parallels in human cognition. Many people can state Fermat's Theorem without having evaluated the proof themselves.
Does this mean LLMs just parrot humans when answering without chain-of-thought reasoning? No.

LLMs don't mimic a single human's answers. They aggregate over many human answers, weighted by relevance and whether the source is correlated with truthfulness.
This is loosely analogous to mechanisms that aggregate many human judgments and outperform most individual humans, such as ensembling forecasts, markets, PageRank, and Bayesian Truth Serum.
Moreover, LLMs have some conceptual understanding of their answers, even if they did not evaluate the answers before giving them.

Epistemic Status:
This essay is framed as a dialogue. There are no new experimental results but only my quick takes. Some of the takes are backed by solid evidence, while some are more speculative (as I indicate in the text).

How do LLMs give truthful answers?

Q: We’d like to have LLMs that are

truthful

, i.e. that systematically say true things and avoid saying false or inaccurate things wherever possible. Can we make LLMs like this?

Owain: Current finetuned models like GPT-4 and Claude 3 still make mistakes on obscure long-tail questions and on controversial questions. However, they are substantially

more

truthful

than earlier LLMs (e.g. GPT-2 or GPT-3). Moreover, they are more truthful than their own base models, after being finetuned specifically for truthfulness (or “honesty” or “factuality”) via RLHF.
In general, scaling up models and refining the RLHF finetuning leads to more truthful models, i.e. mod...
UDT1.01: The Story So Far (1/10)
by Diffractor — 2024-03-27T23:22:35.170Z

We now resume your regularly scheduled LessWrong tradition of decision theory posting.

This is a sequence, be sure to note.

Just the first and last post will be on Alignment Forum, and the whole thing will be linked together.

Epistemic Status: This is mostly just recapping old posts so far. If you're a decision-theory veteran, new stuff only starts arising in the "Computational Intractability" section and further down.

You may have heard of a thing called Updateless Decision Theory. It's been discussed for over a decade by now, but progress on rendering it down into something that could maybe someday be run on a computer has been very slow. Last winter, I decided to try helping out Abram and Martin with their efforts at actually formalizing it into some sort of algorithm that nice things could be proved about. I didn't quite succeed, but did manage to have a fair few novel insights that feel like they got a lot further than previous attempts, and I do see the basic outlines of an algorithm here.

So this is post 1 of a 10-part series which will try to cram some of the insights into the heads of people who are not me.

Why UDT?

Summarizing some old history, standard decision theory works great in a single-player environment, where your only pathway to influence reality is what you actually do. Once you transition to a multiplayer setting, where there are things running around that can make predictions about you, and select their actions based on what you will do, or would do in a different situation, things get a lot spicier.

Four examples of "things selecting their actions based on what you will do" are:

A human going "I will help you gain power iff it's very likely that you'll use some of that power to leave me better off for having done so" to another human.

A bank with a brain scanner going "I will give you a loan iff you seem like the sort of person who would predictably pay back the loan"

An AI thinking about another AI and concluding "This other AI, with a different utility function, will predictably fight me in the future if they get stronger, and won't accept binding precommitments. I can do math about their source code which strongly suggests this. Time to preemptively delete their ass."

A human going "It's late and I don't want to brush my teeth but there's a bunch of future-me's who are in the same situation and will make the same decision for the same reasons...
Some Things That Increase Blood Flow to the Brain
by romeostevensit — 2024-03-27T21:48:46.244Z

Epistemic status: very shallow google scholar dive. Intended mostly as trailheads for people to follow up on on their own.

previously: https://www.lesswrong.com/posts/h6kChrecznGD4ikqv/increasing-iq-is-trivial

I don't know to what degree this will wind up being a constraint. But given that many of the things that help in this domain have independent lines of evidence for benefit it seems worth collecting.

Food:

dark chocolate, beets, blueberries, fish, eggs. I've had good effects with strong hibiscus and mint tea (both vasodilators).

Exercise:

Regular cardio, stretching/yoga, going for daily walks.

Learning:

Meditation, math, music, enjoyable hobbies with a learning component.

Light therapy:

Unknown effect size, but increasingly cheap to test over the last few years. I was able to get Too Many lumens for under $50. Sun exposure has a larger effect size here, so exercising outside is helpful.

Cold exposure:

this might mostly just be exercise for the circulation system, but cold showers might also have some unique effects.

Chewing on things:

Increasing blood flow to the large masseter muscles seems to have a broadly stimulating effect on blood flow to the head in general. You can buy unflavored gum for cheap, or xylitol gum which has a positive side effect on decreasing cavity formation.

Things to avoid:

vasoconstrictors, notably cigarettes, alcohol, stimulants. Caffeine seems to cause acute vasoconstriction followed by supranormal vasodilation? Not clear on the research here.

Conjecture/supplements:

I consider the evidence on these a bit weak.

Bacopa is the only nootropic I've taken besides stimulants that has a seemingly noticeable effect (most notably on recall).

Vinpocetine has supposedly shown increased blood oxygenation in the brain.

Pyrroloquinoline Quinone (PQQ) has shown working memory effects with increased blood flow to the prefrontal cortex...in the elderly.
[SP] The Edge of Morality
by Zane — 2024-03-27T21:38:51.827Z

Scott Alexander writes that philosophy is the art of exploring the edge cases of our ethics. It is clear to most that one should not kill innocent people for pleasure, but that will not help us uncover new insights. Instead, we look at the edge cases where it is not clear what morality says. We think of killing one person and harvesting their organs to save many, and then two different parts of us say that it is right, and that it is wrong. It is there that we must look to truly understand our own values.

It is intuitively obvious, not even worth debating in the first place, that it is a moral abomination to drop food onto the ground and then eat it. Dropping food on the ground and then giving it to someone else to eat would be tantamount to shooting a baby. But there are always edge cases, points at the boundaries of what is right and wrong, where two intuitions collide head-on.

At my house, there is a large container of plain oatmeal, easily a gallon or more. I of course do not simply eat the flavorless oatmeal on its own; I eat separate packages of oatmeal, with flavoring and bits of fruit. But when I pour too much water into my oatmeal, as is a common occurrence, I add a bit of plain oatmeal, so as not to turn the whole thing into a soup.

Only a few days ago, in my absentmindedness, I poured an entire bowlful of plain oatmeal out for myself, without even opening the strawberry oatmeal package or heating up the water. I of course realized my error at once, and then, with a furtive look around the room - and I hope, dear reader, that you will forgive me, even if I cannot forgive myself - I poured the plain oatmeal back into the container.

Now, the part of me that reasons with words and logic - System 2, as you may call it - raises a great many points in my defense. It argues that I had not yet put anything else into the bowl, and that the oatmeal, having not even touched my own hands, was at no risk at all of contamination. It argues that the bowl, having been recently washed, was likely just as clean as, if not cleaner than, the original oatmeal container itself. It argues that it is absurd for me to be paying attention to such a trivial matter when on the other side of the planet, there are malaria-carrying mosquitos drinking the blood of African children without so much as washing their hands first. And all this is true.

And yet the other side of me - System...
Come to Manifest 2024 (June 7-9 in Berkeley)
by Saul Munn — 2024-03-27T21:30:17.306Z
Link: https://news.manifold.markets/p/come-to-manifest-2024-june-7-9-in

TLDR

Manifold is hosting a festival for prediction markets: Manifest 2024! We’ll have serious talks, attendee-run workshops, and fun side events over the weekend. Chat with special guests like Nate Silver, Scott Alexander, Robin Hanson, Dwarkesh Patel, Cate Hall, and more at this second in-person gathering of the forecasting & prediction market community!

Tickets & more info: manifest.is

WHEN: June 7-9, 2024, with LessOnline and Summer Camp starting May 31

WHERE: Lighthaven, Berkeley, CA

WHO: Hundreds of folks, interested in forecasting, rationality, EA, economics, journalism, tech and more. If you’re reading this, you’re invited!

People

Manifest is an event for the forecasting & prediction market community, and everyone else who’s interested. We’re aiming for about 500-700 attendees (you can check the markets here!).

Current speakers & special guests include:

Content

Everything’s optional, and there’ll always be a bunch of sessions running concurrently. Like last year, we’ll host a mix of talks from experts, attendee-run workshops, and fun side events. We’re especially excited about attendee-run sessions — people loved hosting their own and attending their friends’. Topics range from:

forecasting in journalism & government
legalizing prediction markets
AI forecasting (& AIs making forecasts)
the case against forecasting/prediction markets
adjacent experimental stuff (impact certificates, quadratic voting, etc)

…and much, much more!

Since Manifest is forecasted to be bigger this year, we’re putting extra effort into making it easier for the right people to connect. We’ll have interest-specific meetups for politics, journalism, AI, mechanism design, etc — and plenty of infrastructure for you to run your own sessions on topics you’re excited about.

Last year’s content

To give you a sense of what to expect, here were some talks from last year:

Prediction Markets in Journalism — Scott Alexander (ACX) and Dylan Matthews (Vox)
Fireside Chat — Nate Silver (538)
Biggest Questions in AI Forecasting — Matthew Barnett (Epoch AI)
The Future of Trust and Evidence in the Age of AI — Emmett Shear (Twitch, YC, OpenAI)
Improving AI Benchmarks — Isabel Juniewicz (Open Philanthropy)
Revolution Strategies — Robin Hanson (GMU, Mercatus Center)
Prediction Markets vs. Financial Markets — Byrne Hobart (The Diff)
Genetic Enhancement: Prediction Markets for Future People — Jonathan

...
The Cognitive-Theoretic Model of the Universe: A Partial Summary and Review
by jessicata — 2024-03-27T19:59:27.893Z
Link: https://unstablerontology.substack.com/p/the-cognitive-theoretic-model-of

About 15 years ago, I read Malcolm Gladwell's Outliers. He profiled Chris Langan, an extremely high-IQ person, claiming that he had only mediocre accomplishments despite his high IQ. Chris Langan's theory of everything, the Cognitive Theoretic Model of the Universe, was mentioned. I considered that it might be worth checking out someday.

Well, someday has happened, and I looked into CTMU, prompted by Alex Zhu (who also paid me for reviewing the work). The main CTMU paper is "The Cognitive-Theoretic Model of the Universe: A New Kind of Reality Theory".

CTMU has a high-IQ mystique about it: if you don't get it, maybe it's because your IQ is too low. The paper itself is dense with insights, especially the first part. It uses quite a lot of nonstandard terminology (partially because the author is outside the normal academic system), having few citations relative to most academic works. The work is incredibly ambitious, attempting to rebase philosophical metaphysics on a new unified foundation. As a short work, it can't fully deliver on this ambition; it can provide a "seed" of a philosophical research program aimed at understanding the world, but few implications are drawn out.

In reading the work, there is a repeated sense of "what?", staring and looking at terms, and then "ohhh" as something clicks. These insights may actually be the main value of the work; at the end I still don't quite see how everything fits together in a coherent system, but there were a lot of clicks along the way nonetheless.

Many of the ideas are similar to other intellectual ideas such as "anthropics" and "acausal interaction", but with less apparent mathematical precision, such that it's harder to see exactly what is being said, and easier to round off to something imprecise and implausible.

There is repeated discussion of "intelligent design", and Langan claims that CTMU proves the existence of God (albeit with a very different conceptualization than traditional religions). From the perspective of someone who witnessed the evolution / intelligent design debate of the 90s-00s, siding with the "intelligent design" branch seems erroneous, although the version presented here differs quite a lot from more standard intelligent design argumentation. On the other hand, the "evolutionists" have gone on to develop complex and underspecified theories of anthropics, multiverses, and simulations,...
[Linkpost] Leif Wenar's The Deaths of Effective Altruism
by Arden — 2024-03-27T19:17:28.170Z
Link: https://www.wired.com/story/deaths-of-effective-altruism/

cross-posted on EA Forum

Leif Wenar thoughtfully critiqued EA in "Poverty is No Pond" (2011) & just wrote a critique in WIRED. He is a philosophy professor at Stanford & author of Blood Oil.

Edit:

My initial thoughts

(which are very raw & will likely change & I will accordingly regret having indelibly inscribed on the Internet):

Initially, after a quick read-through, my take is he does a great job critiquing EA as a whole & showing the shortfalls are not isolated incidents. But none of the incidents were news to me. I think there's value in having these incidents/critique (well) written in a single article.

But, really, I'm interested in the follow-up piece / how to reform EA or else the alternative to EA / what’s next for the many talented young people who care, want to do good, & are drawn to EA. I'd love to hear y'all's thoughts on this.

Thank you, M, for sharing this with me & encouraging me to connect.
Plausibility of cyborgism for protecting boundaries?
by Chris Lakin — 2024-03-27T18:53:38.615Z

Most of my boundaries work so far has been focused on protecting boundaries "from the outside". For example, maybe davidad's OAA could produce some kind of boundary-defending global police AI.

But, imagine parenting a child and protecting them by keeping them inside all day. Seems kind of lame. Something else you could do, though, is not restrict the child and instead allow them to become stronger and better at defending themselves.

So: you can defend boundaries "from the outside", or you can empower those boundaries to be better at protecting themselves "from the inside". (Because, if everyone could defend themselves perfectly, then we wouldn't need AI safety, lol)

Defending boundaries "from the inside" has the advantage of encouraging individual agents/moral patients to be more autonomous and sovereign.

I put some examples of what this might look like in Protecting agent boundaries:

Empower membranes to be better at self-defense

Empower the membranes of humans and other moral patients to be more resilient to collisions with threats. Examples:

Manipulation defense: You have an AI assistant that filters potentially-adversarial information for you.
Crime defense: Police have AI assistants that help them predict, deduce, investigate, and prevent crime.
Physical threat defense: (If nanotech works out) You have an AI assistant that shields you from physical threats.
Biological defense: Faster better vaccines, personal antibody printers, etc.
Cybersecurity defense: Good security practices and strong encryption. Software encryption can be arbitrarily strong.

c.f. writing about this from Foresight Institute:

(1)

,

(2)

,

(3)

…

Legal defense: personal AI assistants for e.g. interfacing with contracts and the legal system.
Bargaining: personal AI assistants for negotiation.

Human intelligence enhancement

Cyborgism

Mark Miller and Allison Duettmann (Foresight Institute) outline more ideas in the form of “Active Shields” here:

7. DEFEND AGAINST PHYSICAL THREATS | Multipolar Active Shields

. Cf Engines of Creation by Eric Drexler.
Related:

We have to Upgrade

– Jed McCaleb

I'm looking to talk to people about the plausibility of empowering boundaries to be better at defending themselves / cyborgism. Let me know, or leave a comment if you know anyone who's thinking about this.
Towards White Box Deep Learning
by Maciej Satkiewicz — 2024-03-27T18:20:57.611Z
Link: https://arxiv.org/abs/2403.09863

Hi, I’d like to share my paper that proposes a novel approach for building white box neural networks.

The paper introduces semantic features as a general technique for controlled dimensionality reduction, somewhat reminiscent of Hinton’s capsules and the idea of “inverse rendering”. In short, semantic features aim to capture the core characteristic of any semantic entity - having many possible states but being at exactly one state at a time. This results in regularization that is strong enough to make the PoC neural network inherently interpretable and also robust to adversarial attacks - despite no form of adversarial training! The paper may be viewed as a manifesto for a novel white-box approach to deep learning.

As an independent researcher I’d be grateful for your feedback!