trentmkelly/LessWrong-43k huggingface dataset:
An Ontology for Strategic Epistemology
by StrivingForLegibility — 2023-12-28T22:11:56.510Z

In order to apply game theory, we need a model of a strategic interaction. And that model needs to include data about what agents are present, what they care about, and what policies they can implement. Following the same aesthetic as the Embedded Agency sequence, I think we can build world models which at a low level of abstraction do not feature agents at all. But then we can apply abstractions to that model to draw boundaries around a collection of parts, and draw a system diagram for how those parts come together to form an agent.

The upshot of this sequence is that we can design systems that work together to model the world and the strategic landscape it implies. Even when those systems were designed for different purposes using different ways to model the world, by different designers with different ways they would like the world to be.

Automating Epistemology

A software system doesn't need to model the world in order to be useful. Compilers and calculators don't model the world. But some tasks are most naturally decomposed as "build up a model of the world, and take different actions depending on the features of that model." A thermostat is one of the simplest systems I can think of that meaningfully implements this design pattern: it has an internal representation of the outside world ("the temperature" as reported by a thermometer), and takes actions that depend on the features of that representation (activate a heater if "the temperature" is too low, activate a cooler if "the temperature" is too high).

A thermostat's control chip can't directly experience temperature, any more than our brains can directly experience light bouncing off our shoelaces and hitting our retinas. But there are lawful ways of processing information that predictably lead a system's model of the world to be correlated with the way the world actually is. That lead a thermostat to register "the temperature is above 78° F" usually only in worlds where the temperature is above 78° F.

Given the right conditions, well-founded beliefs are also contagious.

If your tongue speaks truly, your rational beliefs, which are themselves evidence, can act as evidence for someone else. Entanglement can be transmitted through chains of cause and effect—and if you speak, and another hears, that too is cause and effect. When you say “My shoelaces are untied” over a cellphone, you’re sharing your

...
Psychology of AI doomers and AI optimists
by Igor Ivanov — 2023-12-28T17:55:31.686Z

What is this dialogue about

This is a dialogue about complicated relationships between AI safety and psychology.

AI safety field consists of people, and people have all sorts of motivations, cognitive biases, and coping mechanisms that are interesting to explore.

This is a discussion about psychological reasons for not taking AI risks seriously, as well as being an AI doomer. We also discuss why people might start working on AI safety, and what are the relationships between AI risks and existential questions.

The partisipants

Joep Meindertsma, a founder of PauseAI.

Igor Ivanov, a psychotherapists helping people working on AI safety, as well as an independent AI governance researcher.

(To some reason it's impossible to make a dialogue with one of the participants have low karma, so this is posted as a post, not a dialogue.)

Spoiler

Both participants are AI doomers, and we discuss psychology and AI risk through the lens of our point of view, so we are prone to pay attention to the coping mechanisms, wishful thinking and fallacies of AI optimists, but we don't mean that they don't have strong and valid arguments. AI doomers also have all sorts of unhealthy coping mechanisms, weird believes, and delusions, but it's easier to see problems of the people from the other camp.

I believe that if a couple of skeptics discussed delusions of doomers, they would probably find a lot of interesting things.

The discussion

Igor
As a person who is concerned about the x-risks and who believes that the rate of AI development is extremely fast, I feel worried about all these risks. Someone says that there is a very high chance that AI will pose existential risks. Someone disagrees and says that the chances of existential risks are not that high, but they still say that it can be used for misinformation or cyber attacks. In any case, I don't think that anybody disagrees that AGI will be very powerful and it will pose a lot of risks to our society, and it seems like most people outside of the AI safety community don't realize what's going on. They live their own lives.

For example, a friend of mine started an online school to teach people Python. She wants to produce new junior software developers in 2023. As far as I know, many companies automate their software development and the first people who they won't hire anymore are juni...
AI #44: Copyright Confrontation
by Zvi — 2023-12-28T14:30:10.237Z

The New York Times has thrown down the gauntlet, suing OpenAI and Microsoft for copyright infringement. Others are complaining about recreated images in the otherwise deeply awesome MidJourney v6.0. As is usually the case, the critics misunderstand the technology involved, complain about infringements that inflict no substantial damages, engineer many of the complaints being made and make cringeworthy accusations.

That does not, however, mean that The New York Times case is baseless. There are still very real copyright issues at the heart of Generative AI. This suit is a serious effort by top lawyers. It has strong legal merit. They are likely to win if the case is not settled.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Entrepreneurial advice.

GPT-4 Real This Time. What will we get in the coming year?

Fun With Image Generation. MidJourney wants you to speak (creative) English.

Copyright Confrontation. The New York Times versus OpenAI.

Deepfaketown and Botpocalypse Soon. ChatGPT used to spot plagiarism? Good.

Going Nuclear. Wait, you don’t want AI involved in nuclear safety?

In Other AI News. Nancy Pelosi buys Nvidia options.

Quiet Speculations. Will scaling LLMs lead to AGI? Dwarkesh Patel ponders.

The UN Reports. UN says UN things, most of you can skip this.

The Week in Audio. Shapira, Lebenz,Bloom and the Crystal Society audiobook.

Rhetorical Innovation. They are building a religion. They are building it bigger.

AI With Open Model Weights Is Unsafe and Nothing Can Fix This. Them too.

Aligning a Human Level Intelligence is Still Difficult. Chinese alignment paper.

Please Speak Directly Into the Microphone. Daniel Faggella.

The Wit and Wisdom of Sam Altman. Mostly being rather wise recently. Also rich.

The Lighter Side. A warning in song.

Language Models Offer Mundane Utility

A game called Thus Spoke Zaranova where you have to pretend to be an AI, Tweet thread, design notes. Premise is of course rather silly, but is the game interesting or fun? I do not know.

In previous studies, we consistently find an equalizing effect from use of LLMs. High performers improve, but low performers improve a lot more.

Now we have a study that finds the opposite effect. Entrepreneurs in Kenya were given AI ‘mentor’ access via WhatsApp. High performers benefited, lo...
How to develop a photographic memory 1/3
by PhilosophicalSoul — 2023-12-28T13:26:36.669Z

Mastery of research, memory and logic is key to rebuilding a metaphorical Library of Alexandria. The first asks: how do we find knowledge? The second asks: how do we remember the knowledge we find? The third asks: how do we apply that knowledge accurately? I will be dealing with the second question in a series of three (3) posts.

INTRODUCTION

@TurnTrout in How do you feel about LessWrong these days? [Open feedback thread] — LessWrong posits two (2) things to do when making a post. First, to outline your claims in the beginning. Second, to keep your post concise and understandable. I tried my best throughout this post to follow these warnings. I also read both @Eliezer Yudkowsky books from Rationality: From AI to Zombies in the hopes that I avoid some of the more common biases and fallacies. Correct me if I've failed there. Thank you, moderators!

DISCLAIMER

You will be able to understand this post even if you haven't read any of the ones I link.

I encourage readers to interact as they see fit. I am open to alternative ideas and developing this. I am genuinely eager to learn from the LessWrong community.
This trio of posts will be divided into five (5) chapters; orthodox techniques, unorthodox techniques, memory modules, application, and the Solakios technique (yes, that's what I'm going to call it). The use of images below will be explained briefly in [Chapter I, Part 2], then more extensively in [Chapter IV].

For those familiar with memory, you'll find tidbits of new knowledge spread about this post. If not, [Chapter IV] and [Chapter V] will be most beneficial. If you're in a hurry for an upcoming test, exam or thesis I'd suggest skipping to [Chapter V].

CLAIMS

I want you to leave this post able to devour the sacred texts of your field more easily, and more effectively. I want to guide you to techniques that enhance your ability to compete with artificial intelligence. Thus, my claims are:

This is a guide to developing photographic memory, not eidetic memory

What is the difference?
It means that you will still be able to recognise patterns and understand how concepts are related. Comparatively, those with eidetic memory often suffer the fate of remembering the whole puzzle but having no idea how individual pieces came together to form it.[1]

This is a practical step-by-step that can be applied immediately
What does that entail?
I am not going

...
Gunpowder as metaphor for AI
by Nathan Helm-Burger — 2023-12-28T04:31:40.663Z

The exothermic reaction of combusting gunpowder is a continuous but fast process. Sufficiently fast that a human will perceive it as discontinuous.

This is just refining a metaphor for a subject that's been hashed out many times before.

I'm basically just agreeing with JBlack's comment on the linked post:

There isn't really as much difference as it might seem.

This is true in a prosaic, boring sense: a steep continuous change may have exactly the same outcome as a discontinuous change on some process, even if technically there is some period during which the behaviour differs.

It is also true in a more interesting sense: the long term (technically: asymptotic) outcome of a completely continuous system can be actually discontinuous in the values of some parameter. This almost certainly does apply to things like values of hyperparameters in a model and to attractor basins following learning processes. It seems likely to apply to measures such as capabilities for self-awareness and self-improvement.

So really I'm not sure that the question of whether the best model is continuous or discontinuous is especially relevant. What matters more is along the lines of whether it will be faster than we can effectively respond.

Most of my personal experience with the combustion of gunpowder comes from playing with or watching fireworks as a kid. So here's a metaphor drawing on that experience.

Fuse

You can easily halt the process as it has been designed for that. It is slow, steady, safe, contained.

Example: the proposal I've made for safe, contained testing on models designed from the ground up for alignability trained on censored data (simulations with no mention of humans or computer technology)

Sparkle Fountain

You can stomp it out with significant effort and risk. It is flashy, showy, hot.

It seems uncontained, but actually has been carefully designed to move more slowly than gunpowder normally would.

Example: This is the mode that I currently think frontier models are operating in, in late 2023. GPT-4, Gemini, Claude, etc.

M80

You cannot stomp it out. It hasn't been designed to go fast, per say, it is just the default behavior of gunpowder. On a short enough time scale, it can theoretically be seen that the gunpowder combustion is continuous, but practically for humans, it is not. The process is faster than your OODA loop. Once started, you no longer have control. ...
E.T. Jaynes Probability Theory: The logic of Science I
by Jan Christian Refsgaard — 2023-12-27T23:47:52.579Z

This book (available as a free pdf here) is one of the foundational texts for Rationalists; Eliezer refers to Jaynes as The Level Above Mine. While we agree that Jaynes is very lucid and the book is wonderful, we also think the book's target audience are graduate physics students. It occupies a sparse area of the literature, between statistics, math, causality, and philosophy of science that contains real, practical lessons about how to think about the world and its uncertainty. Here, we will try to give a review where we share the core insight of the book while skipping all the integrals and derivations. The book is basically 50% high level math and 50% cool stories and rants that you can mostly understand without the math. It also covers a lot of material, and this review covers just the first 6 chapters, which introduces the fundamental concepts and problems.

The book has 2 parts:

Part I (Chapter 1-10): Principles and elementary applications

Part II (Chapter 11-22): Advanced applications

This is a review of chapters 1-6

Part 1: A new Foundation of Probability Theory

The first two chapters of the book are by far the toughest. In chapter one, Jaynes gives a half chapter refresher of formal logic, and introduces the concept of probability as logic. Here the goal is to extend the rules of formal logic such that they also apply to probabilistic reasoning. This is a complicated task: going from strict binary truth values to continuous probabilities without assuming anything about what a probability is. This is also where the book gets its name, the logic of science, because much like Eliezer, Jaynes views reasoning under uncertainty as the foundation of science.

Chapter 1: Plausible Reasoning

Jaynes opens the chapter with the following:

Suppose some dark night a policeman walks down a street, apparently deserted. Suddenly he hears a burglar alarm, looks across the street, and sees a jewelry store with a broken window. Then a gentleman wearing a mask comes crawling out through the broken window, carrying a bag which turns out to be full of expensive jewelry. The policeman doesn’t hesitate at all in deciding that this gentleman is dishonest. But by what reasoning process does he arrive at this conclusion? Let us first take a leisurely look at the general nature of such problems.

In formal logic we have statements such as "If A, Then B; A is true, therefore B i...
Free agents
by Michele Campolo — 2023-12-27T20:20:59.855Z

Posted also on the EA Forum.

Shameless attempt at getting your attention:

If you’ve heard of AI alignment before, this might change your perspective on it. If you come from the field of machine ethics or philosophy, this is about how to create an independent moral agent.

Introduction

The problem of creating an AI that understands human values is often split into two parts: first, expressing human values in a machine-digestible format, or making the AI infer them from human data and behaviour; and second, ensuring the AI correctly interprets and follows these values.

In this post I propose a different approach, closer to how human beings form their moral beliefs. I present a design of an agent that resembles an independent thinker instead of an obedient servant, and argue that this approach is a viable, possibly better, alternative to the aforementioned split.

I’ve structured the post in a main body, asserting the key points while trying to remain concise, and an appendix, which first expands sections of the main body and then discusses some related work. Although it ended up in the appendix, I think the extended Motivation section is well worth reading if you find the main body interesting.

Without further ado, some more ado first.

A brief note on style and target audience

This post contains a tiny amount of mathematical formalism, which should improve readability for maths-oriented people. Here, the purpose of the formalism is to reduce some of the ambiguities that normally arise with the use of natural language, not to prove fancy theorems. As a result, the post should be readable by pretty much anyone who has some background knowledge in AI, machine ethics, or AI alignment — from software engineers to philosophers and AI enthusiasts (or doomers).

If you are not a maths person, you won’t lose much by skipping the maths here and there: I tried to write sentences in such a way that they keep their structure and remain sensible even if all the mathematical symbols are removed from the document. However, this doesn’t mean that the content is easy to digest; at some points you might have to stay focused and keep in mind various things at the same time in order to follow.

Motivation

The main purpose of this research is to enable the engineering of an agent which understands good and bad and whose actions are guided by its understanding of good and bad.

I’ve already giv...
Natural Latents: The Math
by johnswentworth — 2023-12-27T19:03:01.923Z

Natural latents are a relatively elegant piece of math which we figured out over the past year, in our efforts to boil down and generalize various results and examples involving natural abstraction. In particular, this new framework handles approximation well, which was a major piece missing previously. This post will present the math of natural latents, with a handful of examples but otherwise minimal commentary. If you want the conceptual story, and a conceptual explanation for how this might connect to various problems, that will probably be in a future post.

While this post is not generally written to be a "concepts" post, it is generally written in the hope that people who want to use this math will see how to do so.

2-Variable Theorems

This section will present a simplified but less general version of all the main theorems of this post, in order to emphasize the key ideas and steps.

Simplified Fundamental Theorem

Suppose we have:

A distribution P[X] over random variables X=(X1,X2)
A latent variable Λ which induces independence between X1 and X2 (first diagram below)
Another latent variable Λ′ about which X1 and X2 give the same information (second diagram below)

Further, assume that X mediates between Λ and Λ′ (third diagram below). This last assumption can typically be satisfied by construction in the minimality/maximality theorems below.

Λ induces independence between X1 and X2

X1 and X2 give the same information about Λ′

X mediates between Λ and Λ′

Then, claim: Λ mediates between Λ′ and X.

Intuition

Picture Λ as a pipe between X1 and X2. The only way any information can get from X1 to X2 is via that pipe (that’s the first diagram). Λ′ is a piece of information which is present in both X1 and X2 - something we can learn from either of them (that’s the second pair of diagrams). Intuitively, the only way that can happen is if the information Λ′ went through the pipe - meaning that we can also learn it from Λ.

The third diagram rules out three-variable interactions which could mess up that intuitive picture - for instance, the case where one bit of Λ′ is an xor of some ind...
NYT is suing OpenAI&Microsoft for alleged copyright infringement; some quick thoughts
by Mikhail Samin — 2023-12-27T18:44:33.976Z

Unpaywalled article, the lawsuit.

(I don't have a law degree, this is not legal advice, my background is going through a US copyright law course many years ago.) I've read most of the lawsuit and skimmed through the rest, some quick thoughts on the allegations:

Memorisation: when ChatGPT outputs text that closely copies original NYT content, this is clearly a copyright infringement. I think it's clear that OpenAI & Microsoft should be paying everyone whose work their LLMs reproduce.
Training: it's not clear to me whether training LLMs on copyrighted content is a copyright infringement under the current US copyright law. I think lawmakers should introduce regulations to make it an infringement, but I wouldn't think the courts should consider it to be an infringement under the current laws (although I might not be familiar with all relevant case law).
Summarising news articles found on the internet: copyright protects expression, not facts (if you read about something in a NYT article, the knowledge you received isn't protected by copyright, and you're free to share the knowledge); I think that if an LLM summarises text it has lawful access to, this doesn't violate copyright if it just talks about the same facts, or might be fair use. NYT alleges damage from Bing that Wikipedia also causes by citing facts and linking the source. I think to the extent LLMs don't preserve the wording/the creative structure, copyright doesn't provide protection; and some preservation of the structure might be fair use.
Hallucinations: ChatGPT hallucinating false info and attributing it to NYT is outside copyright law, but seems bad and damaging. I'm not sure what the existing law around that sort of stuff is, but I think even if it's not covered by the existing law, it'd be great to see regulations making AI companies liable for all sorts of damage from their products, including attributing statements to people who've never made them.
Extropy magazine review 
by Peter lawless  — 2023-12-27T18:37:36.256Z

I've been reading extropy magazine(a pioneering futuristic publication in the 90s), some topics like decentralized payment systems are no longer as innovative as they were back then. but I discovered several interesting articles:

A)In one of his rare fiction texts we have a fake advertisement for The Galactomatic-1000 (TM) Basement Universe( The galactomatic-1000) Written by Carl Feynman, computer engineer and son of Richard Feynman.Science comedy at its best!

B) Perhaps more curious is this article(https://arch-anarchism.blogspot.com/2023/12/the-thermodynamics-of-death.html?m=1) discussing whether the scientific search for immortality is restricted by the laws of thermodynamics.

C)arch-anarchy The view that we should seek to void all limits on our freedom, including those imposed by the laws of nature.(Article: Arch-Anarchy)

D) Smart Contracts: Building Blocks for Digital Markets(

http://www.alamut.com/subj/economics/nick_szabo/smartContracts.html

). Obs:Yes, I recognize that as I said above it is no longer as innovative as it was at the time it was published, it is still worth a look.