trentmkelly/LessWrong-43k huggingface dataset:
Mnestics
by Jarred Filmer — 2022-10-23T00:30:11.159Z

Take your mnestic

Part IV

In the web series "There is No Antimemetics Division" our heroes must fight a great evil, monsters that make you forget they exist. In the course of such adventures, they find it necessary to imbibe "mnestics" (the opposite of an amnesiac), drugs for remembering.

Every morning since I left that cabin, I wake up having to remember only one thing.

Take. your. mnestic

To do this, I sit with my laptop and place a leaf between my lips. This looks odd if I happen to be in a cafe, but serves to remind me I was doing something important if I get distracted. It also helps to turn interruptions like "do you have a second?", into "Jarred, why are you eating a leaf?".

I hit "ctrl+m" to pull up my desktop forever dedicated to "mnestic.txt" and my calendar. The contents of mnestic.txt have changed over time, but the quote at the top is always the same:

Humans can forget anything. It's okay to forget some things, because we are mortal and finite. But some things we have to remember. It's important that we remember. Write to yourself something which will make you remember.

- There Is No Antimemetics Division

I read it one word at a time, and try to recapture that sense of dread. The feeling of recalling something existentially important that needs sustained action over years. Knowing I'll forget.

I read the second line, still a word at a time:

A routine of self perpetuating, undistracted time to think is necessary to respond well to being alive.

I try to recall the feeling I had sitting in that cabin, that "Space" is something both vital and fragile, and while I have it, I need to use it to ensure I don't lose it. It's at this stage I check my calendar to confirm the week's mnestics are all lined up, coming to me free and clear down the river of time.

I read the remaining 8 lines from past selves who had access to a wealth of space, reminding me of how I want live. With my priorities freshly re-downloaded into my brain, I plan my day, set intentions, and make a todo list. Altogether this takes around 30 minutes.

To finish, I try to capture in time who I feel I am now, and verbally extend a bridge to my future self. Tomorrow when sitting down to take his own mnestic, he will think of this moment and verbally accept that bridge from the past, creating a chain throughout time built of 30-minute pockets of daily space-taking. A highway down which c...
Telic intuitions across the sciences
by mrcbarbier — 2022-10-22T21:31:28.672Z

[WORK IN PROGRESS - I will keep editing this post over time]

See the introductory post for a motivation.

Explaining a phenomenon in terms of function, role, purpose, goal or meaning is extremely common throughout, let's say, most of the sciences that are not physics or chemistry. Yet it is almost always done qualitatively, verbally, without real support from any kind of math.

In this post, I will try to list every important intuition about this class of concepts that I have stumbled upon so far. This probably does not make for a great page turner/scroller, but this is more of a reference post for future ones to quote.

Bias/competence disclosure: Having been trained all the way as a physicist and half-way as a linguist, my main inspirations are grammar and statistical mechanics. My understanding of other topics will be far more cursory, besides theoretical ecology which is where I work now. That being said, I do think everyone should learn a bit of grammar and stat mech, as those are virtuous pursuits.

General intuitions in existing theories

Generative grammar

Why grammar is an important example

Grammar is the only science I know which has been consistently good at not confusing "nature" and "function":

Nature/category is what a word is, e.g. noun or adjective or verb... I can ask this question of the word in itself: is cat a noun or a verb?
Function is what role a word serves in a larger context, e.g. cat serves as the subject in "Cat bites man." You cannot take the word out of context and ask of it, apart from any sentence: is cat a subject or a complement?

The same function can be served by many different natures (e.g. a verb-like thing serves as the subject in "To live forever sounds exciting/tiring."), and the same nature can perform many different functions.

Grammar is also the only science I know which has tried to propose autonomous and non-trivial theories of functions.

Quick recap: how grammar works

Many of you will have seen a parse tree such as the following (from Wikipedia)

Each node on such a tree is labeled with its nature (S: sentence, N: noun, V: verb, D: determiner; XP: phrase of type X). The position of a node in the tree indicates its grammatical function: the noun closest to the sentence is the grammatical subject, the noun phrase inside the verb phrase is the object.

How do we come up with any of this? There is a quite fascinating empirical ...
A basic lexicon of telic concepts
by mrcbarbier — 2022-10-22T21:28:10.475Z

See the introductory post of this sequence to make any sense of the following.

Universal tools : subtraction and substitution

The many sciences that deal with telic questions, from linguistics to anatomy to genetics to sociology, have at least two basic tools in common:

Subtraction/deletion/knockout

Since reductionism is the science of adding up parts to get a whole, it is natural to expect telism to behave a bit like subtraction.

If I want to know the role of a part, I can remove it to see how the whole changes:
* linguistics: remove a word from a sentence and see if it still makes sense
* genetics: knock out a gene and see if the organism survives
* neuroscience: remove parts of the brain to see what cognitive functions are affected

This suggests that there is at least a mapping between the role of a part and the way that the whole fails without it. This notion of failure indicates that we consider the whole itself to be under selection (to make sense, to survive, to think). We thus try to explain the part by showing how selection on the whole translates into selection on the part.

Substitution

If I want to know if two objects have the same role, I can substitute an object for another and see if the whole does *not* change:
* anatomy: replacing a heart by a mechanical pump, without dying
* linguistics: replacing a noun by a pronoun, or by a phrase, without losing meaning or grammaticality.

This allows us to identify objects that have the same function in the context of a given whole, even when they have very different natures. The same nature can hold different functions, and the same function can be achieved by different natures. We can use this to build a many-to-many relationship of natures and functions.

Taxonomy of related concepts

In other posts, I have grouped together a bunch of concepts like function, role, goal, purpose and meaning, claiming they are all more-or-less related to telos. I wish to give them more precise technical definitions that also clarify distinctions.

Function/role

Here, function is to be understood as it is used in biology or sociology, and probably quite unlike its common mathematical sense (which is deeply embedded in set theory, w...
Do we have the right kind of math for roles, goals and meaning?
by mrcbarbier — 2022-10-22T21:28:04.935Z

Preamble on LessWrong relevance: Agentic goals and values, which are of some interest here at LW, are a special case of a broader class of objects that have interested linguists, biologists, and social scientists for a while. This class of objects should really have a common basic mathematical language. It does not, by historical accident. As a consequence, low-hanging fruits and basic confusions abound: research relies on verbal intuitions, gets hung up on domain-specific peculiarities, and misses fundamental generalizations across systems and fields. I suspect potential for progress ranges from Shannon-level to Newton-level, and unfortunately, no Shannons or higher are currently to be found working on it.

This is a broad introduction to give context for an upcoming sequence of increasingly technical posts. You can also try to skip ahead to What should a telic science look like for a different take on how to introduce the project.

Motivation: A mental black hole

The way mathematical models were introduced into the biological, cognitive and social sciences should be recognized as a great redirection: it spawned many new fascinating questions, but didn't make it any easier to answer the old questions.

The old questions all have to do with something like function, role, goal, purpose or meaning: what's the point of this organ, this gene, this word, this institution?

We have been made to believe that these questions are either so trivial as to admit simple verbal answers, or unscientific until converted into other questions that are the proper domain of science (such as "how is this implemented?" or "how could this evolve?"). This mindset is such a strong attractor that it has become a kind of intellectual black hole: all our attempts to move away lead back there, forgetting our original intent.

Everything I will say next is thus both exceedingly simple, and exceedingly hard - researchers who try to address this seriously tend to either go crackpot, or fall back into the black hole. It's hard to imagine what success even looks like when it has never ever happened.

1) Two basic directions of scientific explanation

The behavior of many systems can be explained from at least two directions: what's happening inside them, and what's influencing them from the outside.

At one extreme, there's usual physics: it deals with isolated systems or simple boundary conditio

...
The Last Year  - is there an existing novel about the last year before AI doom?
by Luca Petrolati — 2022-10-22T20:44:58.055Z

Does anybody know of any kind of novel which is set in a plausible version of the world a year or so before humanity id ended by AI, at least as we currently understand the term "humanity"? Maybe not something as technical as Gwern's doom scenario, but still something that would be classified as pretty hard sci-fi and at the same time explore more the perspective of actual human characters on how living in such a world does feel like.

Even just very preliminary drafts would be very valuable I think.
The highest-probability outcome can be out of distribution
by tailcalled — 2022-10-22T20:00:16.233Z

"Out of distribution" is a phenomenon where a data point x does not come from the distribution you had in mind. To give some examples, here is what might be considered OOD:

You play a dice game against a cheater, and the cheater uses a die that has been weighted to always roll six. Thus their rolls look like 666666..., whereas your rolls look like 312265....
You order an apple from a store and the store decides to drive a truck into your house in order to murder you.
You are reading an interesting post on LessWrong, and th2u
mCXh4:xaapdQA
r bKv4ztPktiR)pYpXwT-j
%?:Ta4Or
XXdicVAQ?;DHJFh?U4wLW! iCmVD~z1iQT5cB4x&J_A-2MmUxCCxDdA3hYXjomA "m5mID;9-C9~=%~-j6ba"2N!fV~jikXZOQ2iEsnNxR0nggVYR:ZF:9&p
m#R"Xf
j~=vFiM_N(&~M%8amq2q5z5

Possibilities 2 and 3 seem to be based on something that is unlikely or surprising happening, so they seem to suggest that being "out of distribution" is about being low-probability. But if you think about it for possibility 1, then every sequence of dice rolls of equal length should have the same probability when using fair dice, and therefore low probability cannot account for possibility 1.

In fact, we can construct a scenario where a point is "out of distribution" despite being the highest-probability point in the distribution. Imagine a gaming place where everyone is expected to cheat, playing with dice that have 58% chance of rolling a 6. In that case, an ordinary roll sequence would look like 624646662443.... However, if someone got a roll sequence full of 6s, like 666666666666..., then their rolls would be out of distribution, and it would look like they were cheating beyond the accepted weighted dice.

Conceptually, what is happening here is that even though 666666666666... is the most likely roll sequence, there are so many other sequences of rolls that have fewer 6s that their lower likelihood is counterbalanced (and then some) by their greater number. The sum of the probabilities of other sequences ends up being greater than the probability of 666666666666....

However, this doesn't explain the full story. After all, the sum of the probabilities of the non-624646662443... roll sequences is also much greater than the probability of 624646662443.... Any one specific roll sequence is unlikely, so why do we consider some to be "out of distribution" and others not?

This is probably not a fully solved problem. Perhaps in a later post, I will...
Newsletter for Alignment Research: The ML Safety Updates
by Esben Kran — 2022-10-22T16:17:18.208Z

Introducing the ML Safety Updates

TLDR; We present a new AI safety update series in

podcast

,

YouTube

and

newsletter

format released weekly to stay updated in alignment and ML safety research and get exposed to Ai safety opportunities. Read the latest newsletter

further down in this post

.

Our motivations for this are two-fold:

It has never been easy to stay updated on the latest developments in specific research fields and in the past couple of years, the amount of alignment research has

increased significantly

. On top of that, much safety-relevant AI work is not to be found in legible EA / rationalist channels, e.g. cybersecurity, AI legislation, robustness, and monitoring.
Existing newsletters in alignment research are focused on deep examinations of theory and give detailed insights to the reader. However, there is no newsletter series for up-to-date, weekly events.

Our newsletters cover a summary of the past week’s research, both within alignment and safety-relevant AI work, as well as promote opportunities in the AI safety space.

The past 7 weeks, we have released these updates as a YouTube video series summarizing novel AI and ML safety research in 4-6 minutes. This week, we released them in

podcast

and

newsletter

format, and future updates will also be released to LessWrong. Subscribe

here

.

The case for an AI safety update series

There are already a few amazing resources on AI safety similar to newsletters. However, the ones that exist are either biased towards specific topics or have not been kept up to date that past year. See our summary below.

Alignment Newsletter

: Rohin Shah has kept the Alignment Newsletter running for a long while and Rob Miles has recorded its entries as

podcast episodes

. It is released in

Chinese

, See the whole team on

the website

and their

spreadsheet of all newsletters

. It started on

April 9, 2018

and was released once a week. There are a total of 3 episodes in 2022.

ML Safety Newsletter

: Dan Hendrycks sends out a Substack newsletter every month with summaries of new ML safety research.

AGISF Newsletter

: This newsletter managed by the Blue Dot Impact team shares opportunities to over 1,000 subscribers somewhat regularly (~monthly+).

Quintin’s Alignment Paper Review

: Quintin releases a wonderfully comprehensive ~weekly review of an AI safety adjacent or relevant field

...
Crypto loves impact markets: Notes from Schelling Point Bogotá
by Rachel Shu — 2022-10-22T15:58:39.101Z

Thanks to Rhys Lindmark, Noah Chon Lee, Dawn Drescher and Sinclair Chen for assistance with this post.

Last week Dony and I went down to the

Schelling Point public goods conference

in Bogota, Colombia, where Dony was leading a workshop on impact markets. Gitcoin hosted the event, which felt very professionally put on and their AV team was fantastic – here is a

link to their official photos and videos

in lieu of any of our own. Gitcoin is also writing its own retrospective which I’ll also link. I was surprised and taken in by the level of interest in public goods funding on display at the conference, which I think looked like it had at least 500 attendees. Even more unexpected was how much of the day’s discussion revolved around impact markets.

Impact markets as a solution for efficiently funding speculative interventions has been

a topic of discussion (click for an explanation of the idea)

in a weakly positive light (with some

particular attention to drawbacks

) on this forum from Paul Christiano’s

introduction of the idea

in 2014. More recently it has been discussed in the larger EA community, such as with Scott Alexander’s

previous writing on the topic

, experiments done by Clearer Thinking and Manifold Markets on

market-directed funding mechanisms

, and previous work by my co-author Dony alongside Dawn Drescher and Matt Brooks on impact certificates on this forum and at

https://impactmarkets.io

. However, the space has remained fairly small within EA for such projects.

In contrast, in the past year the web3 community has quickly taken to the concept of impact markets, and several startups might be close to developing workable impact market implementations. It seems like this work hasn’t been discussed much on the forums, so I wanted to make a brief sketch of the emerging public goods scene within web3 to make this work more visible to EAs, and to think about how it might fit in with EA work. Some readers will be familiar with this development already, perhaps from some of Dawn and Dony’s posts and comments; I think most will be hearing about it for the first time, as I had just three weeks ago. At any rate, I think it is worth having another post written to bring it more to attention.

What’s web3

? Web3 is a collection of internet projects which interface with decentralized ledgers (blockchains) such as Ethereum, aiming to i...
When trying to define general intelligence is ability to achieve goals the best metric?
by jmh — 2022-10-22T03:09:51.923Z

This is a rather loosely thought out position but one I've held for a long time. For me, I will consider some system an artificial intelligence when it starts asking why type questions and then sets out to get the answers (how type questions) on its own.

Put a bit differently, that is problem identification over problem solving. It seems to me that most of the AI researchers and AI alignment people use definitions that only focus on problem solution/goal attainment without actually requiring the "intelligence" to have its own goals.

Is that take on the working definition incorrect?

If not, should that line of inquiry be fit into someone's investigations? If so, where in the priority ordering? If not, is that because people think it would be largely intractable or perhaps just low returns?

I know at least one on the forum has raised the question about why an AI would ever "want" anything. That view clearly fits with the view we don't need to worry about my view of what intelligence is. But I think the recent post by John about predication markets and helping to identify good questions perhaps suggests we should consider that. The questions asked do seem to be very important to success and growth/innovation and general problem solving results. To me that points towards ability to come up with insightful questions might be a key part of intelligence (though I suspect there is a bit of a bias in though by me here on that "conclusion").
Moorean Statements
by David Udell — 2022-10-22T00:50:52.138Z

Moorean statements are statements like:

It's not raining but I believe it is.

These statements sound strange because any agent that outright tells you it's not raining must already at least tacitly represent that fact in their world model. That agent is plugged into their world model well enough to report what it says, but not well enough to accurately model their model. For this explicit a Moorean statement, the epistemic strangeness is so obvious that basically no one will have that combination of access to and confusion about their world model.

An Eliezerism my Eliezer-model often generates is that many social scripts involve expressing Moorean propositions. They're subtler, but the essential confusion is the same.

I'm a committed Christian because my parents are -- that's just how I was raised.

Well, if intuitions aren't epistemically admissible in philosophy, philosophers would be out of a job!

What? How can you simultaneously recognize the non-epistemic generator of your belief and hold the belief?

Can you generate more instances?
Wisdom Cannot Be Unzipped
by Sable — 2022-10-22T00:28:25.476Z
Link: https://affablyevil.substack.com/p/wisdom-cannot-be-unzipped

Introduction

The year was 2016 (or close enough), and a friend of mine’s younger brother - we’ll call this younger brother Luke - was about to start college.

Having completed college not that long ago, I thought it would be nice if I could make a list of some of the things that had helped me - the strategies and disciplines that I had made use of - to hand to this prospective scholar.

I made my list, including links to productivity resources and guides to How To Get Things Done, and prepared to hand it off to Luke with my best wishes.

I remember how it felt, handing him that list. I was envious of Luke, in a way; I wish that I had had someone hand me this list back when I started college. It would have made things so much easier! And here I was, just handing Luke the keys to success on a silver platter.

He didn’t even know how lucky he was. I could picture how much easier he’d find things than I did, with the steps on his path helpfully guided by my concise and helpful list.

A few months later, I check in with my friend as to how his younger brother is doing, sure that I am about to hear a tale of effortless success and, well…

Luke was failing his classes.

I later learned that Luke had dropped out after his first year.

Nowadays Luke’s doing his own thing, and I have, shall we say, different opinions about the utility of college in general. And I can look back at the list I gave him and admit that it was not a perfect distillation of how to Mange Time Effectively and Really Get Things Done.

At the time though, I was aghast, and spent a great deal of thought on how my Super Awesome List failed to teach Luke the secrets I had learned throughout college on How To Get Things Done.

What I realized, after a great deal of introspection, was that it was inherent to the nature of wisdom itself that it cannot be decompressed. What follows is an examination of that idea.

Part I: Gird Your Metaphors

Compression as a Technology

Compression is a key technology in digital systems. It’s what happens when you ‘zip’ a file - the file size is reduced, like packing a puffy winter jacket in a suitcase by sitting on the suitcase until you can get it closed.

We compress data in digital systems because data transmission and storage aren’t free, in money or in time. It takes longer to send a gigabyte than a megabyte, and you can (by definition) store a thousand of the latter for every one of the fo...