trentmkelly/LessWrong-43k huggingface dataset:
A review of "Don’t forget the boundary problem..."
by jessicata — 2024-02-08T23:19:49.786Z
Link: https://unstablerontology.substack.com/p/a-review-of-dont-forget-the-boundary

Introduction

This is a review of "Don't forget the boundary problem! How EM field topology can address the overlooked cousin to the binding problem for consciousness", by Andrés Gómez-Emilsson. This article seeks to address two problems with theories of consciousness, using an electromagnetic theory:

The binding problem asks how a unified first person perspective (1PP) can bind experiences across multiple physically distinct activities, whether billions of individual neurons firing or some other underlying phenomenon. To a first approximation, the boundary problem asks why we experience hard boundaries around those unified 1PPs and why the boundaries operate at their apparent spatiotemporal scale.

The binding problem is fairly easy to understand the basics of: it's about how "our consciousness somehow binds multiple discrete features into a single unified awareness." The boundary problem is elaborated on:

At the same time, our consciousness does not "bind" features without limit---what we experience varies over time and is thus always strictly a subset of what could be experienced. There is an edge to our awareness, a boundary around us that is generally felt to exist at the human-scale of experience, rather than at the cellular or societal level.

To me this seems not to be a problem with my preferred functionalist-ish theories of consciousness. A single brain mainly processes information from itself, and processes information from outside through fairly limited channels such as sensory perception. Therefore, it is unsurprising that no one experiences everything, because no one knows everything.

One could speculate about there being super-entities, such as corporations, that have consciousness spanning many humans. This is not a major challenge to functionalism. If McDonalds experiences things its employees do not experience, it does not follow that any McDonalds employee writing papers about consciousness would report experiencing whatever McDonalds knows. That information is simply not present within the physical sub-system of the McDonalds employee writing a paper about consciousness.

One can also speculate about the anthropic question of why we experience being a human rather than a corporation. Here a few things can be said:

Some level of complexity of mind is required to be conscious in the relevant meta-aware sense (see HOT the

...
aintelope project update
by Gunnar_Zarncke — 2024-02-08T18:32:00.000Z

Since the announcement of the Brain-like AGI project "aintelope" in 2022, we have been busy working continuously on the project. The project was entirely based on volunteer work from the start until September 2023. Our principle was sustainability. The project had to work with low or interrupted contributions from participants with a full-time job and family. And it worked. The project has progressed and grown steadily. This is a first status report of project aintelope. You can also read it as a story of how slow and incremental progress leads to a prospering project.

Participants

Andre Kochanke - Data Scientist, founder of aintelope UG (non-profit)
Hauke Rehfeld - Python, founding member
Gunnar Zarncke - Managing Director and founder of aintelope UG (non-profit)
Joel Pyykkö - AI Researcher, member since 10/2022
Roland Pihlakas - AI Safety Researcher, member since 1/2023
Rasmus Herlo - Neuroscience Researcher and advisor since 11/2023
Fabian Zarncke - Student, founder of aintelope UG (non-profit)

TL;DR;

In 2023, aintelope made progress in agent learning, simulations, and discussions of AI behaviors like the Waluigi Effect. We virtually hosted multiple successful hackathons, aligning our code with brain-like AGI concepts. Conversations at EAGxNordics 2023 sparked ideas for better development and teamwork.

We put together an information hazard policy. Significant effort was spent on improving the platform and code base. Collaborating with the AI Alignment community, especially since recently with neuroscience researcher Rasmus Herlo as an advisor, has started promising.

First quarter

In the first quarter of 2023, aintelope was focused on exploration and discussion culminating in a hackathon and retrospective. The team explored various aspects of AI development, the exploration of 3D environment simulation and many discussions of AI behavior patterns including the Waluigi Effect and human-like aspects in AI, like cultural learning and instinct encoding. Papers and books on emotion in reinforcement learning were shared.

Practical topics were improving the project structure, the savanna environment, agents modules, and managing GitHub effectively. And how to make the AI training process more transparent and monitoring-friendly, with enhancements in model inference and training workflow documentation. One frustration was administrative delays in tax exe...
Updatelessness doesn't solve most problems
by Martín Soto — 2024-02-08T17:30:11.266Z

In some discussions (especially about

acausal trade

and

multi-polar conflict

), I’ve heard the motto “X will/won’t be a problem because superintelligences will just be Updateless”. Here I’ll explain (in layman’s terms) why, as far as we know, it’s not looking likely that a super satisfactory implementation of Updatelessness exists, nor that superintelligences automatically implement it, nor that this would drastically improve multi-agentic bargaining.

Epistemic status: These insights seem like the most robust update from my work with Demski on

Logical Updatelessness

and discussions with CLR employees about

Open-Minded Updatelessness

. To my understanding, most researchers involved agree with them and the message of this post.

What is Updatelessness?

This is skippable if you’re already familiar with the concept.

It’s easier to illustrate with the following example: Counterfactual Mugging.

I will throw a fair coin.

If it lands Heads, you will be able to freely choose whether to pay me $100 (and if so, you will receive nothing in return).
If it lands Tails, I will check whether you paid me the $100 in the Heads world[1], and if so, I will pay you $1000.

In this picture, I am Ω (throwing the coin) and you are A (deciding whether to Pay).

When you find yourself in the Heads world, one might argue, the rational thing to do is to not pay. After all, you already know the coin landed Heads, so you will gain nothing by paying the $100 (assume this game is not iterated, etc.).

But if, before knowing how the coin lands, someone offers you the opportunity of committing to paying up in the Heads world, you will want to accept it! Indeed, you’re still uncertain about whether you’ll end up in the Heads or the Tails world (50% chance on each). If you don’t commit, you know you won’t pay if you find yourself in the Heads world (and so also won’t receive $1000 in the Tails world), so your expected payoff is $0. But if you commit, your payoff will be -$100 in the Heads world, and $1000 in the Tails world, so $450 in expectation.

This is indeed what happens to the best-known decision theories (CDT and EDT): they want to commit to paying, but if they don’t, by the time they get to the Heads world they don’t pay. We call this dynamic instability, because different (temporal) versions of the agent seem to be working against each other.

Why does this happen? Because,...
Predicting Alignment Award Winners Using ChatGPT 4
by Shoshannah Tekofsky — 2024-02-08T14:38:37.925Z

This experiment explores the Enhancing Selection branch of the

Collective Human Intelligence

(CHI) research agenda. The hypothesis under evaluation was:

There are testable traits that make people better at alignment research such that we could use them for funding and training selection.

The conclusion is “Maybe” - At least according to the data I got from prompting ChatGPT 4 to detect the winners of the

Alignment Award

competitions on the

Goal Misgeneralization Problem

and the

Shutdown Problem

. Initially it seemed neither structured prompt exploration nor prompts generated by ChatGPT 4 could consistently detect the winners of either competition. However, running a simple tournament prompt - comparing two research summaries and then promoting the winner to the next round where the process is repeated - did actually result in detecting the winner in 5 out of 10 runs, and putting the winner in the semi-finals in 3 out of the 5 remaining runs for the Shutdownability contest. In contrast, the same prompt had earlier failed to detect the winner on the Goal Misgeneralization contest across 10 runs.

Overall, my intuition is that we can get better results with data that more directly represents the candidates’ research prowess (e.g., using

recursive summarization of a larger body of work

) and using success measures that are less noisy (e.g. wide-scale adoption of a proposed alignment technique). This could be applied to pre-filter grant proposals or sift for promising new talent among applicants of training programmes like

MATS

or

AI Safety Camp

.

Background

Aka, why did you run this experiment?

Cause I’m not entirely sure how to solve the alignment problem myself, but maybe I am able to boost humanity as a whole in solving the alignment problem together. One way to provide this boost to our collective human intelligence is to find new talent more quickly and cost effectively. And one way to do that, is by checking if LLMs can do the finding for us.

Notably I’m assuming that baby geniuses look different from baby everyone else, such that failures of LLMs to predict world events (

Zou et al., 2022

) may not apply to failure to predict excellence in AIS research. I spent some time trying to figure out what a Da Vinci, Einstein, or Kahneman looks like before they become notable. Ideally we’d prompt ChatGPT 4 to check...
AI #50: The Most Dangerous Thing
by Zvi — 2024-02-08T14:30:13.168Z

In a week with two podcasts I covered extensively, I was happy that there was little other news.

That is, until right before press time, when Google rebranded Bard to Gemini, released an app for that, and offered a premium subscription ($20/month) for Gemini Ultra.

Gemini Ultra is Here

I have had the honor and opportunity to check out Gemini Advanced before its release.

The base model seems to be better than GPT-4. It seems excellent for code, for explanations and answering questions about facts or how things work, for generic displays of intelligence, for telling you how to do something. Hitting the Google icon to have it look for sources is great.

In general, if you want to be a power user, if you want to push the envelope in various ways, Gemini is not going to make it easy on you. However, if you want to be a normal user, doing the baseline things that I or others most often find most useful, and you are fine with what Google ‘wants’ you to be doing? Then it seems great.

The biggest issue is that Gemini can be conservative with its refusals. It is graceful, but it will still often not give you what you wanted. There is a habit of telling you how to do something, when you wanted Gemini to go ahead and do it. Trying to get an estimation or probability of any kind can be extremely difficult, and that is a large chunk of what I often want. If the model is not sure, it will say it is not sure and good luck getting it to guess, even when it knows far more than you. This is the ‘doctor, is this a 1%, 10%, 50%, 90% or 99% chance?’ situation, where they say ‘it could be cancer’ and they won’t give you anything beyond that. I’ve learned to ask such questions elsewhere.

There are also various features in ChatGPT, like GPTs and custom instructions and playground settings, that are absent. Here I do not know what Google will decide to do.

I expect this to continue to be the balance. Gemini likely remains relatively locked down and harder to customize or push the envelope with, but very good at normal cases, at least until OpenAI releases GPT-5, then who knows.

There are various other features where there is room for improvement. Knowledge of the present I found impossible to predict, sometimes it knew things and it was great, other times it did not. The Gemini Extensions are great when they work and it would be great to get more of them, but are finicky and made s...
How to develop a photographic memory 3/3
by PhilosophicalSoul — 2024-02-08T09:22:07.918Z

Under advisement from @Algon and @Seth Herd, I've included a TLDR:

The actual photographic memory technique starts at [Chapter V].
Early 'photographic memory' and results [Chapter III: Cont'd > (2) Tachisto...].
Is this trio of posts overwhelming and difficult to remember? I've reduced it to the most important points in [Chapter IV: Application > (2) Essentials].
Problems beyond my scope, or I hadn't the time to solve in the 'field of mnemonics' can be read in [Chapter IV: Application > (2) Essentials > (b) Queries].

I intend to code and freely distribute an app which can streamline the Solakios technique some point in the future. At present, I have only my prototype TM. You'll find that the technique itself is quite simple to understand, difficult to master.

Interestingly, Dr Charan Ranganath seems to have been inspired by my trio of posts. I have no way of knowing this, I can only guess and hope. He published his book "Why we Remember" shortly after I originally created this post. From what I've read of the preview, there's also anecdotes that use sentences and jargon I employed. Either way, pretty cool that my experimenting has been cemented by a more qualified individual.

CHAPTER III: CONT'D

I have discovered or been informed of additional techniques since we last spoke. It would make little sense to edit them into my previous post.

(1) PALACE TO KINGDOM

I devised the best possible version of a memory palace, inspired by @DirectedEvolution. This technique is effective, even for complex concepts. I'll also point out a digital conception; primed for efficiency purposes.

(a) PROBLEM-ORIENTATED APPROACH

Memory palaces work because they 'hit the correct beats' (as in, they embrace the principles of memory discussed previously.) These beats can be amplified by the lessons learned from anecdotal evidence in this post:

The headings of scholarships (textbooks, articles, company memo's) must be mapped to the label given to each structure.
E.g. criminal legislation would be mapped to a building called 'Prison'.
Personalising and 'coolifying' the room.
E.g. concepts are mapped to NPC's, typically authors from the book or celebrities from the article. They must discuss with and interrogate you.
Logical flow that connects each structure.
E.g. in your mind, it makes sense to walk from the poetry room to the literature room, then to the essay room to intertwine bo

...
Believing In
by AnnaSalamon — 2024-02-08T07:06:13.072Z

“In America, we believe in driving on the right hand side of the road.”

Tl;dr: Beliefs are like bets (on outcomes the belief doesn’t affect). “Believing in”s are more like kickstarters (for outcomes the believing-in does affect).

Epistemic status: New model; could use critique.

In one early CFAR test session, we asked volunteers to each write down something they believed. My plan was that we would then think together about what we would see in a world where each belief was true, compared to a world where it was false.

I was a bit flummoxed when, instead of the beliefs-aka-predictions I had been expecting, they wrote down such “beliefs” as “the environment,” “kindness,” or “respecting people.” At the time, I thought this meant that the state of ambient rationality was so low that people didn’t know

“beliefs” were supposed to be predictions

, as opposed to

group affiliations

.

I’ve since changed my mind. My new view is that there is not one but two useful kinds of vaguely belief-like thingies – one to do with predictions and Bayes-math, and a different one I’ll call “believing in.” I believe both are lawlike, and neither is a flawed attempt to imitate/parasitize the other. I further believe both can be practiced at once – that they are distinct but compatible.

I’ll be aiming, in this post, to give a clear concept of “believing in,” and to get readers’ models of “how to ‘believe in’ well” disentangled from their models of “how to predict well.”

Examples of “believing in”

Let’s collect some examples, before we get to theory. Places where people talk of “believing in” include:

An individual stating their personal ethical code. E.g., “I believe in being honest,” “I believe in hard work,” “I believe in treating people with respect,” etc.
A group stating the local social norms that group tries to practice as a group. E.g., “Around here, we believe in being on time.”
“I believe in you,” said by one friend or family member to another, sometimes in a specific context (“I believe in your ability to win this race,”) sometimes in a more general context (“I believe in you [your abilities, character, and future undertakings in general]”).
A difficult one-person undertaking, of the sort that’ll require cooperation across many different time-slices of a self. (“I believe in this novel I’m writing.”)
A difficult many-perso

...
Measuring pre-peer-review epistemic status
by Jakub Smékal — 2024-02-08T05:09:01.418Z

Epistemic status: Exploratory, speculative, half-baked thought

It’s a worldwide optimization problem. What content to consume under what conditions to reach a particular goal? Taking a step back and looking only at academia, and constraining ourselves to only academic papers (and preprints for simplicity), the question boils down to What papers should I read in what sequence to understand this field/problem, to figure out what the relevant problems are, etc.? (slight abuse of the semantics of “boils down to”, but what can you do).

One clear metric is to look at the citations of a paper, but that will be of no use when the field in question is evolving at a rapid pace (e.g. AI) and most of the relevant papers are recent preprints on arXiv. There are of course other metrics (number of retweets on Twitter, endorsements from the community on discussion forums, etc.), but as far as I know, there isn’t a widely accepted measure of the epistemic status of scientific papers. I saw many posts on LessWrong declaring their epistemic status explicitly before the main content. This seems like a great proxy for figuring out whether you want to read something or not. For instance, if you’re looking for a tutorial on training sparse autoencoders on transformer activations, you’re not likely going to find what you’re looking for in a 20 minute read-long post with Epistemic status: philosophical, speculative, even if it mentions sparse autoencoders in the title. It’s fair to say that the epistemic status of a particular post is implicit in the platform it’s published on, i.e. every platform attracts a particular type of content, but I’d argue that this is not the case for preprints on arXiv, where you find both ML papers of the type “we tried a new architecture and it performed X% better on this benchmark” as well as those of the type “we propose this framework for thinking about consciousness”. And the greater issue at hand, which is why I think an explicit measure of epistemic status might be useful, is that you often can’t tell the epistemic status of the paper simply by reading the title, abstract, and perhaps not even after you skim through it (though I leave open the possibility that this is not as common as I make it to be). Further, given the large quantity of preprints, it’s highly likely that certain works will overlap without each other's knowledge, which is another ...
A Chess-GPT Linear Emergent World Representation
by Adam Karvonen — 2024-02-08T04:25:15.222Z
Link: https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html

A Chess-GPT Linear Emergent World Representation

Introduction

Among the many recent developments in ML, there were two I found interesting and wanted to dig into further. The first was gpt-3.5-turbo-instruct's ability to play chess at 1800 Elo. The fact that an LLM could learn to play chess well from random text scraped off the internet seemed almost magical. The second was Kenneth Li's Emergent World Representations paper. There is an excellent summary on The Gradient and a follow-up from Neel Nanda. In it, they trained a 25 million parameter GPT to predict the next character in an Othello game. It learns to accurately make moves in games unseen in its training dataset, and using both non-linear and linear probes it was found that the model accurately tracks the state of the board.

However, this only worked for a model trained on a synthetic dataset of games uniformly sampled from the Othello game tree. They tried the same techniques on a model trained using games played by humans and had poor results. To me, this seemed like a major caveat to the findings of the paper which may limit its real world applicability. We cannot, for example, generate code by uniformly sampling from a code tree. There was also discussion on the implications of this on LessWrong, such as if pretraining should begin with synthetic data to improve interpretability.

So I dug into it. I trained some models on chess games and used linear probes on the trained models. My results were very positive, and answered all of my previous questions (although of course, more questions were generated).

A 50 million parameter GPT trained on 5 million games of chess learns to play at ~1300 Elo in one day on 4 RTX 3090 GPUs. This model is only trained to predict the next character in PGN strings (1.e4 e5 2.Nf3 ...) and is never explicitly given the state of the board or the rules of chess. Despite this, in order to better predict the next character, it learns to compute the state of the board at any point of the game, and learns a diverse set of rules, including check, checkmate, castling, en passant, promotion, pinned pieces, etc. In addition, to better predict the next character it also learns to estimate latent variables such as the Elo rating of the players in the game.

All code, data, and models have been open sourced.

Training Chess GPT

My initial hypothesis was that Othello-GPT trained on human games...
Domestic Production vs International Wealth Creation
by 100YearPants — 2024-02-08T04:25:03.334Z

First post here but have been in and around the scene for some time. The first rationalist meetup I went to was in 2019 in SF. Glad to be here. I wanted to make my first post something short and sweet that I've been thinking about for some time and would love input from the group.

Consumer goods produced in Asia have endowed us with cheap and wide ranging products. This is good because there is more optionality for consumers and there's greater wealth creation via trade. The tradeoff though is that 1) the goods are of worse quality ie their lifetimes are shorter, 2) the wealth is captured by a foreign body instead of domestic workers, and 3) harder to measure facets such as cultural attributes baked into products are lost.

The question I'd pose to everyone here: is it better to have cheap and varied goods with the aforementioned tradeoffs or is it better to have a more narrow range of goods that are domestically produced?

Curious to hear any and all thoughts.