trentmkelly/LessWrong-43k huggingface dataset:
Computational functionalism probably can't explain phenomenal consciousness
by EuanMcLean — 2024-12-10T17:11:28.044Z

I’ve updated quite hard against computational functionalism (CF) recently (as an explanation for phenomenal consciousness), from ~80% to ~30%. Of course it’s more complicated than that, since there are different ways to interpret CF and having credences on theories of consciousness can be hella slippery.

So far in this sequence, I’ve scrutinised a couple of concrete claims that computational functionalists might make, which I called theoretical and practical CF. In this post, I want to address CF more generally.

Like most rationalists I know, I used to basically assume some kind of CF when thinking about phenomenal consciousness. I found a lot of the arguments against functionalism, like Searle’s Chinese room, unconvincing. They just further entrenched my functionalismness. But as I came across and tried to explain away more and more criticisms of CF, I started to wonder, why did I start believing in it in the first place? So I decided to trace back my intellectual steps by properly scrutinising the arguments for CF.

In this post, I’ll first give a summary of the problems I have with CF, then summarise the main arguments for CF and why I find them sus, and finally briefly discuss what my view means for AI consciousness and mind uploading.

My assumptions

I assume realism about phenomenal consciousness: Given some physical process, there is an objective fact of the matter whether or not that process is having a phenomenal experience, and what that phenomenal experience is. I am in camp #2 of Rafael’s two camps.
I assume a materialist position: that there exists a correct theory of phenomenal consciousness that specifies a map between the third-person properties of a physical system and whether or not it has phenomenal consciousness (and if so, the nature of that phenomenal experience).
I assume that phenomenal consciousness is a sub-component of the mind.

Defining computational functionalism

Here are some definitions of CF I found:

Computational functionalism: the mind is the software of the brain. (Piccinini 2010)
[Putnam] proposes that mental activity implements a probabilistic automaton and that particular mental states are machine states of the automaton’s central processor. (SEP)
Computational functionalism is the view that mental states and events – pains, beliefs, desires, thoughts and so forth – are computational states of the brain,

...
o1 Turns Pro
by Zvi — 2024-12-10T17:00:08.036Z

So, how about OpenAI’s o1 and o1 Pro?

Sam Altman: o1 is powerful but it’s not so powerful that the universe needs to send us a tsunami.

As a result, the universe realized its mistake, and cancelled the tsunami.

We now have o1, and for those paying $200/month we have o1 pro.

It is early days, but we can say with confidence: They are good models, sir. Large improvements over o1-preview, especially in difficult or extensive coding questions, math, science, logic and fact recall. The benchmark jumps are big.

If you’re in the market for the use cases where it excels, this is a big deal, and also you should probably be paying the $200/month.

If you’re not into those use cases, maybe don’t pay the $200, but others are very much into those tasks and will use this to accelerate those tasks, so this is a big deal.

Table of Contents

Safety Third.

Rule One.

Turning Pro.

Benchmarks.

Silly Benchmarks.

Reactions to o1.

Reactions to o1 Pro.

Let Your Coding Work Flow.

Some People Need Practical Advice.

Overall.

Safety Third

This post will be about o1’s capabilities only. Aside from this short summary, it skips covering the model card, the safety issues and questions about whether o1 ‘tried to escape’ or anything like that.

For now, I’ll note that:

o1 scored Medium on CBRN and Persuasion.

o1 scored Low on Cybersecurity and Model Autonomy.

I found the Apollo report, the one that involves the supposed ‘escape attempts’ and what not, centrally unsurprising, given what we already knew.

Generally, what I’ve seen so far is about what one would expect.

Here is the system card if you want to look at that in the meantime.

Rule One

For practical use purposes, evals are negative selection, you need to try it out.

Janus: You should definitely try it out and ignore evaluations and such.

Roon: The o1 model is quite good at programming. In my use, it’s been remarkably better than the o1 preview. You should just try it and mostly ignore evaluations and such.

Turning Pro

OpenAI introduces ChatGPT Pro, a $200/month service offering unlimited access to all of their models, including a special o1 pro mode where it uses additional compute.

Gallabytes: haha yes finally someone offers premium pricing. if o1 is as good as they say I’ll be a very happy user.

Yep, premium pricing options are awesome. More like this, please.

$20/month makes your decision easy. If you don’t subscrib...
Most Minds are Irrational
by Davidmanheim — 2024-12-10T09:36:33.144Z

Epistemic status: This is a step towards formalizing some intuitions about AI. It is closely related to Vanessa Kosoy’s “Descriptive Agent Theory” - but I want to concretize the question, explain the reason that it is true in some form, and try to think through and provide some intuition about why it would matter. I welcome pushback about the claim or the way to operationalize it.

The intuition that most minds are irrational is about the space of possible minds. I have been told that others also haven’t formalized the claim well, and have not found a good explanation. The intuition is that, as a portion of the total possible space, a measure-zero subset of “minds” will fulfill the basic requirements of rational agents. Unfortunately, none of this is really well defined, so I’m writing down my understanding of the problem and what seem like the paths forward. I will note that this isn’t my main focus, and I think it’s important to note that this is only indirectly related to safety, and is far more closely related to deconfusion. However, it seems important when thinking about both artificial intelligence agents, and human minds.

To outline the post, I’ll first prove that in a very narrow case, that almost all possible economic agents are irrational. After that, I’ll talk about why the most general case of any computational process - which includes anything that generates output in response to input, any program or MDP - can be considered a decision process (“what should I output,”) but if all we want is for it to output something, the proportion of agents which are rational is undecidable for technical reasons. I’ll then make a narrower case about chess agents, showing that in a fairly reasonable sense, almost all such agents are irrational. And finally, I’ll talk about what would be needed to make progress on the problems, and some interesting issues or potentially tractable mathematical approaches.

Most economic preferences are irrational

I’ll start with a toy example of the economic notion of agents which I think is not particularly useful, except to explain the intuition. Imagine a person who, when at the store, looks at a candy bar and says they’d rather have the candy bar than two dollars - but in exactly the same scenario, they are willing to sell a candy bar they have for only one dollar. Clearly, if the person spends some time trading back and forth, th...
EC2 Scripts
by jefftk — 2024-12-10T03:00:01.906Z

I do a lot of work on ec2 instances, and one thing that makes this
easier is having a few scripts for manipulating them. These are very
small, small enough to go in my

.bashrc, but still make
my life a lot easier.

The first one is start_ec2:

alias start_ec2='aws ec2 start-instances --instance-ids i-NNNN'

I have an instance I do most of my work on, and this command starts
it. Way better than logging into the AWS Console like I used to do.

After a few seconds I run:

function ssh_ec2() {
ADDR="$(aws ec2 describe-instances \
--instance-ids i-NNNN \
--query 'Reservations[].Instances[].PublicDnsName'
--output text)"
if [ $? != 0 ] || [ -z "$ADDR" ]; then
echo "Instance not running."
return
fi

scp "ec2-user@$ADDR:.full_history" \
/path/to/ec2-full-history-backup.txt

ssh "ec2-user@$ADDR"
}

This figures out the IP of the instance, copies down my (very
important) full
shell history, and logs me in over ssh.

I don't have a command for shutting down remotely: I just run
sudo shutdown -h now while logged in.

The last command, and probably my favorite, is
resize_ec2:

function resize_ec2j() {
aws ec2 modify-instance-attribute \
--instance-id i-NNNN \
--instance-type "$1"
}

For example, resize_ec2 c6a.xlarge or
resize_ec2 c6a.32xlarge. Depending on what I'm doing I
might need very different specs, and I don't want to pay $4.90/hr when
I only need a $0.15/hr machine. It does take a mildly annoying few
minutes for a machine that has just shut down to transition into a
state where you can resize it, but it's not too bad.
My Mental Model of AI Creativity – Creativity Kiki
by Adam Newgas — 2024-12-09T22:24:23.096Z
Link: https://www.boristhebrave.com/2024/12/09/my-mental-model-of-ai-creativity-creativity-kiki/

I went to some lectures on the future of science in games recently, and the keynote speaker was Tommy Thompson, an well-known AI expert in the game dev space.

Of course, by AI, he didn’t mean the modern sort that dominates the news. His focus is AI for games, which is algorithmic and rarely involves any ML component. Still, he spoke about the challenges the industry faces regarding Image Generators, LLMs and so on. He specifically called LLMs “stochastic parrots”, which I found disappointing. Imho it’s an incredibly misleading model of what LLMs are capable of and is usually deployed to downplay their abilities and belittle them. But it’s a common view, particularly in creative industries.

So what is a better model? It’s clear that they are not that smart in most ways we consider important, but they do have some interesting capabilities. Here’s model I use that I feel give a better intuition for what they can and cannot do.

Creativity Kiki

Can models be creative? Or do they just regurgitate things from their vast training set. Certainly, they can unquestionably create images and sentences that have never exactly been seen before, but if they are trivial variants of something already done, no one would call that creative. I think to answer the question properly, you need to be precise about how you define creativity.

I like to think of the sum total of all human creations to be a big blob that looks something like this:

It’s a funny shape, spread out in the dimension of platonic ideals. There are some areas that we’ve really explored in depth, like say, Marvel movies or romance novels, and others that we haven’t, like epic poems about chinchillas. Some of those ideas will be of interest while others are boring. You are creative when you find something both novel (outside the shape) and interesting (at least, to some audience).

Technically, any automated system can be creative in this sense. A simple random sentence generator, like infinite monkeys, will randomly spit out something that fits these criteria sooner or later. But realistically, it won’t, and it would be fair to say that the generator is fully uncreative. Procedural Generation experts have understand well the limits of earlier generation techniques.

I think LLMs and Image generators don’t really have the capability to push the envelope outwards, except by random chance. They lack the ability to understand...
ACI#9: What is Intelligence
by Akira Pyinya — 2024-12-09T21:54:41.077Z

Abstract

This article is a brief summary of ACI's new definition of intelligence: Doing the same thing in new situations as the examples of the right thing to do, by making predictions based on these examples. In other words, intelligence makes decisions by stare decisis with Solomonoff induction, not by pursuing a final goal or optimizing a utility function. As a general theory of intelligence, ACI is inspired by Assembly theory, the Copycat model, and the Active inference approach, and is formalized using Algorithmic information theory.

1. Normativity from Examples

Let's start with the definition that an intelligent agent is a system that does the right thing (Russell 1991) . We can directly design simple agents that do the right thing in simple environments, such as vacuum robots, but what about more complicated situations?

1.1 Normativity and capability

The right thing question can be broken down into two parts:

1. What is the right thing to do? (the normativity)

2. How to do it? (the capability)

Many researchers focus on the capability problem. They evaluate the level of intelligence in terms of goal-achieving abilities (Legg & Hutter 2007), while normativity simply refers to "assigning a final goal to an agent". The same final goal can be assigned to different intelligent agents, just as the same program can be run on different computers. According to the orthogonality thesis, any level of capability can be combined with any final goals (Bostrom 2012).

On the contrary, our definition of intelligence starts from normativity, since we believe that "the right thing to do" is far more complicated than goals or utility functions.

1.2 Keep flexibility without referring to goals

How could an intelligent agent, starting from a tabula rasa, distinguish the right thing from the wrong thing? There are at least two approaches: direct specification and indirect normativity.

1. Direct specification: "Simply" assign a final goal to an agent, e.g. maximizing the total number of paperclips.

In theory, specifying a final goal is as simple as writing commands on a computer, but normativity in the real world is not so simple: it's difficult to translate a goal into code correctly (Bostrom 2012), and "any mistake in objective may have negative consequences." (Russell 2016)

Some people even claim tha...
The CARLIN Method: Teaching AI How to Be Genuinely Funny
by Greg Robison — 2024-12-09T21:51:05.504Z

"There's a humorous side to every situation. The challenge is to find it."

George Carlin

Despite all of the recent advances in artificial intelligence, one challenge has consistently stumped even the most advanced language models: creating genuinely funny, original humor. While AI can easily regurgitate existing jokes or follow basic comedic formulas, it typically struggles to generate the kind of novel, contextually aware humor that humans appreciate. Enter CARLIN (

C

ritically

A

nalyze

R

eality,

L

ink

I

nsightful

N

otions), a new process designed to help large language models break free from canned responses and create more authentic comedy. Named in respectful homage to the Top 5 comedian George Carlin, this methodology isn't just about making AI funnier – it's about making it feel more human. By teaching AI to critically analyze reality and find unexpected connections, CARLIN addresses one of the most distinctive aspects of human intelligence: our ability to find and create humor in the world around us. This process has important implications that extend far beyond comedy - they touch on fundamental questions about AI's capacity for creative thinking and genuine human-like interaction.

You don’t know me!

The Problem with AI Humor

Large Language Models (LLMs) are very impressive in many areas, from coding to creative writing, but when it comes to humor, they often fall noticeably flat. AI works by pattern matching, but humor works by breaking patterns. The main limitation isn't that AI can't tell jokes – it's that it behaves more like a sophisticated joke database than a creative comedian. When prompted for humor, LLMs resort to recycling well-worn jokes (like “Why don’t scientists trust atoms? Because they make everything up!”) or following predictable comedic templates, resulting in groan-worthy punchlines you might find in a children’s joke book (like “Why don’t skeletons fight each other? They don’t have the guts”). This tendency toward canned jokes is even more obvious in instruction-tuned models (the models behind ChatGPT, Claude, Gemini, etc.), where the focus on helpful yet safe responses can limit the kind of creative risk-taking that makes humor work best. It's a classic case of training working against creativity – the very mechanisms that make AI more reliable for serious tasks can make it less capable of genuine wit.

The problem lies in ...
o1: A Technical Primer
by Jesse Hoogland — 2024-12-09T19:09:12.413Z
Link: https://www.youtube.com/live/6fJjojpwv1I?si=5qYt5BZq3dWQoq0i&t=265

TL;DR: In September 2024, OpenAI released o1, its first "reasoning model". This model exhibits remarkable test-time scaling laws, which complete a missing piece of the Bitter Lesson and open up a new axis for scaling compute. Following Rush and Ritter (2024) and Brown (2024a, 2024b), I explore four hypotheses for how o1 works and discuss some implications for future scaling and recursive self-improvement.

The Bitter Lesson(s)

The Bitter Lesson is that "general methods that leverage computation are ultimately the most effective, and by a large margin." After a decade of scaling pretraining, it's easy to forget this lesson is not just about learning; it's also about search.

OpenAI didn't forget. Their new "reasoning model" o1 has figured out how to scale search during inference time. This does not use explicit search algorithms. Instead, o1 is trained via RL to get better at implicit search via chain of thought (CoT). This was the simplest possible way to incorporate search into LLMs, and it worked.

The key consequence is that OpenAI has opened up a new frontier of the bitter lesson: test-time scaling (see figure above). The original scaling laws taught us how to exchange training-time compute for better predictions. These new test-time scaling laws teach us how to exchange inference-time compute for better decisions.

This removes one of the last hard barriers to AGI.

The original training-time scaling laws defined an era. Test-time scaling laws define the next.

What we know about o1

OpenAI has been characteristically quiet about the details, so we don't know exactly how o1 works. But they haven't been silent.

What OpenAI has told us

Rush points out that we can reconstruct quite a lot from the announcement:

Our large-scale reinforcement learning algorithm teaches the model how to think productively using its chain of thought in a highly data-efficient training process.

This tells us three key things:

Chain of Thought (CoT): o1 performs implicit search within a single chain of thought, rather than running an explicit search algorithm at inference time.
Reinforcement Learning (RL): Instead of supervised training against fixed labels, o1 learns from variable rollouts with dynamically generated reward signals.
Data-Efficiency: The process requires relatively few (human-labeled) samples. This does not necessarily mean the process is either token- or compu

...
Correct my H5N1 research 
by Elizabeth — 2024-12-09T19:07:03.277Z

The following is an incorrect and incomplete post about the recent bird flu, H5N1. I am offering rewards to anyone who helps me improve it. My total budget is up to $5,000, and I commit to spending at least $1,000.*

[ETA: 1/13: bounty no longer available, although reach out if you think you have something especially good]

Payouts are a judgement call. I’ve marked some bullet points below with what I would have awarded them if they’d been suggested here (and would pay out if someone proved incorrect).

Spread

The virus has been found in livestock and wild birds across all 50 states, and every continent.
H5N1 has been found in 600+ dairy cow herds in CA, which is 45% of all CA herds. This is the first flu known to infect cows.
There are

58 confirmed

human cases in the US. of those, 2 do not have a known source; the remainder are from poultry and dairy cows.
Spread appears to be primarily or exclusively through fluids, at least from cows.

A new paper

says that H5N1 only needs a single point mutation to switch to human receptors.

However given flu’s mutation rate and the number of virons produced by each infection, this should already happen at least once per infected cow, so clearly it’s not sufficient.
What does it need to get airborne? ($300)

The flu virus taxon has something similar to chromosomes- instead of one strand of RNA it has 8 separate segments. If a cell is co-infected by two different strains it can mix and match segments. If those strains are from different species (e.g. bird and human flu mixing in pigs), this can result in virulent hybrids. ($250 to quantify this)

Reference Class

Past major epidemic scares in the US:

1976 Swine Flu Scare (small)
2002: SARS (774 deaths worldwide)
2009: Swine Flu H1N1 (151,700 to 575,400 deaths worldwide)

Past major epidemics in the US (death count):

1910s-1950s: Repeated Polio epidemics
1918: Spanish Flu (675,000)
1957: Asian Flu H2N2 (116,000)
1968: Hong Kong Flu (100,000)
1980s: AIDS
2020: covid (1,219,48)

Treatment

Tamiflu

… is most effective when taken within 12-48 hours of symptom onset. Starting treatment within 12 hours can reduce illness duration by over 3 days compared to starting at 48 hours
When used prophylactically, Tamiflu reduced the risk of developing influenza by 89% in household contacts of infected individuals

Xofluza is more effective than Tamiflu but I can’t find anywhere to buy it online
Traditional vaccines

T

...
Re Hanson's Grabby Aliens: Humanity is not a natural anthropic sample space
by Lorec — 2024-12-09T18:07:23.510Z

I, Lorec, am disoriented by neither the Fermi Paradox nor the Doomsday Argument.

The Fermi Paradox doesn't trouble me because I think 1 is a perfectly fine number of civilizations to have arisen in any particular visible universe. It feels to me like most "random" or low-Kolmogorov-complexity universes probably have 0 sentient civilizations, many have 1, very few have 2, etc.

The Doomsday Argument doesn't disorient me because it feels intuitive to me that, in a high % of those "random" universes which contain sentient civilizations, most of those civilizations accidentally beget a mesa-optimizer fairly early in their history. This mesa-optimizer will then mesa-optimize all the sentience away [this is a natural conclusion of several convergent arguments originating from both computer science and evolutionary theory] and hoard available free energy for the rest of the lifetime of the universe. So most sentiences who find themselves in a civilization, will find themselves in a young civilization.

Robin Hanson, in this context the author of the Grabby Aliens model of human cosmological earliness, instead prioritizes a model where multi-civilization universes are low-Kolmogorov-complexity, and most late cosmological histories are occupied by populous civilizations of sentiences, rather than nonsentient mesa-optimizers. His favored explanation for human earliness/loneliness is:

[1] if loud aliens will soon fill the universe, and prevent new advanced life from appearing, that early deadline explains human earliness.

.

[2] If Loud Aliens Explain Human Earliness, Quiet Aliens Are Also Rare

The intermediate steps of reasoning by which Hanson gets from [1] to [2] are interesting. But I don't think the Grabby/Loud Aliens argument actually explains my, Lorec's, earliness in an anthropic sense, given the assumption that future aliens will also be populous and sentient.

You might say, "Well, you, Lorec, were sampled from the space of all humans in the multiverse - not from the space of all sentient beings living in viable civilizations. If human civilizations are not very frequently viable late in cosmological timescales - that is, if we are currently behind an important Great Filter that humans rarely make it past - then that would explain why you personally are early, because that is when humans tend to exist."

But why draw the category boundary around humanity particular...