trentmkelly/LessWrong-43k huggingface dataset:
What Depression Is Like
by Sable — 2024-08-27T17:43:22.549Z
Link: https://affablyevil.substack.com/p/what-depression-is-like

I was thinking to myself about the difficulties I have explaining depression to people, when I thought of a relatively good metaphor for it.

Bear with me.

Sudoku

For anyone unaware, Sudoku is a puzzle where one tries to fill in a 9x9 grid of numbers according to certain rules:

Each row, column, and 3x3 square must have the numbers 1-9 in them, without repeating any numbers. Black numbers are given, red numbers start as blank squares and must be solved by the puzzler.

It’s a common form of brain teaser, much like a crossword puzzle or logic puzzle. Some Sudoku puzzles are difficult and some are easy; for our purposes we’ll think about ones that are relatively easy.

Brain App

Imagine, for a moment, that someone hacked your brain, and installed an app in it (don’t worry about the how). What this app does is force you to - whenever you want to do something - solve a mild Sudoku puzzle first. Not a hard one, it’s not difficult, just annoying. Want to get out of bed? Solve a Sudoku puzzle. Want to start work in the morning? Solve a Sudoku puzzle.

Want to get dressed, workout, eat, talk to someone, etc.?

First you’ve got to solve the puzzle.

At first it’s irritating, but you adapt. You figure out shortcuts for solving Sudoku puzzles. It’s brainpower you’re not expending on anything useful, but you get by.

This is the base case, the core of the metaphor.

Now we expand it.

There are two dimensions along which this nefarious app gets more annoying as time goes on:

It decreases the granularity of the actions to which it applies. In other words, where before you had to solve a Sudoku puzzle to go to work, now you’ve got to solve a puzzle to get dressed, a puzzle to get in the car, a puzzle to drive, and a puzzle to actually get started working. Before all of those counted as a single action - ‘go to work’ - now they’re counted separately, as discrete steps, and each requires a puzzle.
It increases the number of puzzles you have to solve to do anything. At first it’s just one Sudoku puzzle; eventually, it’s two, then three, and so on. Having to solve a single Sudoku puzzle whenever you want to do anything is annoying; having to solve five is downright irritating.

So what happens to you - what does your life look like - with this app running in your head?

Dimension 1

As the depression gets worse, the granularity of the actions requiring Sudoku solves gets smaller.

What does this look ...
Unit economics of LLM APIs
by dschwarz — 2024-08-27T16:51:22.692Z

Disclaimer 1: Our calculations are rough in places; information is sparse, guesstimates abound.

Disclaimer 2: This post draws from public info on FutureSearch as well as a paywalled report. If you want the paywalled numbers, email

dan@futuresearch.ai

with your LW account name and we’ll send you the report for free.

Here’s our view of the unit economics of OpenAI’s API. Note: this considers GPT-4-class models only, not audio or image APIs, and only direct API traffic, not usage in ChatGPT products.

As of June 2024, OpenAI's API was very likely profitable, with surprisingly high margins. Our median estimate for gross margin (not including model training costs or employee salaries) was 75%.
Once all traffic switches over to the new August GPT-4o model and pricing, OpenAI plausibly still will have a healthy profit margin. Our median estimate for the profit margin is 55%.
The Information implied that OpenAI rents ~60k A100-equivalents from Microsoft for non-ChatGPT inference. If this is true, OpenAI is massively overprovisioned for the API, even when we account for the need to rent many extra GPUs to account for traffic spikes and future growth (arguably creating something of a mystery).
We provide an explicit, simplified first-principles calculation of inference costs for the original GPT-4, and find significantly lower throughput & higher costs than Benjamin Todd’s result (which drew from Semianalysis).

Summary chart:

What does this imply? With any numbers, we see two major scenarios:

Scenario one: competition intensifies. With llama, Gemini, and Claude all comparable and cheap, OpenAI will be forced to again drop their prices in half. (With their margins FutureSearch calculates, they can do this without running at a loss.) LLM APIs become like cloud computing: huge revenue, but not very profitable.

Scenario two: one LLM pulls away in quality. GPT-5 and Claude-3.5-opus might come out soon at huge quality improvements. If only one LLM is good enough for important workflows (like agents), it may be able to sustain a high price and huge margins. Profits will flow to this one winner.

Our numbers update us, in either scenario, towards:

An increased likelihood of more significant price drops for GPT-4-class models.
A (weak) update that frontier labs are facing less pressure today to race to more capable models.

If you thought that GPT-4o (an...
Soft Nationalization: how the USG will control AI labs
by Deric Cheng — 2024-08-27T15:11:14.601Z
Link: https://www.convergenceanalysis.org/publications/soft-nationalization-how-the-us-government-will-control-ai-labs

Crossposted to the EA Forum.

We have yet to see anyone describe a critical element of effective AI safety planning: a realistic model of the upcoming role the US government will play in controlling frontier AI.

The rapid development of AI will lead to increasing national security concerns, which will in turn pressure the US to progressively take action to control frontier AI development. This process has already begun,[1] and it will only escalate as frontier capabilities advance.

However, we argue that existing descriptions of nationalization[2] along the lines of a new Manhattan Project[3] are unrealistic and reductive. The state of the frontier AI industry — with more than $1 trillion[4] in private funding, tens of thousands of participants, and pervasive economic impacts — is unlike nuclear research or any previously nationalized industry. The traditional interpretation of nationalization, which entails bringing private assets under the ownership of a state government,[5] is not the only option available. Government consolidation of frontier AI development is legally, politically, and practically unlikely.

We expect that AI nationalization won't look like a consolidated government-led “Project”, but rather like an evolving application of US government control over frontier AI labs. The US government can select from many different policy levers to gain influence over these labs, and will progressively pull these levers as geopolitical circumstances, particularly around national security, seem to demand it.

Government control of AI labs will likely escalate as concerns over national security grow. The boundary between "regulation" and "nationalization" will become hazy. In particular, we believe the US government can and will satisfy its national security concerns in nearly all scenarios by combining sets of these policy levers, and would only turn to total nationalization as a last resort.

We’re calling the process of progressively increasing government control over frontier AI labs via iterative policy levers soft nationalization.

It’s important to clarify that we are not advocating for a national security approach to AI governance, nor yet supporting any individual policy actions. Instead, we are describing a model of US behavior that we believe is likely to be accurate to improve the effectiveness of AI safety a...
"Real summer"?
by duck_master — 2024-08-26T22:11:51.616Z

(epistemic status: personal/metaphysical rambling)

Today - yes, that's August 26, 2024 - I took a really, really long nap. The first phase was from about 11:30am to 1pm; the second phase lasted from about 3pm to 4:30pm.[1] The dream itself was kind of incoherent and not worth retelling, except for one thing - bits and pieces of one particular song (in the dream itself, this was mainly the chorus).

After I woke up from the second phase, I decided to listen to that song. It was, as I soon learned, "Mayday" by TheFatRat. I listened to it twice, and...

... it was oddly cathartic. During the first playthrough I almost even started picturing it as a sort of theme song for ducktopia, my medianworld.[2] It was as if today encapsulated the summer of 2024 better than any other day could. Or, as I would soon put it[3], it was as if today was the first real day of summer, and every other day in the past was a sort of fake summer.[4]

What is a real summer, anyways?

This is an odd question to ask, but as a first stab, I might answer it like this:

It's relaxing.
It's a sense of inner peace.
It's a sense of not having anything to do, yet having endless possibilities of what you could do.

Epilogue: the song

As one might put it, the best music goes beyond notes and words; it reaches into your emotions. In other words, it's true poetry. So, without further ado, here's the song itself, plus the verbatim-quoted lyrics of the first verse and the chorus.[5]

[verse]

Can you hear, can you hear, can you hear my voice
Coming through, coming through, coming through the noise?
I'm floating through outer space; I'm lost and I can't find a way
All the lights going dark and my hope's destroyed

[pre-chorus]
Help me, is anybody there?
(Is anybody there? Is anybody there?)
Save me, I'm running out of air
Ooh, [chorus] Calling out mayday

[instrumental]
[instrumental] Calling out mayday
[instrumental]
[instrumental]

Footnotes + post-script

(P.S. Maybe I should write more wishy-washy posts like this. I feel like I often overestimate the amount of depth and rigor that I need to make a proper LessWrong post.)

^

Why the break? It was for lunch - or, rather, brunch. Also, I had only had 6 hours of sleep anyways.

^

Literally, a medianworld is an alternate world where you are the perfect median of the population across all axes of variation. More figuratively, it could be described as the world th

...
Metaculus's 'Minitaculus' Experiments — Collaborate With Us
by ChristianWilliams — 2024-08-26T20:44:32.125Z
Link: https://www.metaculus.com/notebooks/27612/experimenting-with-minitaculuses--collaborate-with-us/

Metaculus is experimenting with hosting more specialized forecasting communities called "minitaculuses" managed by volunteer curators. A minitaculus might be dedicated to a particular area of inquiry — like the workings of a particular industry, or questions taking a deep dive on a topic. A minitaculus can be the venue for a series of niche economic forecasts, or a collection of questions focused on a single city or region. And importantly, minitaculus curators can make questions public in a matter of minutes and can resolve them at will.

Metaculus is transitioning to open source development soon, and we're interested in experimenting with more ways to decentralize and diversify forecasting. Giving more forecasters the opportunity to manage their own spaces and create questions for others to forecast can lead to a richer variety of forecasting questions on offer.

We've invited a few forecasters to create their own spaces, and you can find these below. Right now these are very much an experiment, but we're investigating what tools we can give curators, and how to best present these spaces to give them their own look and feel.

Abstraction — curated by Jonathan Mann, a member of Samotsvety, an INFER All-Star, and a Good Judgment Superforecaster.
Zvi Mowshowitz's Minitaculus — Zvi writes on AI, rationality, public policy, and more at Don’t Worry About the Vase.
David Mathers' Minitaculus, is a seasoned forecaster and researcher at Arb Research.
The Sudan Minitaculus — Sudan-focused questions by M.M. Elrashid, who led the Forecasting Society at KU Leuven.
More to come

Are you interested in setting up your own space? What do you want to collaborate on? Reach out.

Note that minitaculuses operate largely independently from the usual Metaculus moderation process. You might find different styles of question operationalization; curators will vary in how detailed their resolution criteria are and how they approach question resolution. To that, your scores on minitaculus questions do not contribute to your track record or standing on the Metaculus leaderboards. We're still considering what scoring regime is most sensible for minitaculuses. Let us know what you'd like to see.
My Apartment Art Commission Process
by jenn — 2024-08-26T18:36:44.363Z
Link: https://jenn.site/2024/06/my-apartment-art-commission-process/

When I know that I’m going to be moving out from an apartment soon, I commission a digital artist to draw it for me. Then I print it out and I have a cool art piece. If you love your current place but you don’t think you’ll spend the rest of your life there, you should consider doing the same.

Digital artists are much cheaper than I think they should be. I’ve paid artists between $200-$500 CAD[1] for my commissions, generally spread across one or two additional housemates. (You should expect to pay more – I limit my own commissions to the common areas since my bedrooms tend to be very plain, and solely used for sleep and other private activities. Also inflation exists.)

You can also consider hiring artists from developing countries if you want your dollar to go further, but I don’t have any advice on how to seek those folks out specifically.

You’ll be looking at around 10 hours of effort on your end, frontloaded but spread out across 2-4 months. I detail my process below.

But first, here are the pieces that I’ve commissioned so far:

bigger version, 2019 artist

bigger version, 2021 artist

bigger version, 2024 artist

Aren’t they sick as hell??? I love them so much. Okay, let’s get you started on yours.

I’ll stick a sample email script at the bottom too.

Commissioning An Art Of Your Living Space, Step By Step

come up with a budget

talk to your roomies if you have them, and come up with a price you’re willing to pay altogether. i think my apartment commissions are probably 15-30? hours of work, multiply that by how much you’re willing to pay a skilled artisan for an hour of work. (i should note that in 3/3 cases for me, the minimum budget ended up being like 30-100% more than what the artist was willing to accept. digital artists often decline to charge reasonable rates for their labour.)

find 2-3 viable artists

endorsed strategies involve browsing r/wimmelbilder, the twitter/tumblr hashtag #isometricart, and google imagesing “isometric apartment layout” and clicking around. for maximal exposure to artists that are open to work, search dribbble.com for “isometric”, but note that the pickings there are fairly slim. in many isometric tags i find a lot of rendered stuff but i prefer to go for more trad art forms as i expect renderings to be more time consuming (expensive), harder to edit, and worse for the amount of detail i want[2]. also, you don’t need to commission speci

...
My (current) model of what an AI governance researcher does
by Johan de Kock — 2024-08-26T17:58:11.533Z

Purpose of this post: We need more people working as AI governance researchers. This post establishes a basic framework for thinking about what this career path entails, and how you might test your fit. I hope to receive feedback on this framework.

Epistemic status and disclosure: I have thought about and researched this by myself for about 15 hours, and while I have obtained an academic degree, I don’t consider myself as an expert yet. A partial motivation of writing this post is to gather feedback and improve my own thinking.

Photo by Johannes Plenio on Unsplash

Introduction

Alongside the many positive developments that AI brings, there are significant risks it poses. To address both the risks and benefits of AI, we have the still relatively nascent field of AI safety. Contributions to this field range from technical alignment research and AI governance to advocacy and field-building. But how does an individual choose which pathway to pursue?

A strong starting point is to choose an area with a good personal fit and to understand your unique capacity to contribute. However, making sense of this can be challenging. Experimenting with different roles and projects is a promising way to begin. This approach allows one to learn more about the nature of the work and provides real-world evidence to test your hypotheses about what might be the best fit for you.

The perspective that high-quality research is a critical area to work on inspired me to test my personal fit for AI governance research. Therefore, this post will focus on this career path, though I expect most (if not all) of the steps to be quite similar to technical research.

The aims of an AI governance researcher

To determine whether a career in AI governance research is right for you, it’s relevant to understand the purpose of this career path. I define the high-level goal of an AI governance researcher as follows: to generate and share useful insights that will better position society to effectively manage the development of increasingly advanced AI systems. But what does this mean in concrete terms?

To achieve this overarching aim effectively, an AI governance researcher seems to typically engage in seven different activity clusters.

1. Developing a deep understanding of relevant areas

1.1 The field of AI governance

Developing an extensive, detailed understanding of the AI governance landscape, including k

...
Would catching your AIs trying to escape convince AI developers to slow down or undeploy?
by Buck — 2024-08-26T16:46:18.872Z

I often talk to people who think that if frontier models were egregiously misaligned and powerful enough to pose an existential threat, you could get AI developers to slow down or undeploy models by producing evidence of their misalignment. I'm not so sure. As an extreme thought experiment, I’ll argue this could be hard even if you caught your AI red-handed trying to escape.

Imagine you're running an AI lab at the point where your AIs are able to automate almost all intellectual labor; the AIs are now mostly being deployed internally to do AI R&D. (If you want a concrete picture here, I'm imagining that there are 10 million parallel instances, running at 10x human speed, working 24/7. See e.g. similar calculations here). And suppose (as I think is 35% likely) that these models are egregiously misaligned and are actively looking for ways to subvert your safety measures and escape your control.

You caught your AI attempting to backdoor some code that controls access to a security-critical service. You investigated this by running other instances on the inputs they'd see if that backdoored code had actually been deployed; the other AI instances never point out the security flaw, and they often try to exploit that code to escalate their privileges. (See

here

for more on this.)

At this point you're almost sure the AI is scheming. So you try to call the US President and they hear you out, and then they go off and call one of your competitors to ask for their take. They say something like this:

It would be extremely costly to shut down all the AI projects. China is racing to catch up and won't stop for this. If you force us to stop deploying our frontier models, they'll outcompete us within a year, and you won't be able to get China to pause without substantial risk of war.
AIs are well known to do weird stuff. It would be irresponsible to assume one instance of anomalous behavior meant AIs were systematically going to act that way. I can put you in touch with top AI experts and they'll back me up.
Even if the AI is indeed doing something systematically funny, we have no evidence that it has the ambition to seize huge amounts of power, and it's not clear that it would be able to. It seems implausible that AIs would do that; there's been no concrete evidence that the AIs are in fact power-hungry.
Maybe our competitor just messed up their alignment, which would mak

...
... Wait, our models of semantics should inform fluid mechanics?!?
by johnswentworth — 2024-08-26T16:38:53.924Z

This post is styled after conversations we’ve had in the course of our research, put together in a way that hopefully highlights a bunch of relatively recent and (ironically) hard-to-articulate ideas around natural abstractions.

John: So we’ve been working a bit on semantics, and also separately on fluid mechanics. Our main goal for both of them is to figure out more of the higher-level natural abstract data structures. But I’m concerned that the two threads haven’t been informing each other as much as they should.

David: Okay…what do you mean by “as much as they should”? I mean, there’s the foundational natural latent framework, and that’s been useful for our thinking on both semantics and fluid mechanics. But beyond that, concretely, in what ways do (should?) semantics and fluid mechanics inform each other?

John: We should see the same types of higher-level data structures across both - e.g. the “geometry + trajectory” natural latents we used in the

semantics post

should, insofar as the post correctly captures the relevant concepts, generalize to recognizable “objects” in a fluid flow, like eddies (modulo adjustments for nonrigid objects).

David: Sure, I did think it was intuitive to think along those lines as a model for eddies in fluid flow. But in general, why expect to see the same types of data structures for semantics and fluid flow? Why not expect various phenomena in fluid flow to be more suited to representation in some data structures which aren’t the exact same type as those used for the referrents of human words?

John: Specifically, I claim that the types of high-level data structures which are natural for fluid flow should be a subset of the types needed for semantics. If there’s a type of high-level data structure which is natural for fluid flow, but doesn’t match any of the semantic types (noun, verb, adjective, short phrases constructed from those, etc), then that pretty directly disproves at least one version of the natural abstraction hypothesis (and it’s a version which I currently think is probably true).

David: Woah, hold up, that sounds like a very different form of the natural abstraction hypothesis than our audience has heard before! It almost sounds like you’re saying that there are no “non-linguistic concepts”. But I know you actually think that much/most of human cognition routes through “non-linguistic concep...