trentmkelly/LessWrong-43k huggingface dataset:
$300 for the best sci-fi prompt: the results
by RomanS — 2024-01-03T19:10:19.353Z

Six months ago we announced:

We would like to find the best prompt to make GPT-4 do the following:

write the first chapter of a science fiction novel
the result should be good enough to make seasoned sci-fi readers (us) crave for a continuation

Why?

to create a "midjourney moment" for writers. The sooner we have an AI-written Nebula-quality novel, the more people will realize that the actually intelligent AI is already here
to better evaluate the creativity of GPT-4
to blow our minds with a surprisingly good AI-generated sci-fi story (which is the ultimate science fiction moment by itself).

Today we announce the winner: Waldvogel. The stories generated with their prompt turned out to be the most captivating and the most enjoyable to read. We ask them to message us so we discuss how to better transfer the $300.

At the end of this post are 3 stories that GPT-4 has written, using the winner’s prompt plus our custom plot summaries. You can find the prompt and the plot summaries at the very end.

How we evaluated the prompts:

Generate 3 stories for each prompt, using our plot summaries. 1 shot.
Read each story and rate it from 1 to 10. The criterion: how strongly we want to read the continuation.
Sum up the scores for each prompt.
The prompt with the highest total score – wins.

Some conclusions we drew from reading the generated stories

The whole experience is liberating. If you’re writing a story, you can now write a decent first draft in minutes. The gap between your imagination and a decent text is much smaller now.

One learns to recognize the prose generated by GPT-4. The plot twists may be original and interesting, but many story elements and many phrases are often reused.

A competent editor is required to make a publishable material from it. But perhaps one can partially replace the editor with GPT-4 itself.

The overall quality strongly depends on the prompt. It’s possible that some better prompt can squeeze even better stories out of GPT-4.

The participants have tried several prompting strategies: write a long enumeration of requirements, ask GPT-4 itself to write the requirements, just ask GPT-4 to pretend to be a writer, add some quirky personality traits, etc. At the end, the strongest prompt turned out to be one of the simplest (59 words). But the second-strongest prompt contains 1100 words, and it was not far behind by the total score.

We used the v...
Agent membranes/boundaries and formalizing “safety”
by Chris Lakin — 2024-01-03T17:55:21.018Z

It might be possible to formalize what it means for an agent/moral patient to be safe via

membranes/boundaries

. This post tells one (just one) story for how the membranes idea could be useful for thinking about existential risk and AI safety.

Formalizing “safety” using agent membranes

A few examples:

A bacterium uses its membrane to protect its internal processes from external influences.

A nation maintains its sovereignty by defending its borders.

A human protects their mental integrity by selectively filtering the information that comes in and out of their mind.

A natural abstraction for agent safety?

Agent boundaries/membranes seem to be a natural abstraction representing the safety and autonomy of agents.

A bacterium survives only if its membrane is preserved.
A nation maintains its sovereignty only if its borders aren’t invaded.
A human mind maintains mental integrity only if it can hold off informational manipulation.

Maybe the safety of agents could be largely formalized as the preservation of their membranes.

Distinct from preferences!

Boundaries are also cool because they show a way to respect agents without needing to talking about their preferences or utility functions. Andrew Critch has said the following about this idea:

my goal is to treat boundaries as more fundamental than preferences, rather than as merely a feature of them. In other words, I think boundaries are probably better able to carve reality at the joints than either preferences or utility functions, for the purpose of creating a good working relationship between humanity and AI technology (

«Boundaries» Sequence, Part 3b

)

For instance, respecting the boundary of bacterium would probably mean “preserving or not disrupting its membrane” (as opposed to knowing its preferences and satisfying them).

Protecting agents and infrastructure

By formalizing and preserving the important boundaries in the world, we could be in a better position to protect humanity from AI threats. Examples:

Critical computing infrastructure could be secured by creating strong boundaries around them. This can be enforced by cryptography and formal methods such that only the subprocesses that need to have read and/or write access to a particular resource (like memory) have the encryption keys to do so.

Related: Object-capability model, Principle of least privilege, Evan Miyazono’s Atlas Co

...
Safety First: safety before full alignment. The deontic sufficiency hypothesis. 
by Chris Lakin — 2024-01-03T17:55:19.825Z

It could be the case that these two goals are separable and independent:

“AI safety”: avoiding existential risk, s-risk, actively negative outcomes
“AI getting-everything-we-want” (

CEV

)

This is what Davidad calls this the Deontic Sufficiency Hypothesis.

If the hypothesis is true, it should be possible to de-pessimize and mitigate the urgent risk from AI without necessarily ensuring that AI creates actively positive outcomes. Because, for safety, it is only necessary to ensure that actively harmful outcomes do not occur. And hopefully this is easier than achieving “full alignment”.

Safety first! We can figure out the rest later.

Quotes from Davidad's The Open Agency Architecture plans

This is Davidad’s plan with the

Open Agency Architecture

(OAA).

A list of core AI safety problems and how I hope to solve them

(2023 August)

1.1. First, instead of trying to specify "value", instead "de-pessimize" and specify the absence of a catastrophe, and maybe a handful of bounded constructive tasks like supplying clean water. A de-pessimizing OAA would effectively buy humanity some time, and freedom to experiment with less risk, for tackling the CEV-style alignment problem—which is harder than merely mitigating extinction risk.

Davidad's Bold Plan for Alignment: An In-Depth Explanation — LessWrong

(2023 April)

Deontic Sufficiency Hypothesis: This hypothesis posits that it is possible to identify desiderata that are adequate to ensure the model doesn't engage in undesirable behavior. Davidad is optimistic that it's feasible to find desiderata ensuring safety for a few weeks before a better solution is discovered, making this a weaker approach than solving outer alignment. For instance, Davidad suggests that even without a deep understanding of music, you can be confident your hearing is safe by ensuring the sound pressure level remains below 80 decibels. However, since the model would still be executing a pivotal process with significant influence, relying on a partial solution for decades could be risky.

Getting traction on the deontic feasibility [sic] hypothesis

Davidad believes that using formalisms such as Markov Blankets would be crucial in encoding the desiderata that the AI should not cross boundary lines at various levels of the world-model. We only need to “imply high probability of existential safety”, so according to davida

...
Practically A Book Review: Appendix to "Nonlinear's Evidence: Debunking False and Misleading Claims" (ThingOfThings)
by tailcalled — 2024-01-03T17:07:13.990Z
Link: https://thingofthings.substack.com/p/practically-a-book-review-appendix

Subtitle: Taking nonprescription amphetamines across the U.S.-Mexico border is a felony

I haven't followed the controversy long enough to be able to tell how correct Ozy is, but what I like about this post is that I find it better structured than some posts, and with better focus on strategically relevant information.

For instance, in response to Nonlinear's claim that they weren't isolating Alice and Chloe, Ozy wrote:

Kat believes in the importance of digital nomads remaining socially connected to others. However, Kat and Emerson had a consistent pattern of encouraging Alice and Chloe to spend time with people they considered high value (i.e. effective altruists, especially those working in AI safety) instead of people they considered low-value. To be clear, Kat and Emerson didn't think Alice and Chloe should completely isolate themselves from people who weren’t effective altruists. Kat encouraged Alice and Chloe to call their families regularly. She explicitly supports spending some time with locals. Friends and family who didn't work in AI safety were invited to travel with Nonlinear, although they were lower priority to invite than AI safety people.

However, the vast majority of Kat’s evidence that she didn't isolate Alice and Chloe is evidence that she didn't isolate Alice and Chloe from effective altruists, particularly "top" effective altruists working in AI safety. Alice and Chloe were given lots of access to so-called top effective altruists: there was an average of 7 people living in the house. Nonlinear encouraged networking with FTX people. They traveled with Chloe's boyfriend, whom Kat Woods considered to "have high potential." Inviting people to travel with Nonlinear was framed as "one of the highest ROI things you can do”—that is, as an important means of bettering the world.

Kat and Emerson discouraged Alice from visiting her family because her trip overlapped with "some of the top figures in the field" coming to visit. (The chatlogs are suggestive that Alice timed her visit around a family emergency, but Kat doesn't explicitly mention this.) Kat also discouraged Alice from spending too much time socializing with locals, saying that she would have higher impact if she spent time with higher-value people.

It doesn't seem to have occurred to Nonlinear that Alice and Chloe might have been complaining about being isolated from people who aren't effecti

...
Trivial Mathematics as a Path Forward
by ACrackedPot — 2024-01-03T16:41:32.537Z

A gentleman by the name of Dan Willard, who recently passed, was a pioneer in a relatively unexamined area of mathematics: Mathematical frameworks that are just trivial enough to bypass Gödel. That is, mathematical frameworks that can prove their own consistency. He did so for a reason which may be of interest here: Creating AI that can verify itself.

I stumbled across him for an entirely unrelated reason, related to my username. (Noticing that, insofar as the universe is a mathematical system, things seem like they might get kind of weird if Gödel applies. I had this weird idea about a perfectly bijective mathematical framework and started investigating whether or not it would be subject to Gödel - answer: unclear, as I found this guy instead, and started thinking about his ideas.)

Willard's approach is of course one option - creating mathematics that simply lacked the precise mechanics that run into Gödel, namely, multiplication and addition (he has them, but they're defined in a way that avoids the issue). Thus, the approach is necessarily incomplete - that is, it cannot prove every true statement about the arithmetic of the natural numbers.

But since no finite (compressible to a finite algorithmic basis or set of bases) mathematical system can, we can't exactly hold that against it. (And also, I'm left with a personally annoyed sense of "Wait, that was always an option?")

Thus, contrary to a naive understanding of Gödel, we can in fact create a mathematical framework that can prove its own consistency!

This doesn't imply an easy solution to the Halting Problem, which would be the next step; we could cripple a language in a similar fashion to avoid the disproofs - say, by assigning a finite integer index N to each function with a lower bound of 0, and only permitting functions to call functions with a lower index. Is the halting problem decidable?

Well, if you can express the Goldbach Conjecture in it, I'm going to lean towards "No." The way that addition and multiplication are defined may preclude the expression of the Goldbach Conjecture, but it doesn't quickly look like this is the case to my eye. (There are some modifications to the Turing Machine I have considered that look like they might make the problem tractable, including the above indices, but they also complicate the heck out of any proofs, as th...
Copyright Confrontation #1
by Zvi — 2024-01-03T15:50:04.850Z

Lawsuits and legal issues over copyright continued to get a lot of attention this week, so I’m gathering those topics into their own post. The ‘virtual #0’ post is the relevant section from last week’s roundup.

Four Core Claims

Who will win the case? Which of New York Times’s complaints will be convincing?

Different people have different theories of the case.

Part of that is that there are four distinct allegations NYT is throwing at the wall.

Arvind Narayanan: A thread on some misconceptions about the NYT lawsuit against OpenAI. Morality aside, the legal issues are far from clear cut. Gen AI makes an end run around copyright and IMO this can’t be fully resolved by the courts alone.

As I currently understand it, NYT alleges that OpenAI engaged in 4 types of unauthorized copying of its articles:

The training dataset

The LLMs themselves encode copies in their parameters

Output of memorized articles in response to queries

Output of articles using browsing plugin

Key Claim: The Training Dataset Contains Copyrighted Material

Which, of course, it does.

The training dataset is the straightforward baseline battle royale. The main event.

The real issue is the use of NYT data for training without compensation … Unfortunately, these stand on far murkier legal ground, and several lawsuits along these lines have already been dismissed.

It is unclear how well current copyright law can deal with the labor appropriation inherent to the way generative AI is being built today. Note that *people* could always do the things gen AI does, and it was never a problem.

We have a problem now because those things are being done (1) in an automated way (2) at a billionfold greater scale (3) by companies that have vastly more power in the market than artists, writers, publishers, etc.

Bingo. That’s the real issue. Can you train an LLM or other AI on other people’s copyrighted data without their permission? If you do, do you owe compensation?

A lot of people are confident in very different answers to this question, both in terms of the positive questions of what the law says and what society will do, and also the normative question what society should decide.

Daniel Jeffries, for example, is very confident that this is not how any of this works. We all learn, he points out, for free. Why should a computer system have to pay?

Do we all learn for free?...
Theoretically, could we balance the budget painlessly?
by Logan Zoellner — 2024-01-03T14:46:04.753Z

Economically, all government spending takes the form of forgone private consumption. This implies that all deficit spending is in fact a sort of tax.
Suppose we were to raise taxes to make this implicit tax explicit. For the purpose of this hypothetical, let's image the US imposes a VAT exactly equal to the government deficit. It seems like the following would happen:

A bunch of people would have much higher taxes, and would lower their consumption to be able to pay those taxes
this would send the economy into a recession as aggregate consumption dropped.
The federal reserve would lower interest rates (or do quantitative easing) to stimulate demand.
Eventually the economy would reach a new equilibrium (which presumably would contain the same amount of private consumption as the old equilibrium).

As a mini example of this, consider the recent Japanese VAT hike.

The question is, could we skip straight from step 2. to step 4. without all of the intervening suffering (usually a recession causes unemployment, lowered consumption, etc.)? What would be the most effective way to do so?
Johannes' Biography
by Johannes C. Mayer — 2024-01-03T13:27:19.329Z

Schedule a meeting with me.

Tsuyoku Naritai! I operate on a modified version of Crocker's rules. I am working on making AGI not kill everyone aka AGI notkilleveryoneism.

Read here about what I would do after taking over the world.

"Maximize the positive conscious experiences, and minimize the negative conscious experiences in the universe", is probably not exactly what I care about, but I think it roughly points in the right direction.

I recommend the following:

Believe in yourself.

Think for yourself.

Don't give up because it gets difficult.

This is how I look like:

I just released a major update to my LessWrong Bio. I have rewritten almost everything and added more stuff. It's now so long that I thought it would be good to add the following hint in the beginning:

(If you are looking for the list of <sequences/posts/comments> scroll to the bottom of the page with the END key and then go up. This involves a lot less scrolling.)

(If you'd like to <ask a question about/comment on/vote> this document, you can do so on the shortform announcing the release of this version of the bio.)

Feedback

I appreciate any positive or negative feedback. Especially if it is constructive criticism which helps me to grow. You can do this in person, use this (optionally anonymous) feedback form, or any other way you like.

Buck once said that he avoids critical feedback, because he worries about people's feelings, especially if he does not know what they could do instead (in the context of AGI notkilleveryoneism). If you are also worried that your feedback might harm me, you might want to read about my strategy for handling feedback. I am not perfect at not being hurt, but I believe myself to be much better than most people. If I am overwhelmed, I will tell you. That being said, I appreciate it if your communication is optimized to not hurt my feelings, all else equal. But if that would make you not give feedback, or would be annoying, don't worry about it. Really.

What GPT-4 thinks of me

I used the obsidian-smart-connections plugin to generate an embedding for all my written notes in obsidian (~1,000,000 words). Then I ask GPT-4 (again using the plugin) to tell me what kind of person it thinks I am. The plugin then searched for relevant excerpts (according to the embedding) and gave it to GPT-4 to answer the question:

Based on your notes, you appear to be a highl

...
What Helped Me - Kale, Blood, CPAP, X-tiamine, Methylphenidate
by Johannes C. Mayer — 2024-01-03T13:22:11.700Z

Here are some things that in the last 6 months have helped me a lot, mainly with handling chronic fatigue and ADHD. This post is very quickly thrown together and it seems more appropriate for a short form, however just in case this is exactly the information that somebody needs to hear, I'd like to increase the visibility by making it a regular post.

I did take bupropion in the past as an antidepressant. However, I have discovered that eating 500g of well-cooked kale works better than Bupropion an antidepressant. Bupropion works ok but gives me insomnia. I put frozen pre-chopped kale in a pressure cooker and heat it at maximum heat until the maximum pressure is reached. Then I continue to heat the pot such that it stays at approximately maximum pressure for 15 minutes, and then I let it cool down on its own.

Starting to take 600mg benfotiamine daily might have had a big positive effect but I have not confirmed this rigorously. I am borderline diabetic which is why I think this might be important.

I am also eating 200ml of pig's blood once or twice a week. I put it in a pod and steam it for 15-25 minutes until it turns into a solid pudding-like substance. Nuking it also works. It might be the case that I am very bad at absorbing the iron supplements that I tried so far. Or maybe blood has another effect like raising IGF-1 or something like that. I think I fucked myself by not eating meat for many years. Empirically it seems that I am more awake when eating meat, but when eating blood this effect is much stronger, which seems like weak evidence that iron is the issue as there is much more in blood.

I also got a CPAP device, which probably helps significantly.

I know the blood thing sounds probably ridiculous but I am very sure that it has a very strong effect. Even before the CPAP I ate blood every day for some time, and it made me think that I cured myself because I was just so much more awake. But with CPAP and potentially the other things it got even better.

Also, I am very sure that the kale has an antidepressant effect. First I stopped eating kale two times and each time I could feel my depression coming back. This effect was so strong that I googled if kale had an antidepressant effect which Google told me it has. I did not start eating kale because I thought it would cure my depression but just because I thought it was a generally healthy thing to do.

Also, I ...
Does LessWrong make a difference when it comes to AI alignment?
by PhilosophicalSoul — 2024-01-03T12:21:32.587Z

I see LessWrong is currently obsessed with AI Alignment. I spoke with some others on the unofficial LessWrong discord, and agreed that LessWrong is becoming more and more specialised, thus scaring off any newcomers who aren't interested in AI.

That aside. I'm genuinely curious. Do any of the posts on LessWrong make any difference in the general psychosphere of AI alignment? Does anyone who has actual control on the direction of AI and LLM's follow LessWrong? Does Sam Altman or anyone at OpenAI engage with LessWrongers?

Not being condescending here. I'm just asking this since there's two (2) important things to note: (1) Since LessWrong has very little focus on anything other than AI at the moment, are these efforts meaningful? (2) What are some basic beginner resources someone can use to understand the flood of complex AI posts currently on the front page? (Maybe I'm being ignorant, but I haven't found a sequence dedicated to AI...yet.)