trentmkelly/LessWrong-43k huggingface dataset:
Bit Flip
by Charlie Sanders — 2023-04-16T07:30:55.720Z

In the peaceful and harmonious city of Elysium, a young man named Lucas stands before the Council of Wisdom, his heart filled with excitement. The council members, known for their benevolence and wisdom, have gathered to offer Lucas a once-in-a-lifetime opportunity.

"Lucas," the council leader begins, her voice warm and compassionate. "You have proven yourself to be an exemplary citizen, always putting others before yourself and contributing positively to our community. As a reward, we are offering you two options. You may either spend a year traveling the world, sharing your kindness and generosity, or you may enter a virtual world designed to bring you unparalleled joy and pleasure. You have five minutes to make your decision."

Lucas's mind races as he weighs his options. Traveling the world and spreading kindness seems fulfilling, but the thought of experiencing a virtual paradise is alluring. He wonders if the virtual world might give him the chance to explore his own desires and experience new sensations.

As the minutes tick away, Lucas makes his decision. "I choose the virtual world, esteemed council members," he says, his voice filled with anticipation.

The council leader nods approvingly, and Lucas is escorted to the Virtual Experience Chamber to begin his journey of indulgence.

Six months later, Lucas finds himself immersed in a vibrant, seemingly perfect virtual world. Every day, he experiences unimaginable pleasures and indulges in his wildest fantasies. At first, Lucas revels in the freedom and joy that the virtual world offers, but as time passes, he begins to feel a growing emptiness inside him.

One day, he discovers a hidden door within the virtual landscape, leading to a realm of darkness and chaos. This new realm offers him the opportunity to explore the darker side of life, to feel the thrill of transgression and experience the consequences of his actions.

"I never thought I'd feel so empty in paradise," Lucas whispers to himself as he contemplates his decision. "Maybe this new realm holds the key to truly understanding myself."

With a mix of fear and exhilaration, Lucas chooses to enter the realm of darkness, embracing the unknown in search of a deeper understanding of his desires and the human experience.

Daily Micro Fiction: Hello. I’d like to write a story together. We’ll start by me talking you through the type of story, the characters, the set...
Double-negation as framing
by Stuart Johnson — 2023-04-16T06:59:25.189Z

Any claim can be presented as positive or negative via the law of double negation. Not not P is P. This is only true for as long as you work within the axioms of formal logic. Disputes amongst humans do not abide by these axioms. Amongst humans, not not P is a very different proposition to P. It’s possible to completely shift the tone of a conversation by doubly negating a proposition, and if you ignore the double negation, you’ll instantly lose the argument. I think examining the axioms of rhetoric is a good way to understand people better.

Some of this ceases to apply if both parties try extra hard to make their arguments abide by mathematical logic, but when argumentation is war, that's never guaranteed.

What does it mean to win an argument anyway? In most situations, I think it means a combination of three things:

Getting your way
Convincing an audience that you’re right
Getting the audience to see your character positively

The last two things might seem the same, but they’re not. The competent jerk is right a lot, but diminishes his reputation with each spat. If you’re looking for a promotion, convincing spectators that you’re of good character is the only one that matters. If you ever get the opportunity to trade off either of the first two for the third, take it. It’s a good deal.

This is, of course, only true if you don’t have strong intrinsic motivations of your own. Many people earnestly care about what they perceive as optimality even if there’s no material benefit to pursuing it, and will sacrifice their reputation to argue it. Those people will ignore the illogicality of arguments between humans, for better or for worse.

I.

The size of the search space for doing things is very big. It’s much easier to identify the things that you’re not than the things that you are. It’s why comparison is the theft of joy. Deciding not to do things is often a more effective method for accumulating determination than deciding to do them. When trying to persuade an audience, stating that you believe something has better optics than saying you oppose something. Negative values signal fear or reactivity, which have bad optics. It’s a common form of political mudslinging to say something like “they don’t support[thing], they just hate [other thing].”

Framing is a device for changing a conversation by encouraging a specific interpretation of a piece of data. Rhetorically, a neg...
[Link/crosspost] [US] NTIA: AI Accountability Policy Request for Comment
by  — 2023-04-16T06:57:59.842Z
Link: https://forum.effectivealtruism.org/posts/GcrKndFY2oSKEFLub/us-ntia-ai-accountability-policy-request-for-comment

The Department of Commerce’s National Telecommunications and Information Administration (NTIA)[1] has launched an inquiry into what policies will help businesses, government, and the public be able to trust that Artificial Intelligence (AI) systems work as claimed – and without causing harm.

In line with this, the NTIA announced today, April 13, 2023, a request for public comments on Artificial Intelligence (“AI”) system accountability measures and policies.

Summary:

The National Telecommunications and Information Administration (NTIA) hereby requests comments on Artificial Intelligence (“AI”) system accountability measures and policies. This request focuses on self-regulatory, regulatory, and other measures and policies that are designed to provide reliable evidence to external stakeholders—that is, to provide assurance—that AI systems are legal, effective, ethical, safe, and otherwise trustworthy. NTIA will rely on these comments, along with other public engagements on this topic, to draft and issue a report on AI accountability policy development, focusing especially on the AI assurance ecosystem.[2]

NTIA is seeking input on what policies should shape the AI accountability ecosystem, including topics such as:

What kinds of data access is necessary to conduct audits and assessments
How can regulators and other actors incentivize and support credible assurance of AI systems along with other forms of accountability
What different approaches might be needed in different industry sectors—like employment or health care

If you have relevant knowledge regarding AI technical safety and/or governance, please consider submitting a comment. This is a notable opportunity to positively inform US policymaking.

You can find more information and formally submit your comments here.

Comments can be submitted as a known individual, on behalf of an organization, or anonymously.

The deadline to submit comments is June 12, 2023.
Who is testing AI Safety public outreach messaging?
by yanni kyriacos — 2023-04-16T06:57:46.232Z

Does anyone know who is testing AI Safety public outreach messaging? Or where I can find relevant research on this?

Things that would be interesting to test:

Salience (which messages are remembered and correctly attributed)
Persuasion (which messages get people to act)
Likability (which messages improve sentiment on topic X or for brand Y)
Attention (which messages get peoples attention)
Associations (what attributes come to mind when X message is delivered)

It may also be worth considering channels too. I.e. outdoor vs social vs audio vs OLV but that is a separate thing.

Thanks!

Yanni
Features of Emacs that I only recently discovered
by EmacsScrub — 2023-04-16T06:57:21.374Z

I am an Emacs user of several years, but only recently discovered some features that I find incredibly convenient. I am somewhat embarrassed by my ignorance (and for that reason write this post anonymously), but suspect that others might not be familiar with these features, and so hope that sharing them here might be of use. Note that for most of this post, I'll write assuming you know the basics of Emacs -- if you don't, some of the following may be mysterious to you.

Magit

The feature I've found most useful, but is most widely-discussed, is Magit. Magit is an interface to git inside Emacs. Why do you need Magit when you already know how to use git on the command line?

It's nice not to have to switch to the terminal to commit/pull/push.

Magit commands are shorter than git commands (in the right context, you can just press "c c" to commit, rather than typing "git commit").

Magit nicely shows you the state of your repository: Which changes have been staged, which staged changes have not yet been committed, the state of the remote branch, etc.

Magit also nicely displays which things you can do with it. For instance, the first time you press "c" to commit, you are shown a window of options for your commit -- for example, would you like to override the author? To allow an empty commit? To disable running hooks? By default you can immediately press "c" again to do none of these things, but if you want to do something unusual, Magit shows you how.

In general, it is a much nicer experience than using git on the command line, and I strongly recommend it to Emacs users who regularly interface with git.

Tramp

A situation I am regularly in is that I want to edit files on a remote machine. Previously, to do this, I would SSH into the remote machine, open Emacs in the terminal, and edit that way. This was functional, but meant that I had to set up Emacs on every machine I wanted to work on, which was sometimes impractical. It turns out that there is a better way: when you open a new file in Emacs with C-x C-f, you can type in a path like /ssh:my_remote_machine:/home/my_name/my_dir/my_file.txt, and make edits to the file via ssh. This does not require installing anything, it's a built-in feature of Emacs called "Tramp", and it just works: for instance, when editing a file on a remote machine, you can use Magit to take actions in that machine's copy of the repository.

Adm...
SmartyHeaderCode: anomalous tokens for GPT3.5 and GPT-4
by AdamYedidia — 2023-04-15T22:35:30.039Z

TL;DR: There are anomalous tokens for GPT3.5 and GPT4 which are difficult or impossible for the model to repeat; try playing around with SmartyHeaderCode, APolynomial, or davidjl. There are also plenty which can be repeated but are difficult for the model to spell out, like edTextBox or legalArgumentException.

A couple months ago, Jessica Rumbelow and mwatkins posted about anomalous tokens that cause GPT-2 and GPT-3 to fail. Those anomalous tokens don't cause the same failures on newer models, such as GPT-3.5 Default or GPT-4 on the ChatGPT website, or gpt-3.5-turbo over the API, because the newer models use a different tokenizer. For a very brief explanation of what a tokenizer is doing, the tokenizer has a large vocabulary of tokens, and it encodes ordinary text as a sequence of symbols from that vocabulary.

For example, the string Hello world! gets encoded by the GPT-2 tokenizer as the sequence [15496, 995, 0], meaning that it's a sequence of three tokens, the first of which is the 15,946th token of the vocabulary, or Hello, the second of which is the 995th token of the vocabulary, or world, and the third of which is the 0th token of the vocabulary, or !. In general, a long string being represented by a single token implies that that string appears a lot in the training set (or whatever corpus was used to build the tokenizer), because otherwise it wouldn't have been "worth it" to give that string its own token.

Because of the change in tokenizers, almost all of the tokens which produce anomalous behavior in GPT-2 and GPT-3 don't produce anomalous behavior in the later models, because rather than being a single weird token, they're broken up into many, more normal tokens. For example, SolidGoldMagikarp was encoded as a the single token SolidGoldMagikarp by the old tokenizer, but is encoded as five tokens by the new tokenizer: [' Solid', 'Gold', 'Mag', 'ik', 'arp']. Each of those five tokens is normal and common, so GPT-4 handles them just fine.

Also, it's conveniently the case that tokenizers are released to the public and generally seem to be ordered, with earlier tokens in the vocabulary being shorter, more common, and more ordinary, and later tokens in the vocabulary being longer, less common, and weirder.

The old tokenizer, r50k_base, used a vocabulary of about 50,000 tokens and was used by GPT-2 and GPT-3 (and possibly G...
Open-source LLMs may prove Bostrom's vulnerable world hypothesis
by Roope Ahvenharju — 2023-04-15T19:16:10.398Z

In short, Nick Bostrom's vulnerable world hypothesis states that humanity may in the future invent a technology that is potentially very destructive and very cheap to make and easily implementable. As a thought experiment, he uses something called "easier nukes" that would not have to be made from plutonium as in real life but could instead be made from battery and two pieces of glass. Bostrom says that if everyone could make nuclear weapons in their own home, civilization would be destroyed by default because terrorists, malcontent people and "folk who just want to see what would happen" would blow up most cities.

I can see similarities between Bostrom's hypothesis, and the way powerful LLM-models have recently been open sourced. It did not take long after the publication of ChatGPT and GPT-4 for several "open-source alternatives" to appear on the internet. Some of those you can even download to your own computer for offline use. And then we did have ChaosGPT ordering such a model to "destroy humanity". In my mind, he was one of the "folks who just wanted to see what would happen".

Recently I have been thinking that, in the long term, the biggest challenge in AI safety is the potential wide availability of the future AGI-systems. If we can create a safely aligned AGI, what would prevent some people from creating open-source alternative AGIs that are free from safety constraints and more powerful because of that? Advanced technology can't generally stay secret forever. And then we would have many future ChaosGPTs "who just want to see what would happen" and who could tell such a system to destroy humanity.

Old analogy used in AI safety communities about making a wish to genie and "getting what you asked but not what you wanted" would no longer be relevant in this scenario. Instead, the potential future ChaosGPTs would get exactly what they asked and perhaps even what they truly wanted.

Does the community here also think that this is a reasonable concern? I would like to know that in the comments and maybe start a discussion about the future priorities in AI safety. Because if, in the far future, practically everyone could be able to use open-sourced AGI-system and order it to destroy humanity, it would probably not be possible to completely prevent such malicious applications. Instead, the focus should be about increasing the societal resilience against de...
[linkpost] Elon Musk plans AI start-up to rival OpenAI
by Hatfield — 2023-04-15T19:06:09.223Z
Link: https://www.ft.com/content/2a96995b-c799-4281-8b60-b235e84aefe4

A few quotes:

Musk incorporated a company named X.AI on March 9, according to Nevada business records.

For the new project, Musk has secured thousands of high-powered GPU processors from Nvidia, said people with knowledge of the move.

Musk is recruiting engineers from top AI labs including DeepMind, according to those with knowledge of his plans, who said he began to explore the idea of a rival company earlier this year in response to the rapid progress of OpenAI.

Musk has brought on Igor Babuschkin, a former DeepMind employee, and roughly half a dozen other engineers. The Information previously reported Babuschkin’s early talks with Musk.

The new company would allow Musk to take on OpenAI, the Microsoft-backed group that he co-founded in 2015. He left the board three years later amid clashes with its management, including over attitudes to AI safety, according to two people who were involved in OpenAI at the time. Shortly afterwards, the organisation morphed into a for-profit start-up and raised a $1bn investment from Microsoft.

Since then, Musk has become increasingly vocal in his fears of broader existential threats from AI systems. He has also publicly criticised OpenAI for becoming, in his view, less transparent and too commercially minded in its pursuit of advanced AI. Musk is particularly concerned about the threat of models such as GPT-4, OpenAI’s latest release, to spew falsehoods and show political bias.
FLI report: Policymaking in the Pause
by Zach Stein-Perlman — 2023-04-15T17:01:06.727Z
Link: https://futureoflife.org/wp-content/uploads/2023/04/FLI_Policymaking_In_The_Pause.pdf

this policy brief provides policymakers with concrete recommendations for how governments can manage AI risks.

Policy recommendations:
1. Mandate robust third-party auditing and certification.
2. Regulate access to computational power.
3. Establish capable AI agencies at the national level.
4. Establish liability for AI-caused harms.
5. Introduce measures to prevent and track AI model leaks.
6. Expand technical AI safety research funding.
7. Develop standards for identifying and managing AI-generated content and recommendations.
Reflective journal entries using GPT-4 and Obsidian that demand less willpower.
by Solenoid_Entity — 2023-04-15T12:45:25.982Z

Something I've wanted to do for years, but can't make myself do consistently because I'm bad at forming new habits and seem to have a limited reservoir of willpower that is generally depleted by the end of the day:

Write a short journal entry at the end of every day, summarizing the day and reflecting on it.
Put this in Obsidian, and identify key concepts and make them back-links.
Enjoy as, over several years of consistent writing, the map of my life unfolds through the graph in Obsidian. Maybe this generates helpful insights, shows me patterns. Mostly it's just useful as a resource.

It's a lot of effort to be detailed and reflective, and it's a lot of effort to add the back-links.

The solution:

Rather than having to create a detailed, reflective entry from whole cloth, I can just write some short, abbreviated, incoherent mumbled paragraph, like a surly teenager at family dinner, and then ask ChatGPT to ask more questions to flesh out the story.

This works shockingly well.

It's surprisingly easy to write detailed answers to specific QUESTIONS about your day. It's like a good listener, it can draw you out with its insightful questions.

Then you can ask it to mark all the key words, names, etc, with double square brackets.

The end result isn't perfect but it's pretty good, and much, much better than I would ever do all by myself. It takes less than half the time and virtually no willpower.

Here's what I prompted ChatGPT today (That diary entry is roughly the level of writing I'm capable of at the end of the day.)[1]

Ask me 10 questions about the following diary entry, aiming to 'flesh it out' with extra details. If there are inconsistencies or missing information/context, you can ask for clarification or further information. Your aim is to ask questions that help make this a more complete and coherent account of the day. Some of the questions should relate to the theme of 'gratitude', and help provide more context and identify things to be grateful about in the text. Here's the text:[2]
Woke up early for no reason, lay in bed for 20 minutes thinking, then got up, said goodbye to Xena, headed off at around 7am for the train up to Waitara. Ended up missing the stupid thing because I was waiting on the wrong platform like an idiot. Still got to coffee in time with Adele, which was a relief since I've been an unreliable friend recently. Good coffee with

...
An example elevator pitch for AI doom
by laserfiche — 2023-04-15T12:29:17.303Z

I have been surprised to repeatedly see the claim that there isn't even an argument for concern about AI. That the claim has been made without evidence and can therefore be dismissed.

Obviously, there is an extensive library of evidence and arguments that have been made for decades. Additionally, I would argue that it is the default assumption. However, there is clearly still a need to have a concise argument that can be produced on the fly with no need to understand terminology or any additional background. Here is another attempt at that:

Humans obviously have human values. By definition, we are the most humanly aligned thing possible. And we still have a history of eradicating or subjugating any weaker subpopulation we come across. Neanderthals, previous hominids, populations in Africa, India, the Americas.
There is limited effort to align AIs to human values. GPT-4 is only fractionally aligned at best, so domination by a similar AI would obviously be worse than above.
New LLMs are not aligned at all. When GPT-4 was first red teamed, it was just as happy to give detailed instructions on genociding a population as it was to provide instructions to baking a cake. If it were possible for an LLM or LLM successor to FOOM or otherwise be released prior to further refinement, this is extremely relevant.
In agentized LLMs, the outer monologue IS the inner monologue. The model will say (out loud) "I need to come up with ideas about how to make money." If it then answers itself "Infiltrating systems and stealing money is the most effective method", it will then do that. Period.
An agentized LLM is already capable of training successor versions of itself, which would almost certainly be less aligned than itself (twice removed from humans).
There are plenty of resourceful companies training powerful Ais with even less of a concern for safety than OpenAI. There are companies and governments training powerful Ais with a complete disregard for safety. Since a concern for safety is a competitive disadvantage, this behavior is encouraged.

Does this mean that Ais are 100% certain to wipe out humanity? No, of course not. That's an absurd bar. Rather, the burden of proof should be to show that AIs are 99% certain not to cause catastrophe. If there...