trentmkelly/LessWrong-43k huggingface dataset:
If you're not a morning person, consider quitting allergy pills
by Brendan Long — 2023-05-24T20:11:07.131Z

This is probably obvious to a lot of people, but I thought it still might be useful since I didn't think of it myself for years. Since sometime in college, I've been the kind of person who wakes up groggily at 10 am and then drags myself out of bed. I've always been a late sleeper so I didn't think much about it, but when I stopped taking allergy pills, I fairly quickly became someone who woke up around 8:30 am without much effort (still not early but not crazy-late).

I was aware that allergy pills make you sleepy, but the actual effect when I take them has never been that strong for me, so I never put together that they might still be having a major effect 10+ hours later. I still occasionally take allergy pills in certain cases, and now that I'm looking for it, it's really obvious how they effect my next morning.

For what it's worth, of the two brands that work on my allergies, 6-hour Benadryl and 24-hour Zyrtec seem to have the same effect on my mornings despite claiming different time-periods.

So, if you take allergy pills and find it very difficult to wake up in the morning, consider trying not taking them for a few weeks. There was a reason I was taking them and this was a minor decrease in my quality of life, but getting 2 more hours of time in the morning more than makes up for it.
Adumbrations on AGI from an outsider
by nicholashalden — 2023-05-24T17:41:13.048Z
Link: https://nicholashalden.home.blog/2023/05/24/adumbrations-on-agi-from-an-outsider/

Preamble

A lot of people have written against AI Doom, but I thought it might be interesting to give my account as an outsider encountering these arguments. Even if I don’t end up convincing people who have made AI alignment central to their careers and lives, maybe I’ll at least help some of them understand why the general public, and specifically the group of intelligent people which encounters their arguments, is generally not persuaded by their material. There may be inaccuracies in my account of the AI Doom argument, but this is how I think it’s generally understood by the average intelligent non-expert reader.

I started taking AI alignment arguments seriously when GPT-3 and GPT-4 came out, and started producing amazing results on standardized testing and writing tasks. I am not an ML engineer, do not know much about programming, and am not part of the rationalist community that has been structured around caring deeply about AI risk for the last fifteen years. It may be of interest that I am a professional forecaster, but of financial asset prices, not of geopolitical events or the success of nascent technologies. My knowledge of the arguments comes mostly from reading LessWrong, ACX and other online articles, and specifically I’m responding to Eliezer’s argument detailed in the pages on Orthogonality, Instrumental Convergence, and List of Lethalities (plus the recent Time article).

I. AI doom is unlikely, and it’s weird to me that clearly brilliant people think it’s >90% likely

I agree with the following points:

An AI can probably get much smarter than a human, and it’s only a matter of time before it does
Something being very smart doesn’t make it nice (orthogonality, I think)
A superintelligence doesn’t need to hate you to kill you; any kind of thing-maximizer might end up turning the atoms you’re made of into that thing without specifically wanting to destroy you (instrumental convergence, I think)
Computers hooked up to the internet have plenty of real-world capability via sending emails/crypto/bank account hacking/every other modern cyber convenience.

The argument then goes on to say that, if you take a superintelligence and tell it to build paperclips, it’s going to tile the universe with paperclips, killing everyone in the process (oversimplified). Since the people who use AI are obviously going to tell it to do stuff–we already do that with GPT-4–as ...
Open Thread With Experimental Feature: Reactions
by jimrandomh — 2023-05-24T16:46:39.367Z

This open thread introduces an experimental extension of LessWrong's voting system: reactions. Unlike votes, reactions are public; hovering over the reactions will show a list of users who reacted. For now, this feature is only for comments on this post in particular; after collecting feedback, we might roll out more broadly, or make significant alterations, or scrap it entirely. Reactions to comments in this thread will be preserved while discussions here are active, but they may be lost later if the feature changes in an incompatible way. Using this feature in various ways is planned to have karma minimums, but for this experimental post, those karma minimums are temporarily reduced to zero.

These are similar to the reactions found on other sites such as Slack and Discord, but with a few twists. The first difference is the palette of reactions offered.

The palette of available reactions, and the first page of reactions in particular, is explicitly a claim about what we think people should care about in comments (both positive and negative). On LessWrong, that means ways of relating to comments epistemically, reasons why comments are good and bad contributions.

Reactions currently can only be applied to comments (not posts), only on this thread (for experimentation purposes), and only on entire comments, not individual comment sections. Depending on feedback, we may roll it out to comments on all posts, allow reactions to posts not just comments, and maybe implement something to allow reacting to specific parts of comments and not just to the whole thing.

Antireacts

Unlike other sites that have reactions, LessWrong has antireactions. An antireaction is like downvoting a reaction, or placing -1 reactions. This doesn't affect anyone's karma, but will show your name in the hover-over as someone who disagreed with the react. If there are at least as many antireactions of a given type as there are reactions of that type, then the icon won't be shown to people who don't hover over the reactions icon.

The idea here is that, since we are trying to promote reactions that have more semantic content than the reactions you would find on most sites, many of the reactions in the palette are capable of being incorrect in a way that purely emotive reactions aren't.

Motivation

In my experience, seeing a list of names that I respect, marked as having reacted positively to...
A rejection of the Orthogonality Thesis
by ArisC — 2023-05-24T16:37:51.056Z
Link: https://medium.com/@arisc/there-is-no-good-case-with-agi-792e4995ebb9

I shared a blog post I wrote in Astral Codex, and someone suggested I try and share with the rationalist community (though it's somewhat polemic in tone). So here goes! The blog post is a little wider in scope, but I think the relevant part for the rationalist community is the rejection of the orthogonality thesis. The TL;DR is that the orthogonality thesis is often presented as a fact, but it seems to me that it's mostly a series of assertions, namely:

There is a large mind design space. Do we have any actual reasons for thinking so? Sure, one can argue everything has a large design space, but in practice, there's often an underlying unique mechanism for how things work.
Ethics are not an emergent property of intelligence - but again, that's just an assertion. There's no reason to believe or disbelieve it. It's possible that self-reflection (and hence ethics and the ability to question one's goals and motivations) is a pre-requisite for general cognition - we don't know whether this is true or not because we don't really understand intelligence yet.
The previous two are assertions that could be true, but reflective stability is definitely not true - it's paradoxical. To quote from my post:

This line of reasoning is absurd: it assumes an agent knows in advance the precise effects of self-improvement — but that’s not how learning works! If you knew exactly how an alteration in your understanding of the world would impact you, you wouldn’t need the alteration: to be able to make that judgement, you’d have to be able to reason as though you had already undergone it (of course, you can predict some of the effects of a particular course of study/self-improvement: for instance, you know that if you take a course in accounting, you’ll become better at reading financial statements. But you have no idea what other effects this course will have on your worldview — for instance, an accounting course might cause you to hate reading financial statements. If you did — if you could think exactly as you would after the course - you wouldn’t need the course.)

So if the argument the OT proponents are making is that AI will not self-improve out of fear of jeopardising its commitment to its original goal, then the entire OT is moot, because AI will never risk self-improving at all.

(To tackle the Gandhi analogy head on: obviously Ghandi wouldn’t take a pill if it were sold to him as

...
Aligned AI via monitoring objectives in AutoGPT-like systems
by Paul Colognese — 2023-05-24T15:59:13.621Z

Thanks to Arun Jose, Joseph Bloom, and Johannes Treutlein for feedback/discussions.

Introduction

The release of

AutoGPT

prompted

discussions

related to the potential of such systems to turn non-agentic LLMs into agentic systems that pursue goals, along with the dangers that could follow. The relation of such systems to the

alignment

problem

has also been explored.

In this short post, we investigate a threat model that comes from AutoGPT-like systems pursuing unaligned objectives and explore the potential for alignment via oversight. We briefly consider some key properties of such systems, and then discuss the idea that these systems’ high-level cognition might be interpretable by default and so might allow for sufficient oversight to ensure the system is aligned. Finally, we consider a couple of reasons for why the high-level cognition might be obscured from oversight and highlight ways of preventing this obscuration.

Background

AutoGPT-like systems, or

Scaffolded LLMs

, are systems that:

wrap a programmatic scaffold around an LLM core and chain together a number of individual LLM calls to achieve some larger and more complex task than can be accomplished in a single prompt.[1]

This programmatic scaffold allows for information from the inputs/outputs of LLM calls to be stored in long-term memory as well as used as inputs to

tools/plugins

.

We will state a few assumptions we’ll make about these AutoGPT-like systems. See this post for a similar context.

For the purposes of this post, we assume that a single LLM call is incapable of successfully pursuing objectives. That is, the threat comes from the LLM being used within the scaffolded system.
We assume that LLMs in the system must generate prompts/instructions for other versions of itself (factored cognition/bureaucratic approach)
When information is passed from one LLM call/module/plugin to another, we say that this information is passed through an “information channel”. In particular, an overseer can observe this information (in the form of token sequences, etc.)

The threat model we’ll look at is one in which we have an agentic AutoGPT-like system that pursues unaligned objectives.

We don’t discuss the likelihood of this threat model being realized but note that these systems may suffer from

inefficiencies that come from bureaucratic systems

and hence may not be co...
The Office of Science and Technology Policy put out a request for information on A.I.
by HiroSakuraba — 2023-05-24T13:33:30.672Z
Link: https://www.whitehouse.gov/wp-content/uploads/2023/05/OSTP-Request-for-Information-National-Priorities-for-Artificial-Intelligence.pdf

This request for information does cover some of the possible existential risks, in the first section. I am going to submit a few responses of my own, and I am hopeful others will do the same.

1. What specific measures – such as standards, regulations, investments, and improved trust and safety practices – are needed to ensure that AI systems are designed, developed, and deployed in a manner that protects people’s rights and safety? Which specific entities should develop and implement these measures?

2. How can the principles and practices for identifying and mitigating risks from AI, as outlined in the Blueprint for an AI Bill of Rights and the AI Risk Management Framework, be leveraged most effectively to tackle harms posed by the development and use of specific types of AI systems, such as large language models?

3. Are there forms of voluntary or mandatory oversight of AI systems that would help mitigate risk? Can inspiration be drawn from analogous or instructive models of risk management in other sectors, such as laws and policies that promote oversight through registration, incentives, certification, or licensing?

4. What are the national security benefits associated with AI? What can be done to maximize those benefits?

5. How can AI, including large language models, be used to generate and maintain more secure software and hardware, including software code incorporating best practices in design, coding and post deployment vulnerabilities?

6. How can AI rapidly identify cyber vulnerabilities in existing critical infrastructure systems and accelerate addressing them?

7. What are the national security risks associated with AI? What can be done to mitigate these risks?

8. How does AI affect the United States’ commitment to cut greenhouse gases by 50-52% by 2030, and the Administration’s objective of net-zero greenhouse gas emissions no later than 2050? How does it affect other aspects of environmental quality?
ChatGPT (May 2023) on Designing Friendly Superintelligence
by Mitchell_Porter — 2023-05-24T10:47:16.325Z
Link: https://singularitypolitics.wordpress.com/2023/05/24/chatgpt-on-designing-friendly-superintelligence/

I asked ChatGPT how to make a human-friendly superintelligence, and it gave me a 50-page plan. There are no technical novelties, or any attempt to deal with the dangers that are supposed to be peculiar to superintelligence or self-enhancing systems. The manifesto is really just a potpourri of principles and methods that might be appropriate for any deep learning system that's supposed to be interacting with human beings. And yet I suspect startups have been founded on far less.

To me, this was interesting because this is what ChatGPT comes up with, when you ask it in the laziest possible way, with no prompt engineering at all, how to do this thing. As such, this is representative of its "spontaneous thoughts", the first things that come to its mind, when it is presented with this task. It is therefore also a glimpse of how an LLM-based agent might proceed, when similarly instructed.

I would be interested to see the results of similar experiments to this one. As I said, I made no effort at all, I just asked the question and then got it to expand on its own outline. The people who are actually developing large language models must have conducted similar, but far more sophisticated, experiments by now.
No - AI is just as energy-efficient as your brain.
by Maxwell Clarke — 2023-05-24T02:30:24.526Z

This is simply because AIs run on electricity.

Our brains use sunlight via photosynthesis via dietary energy intake. From sunlight to dietary energy this is about 0.25-0.5% energy efficient. Let's say it's 0.35% efficient. (This is then our complete sunlight-to-capability efficiency for a human)

AI systems use sunlight via solar energy via electrical consumption. From sunlight to GPU this is about 10-18% efficient let's say it's 13% efficient. (Data: typical solar panel efficiency = 15-20% efficient. Typical electricity distribution efficiency = 90% efficient). 13% is then AI sunlight-to-electricity efficiency, but not yet sunlight-to-capability efficiency.

Let's calculate the final part. We need to assume an amount of compute for a human-equivalent AI system. I will assume that (note, at inference time) we can run a human-equivalent AI system on 10 NVIDIA 4070 GPUs at full power, which each consume ~200 Watts, so 2 kW in total. In contrast the typical human consumes 100 watts. 100W divided by 2 kW gives us 5% as AI electricity-to-capability efficiency, which we then multiply to get the total AI efficiency. 5% electricity-to-capability efficiency times 13% sunlight-to-electricity efficiency = 0.65% sunlight-to-capability efficiency.

So humans are 0.35% efficient, and AIs are 0.65% efficient.
By these assumptions, AI is somewhat more efficient in terms of real energy input - here, energy from the sun. This is the number that the economy is going to care about - why use land for crops, when you can use it for solar panels?

Please pick apart my numbers and assumptions in the comments. Thanks.
What projects and efforts are there to promote AI safety research?
by Christopher King — 2023-05-24T00:33:47.554Z

As far as I can tell, the recognition of the existential danger of AI is at an all time. It is going mainstream! Unfortunately, most of the discourse seems very pessimistic. The bulk of the messaging seems to imply that the only thing we can do is wait for nuclear weapon style regulation and bide our time until death.

What projects and efforts are there to promote and recruit for AI existential safety research to those that are just learning about the existential danger? Are there any that unskilled volunteers could contribute to?
My May 2023 priorities for AI x-safety: more empathy, more unification of concerns, and less vilification of OpenAI
by Andrew_Critch — 2023-05-24T00:02:08.836Z

I have a mix of views on AI x-risk in general — and on OpenAI specifically — that no one seems to be able to remember, due to my views not being easily summarized as those of a particular tribe or social group or cluster. For some of the views I consider most neglected and urgently important at this very moment, I've decided to write them here, all-in-one-place to avoid presumptions that being "for X" means I'm necessarily "against Y" for various X and Y.

Probably these views will be confusing to read, especially if you're implicitly trying to pin down "which side" of some kind of debate or tribal affiliation I land on. As far as I can tell, I don't tend to choose my beliefs in a way that's strongly correlated with or caused by the people I affiliate with. As a result, I apologize in advance if I'm not easily remembered as "for" or "against" any particular protest or movement or trend, even though I in fact have pretty distinct views on most topics in this space... the views just aren't correlated according to the usual social-correlation-matrix.

Anyhoo:

Regarding "pausing": I think pausing superintelligence development using collective bargaining agreements between individuals and/or states and/or companies is a good idea, along the lines of FLI's recent open letter, "Pause Giant AI Experiments", which I signed early and advocated for.

Regarding OpenAI, I feel overall positively about them:

I think OpenAI has been a net-positive influence for reducing x-risk from AI, mainly by releasing products in a sufficiently helpful-yet-fallible form that society is now able to engage in less-abstract more-concrete public discourse to come to grips with AI and (soon) AI-risk.
I've found OpenAI's behaviors and effects as an institution to be well-aligned with my interpretations of what they've said publicly. That said, I'm also sympathetic to people other than me who expected more access to models or less access to models than what OpenAI has ended up granting; but my personal assessment, based on my prior expectations from reading their announcements, is "Yep, this is what I thought you told us you would do... thanks!". I've also found OpenAI's various public testimonies, especially to Congress, to move the needle on helping humanity come to grips with AI x-risk in a healthy and coordinated way (relative to what would happen if OpenAI made

...
AI Safety Newsletter #7: Disinformation, Governance Recommendations for AI labs, and Senate Hearings on AI
by Dan H — 2023-05-23T21:47:34.755Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-7

Welcome to the AI Safety Newsletter by the

Center for AI Safety

. We discuss developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions.

---

How AI enables disinformation

Yesterday, a

fake photo generated by an AI tool

showed an explosion at the Pentagon. The photo was falsely attributed to Bloomberg News and circulated quickly online. Within minutes, the stock market declined sharply, only to recover after it was discovered that the picture was a hoax.

This story is part of a broader trend. AIs can now generate text, audio, and images that are unnervingly similar to their naturally occurring counterparts. How will this affect our world, and what kinds of solutions are available?

The fake image generated by an AI showed an explosion at the Pentagon.

AIs can generate personalized scams. When John Podesta was the chair of Hillary Clinton’s 2016 presidential campaign, he fell for a scam that might soon become more common. He received what looked like an email from Google asking him to confirm his password. But after Podesta provided his login credentials, he learned that the email was from Russian hackers, who then

leaked thousands of his emails

and private documents to the public, which hindered Clinton’s campaign.

This is an example of a spear phishing attack, and language models could make it easier to tailor these attacks towards towards specific individuals. A

new paper

used ChatGPT and GPT-4 to write personalized emails to more than 600 British Members of Parliament, referencing the hometowns and proposed policies of each MP. These emails could be sent with phony links to malware or password collectors, allowing hackers to steal private information as in the Podesta case.

There is no silver bullet solution, but the paper proposes two countermeasures. First, AI can be used to identify and filter out spam. He notes that this becomes less effective as language models improve their impersonation of normal human text, and as humans use language models in their own writing. Second, he argues that companies should monitor for misuse of their AI tools and cut off access to anyone using AIs for illegal activities. Yet if companies open source their models to the public, this solution would be impossible.

Drowning out democratic voices. Democratic processes often encourage voters to provide input to lawmaker...