trentmkelly/LessWrong-43k huggingface dataset:
AISN #29: Progress on the EU AI Act Plus, the NY Times sues OpenAI for Copyright Infringement, and Congressional Questions about Research Standards in AI Safety
by Dan H — 2024-01-04T16:09:31.336Z
Link: https://newsletter.safe.ai/p/aisn-29-progress-on-the-eu-ai-act

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions.

Listen to the AI Safety Newsletter for free on Spotify.

A Provisional Agreement on the EU AI Act

On December 8th, the EU Parliament, Council, and Commission reached a provisional agreement on the EU AI Act. The agreement regulates the deployment of AI in high risk applications such as hiring and credit pricing, and it bans private companies from building and deploying AI for unacceptable applications such as social credit scoring and individualized predictive policing.

Despite lobbying by some AI startups against regulation of foundation models, the agreement contains risk assessment and mitigation requirements for all general purpose AI systems. Specific requirements apply to AI systems trained with >1025 FLOP such as Google’s Gemini and OpenAI’s GPT-4.

Minimum basic transparency requirements for all GPAI. The provisional agreement regulates foundation models — using the term “general purpose AI,” or GPAI — in two tiers. In the first tier, all foundation models are required to meet minimum basic transparency requirements. For example, GPAI developers may be required to accompany their models with technical documentation, summaries of training data, and copyright protection policies. GPAI deployers will also have to inform users when they are interacting with AI, and GPAI-generated content is required to be labeled and detectable.

Additional safety requirements for GPAI with systemic risks. The provisional agreement defines models that are trained with computing power greater than 1025 FLOPs as carrying systemic risks. This tier of GPAI — which currently only includes OpenAI’s GPT-4 and Google’s Gemini — is subject to additional safety requirements beyond minimum basic transparency. Developers of such a model are required to:

Evaluate it for systemic risks
Conduct and document adversarial testing
Implement strong cybersecurity practices
Document and report its energy consumption

The agreement similarly regulates non-GPAI in three tiers. Some AI uses — such as social scoring, individual predictive policing, and untargeted scraping of data for facial recognition databases — are considered unacceptable, and are banned outright.

Systems outside of this list, but which ot...
EAG Bay Area Satellite event: AI Institution Design Hackathon 2024
by beatrice@foresight.org — 2024-01-04T15:02:13.582Z

Applications are open!

Apply to attend

here

Deadline: Jan 18, 2024

This event will:

Explore near-term Transformative AI scenarios to identify and evaluate existing and possible future institutions or coordination architectures for the safe and beneficial development of TAI technologies.
Host up to 45 participants, who will be supported by experienced mentors. Application based.
Be held on February 5th - 6th; as a satellite event to EAG Bay Area 2024
Be held at

The Institute

, Salesforce Tower, San Francisco.

Hackathon Format

This is a two-day hackathon where participants will create prototype solutions to AI challenges, with a special focus on formats for new institutions.
Before the event begins, participants will receive a list of suggested readings and will be asked to complete a survey that covers forthcoming challenges in the field and the effectiveness of existing institutions in governing AI.
At the end of the hackathon, the prototypes will be unveiled, and a Development Grant winner will be announced, which includes an opportunity for further development and implementation of their idea. A report summarizing the hackathon’s results will also be produced.

Should you apply?

We invite applications from researchers, funders, and builders in AI, social sciences, economics, mechanism design, game theory, systems thinking, and other relevant areas.

This event is hosted by the

Foresight Institute

and sponsored by the

Future of Life Institute

.
AI #45: To Be Determined
by Zvi — 2024-01-04T15:00:05.936Z

The first half of the week was filled with continued talk about the New York Times lawsuit against OpenAI, which I covered in its own post. Then that talk seemed to mostly die down,, and things were relatively quiet. We got a bunch of predictions for 2024, and I experimented with prediction markets for many of them.

Note that if you want to help contribute in a fun, free and low-key, participating in my prediction markets on Manifold is a way to do that. Each new participant in each market, even if small, adds intelligence, adds liquidity and provides me a tiny bonus. Also, of course, it is great to help get the word out to those who would be interested. Paid subscriptions and contributions to Balsa are of course also welcome.

I will hopefully be doing both a review of my 2023 predictions (mostly not about AI) once grading is complete, and also a post of 2024 predictions some time in January. I am taking suggestions for things to make additional predictions on in the comments.

Table of Contents

Copyright Confrontation #1 covered the New York Times lawsuit.

AI Impacts did an updated survey for 2023. Link goes to the survey. I plan to do a post summarizing the key results, once I have fully processed them, so I can refer back to it in the future.

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Google providing less every year?

Language Models Don’t Offer Mundane Utility. Left-libertarian or bust.

GPT-4 Real This Time. It’s not getting stupider, the world is changing.

Fun With Image Generation. The fun is all with MidJourney 6.0 these days.

Deepfaketown and Botpocalypse Soon. Confirm you are buying a real book.

They Took Our Jobs. Plans to compensate losers are not realistic.

Get Involved. Support Dwarkesh Patel, apply for Emergent Ventures.

Introducing. DPO methods? ‘On benchmarks’ is the new ‘in mice.’

In Other AI News. Square Enix say they’re going in on generative AI.

Doom? As many estimates of p(doom) went up in 2023 as went down. Why?

Quiet Speculations. Some other predictions.

The Week in Audio. Eric Jang on AI girlfriend empowerment.

Rhetorical Innovation. Machines and people, very different of course.

Politico Problems. Some sort of ongoing slanderous crusade.

Cup of Coffee. Just like advanced AI, it proves that you don’t love me.

Aligning a Smarter Than Human Intelli

...
Screen-supported Portable Monitor
by jefftk — 2024-01-04T13:50:06.179Z

I broke my MacBook screen

a few
weeks ago and it

didn't
end up making sense to repair it. I was considering getting a new
computer, but since I almost always use an external monitor maybe a
portable external monitor would work?

I found a well
reviewed 15.6" one for $130, capable of 1920x1080 (1080p / FHD).
It can do power and video over a single USB C cable, and if I fold the
cover back it will slide over my laptop's broken screen:

The plug-and-play was pretty great: I attached the cable and it just
worked! I did have to set the screen to 1920x1080 manually, though;
it defaulted to 960x540:

This is nice, but really I'd like something at eye level so I don't
hurt my neck looking down at the screen. Sometimes I'll be somewhere
I can put the monitor up on something, but it would be nice if I could
use the built-in screen for that:

It seems like it ought to be possible to make some sort of bracket,
possibly 3D printed, that attaches to the portable monitor and is
supported by the laptop screen? It has two VESA holes on the back for
mounting and weighs 1.6lb (726g). I can't find anyone online doing
this; is there a reason it wouldn't work?

Comment via: facebook, lesswrong, mastodon
Which investments for aligned-AI outcomes?
by tailcalled — 2024-01-04T13:28:57.198Z

The text below is a confused ramble to illustrate where my mindset is at.

Suppose we condition on developing some kind of AGI that is reasonably controllable. In that case, the future is going to look a lot different from the present, as I assume the value of a lot of human cognitive labor is going to drop a lot. It seems like it would be wise to invest so one is in a reasonably good position for life, even in that case.

One investment option would be stock in AI companies, under the assumption that this stock would go up with AI. This seems superficially plausible, but I can't help but worry that it's dangerous to put too much money into this, as there's the whole "moat" problem; couldn't other organizations just make their own AIs, thereby driving down the price of the AI itself to ~0?

I'm not sure what else would be worth investing in. Maybe complementary goods or resources (I've seen chip firms mentioned sometimes), but are there any that can be known to predictably increase in value?

Some people say to invest in land because land prices keep going up. But I can't help but wonder if there's a pyramid scheme element to this. In the end, we'd expect land to increase in value because people who need land get more productive and can therefore afford to bid it higher, but if the value of human labor drops, shouldn't we expect the value of land to drop too? Do we think governments could somehow put in some policies to prevent this? If they could, presumably this would lead to economic distortions, but which distortions would be most plausible to occur?

I guess ultimately if the value of human labor drops to ~0, we'd need something like Universal Basic Income to survive. This would have to be funded by taxing whoever owns whatever is generating economic value, and the taxes need to be enforced with capacity for violence, which in turn requires some solid source of economic value.

If we speculate that AI companies are the ones that generate economic value, then this would look something like the state taxing AI companies and then using those taxes partly for UBI and partly for hiring them to make AI systems that enforce the taxes. But if the AI companies are all American, then it would be harder for EU countries to tax them.

I guess the standard take would be that due to uncertainty, it's best to have a diverse set of investments. Which I guess is logical enough, but I...
Non-alignment project ideas for making transformative AI go well
by Lukas Finnveden — 2024-01-04T07:23:13.658Z
Link: https://www.forethought.org/research/project-ideas-for-making-transformative-ai-go-well-other-than-by-working-on-alignment

This is a series of posts with lists of projects that it could be valuable for someone to work on. The unifying theme is that they are projects that:

Would be especially valuable if transformative AI is coming in the next 10 years or so.
Are not primarily about controlling AI or aligning AI to human intentions.[1]

Most of the projects would be valuable even if we were guaranteed to get aligned AI.
Some of the projects would be especially valuable if we were inevitably going to get misaligned AI.

The posts contain some discussion of how important it is to work on these topics, but not a lot. For previous discussion (especially: discussing the objection “Why not leave these issues to future AI systems?”), you can see the section

How ITN are these issues?

from my previous

memo on some neglected topics

.

The lists are definitely not exhaustive. Failure to include an idea doesn’t necessarily mean I wouldn’t like it. (Similarly, although I’ve made some attempts to link to previous writings when appropriate, I’m sure to have missed a lot of good previous content.)

There’s a lot of variation in how sketched out the projects are. Most of the projects just have some informal notes and would require more thought before someone could start executing. If you're potentially interested in working on any of them and you could benefit from more discussion, I’d be excited if you reached out to me! [2]

There’s also a lot of variation in skills needed for the projects. If you’re looking for projects that are especially suited to your talents, you can search the posts for any of the following tags (including brackets):

[ML] [Empirical research] [Philosophical/conceptual] [survey/interview] [Advocacy] [Governance] [Writing] [Forecasting]

The projects are organized into the following categories (which are in separate posts). Feel free to skip to whatever you’re most interested in.

Governance during explosive technological growth

It’s plausible that AI will lead to explosive economic and technological growth.
Our current methods of governance can barely keep up with today's technological advances. Speeding up the rate of technological growth by 30x+ would cause huge problems and could lead to rapid, destabilizing changes in power.
This section is about trying to prepare the world for this. Either generating policy solutions to

...
Fact Checking and Retaliation Against Sources
by jefftk — 2024-01-04T00:41:07.134Z

Let's say you're concerned about someone in your community, and are putting together a post detailing why you think others should be concerned as well. This sort of documentation is a core part of decentralized exclusion, how a large community without any formal leadership can still kick out bad actors. As part of writing this you're talking to people who they've have hurt, and your sources are understandably nervous about retaliation. Some of that is unavoidable, since once the post is public the accused will likely be able to tell a lot about who your sources are from what information you include, but there's also the issue of retaliation against your sources to keep you from publishing.

One way you could handle this is just to publish without giving the accused any indication that this is happening. I think this can very occasionally be the right decision, but not offering the accused an opportunity to point out places where you've gotten the facts wrong gives up a lot:

You only want to be accusing someone of doing things they actually did, and if some of your claims are false or misleading they don't belong in the post.
The stronger your core accusations are when you first go public the more likely your post is to succeed. You want to avoid putting the community into a muddle of "there are tons of claims and who can tell what to make of it all" where many will stop paying attention.
The more you can winnow your claims to what really matters before going public the less community time you waste. You really don't want hundreds of people needing to make up their mind about whether a particular set of screenshots indicates bad behavior if there's some other hard-to-dispute evidence that would be sufficient.
There's a general sense in a lot of my communities that defamation law is too strong and keeps people from publishing true and important information about people who are causing harm. In some cases even if the accused would very likely win a libel suit against the accuser we don't think threatening or filing one is appropriate. But this is not absolute, and if you publish false things because you didn't give the accused a chance to point out mistakes I think resorting to the law shouldn't get our normal disapproval.

But then what can you do? How can you give the accused a chance to privately dispute your claims while minimizing the risk to your sources that the...
Investigating Alternative Futures: Human and Superintelligence Interaction Scenarios
by Hiroshi Yamakawa — 2024-01-03T23:46:46.924Z

Abstract

Given recent advances in AI technology, there is potential for the emergence of general-purpose, autonomous AI. This emerging AI could evolve into Artificial Superintelligence (ASI), an intelligence that surpasses human capabilities in almost all areas. ASI could have significant consequences for humanity, both positive and negative, with the most problematic being its potentially devastating impact on our future. Various thinkers, researchers, and futurists have deeply considered the potential outcomes of human interaction with ASI. However, there seems to be no comprehensive overview detailing the conditions under which I might transition into these potential outcomes. In our study, I constructed a flowchart outlining potential future trajectories for ASI and humankind. This was based on the assumption that ASI with suitable properties can be sustainable in the long term. Two worlds are relatively desirable for humanity: one in which the status quo is maintained and the other in which a benevolent ASI manages humanity. Conversely, I envisioned three more dire worlds where human survival would be difficult or impossible. The branches leading to these worlds were identified as having three human factors related to the development, deployment, and control of ASI and three ethical factors associated with the essential nature and purpose of the created ASI.

(*) This article is an English translation of the Japanese article[1] with some revisions.

1. Introduction

Artificial superintelligence (ASI) may be realized relatively soon if current advances in AI technology continue. ASI refers to AI technology that is autonomous, versatile, and has the potential to outperform human intelligence comprehensively. The realization of ASI can potentially revolutionize our lives, society, and even civilization by exceeding human intelligence. This includes advances in fields as diverse as medicine, economics, and education. At the same time, however, the potential risks and ethical issues that ASIs pose to humanity are of grave concern, including existential risk. Discussing the relationship between humankind and ASI is becoming increasingly important.

To analyze this relationship, it is necessary to consider the influence of both sides. On the one hand, since the initial ASI was created by humanity, humanity's impact is significant until the ASI is completed, and it...
"Attitudes Toward Artificial General Intelligence: Results from American Adults 2021 and 2023" - call for reviewers (Seeds of Science)
by rogersbacon — 2024-01-03T20:11:43.216Z

Abstract

A compact, inexpensive repeated survey on American adults’ attitudes toward Artificial General Intelligence (AGI) revealed a stable ordering but changing magnitudes of agreement toward three statements. From 2021 to 2023, American adults increasingly agreed AGI was possible to build. Respondents agreed more weakly that AGI should be built. Finally, American adults mostly disagree that an AGI should have the same rights as a human being; disagreeing more strongly in 2023 than in 2021.

---

Seeds of Science is a journal (funded through Scott Alexander's ACX grants program) that publishes speculative or non-traditional articles on scientific topics. Peer review is conducted through community-based voting and commenting by a diverse network of reviewers (or "gardeners" as we call them). Comments that critique or extend the article (the "seed of science") in a useful manner are published in the final document following the main text.

We have just sent out a manuscript for review, "Attitudes Toward Artificial General Intelligence: Results from American Adults 2021 and 2023", that may be of interest to some in the LessWrong community so I wanted to see if anyone would be interested in joining us as a gardener and providing feedback on the article. As noted above, this is an opportunity to have your comment recorded in the scientific literature (comments can be made with real name or pseudonym).

It is free to join as a gardener and anyone is welcome (we currently have gardeners from all levels of academia and outside of it). Participation is entirely voluntary - we send you submitted articles and you can choose to vote/comment or abstain without notification (so no worries if you don't plan on reviewing very often but just want to take a look here and there at the articles people are submitting).

To register, you can fill out this google form. From there, it's pretty self-explanatory - I will add you to the mailing list and send you an email that includes the manuscript, our publication criteria, and a simple review form for recording votes/comments. If you would like to just take a look at this article without being added to the mailing list, then just reach out (info@theseedsofscience.org) and say so.

Happy to answer any questions about the journal through email or in the comments below.
What’s up with LLMs representing XORs of arbitrary features?
by Sam Marks — 2024-01-03T19:44:33.162Z

Thanks to Clément Dumas, Nikola Jurković, Nora Belrose, Arthur Conmy, and Oam Patel for feedback.

In the comments of the

post on Google Deepmind’s CCS challenges paper

,

I expressed skepticism

that some of the experimental results seemed possible. When addressing my concerns, Rohin Shah made some claims along the lines of “If an LLM linearly represents features a and b, then it will also linearly represent their XOR, a⊕b, and this is true even in settings where there’s no obvious reason the model would need to make use of the feature a⊕b.”[1]

For reasons that I’ll explain below, I thought this claim was absolutely bonkers, both in general and in the specific setting that the GDM paper was working in. So I ran some experiments to prove Rohin wrong.

The result: Rohin was right and I was wrong. LLMs seem to compute and linearly represent XORs of features even when there’s no obvious reason to do so.

I think this is deeply weird and surprising. If something like this holds generally, I think this has importance far beyond the original question of “Is CCS useful?”

In the rest of this post I’ll:

Articulate a claim I’ll call “representation of arbitrary XORs (RAX)”: LLMs compute and linearly represent XORs of arbitrary features, even when there’s no reason to do so.
Explain why it would be shocking if RAX is true. For example, without additional assumptions, RAX implies that linear probes should utterly fail to generalize across distributional shift, no matter how minor the distributional shift. (Empirically, linear probes often do generalize decently.)
Present experiments showing that RAX seems to be true in every case that I’ve checked.
Think through what RAX would mean for AI safety research: overall, probably a bad sign for interpretability work in general, and work that relies on using simple probes of model internals (e.g. ELK probes or

coup probes

) in particular.
Make some guesses about what’s really going on here.

Overall, this has left me very confused: I’ve found myself simultaneously having (a) an argument that A⟹B, (b) empirical evidence of A, and (c) empirical evidence of ¬B. (Here A = RAX and B = other facts about LLM representations.)

The RAX claim: LLMs linearly represent XORs of arbitrary features, even when there’s no reason to do so

To keep things simple, throughout this post, I’ll say that a model linearl...
Spirit Airlines Merger Play
by sapphire — 2024-01-03T19:25:54.726Z

As far as I can tell buying SAVE is VERY plus EV given the potential merger.

Rough guess is the bet looks like:

+90% on win

-60% on loss (maybe less given P/B of 1.36)

FAVORED to win. Thats a good bet.

Here is a quick primer: https://reddit.com/r/wallstreetbets/comments/18sx0n6/get_saved_with_save/.

More details:

Seeking alpha post - https://archive.ph/SbuXU

Another seeking alpha post - https://archive.ph/rmZOX