trentmkelly/LessWrong-43k huggingface dataset:
AI #16: AI in the UK
by Zvi — 2023-06-15T13:20:03.939Z

It would make strategic sense, even from a selfish perspective, for essentially any nation to get more involved in sensible AI regulation, and to get more involved in capturing AI benefits and being the home to AI and AI-enabled economic engines.

It makes especially good sense for the UK, which is facing numerous economic tailwinds and yet still has the legacy of its history and of London and DeepMind, of Oxford and Cambridge, and has broken free of the absolutely bonkers regulatory regime that is the European Union. This is their chance, and Sunak is attempting to take it.

Will it end up being yet another capabilities push, or will we get real progress towards safety and alignment and sensible paths forward? The real work on that begins now.

In terms of capabilities, this was what passes for a quiet week.

The most exciting recent capability announcement was the Apple Vision Pro, which didn’t mention AI at all despite the obvious synergies. I review the reviews here.

Table of Contents

Table of Contents

Language Models Offer Mundane Utility. Watermarks might even work.

Language Models Don’t Offer Mundane Utility. You only hallucinated that.

Fun With Image Generation. No fun this week, how about stereotypes instead?

Deepfaketown and Botpocalypse Soon. I heard you like scams, so…

GPT Upgrades. Cheaper, faster, better function calls.

The Art of the Super Prompt. I am very intelligent.

They Took Our Jobs. Even better jobs, coming soon?

Introducing. Arc is hiring, Andromeda has compute. Choose your weapon.

In Other AI News. Nature seeks alignment manuscripts.

Potential Alignment Progress. Might the AI have a tell?

Misalignment: Who To Blame? A taxonomy. Blame Critch and Russel for paper.

Visions of 2030. An attempt to not be surprised. I’d be surprised if it worked.

Alignment Difficulty Distribution. Chris Olah on the uncertainty involved.

Quiet Speculations. We must unlock the power of Web3.

AGI Will Do Things You Are Not Imagining. You have not solved physics.

Anthropic Charts a Path. Highly reasonable suggestions, with a notable exception.

The Quest for Sane Regulation. No one said it would be easy.

AI in the UK. 100 million pounds for AI safety. Totally not for capabilities.

What Exactly is Alignment. The will of humanity? Asimov’s laws? Oh no.

Rhetorical Innovation. Always more to throw at the

...
I still think it's very unlikely we're observing alien aircraft
by dynomight — 2023-06-15T13:01:27.734Z
Link: https://dynomight.net/aliens/

Some suggest there might be alien aircraft on Earth now. The argument goes something like this:

(1) A priori, there’s no reason there shouldn’t be alien aircraft. Earth is 4.54 billion years old, but the universe is 13.7 billion years old, and within a billion light years of Earth there are something like 5 × 10¹⁴ stars. Most of those stars have planets, and if an alien civilization arose anywhere and built a von Neumann probe, those probes would spread everywhere.

(2) We have tons of observations that would be more likely if there were alien aircraft around than if there weren’t. These include:

Vast numbers of anecdotal reports from pilots.
Videos that appear to show objects with flight characteristics far beyond known human capabilities.
Senators—with access to classified information—raising concerns about unknown craft near military installations.
Rumors from multiple sources that the government has captured, intact alien craft.

(3) So if we agree that:

the prior probability P[aliens] is pretty high; and
P[gimbal video | aliens] > P[gimbal | no aliens]; and
P[Pheonix lights | aliens] > P[Pheonix | no aliens]; and
P[Trans-en-Provence | aliens] > P[Trans-en-Provence | no aliens]; and
P[2021 Pentagon UAP Task Force report | aliens] > P[report | no aliens]; and
P[UAP Task Force member whistleblower alleges US government has intact non-human vehicles | aliens] > P[whisleblower | no aliens]; and
P[Former intelligence official says four people confirm secret US government program to analyze off-world craft | aliens] > P[official | no aliens],

then don’t we have to conclude that the posterior probability P[aliens | everything] is pretty high?

No.

What I like about this argument

First, I agree the prior probability should be pretty high. Say you just told me:

Okay, say there’s a universe with at least 10²⁴ stars, and most stars have planets, and on one planet somewhere life has evolved and started sending stuff to other planets and has an ever-accelerating pace of technological development.

Then I would agree—the odds that there would be alien aircraft on my planet seem good. It seems like we—or our artificial descendants—might well send aircraft to other planets. So why shouldn’t alien aircraft be here, now?

Second, it takes eyewitness reports seriously. There are a huge number of reports by pilots seeing objects accelerating at insane rates with no obvious win...
Aligned Objectives Prize Competition
by Prometheus — 2023-06-15T12:42:21.682Z
Link: https://app.impactmarkets.io/bounty/clix38oh200003e6s5dem1q3g

The goal of this prize is to generate ideas and plans for a long-term, positive future. The starting prize pool is $500 USD, though others are welcome to add to it.

We are going to pretend for this exercise that the technical part of AI Alignment is already solved. Your mission, should you choose to accept it, is to come up with a design for what you want this AI to be doing.

Relaxations:

The AI will be safe regarding what you're targeting. Example: if you choose to give an AI the sole goal of eliminating cancer, it will actually cure cancer, and not just kill all humans to stop cancers from developing.
You have some degree of influence, politically or otherwise. You can push your idea pretty far, but you’re not all-powerful. If people would normally really hate your idea, or otherwise have no interest at all in it, that’s a problem.

Constraints:

The target of alignment you choose has to be something that major political actors (NATO, China, etc.) would not oppose.
It has to be something that would be within the economic interests of AI companies to implement.
It needs to be something that would not provoke massive public backlash.
You can’t turn off or change the fundamental architecture once implemented. (Meaning if you give it a certain target to be optimizing for, you can’t change the target in the future)
There will be no future intelligence (artificial or otherwise) that can ever surpass it in intelligence. It will continue to run until the stars fade and the Universe grows cold.

The criteria for selecting a winner will be determined by myself and others at AI Safety Strategy, based on a few factors.

Simplicity over complexity: The simpler and more effective strategies will be preferred over more complex ones. For example, an idea such as Extrapolated Human Volition will lose a lot of points for its added complexity. This is because we don’t know how robust a hypothetical alignment method might be, meaning aligning simpler objectives might be far easier than complex ones.

Multi-party interests: Ideas that are more likely to be backed by multiple interests will rank higher than ones that only please a few. Inversely, ideas that are likely to get opposition from multiple parties will be ranked much lower. All else being equal, an idea that pleases fewer parties but is opposed by almost no one will be ranked higher than an idea idea that pleases more parties but would...
A more effective Elevator Pitch for AI risk
by Iknownothing — 2023-06-15T12:39:03.363Z

" 'We're generating algorithms that we don't really understand, that can do more and more things and keep getting even harder to understand.

They often become capable of doing things we didn't realize they'd be able to do- this is also happening more often.

When we get them to solve a problem, we don't know how they're going to solve the problem and we don't know if one of the steps is going to be something we don't want it to do.

We're concerned that eventually, one of the steps might be something that really harms a lot of people and we won't know until it's too late.

This concern gets closer, the more capable we make the algorithms.

Right now, it's very close. "

leave scifi and fantasy stuff out of it. that is how you look like someone obsessed with stories, making a story of your own.

and even if you get through, it might cause emotional fear/panic rather than a problem solving response. the reaction to the fear/panic won't be to problem solve, but to look for a different explanation that will relieve their fear/panic.
Why "AI alignment" would better be renamed into "Artificial Intention research"
by chaosmage — 2023-06-15T10:32:26.094Z

"AI alignment" has the application, the agenda, less charitably the activism, right in the name. It is a lot like "Missiology" (the study of how to proselytize to "the savages") which had to evolve into "Anthropology" in order to get atheists and Jews to participate. In the same way, "AI Alignment" excludes e.g. people who are inclined to believe superintelligences will know better than us what is good, and who don't want to hamstring them. You can think we're well rid of these people. But you're still excluding people and thereby reducing the amount of thinking that will be applied to the problem.

"Artificial Intention research" instead emphasizes the space of possible intentions, the space of possible minds, and stresses how intentions that are not natural (constrained by evolution) will be different and weird.

And obviously "Artificial Intention" is an alliteration and a close parallel with "Artificial Intelligence", so it is very catchy. Catchiness matters a lot when you want an idea to catch on at scale!

Extremely superficially, it doesn't sound "tacked on" to Artificial Intelligence research, it sounds like a logical completion.

The necessity of alignment doesn't have to be in the name, because it logically follows from the focus on intention, with this very simple argument:

Intention doesn't have to be conscious or communicable. It is just a preference for some futures over others, inferred as an explanation for behavior that chooses some future over others. Like, even single celled organisms have basic intentions if they move towards nutrients or away from bad temperatures.
Therefore, anything that selectively acts in the world, including AI systems, can be modeled to have some intent that explains its behavior.
So you're always going to get an intent, and if you don't design it thoughtfully you'll get an essentially random one.
...which is most likely bad (e.g. the paperclip maximizer) because it is random and different and weird.

So this would continue to be useful for alignment. Just like anthropology continued to be useful, and in fact was even more useful than original missiology, to the missionaries.

Having the Intelligence (the I in "AI Alignment") only implicitly part of it (because of the alliteration and the close parallel) might lose some of the focus on how the Intelligence makes the Intention much more relevant? If that isn't obv...
Matt Taibbi's COVID reporting
by ChristianKl — 2023-06-15T09:49:54.272Z
Link: https://www.racket.news/p/on-todays-explosive-coronavirus-story

Michael Shellenberger’s Public today released a blockbuster story, “First Person Sickened By COVID-19 Was Chinese Scientist Who Oversaw “Gain Of Function” Research That Created Virus,” which generously credits Racket. The story cites three government officials in naming scientist Ben Hu, who was in charge of “gain-of-function” research at the Wuhan Institute of Virology, as the “patient zero” of the Covid-19 pandemic.

[...]

The secrets of both the pandemic’s origin and the reason for America’s at-best-sluggish investigation of same have become the mother of all political footballs, and today’s news is likely to be just the first in a series of loud surprises.

[...]

Numerous federal agencies appear to have designed their probes of Covid-19’s origins so as to discount the possibility of lab origin in advance.

We were told, for instance, that despite longstanding interest in the Wuhan Institute as a potential security concern, at least one intelligence agency overruled a majority of its in-house investigators to produce a report on the pandemic’s origin discounting the lab-leak hypothesis.

Two years ago I wrote on LessWrong that my likelihood for the lab leak hypothesis 99% hypothesis. Given that updating on evidence is important I think I'm warranted to update to 99.9%.

I was wrong when I expected that the truth comes out sooner because I underrate the extent to which the intelligence community will try to mislead the public. In retrospect, that seems like a stupid mistake. Nevertheless, we seem now at the point where the public evidence will force more organizations to change their assessments.

We might also come into the phase where the press will start to focus on how the lab leak theory was suppressed.
Looking Back On Ads
by jefftk — 2023-06-15T02:10:04.076Z

It is difficult to get a man to understand something, when his salary
depends on his not understanding it.

Talking about ads online I would often
get a
response that as someone
working in ads I was bound
to support
my employment. I'm now a
year out from working in ads and very unlikely to return to the
industry: with the economic bias removed but still knowing the
industry reasonably well, what do I think of it all now?

My overall
views haven't changed very much. I still think the only serious
options are paywalls ("pay with your money") and ads ("pay with your
attention"), and in the case of the web the value of moving fluidly
from site to site pushes strongly for ads. And for sites that are
supported by ads it's worth working to make them, to the extent
possible, not-bad for visitors and lucrative for the site.

Sometimes technical work can improve ads on the web on both axes
simultaneously. The introduction of intersection
observer and ping
both took something ads really wanted (logging which ads were within
the user's viewport or that an ad had been clicked on) and was
already possible (with the scroll
event or a redirect)
and made the implementation much more efficient. These were already
so valuable for ads that ~all ads did them, so increasing
efficiency didn't increase how often the tracking happened, it just
made the web a bit more responsive. I interpret Chrome's Privacy Sandbox work
as another attempt in this direction, trying to retain the economic
effects of adtech being able to track users from site to site (predicting
which ads users will best respond to, detecting abuse, etc) while
dramatically improving the privacy situation.

In other cases, though, there's really just a fight over whether ads
should be more or less intrusive, numerous, etc. Users want to see
the stuff they came to the site for, publishers want to make more
money. This is held somewhat in check by how if a publisher makes
their site too unpleasant users will leave (or install adblockers) but
only a bit: if I have a few tasteful text-only static ads for relevant
things that only goes so far if other sites have flashing hovering
noisy moving ads. There's a commons
problem where each annoying site makes things worse for all the
sites.

Ideally users could credibly say "if you make your site too unpleasant
we're not going to visit", as some sort of ...
Why libertarians are advocating for regulation on AI
by RobertM — 2023-06-14T20:59:58.225Z

Motivation: some people on the internet seem confused about why libertarians (and others generally suspicious of government intervention) are advocating for regulation to help prevent x-risk from AI. The arguments here aren't novel and generally seem pretty obvious to me, but I wanted a quick reference.

Who is advocating for regulations?

This is obviously the first question: is the premise correct? Are libertarians uncharacteristically advocating for regulations?

They probably aren't the loudest or most numerous among the set of people who are advocating for regulations while being motivated by reducing AI x-risk, but I'm one such person and know multiple others.

I tend to bucket those advocating for regulations into three buckets.

Non-libertarians

Probably the biggest group, though I have wide error bars here. Most of the people I know working on the policy & comms side are either liberal, apolitical, or not usefully described by a label like that. I think people in this group tend to have somewhat more optimistic views on how likely things are to go well, but this isn't a super strong correlation.

Libertarians

Although some of them are focusing primarily on policy & comms, those I know more often dual-class with a technical primary & comms secondary. Tend to have more pessimistic views on our likelihood of making it through.

Eliezer Yudkowsky

I could have put him in the "libertarian" group, but he has a distinct and specific policy "ask". Also often misquoted/misunderstood.

The basic argument

There are many arguments for why various regulations in this domain might seem compelling from a non-libertarian point of view, and people generally object to those on different grounds (i.e. accusations of regulatory capture, corruption, tribalism, etc.), so I'll skip trying to make their case for them.

Why might libertarians advocate for (certain kinds of) regulation on AI, given their general distrust of government?

Straightforwardly, the government is not a malicious genie optimizing for the inverse of your utility function, should you happen to ask it for a favor. Government interventions tend to come in familiar shapes, with a relatively well-understood distribution of likely first- and second-order effects.

If you're mostly a deontological libertarian, you probably oppose government interventions because they violate peopl...
Instrumental Convergence? [Draft]
by J. Dmitri Gallow — 2023-06-14T20:21:41.485Z

This is a draft written by J. Dmitri Gallow, Senior Research Fellow at the Dianoia Institute of Philosophy at ACU, as part of the Center for AI Safety Philosophy Fellowship. This draft is meant to solicit feedback. Here is a PDF version of the draft.

Abstract

The thesis of instrumental convergence holds that a wide range of ends have common means: for instance, self preservation, desire preservation, self improvement, and resource acquisition. Bostrom (2014) contends that instrumental convergence gives us reason to think that ''the default outcome of the creation of machine superintelligence is existential catastrophe''. I use the tools of decision theory to investigate whether this thesis is true. I find that, even if intrinsic desires are randomly selected, instrumental rationality induces biases towards certain kinds of choices. Firstly, a bias towards choices which leave less up to chance. Secondly, a bias towards desire preservation, in line with Bostrom's conjecture. And thirdly, a bias towards choices which afford more choices later on. I do not find biases towards any other of the convergent instrumental means on Bostrom's list. I conclude that the biases induced by instrumental rationality at best weakly support Bostrom's conclusion that machine superintelligence is likely to lead to existential catastrophe.

Section 1: A Means to Most Ends?

According to Bostrom (2014, ch. 7), an intelligent agent could have any desires. There's nothing in the nature of intelligence that makes you more inclined to want some things rather than others. He calls this the orthogonality thesis (intelligence and desire are orthogonal). There is an infinitely large collection of ends which an agent could have. And there's nothing in the nature of intelligence itself which takes any of these ends off the table. Bostrom's point is that, if the orthogonality thesis is true, then we should be wary of anthropomorphising the motivations of future artificial superintelligences. An AI could be as intelligent as you wish, and still want nothing more than to create paperclips, or dance the Macarena, or reshape the Earth into a tetrahedron.
Yudkowsky (2018) agrees: "Imagine a map of mind design space. In one corner, a tiny little circle contains all humans; within a larger tiny circle containing all biological life; and all the r...
On the Apple Vision Pro
by Zvi — 2023-06-14T17:50:08.957Z

Apple is offering a VR/AR/XR headset, Vision Pro, for the low, low price of $3,500.

I kid. Also I am deadly serious.

The value of this headset to a middle class American or someone richer than that is almost certainly either vastly more than $3,500, or at best very close to $0.

This type of technology is a threshold effect. Once it gets good enough, if it gets good enough, it will feel essential to our lives and our productivity. Until then, it’s a trifle.

Thus, like Divia Eden, I am bullish on using the Tesla strategy of offering a premium product at a premium price, then later either people decide they need it and pay up or you scale enough to lower costs – if the tech delivers.

Gaming could be a modest benefit. Mark Zuckerberg points out on the Lex Fridman podcast that with no native controller this could be a poor VR/AR gaming platform. Mark suggested this could drive demand for the more reasonably priced Occulus.

This doesn’t apply to traditional gaming with the VR used to improve the screen and mobility, assuming you can get connected to devices that allow real gaming (e.g. console devices or PCs, not iPhones and Macs.) Apple lets you hook up a PS5 controller or mouse and keyboard if you like, but only directly integrates with Apple devices.

My current impression of existing VR/AR is that it is in the ‘not worth much’ section of the curve. The games and activities are fun to try out, but not worth sustained engagement. Productivity wasn’t there at all.

Can Apple do better? Are we there? There’s definitely a bunch of new tech here.

Your New Computer Interface?

That’s how their presentation pitch starts. ‘All your favorite apps,’ controlled by your eyes and finger scrolls and taps.

Do we want to use a phone or computer in AR or VR? It makes sense to go use case by use case.

Watching movies, shows and other media? Sounds good if quality is there. Automatically darkening the rules and resizing the screen seem like awesome features. Looks like experience quality is there. The question is, what does this offer that’s ten times better than a television screen, other than mobility? Otherwise, cost plus the trivial inconveniences involved will be big problems.

Looking at your old videos and photos, and also taking photos and videos? Experience quality here seems fantastic. Banking memories and vistas for later viewing seems killer, and the amount yo

...