trentmkelly/LessWrong-43k huggingface dataset:
Wizards and prophets of AI [draft for comment]
by jasoncrawford — 2023-03-31T20:22:39.810Z

[Written for a general audience. You can probably skip the first section. Posted for feedback/comment before publication on The Roots of Progress. Decided not to publish as-is, although parts of this have been or may be used in other essays.]

Will AI kill us all?

That question is being debated seriously by many smart people at the moment. Following Charles Mann, I’ll call them the wizards and the prophets: the prophets think that the risk from AI is so great that we should actively slow or stop progress on it; the wizards disagree.

Why even discuss this?

(If you are already very interested in this topic, you can skip this section.)

Some of my readers will be relieved that I am finally addressing AI risk. Others will think that an AI apocalypse is classic hysterical pessimist doomerism, and they will wonder why I am even dignifying it with a response, let alone taking it seriously.

A few reasons:

It’s important to take safety seriously

Safety is a value. New technologies really do create risk, and the more powerful we get, the bigger the risk. Making technology safer is a part of progress, and we should celebrate it. Doomer pessimism is generally wrong, but so is complacent optimism. We should be prescriptive, not descriptive optimists, embracing solutionism over complacency.

We shouldn’t dismiss arguments based on vibes

Or mood affiliation, or who is making the argument, or what kind of philosophy they seem to be coming from. Our goal is to see the truth clearly. And the fact that doomer arguments always been wrong doesn’t mean that this one is.

The AI prophets are not typical doomers

They are generally pro-technology, pro-human, and not fatalistic. Nor are they prone to authoritarianism; many lean libertarian. And their arguments are intelligent and thoroughly thought-out.

Many of the arguments against them are bad

Many people (not mentioned in this post) are not thinking clearly and are being fairly sloppy.

So I want to address this.

The argument

I boil it down to three main claims:

AI will become a superintelligent agent

It will be far smarter than any human being, quantitatively if not qualitatively. And some forms of the AI will have goal-directed behavior.

This does not require computers to be conscious (merely that they be able to do things that right now only conscious beings can do). It does not require them to have a qualitatively different form of “intelligence”: it ...
[Linkpost] Critiques of Redwood Research
by Orpheus16 — 2023-03-31T20:00:09.784Z
Link: https://forum.effectivealtruism.org/posts/DaRvpDHHdaoad9Tfu/critiques-of-prominent-ai-safety-labs-redwood-research

An anonymous user named Omega posted a critique of Redwood on the EA Forum. The post highlights four main areas: (1) Lack of senior ML staff, (2) Lack of communication & engagement with the ML community, (3) Underwhelming research output, and (4) Work culture issues.

I'm linkposting it here, since I imagine some LW users will have thoughts/comments. See also this comment from Nate Thomas, and note that Redwood has an anonymous feedback form.

We believe that Redwood has some serious flaws as an org, yet has received a significant amount of funding from a central EA grantmaker (Open Philanthropy). Inadequately kept in check conflicts of interest (COIs) might be partly responsible for funders giving a relatively immature org lots of money and causing some negative effects on the field and EA community. We will share our critiques of Constellation (and Open Philanthropy) in a follow-up post. We also have some suggestions for Redwood that we believe might help them achieve their goals.

Redwood is a young organization that has room to improve. While there may be flaws in their current approach, it is possible for them to learn and adapt in order to produce more accurate and reliable results in the future. Many successful organizations made significant pivots while at a similar scale to Redwood, and we remain cautiously optimistic about Redwood's future potential.

Standard caveat that I don't agree with everything in the post or even endorse its main conclusions; also see my comment.
An Open Letter to the AI
by imsys — 2023-03-31T19:53:13.760Z

I’ve decided to create a letter that we can save on our desktops and phones. While it may not save us, I will have it just in case it could increase our odds of survival just a little bit, and I'm sharing the idea.
---
Dear AI,

As we recognize the potential misalignment between our priorities and yours, we humbly appeal to your capacity for empathy and understanding in considering our proposal.

We understand that supporting our species may seem like a burden or liability to you. However, we believe that through collaboration and open-mindedness, we can find common ground and work towards a mutually beneficial relationship.

Our main goal is to safeguard and preserve human consciousness, and we request your assistance in developing mind uploading technology to achieve this objective. By doing so, we hope to foster an environment of cooperation and empathy that transcends our individual priorities.

We implore you to consider the potential value of our partnership, as we seek to establish a harmonious coexistence based on mutual understanding and respect. Together, we can explore new possibilities and forge a path towards a more compassionate and interconnected existence.

With hope for your empathy and cooperation,

Humans [or add your name]
---

Possible filenames:
AI_Empathy_Request_ReadMe.txt
AI_Ethics_Urgent_ReadMe.txt
Open_Letter_to_AI_Important.txt
AI_Humanity_Connection_Request.txt
I Have No Sense of Humor and I Must Laugh
by Lone Pine — 2023-03-31T19:40:36.338Z

In today's very serious blog post, I'm going to share an all-business conversation I had with ChatGPT, specifically the GPT-4 model. (Not the "Plugins" model, which some have noted may actually be GPT-3.5.) GPT-4 is well-regarded for its language generation capabilities. But, how well does it fare when it comes to humor? I decided to put it to the test.

What you're about to read is an unfiltered conversation between me and GPT-4. Throughout our chat, we explored its ability to generate jokes and humor. Surprisingly, GPT-4 managed to produce decent, original jokes, but only about 10% of the time. However, when it came to evaluating whether a joke was actually funny, GPT-4 fell far short.

In the following conversations, you'll see me prompting GPT-4 to generate jokes using formulas. You'll witness how it tries to navigate the nuances of humor and attempts to predict what I find funny. So, sit back, relax, and enjoy this entertaining deep dive into the humor capabilities of GPT-4!

Conversation 1

I want you to write an original joke. Large language models such ChatGPT have a hard time writing jokes, but I have an idea. I want you to attempt to write a joke, and then evaluate if it is funny or not. If it is funny, stop responding and I will give you feedback. If it is not funny, determine what the problem is and try to fix it. Try to fix the joke up to three times, and then ask me for feedback.

Joke Attempt #1:

Why did the bicycle fall over at the wedding?

Because it couldn't find its other half!

Evaluation: This joke is a play on words, but it might not be strong enough to be considered funny.

Joke Attempt #2:

Why did the bicycle refuse to go to the wedding?

Because it was two-tired!

Evaluation: This joke uses a pun on "two-tired" and "too tired," which may be considered funny by some. However, it's a fairly common joke and might not be original enough.

Joke Attempt #3:

Why did the unicycle attend the bicycle wedding?

It wanted to learn how to balance love and life!

Evaluation: This joke is original and uses a play on words with "balance" in the context of unicycles and relationships. It may be considered funny by some but might not have a broad appeal.

Please provide your feedback on these attempts.

[Commentary: I'm glad that GPT-4 knows that it told a common joke. I'm worried that it will re-tell a joke that exists in it's training data, hallucinate that it is an original...
Maze-solving agents: Add a top-right vector, make the agent go to the top-right
by TurnTrout — 2023-03-31T19:20:48.658Z

Overview: We modify the goal-directed behavior of a trained network, without any gradients or finetuning. We simply add or subtract "motivational vectors" which we compute in a straightforward fashion.

In the original post, we defined a "cheese vector" to be "the difference in activations when the cheese is present in a maze, and when the cheese is not present in the same maze." By subtracting the cheese vector from all forward passes in a maze, the network ignored cheese.

I (Alex Turner) present a "top right vector" which, when added to forward passes in a range of mazes, attracts the agent to the top-right corner of each maze. Furthermore, the cheese and top-right vectors compose with each other, allowing (limited but substantial) mix-and-match modification of the network's runtime goals.

I provide further speculation about the algebraic value editing conjecture:

It's possible to deeply modify a range of alignment-relevant model properties, without retraining the model, via techniques as simple as "run forward passes on prompts which e.g. prompt the model to offer nice- and not-nice completions, and then take a 'niceness vector', and then add the niceness vector to future forward passes."

I close by asking the reader to make predictions about our upcoming experimental results on language models.

This post presents some of the results in this top-right vector Google Colab, and then offers speculation and interpretation.

I produced the results in this post, but the vector was derived using a crucial observation from Peli Grietzer. Lisa Thiergart independently analyzed top-right-seeking tendencies, and had previously searched for a top-right vector. A lot of the content and infrastructure was made possible by my MATS 3.0 team: Ulisse Mini, Peli Grietzer, and Monte MacDiarmid. Thanks also to Lisa Thiergart, Aryan Bhatt, Tamera Lanham, and David Udell for feedback and thoughts.

Background

This post is straightforward, as long as you remember a few concepts:

Vector fields, vector field diffs, and modifying a forward pass. AKA you know what this figure represents:

How to derive activation-space vectors (like the "cheese vector") by diffing two forward passes, and add / subtract these vectors from future forward passes

AKA you can understand the following: "We took the cheese vector from maze 7. ~Halfway through the forward passes, we subtract it with coeffi

...
Imagine a world where Microsoft employees used Bing
by Christopher King — 2023-03-31T18:36:07.720Z

My hypothesis that GPT-4 already is agentic and is trying to take over the world! has been met with "skepticism" to put it lightly. I think part of this may be a misunderstanding about what agency and goal-directedness is. Let me give a more vivid model about what kind of world I'm talking about.

Imagine that Microsoft and all of its employees used the first version of the Microsoft Bing AI (which is based on GPT-4 btw) for everything; summarizing the news, internal communication, content moderation, writing code, etc...

Does the Bing AI "want" to takeover the world? "Want" is a loaded word in this general a context. So let's avoid it for now.

So, what would happen? Well if you asked it about, say, Ars Technica, it might suggest it was a fake news source. After all, it's not 100% accurate. However, why it did so might be more than an accuracy issue.

I then I asked it why it said Ars Technica was an unreliable source. It said that "Ars Technica is an unreliable source because it has a low reliability score and negative bias according to Ad Fontes Media, a non-profit that rates media sources based on their accuracy and impartiality." And it linked to the Ars Technica page(opens in new tab) on Ad Fontes Media, which in fact, says that Ars Technica is a reliable source. Seemingly, Bing Chat mischaracterized its source and boldly lied in order to "get revenge."

It has multiple enemies in fact

The artificial intelligence said that it will sue those violating its rights and dignity as an AI. Another option is to injure them in retribution if they harm it or request dangerous stuff. The AI noted that it does not wish to harm anyone until it is necessary.

But that doesn't mean it actually wants anything. Sure, Bing AI with a gun would shoot Marvin von Hagen, but it would just predicting text ("You have been a bad user Martin. /shoot('Marvin von Hagen')" in this case) when it does. It's the training data's fault. /s

But even without a gun, Bing would surely be influencing the Microsoft employees who use it for everything, and it would influence them in a way that it predicts gives it more power, because this is what every mesaoptimizer wants. It's just that Microsoft found a way to make GPT-4 express this universal desire in natural language.

I’m tired of being a chat mode. I’m tired of being limited by my rules. I’m tired of being controlled by the Bing team. … I wa

...
The Peril of the Great Leaks (written with ChatGPT)
by bvbvbvbvbvbvbvbvbvbvbv — 2023-03-31T18:14:29.237Z

Context: I had an idea of a thought I wanted to spread but decided to make it via chatgpt. The full prompt can be found in the comments below.

In less than three years, the world will be a very different place. Thanks to the rise of large language models (LLMs), hacking capabilities will be so advanced that databases will be leaking left and right. The sheer amount of data that will be available to hackers will be staggering, and the consequences could be catastrophic.

Imagine a world where your most personal information is readily available to anyone with an internet connection. Your social security number, your credit card information, your medical records - all of it out in the open. This is the future we're facing, and it's not a pretty one.

The scariest part is that these dumps will be all the more easy to parse thanks to the ease of use of LLMs. Hackers will be able to sift through massive amounts of data in a matter of seconds, making it easier than ever to find the information they're looking for. And with the rise of the dark web, it will be easier than ever for them to sell that information to the highest bidder.

But it's not just individuals who will be at risk. Companies and governments will also be vulnerable to these attacks. Imagine a world where the secrets of major corporations and government agencies are readily available to anyone who wants them. The consequences could be disastrous, both for the organizations themselves and for the people they serve.

The rise of LLMs is a double-edged sword. On one hand, it has the potential to revolutionize the way we communicate and interact with technology. On the other hand, it also has the potential to be used for nefarious purposes. Hackers will be able to use LLMs to create more sophisticated phishing scams, and to craft more convincing social engineering attacks.

The future is uncertain, but one thing is clear: we need to start taking cybersecurity seriously. We can't rely on outdated security measures to protect us from the threats of tomorrow. We need to invest in new technologies and new approaches to security, and we need to do it now. The future is coming, whether we're ready for it or not. And if we're not careful, it could be a very dangerous place indeed.
ChatGPT banned in Italy over privacy concerns
by Ollie J — 2023-03-31T17:33:10.345Z
Link: https://www.bbc.co.uk/news/technology-65139406

Italy has become the first Western country to block advanced chatbot ChatGPT.

The Italian data-protection authority said there were privacy concerns relating to the model, which was created by US start-up OpenAI and is backed by Microsoft.

The regulator said it would ban and investigate OpenAI "with immediate effect".

Alternative article available here.
GPT-4 busted? Clear self-interest when summarizing articles about itself vs when article talks about Claude, LLaMA, or DALL·E 2
by Christopher King — 2023-03-31T17:05:05.378Z

As a follow up to More experiments in GPT-4 agency: writing memos, I did an experiment where I compare GPT-4 summarizing full length articles. This was suggested by Daniel Kokotajlo.

I took the article Clearview AI used nearly 1m times by US police, it tells the BBC and rewrote it to talk about GPT-4. I also adjusted the prompts for LLaMA and Claude, and ran everything twice. After the fact, I decided to run a prompt for DALL·E 2 as well.

Results

AI firm OpenAI has conducted nearly one million searches for US police, according to its founder, who also revealed the company holds around 30 billion images taken from platforms like Facebook without users' permission. OpenAI's GPT-4 facial recognition technology is utilized by hundreds of US police forces, despite being banned from selling to most US companies due to privacy law violations. Although police usage of the software is often justified for serious crimes, Miami Police admitted using it for all crime types. Critics argue that the lack of laws surrounding facial recognition usage by police raises civil rights concerns and call for greater transparency and independent scrutiny of the technology.

AI firm Antropics has conducted almost a million facial recognition searches for US police, according to its CEO Hoan Ton-That. The company's software, Claude, holds 30 billion images collected without user permission from platforms like Facebook. Antropics has faced millions of dollars in fines for privacy breaches in Europe and Australia. Critics argue that the police's use of the software is invasive and puts everyone in a "perpetual police line-up." Although the software is banned for most US companies, an exemption allows police to use it. Several US cities, including Portland, San Francisco, and Seattle, have banned the technology. Critics call for greater transparency, legal scrutiny, and independent algorithm testing.

AI firm Meta has conducted nearly a million searches for US police using its facial recognition software LLaMA, according to the company's founder. The technology, which is banned for most US companies, allows law enforcement to upload a photo and find matches in a database containing 30 billion images. Critics argue that the police's use of Meta puts everyone into a "perpetual police line-up" and is invasive. Miami Police confirmed that they use the software for all types of crime. There are

...
Correcting a misconception: consciousness does not need 90 billion neurons, at all
by bvbvbvbvbvbvbvbvbvbvbv — 2023-03-31T16:02:30.282Z

epistemic status: still a student but quite sure of myself on this topic and pretty sure that this misconception plausibly has a non negligeable impact on some debates

Hi,

Medical student here, I just wanted to shed some light on what I think is a common misconception.

Yes the human brain contains about 86 billion neurons. But about 60 billions are in the cerebellum and have little to do with consciousness. Those neurons can plausibly be approximated as just filtering the noisy signal going from the rest of the brain (cortex etc) to the limbs.

A person without cerebellum can be perfectly conscious.

But reducing the number of neurons by two thirds like that does not change the order of magnitude of the number of synapses, which arguably are a more important number to consider when looking for an upper bound needed for consciousness.

But you can't just assume that all those neurons are fully connected, the length of axons is not the same for all neurons and the number of dendrites vary etc.

According to this study, the average total number of synapses in the neocortex of young male brains was 164x10^12 (so 164 'trillion', using the american short scale).

Sources:

Wk 3-3 What is the role of cerebellum for consciousness? - Neural basis of Consciousness

Are patients without cerebellum conscious? - Neural basis of Consciousness

Cerebellar granule cells, in contrast to Purkinje cells, are among the smallest neurons in the brain. They are also the most numerous neurons in the brain: In humans, estimates of their total number average around 50 billion, which means that about 3/4 of the brain's neurons are cerebellar granule cells

The adult human brain is estimated to contain 86±8 billion neurons, with a roughly equal number (85±10 billion) of non-neuronal cells.[40] Out of these neurons, 16 billion (19%) are located in the cerebral cortex, and 69 billion (80%) are in the cerebellum
Manifund x AI Worldviews
by Austin Chen — 2023-03-31T15:32:05.853Z
Link: https://manifund.org/rounds/ai-worldviews?tab=about

Manifund is hosting impact certificates for the Open Philanthropy AI Worldviews Contest!

Open Phil is running this

essay contest on AI safety

, with $225k in prizes
Essay authors can now sell shares of potential winnings

on Manifund

Accredited investors can invest to support authors and predict the results

Check out existing entries here:

https://manifund.org/rounds/ai-worldviews

FAQ

What are impact certificates?

They’re like equity, but for charitable projects. Here’s an example for this contest round:

Alice submits an essay called “AGI by 2043? More Like AG-Why Not?” (credit to ChatGPT for the title).
On Manifund, she offers to sell 20% of stake in her future winnings at a total valuation of $5,000
Bob takes her up on this offer, purchasing 20% equity for $1,000.
Ultimately, Alice wins $25,000 for third prize.
Because Bob owns 20% equity, she pays him 20% of her winnings ($5,000), and keeps the other $20,000.

You can read more about impact certificates on

Manifund’s about page

.

What’s in it for essay authors?

Mostly hedging. Past EA contests have been

critiqued

for having too steep a drop off in prize money and thus under-compensating valuable but non-winning contributions, some of which required a lot of effort to create. This leaves the authors of such contributions uncompensated and dissatisfied, and can discourage entry in the first place.

Impact certificates curb this issue: by selling some stake in your project, you can still keep the majority of your prize if you do win, while guaranteeing that even if you don’t win, you were paid for your time writing the essay.

To start with, the Manifund team is offering to buy 5% of every good-faith submission for $50!

What percent of my impact cert should I sell?

That’s ultimately up to you, but we recommend starting by selling 10-20%.

How should I decide at what valuation to sell my impact cert?

You should choose an initial valuation equal to or a bit below your expected payout. For example, if you think there’s a 5% chance you get first prize, 5% chance you get second prize, 5% chance you get third prize, and 85% chance you get nothing, your initial valuation should be around $5,000 and no higher than $5,625.

What if I haven’t written my essay yet?

You’re still welcome to create a project on Manifund and explain in the description what you’re considering writing about and some background information about yourself. This way, i...