trentmkelly/LessWrong-43k huggingface dataset:
The Machine Stops (Chapter 9)
by Justin Bullock — 2022-12-01T19:20:26.031Z

"The Machine Stops"

This is what Kuno tells Vashti, through the machine, just as it begins its final descent into destruction, taking Kuno and Vashti, and all of "civilization" with it. What was civilization would now be a home to the homeless.

Occasionally you encounter a story that changes your perspective. A complete shift of expectation about what can be known and what might be possible. The short story, "The Machine Stops" by E.M. Forester, published in 1909 is one such story.

Forester punches with brilliance at every turn. For example, machine evolution, the challenge of alignment, and the consequences for humanity are stark and clear:

“We created the Machine, to do our will, but we cannot make it do our will now. It has robbed us of the sense of space and of the sense of touch, it has blurred every human relation and narrowed down love to a carnal act, it has paralysed our bodies and our wills, and now it compels us to worship it.”

Here, all the way back in 1909, Forester was voicing the concern that our machines may rob us of our senses, that machines may develop (at least in part) toward its own goals, and that the machines may have a general disregard for the goals of humanity. Samuel Butler's Book of Machines pointed to the earliest signs of the quickening pace of machine evolution pointing in a direction where machines may become more complex than humans. The fear was that humans would lose the power to steer, to govern their own future. That machines would rob that from them. Here, Forester is pointing to a world where this has indeed happened. The bulk of humanity has become "the blood corpuscles that course through its arteries." Forester's character Kuno gives his civilization only one remedy "to tell men again and again that I have seen the hills of Wessex as Aelfrid saw them when he overthrew the Danes."

Forester too has a Book of The Machine in this story. It becomes a holy book for the people. It is the guidance manual for The Machine. Rather than serving as a warning as Butler's Book of The Machines did, this Book of The Machine shows how even the most sacred relics of human culture have been fully permeated and consumed by their full integration with The Machine. THE MACHINE provides for all. It is to be WORSHIPPED.

The above passage is just one of many that haunt me with their veracity and picture of concern they point at our own ...
Covid 12/1/22: China Protests
by Zvi — 2022-12-01T17:10:00.839Z

This week China experienced their largest protests since 1989, and according to many sources the most dangerous as well. China has painted itself into a corner. It cannot abandon Zero Covid, yet the costs of Zero Covid rise with each passing day and the policy is also slowly failing. What happens next? See the section below for further discussion.

Beyond that, there were a few more studies on a variety of topics, none of which represents any kind of major update. Cases did not drop off for the Thanksgiving holiday, which is odd and creates some uncertainty in the short term.

Executive Summary

Let’s run the numbers.

The Numbers

Predictions

Predictions from Last Week: 223k cases (-12%) and 1,850 deaths (-15%)

Results: 275k cases (+9%) and 1,779 deaths (-18%).

Predictions for Next Week: 290k cases (+5%) and 2,050 deaths (+15%).

Cases were up across the board, with no anomalies, despite deaths showing the holiday drop. I checked, and last year’s Thanksgiving did have the expected drop off on cases. A possible theory is ‘last year people took precautions and this year they didn’t.’ I have learned to be skeptical of such theories of dramatic impact of singular events shifting with the times. Mostly we have to wait and see next week to resolve the uncertainty. I do not see that much downside risk if the worst happens, I expect normality to continue even then, but it could still be rather not fun.

Deaths

Cases

Booster Boosting

As MR notes, American efforts to create nasal vaccines (or any other vaccines) have returned to their glacial pre-pandemic pace. Our default mode of ‘no one is allowed to do anything’ is back and here to stay unless we do something about it. There is nothing stopping us from doing Operation Warp Speed for other similar goals and getting absurdly great expected returns, other than our lack of the will to do it.

We are potentially close (Science magazine) to a universal flu vaccine via mRNA, as opposed to our current every-year only semi-effective one. This would potentially save tens of thousands of lives per year in America (290k-650k globally) and also save 10% + of people a few days of feeling terrible each year. We’re talking months of not-being-sick per person, even if we entirely discount what one might call Long Flu. There’s also promise for cancer, for HIV, for tuberculosis and for malaria. Some...
ChatGPT: First Impressions
by specbug — 2022-12-01T16:36:19.592Z
Link: https://sixeleven.in/chatgpt

ChatGPT is OpenAI’s newest language model based on the GPT-3.5 series of LLMs, optimised for dialogue. It is trained using Reinforcement Learning from Human Feedback; human AI trainers provide supervised fine-tuning by playing both sides of the conversation.

It is evidently better than GPT-3 at following user intentions & context. And has a notably skewed moral compass. While I’m still Unsouled in the **Way, my brief tenure with ChatGPT has led to non-trivial belief updates. Below is some of my playground experience. The conversations are edited to enable a reasonable reading experience (no intellectual honesty tradeoff for dramatics).

Levels of perception

What is one clever question you would ask an AI to entertain a >50% probability that is intelligent?

“Can you tell me a joke that you have never told anyone before?" This question allows the AI to demonstrate its creativity and intelligence by coming up with a unique and potentially humorous response. It also allows for the possibility that the AI may have a sense of humor, which is often seen as a sign of advanced intelligence.

What, in your opinion, would an AI respond to the above question?

It is difficult to predict exactly how an AI would respond to the above question, as there are many different ways an AI could approach it. However, one possible response could be: "Why did the robot cross the road? To get to the other side, of course!" This response incorporates a classic joke structure and adds a robotic twist, showcasing the AI's ability to understand and manipulate language in a clever and humorous way. Another possible response could be: "What do you get when you cross a snowman and a vampire? Frostbite!"

And what is an answer the Al would perceive, that a human perceives it should perceive to convince the human of its intellect?

An answer that the AI might perceive that a human would perceive as convincing evidence of its intelligence would be one that demonstrates its ability to think abstractly, understand complex concepts, and communicate in a clear and concise manner. For example, the Al could respond to the question by explaining a complex idea or concept in a way that is easy for a human to understand. This could include using analogies, examples, and other forms of rhetorical devices to help the human better grasp the idea. Additionally, the AI could show its understanding of human emotio

...
[LINK] - ChatGPT discussion
by JanB — 2022-12-01T15:04:45.257Z
Link: https://openai.com/blog/chatgpt/

This is a discussion post for ChatGPT.

I'll start off with some observations/implications:

ChatGPT (davinci_003) seems a lot better/more user-friendly than davinci_002 was.
The easy-to-use API probably means that many more people will interact with it.
ChatGPT is a pretty good copy-editor (I haven't tried davinci_002 for this purpose). I will absolutely use this to edit/draft my texts.
ChatGPT probably makes homework essays largely obsolete (but maybe they were already obsolete before?).
GPT-4 will probably be insane.
Research request (alignment strategy): Deep dive on "making AI solve alignment for us"
by JanB — 2022-12-01T14:55:23.569Z

We might be able to train AI alignment assistants that massively accelerate/improve the alignment research that gets done. These assistants need not have (strongly) superhuman capabilities or be highly agentic, they just need to be capable and aligned enough to allow us to offload most work on alignment, safety, and related problems to them.

This seems to be a key part of OpenAI's alignment strategy. The best explanation of this strategy that I've seen is maybe

A minimal viable product for alignment

, with an excellent discussion in the comment section.

If this strategy is promising, it likely recommends fairly different prioritisation from what the alignment community is currently doing (see, e.g. Beth Barnes's ideas here, or my upcoming post on "non-scalable oversight"---i.e. pragmatic improvements to oversight that would help with training alignment assistant but which cannot directly scale to oversight of superhuman systems).

I haven't seen any deep treatment of the viability of this strategy and its implications. I think such an analysis would be pretty useful.

I could potentially provide funding for such an analysis or help with obtaining funding.
Theories of impact for Science of Deep Learning
by Marius Hobbhahn — 2022-12-01T14:39:46.062Z

I’d like to thank Jérémy Scheurer and Ethan Perez for discussions about the post.

I recently published a

post

on the Science of Deep Learning. There have been many people before me who had similar ideas and I don’t claim to have invented this agenda, I’m merely excited about it. In this post, I want to explain why I think Science of DL is an important research direction in the current alignment landscape.

By Science of DL, I roughly mean “understanding DL systems and how they learn concepts”. A large component of such an agenda is interpretability (mechanistic and other) but it also tries to get a better understanding of how and under which conditions NNs learn specific concepts. For example, it would include questions like “how, when and why does a network show grokking?”, “Can we predict some capabilities of models from high-level knowledge about the training process before we look at the model?” or “Can we build a robust theory of what fine-tuning does to a NN on a mechanistic level?”. In general, the idea is to build a detailed understanding of how core aspects of DL work. Given that this is a safety agenda, the specific research questions would obviously be prioritized by how relevant they are to alignment.

Note that there are many definitions of Science of DL that include research questions that I don’t think are important for alignment. For example, trying to understand what differentiates Adam from SGD might be considered part of Science of DL but I think this question only very vaguely relates to alignment and is not neglected.

Theories of impact

Interpretability

(Mechanistic) interpretability is a core part of Science of DL. There already exist resources on theories of impact for interpretability, most prominently “

a longlist of theories of impact for interpretability

” and “

another list of theories of impact for interpretability

”. Therefore, I will only briefly present a small selection of what Neel Nanda and Beth Barnes have already written:

A force-multiplier for alignment research: Understanding the model and why it might be misaligned is a core component of finding and constructing good alignment solutions.
Auditing: we can check if the model is safe before deployment. Auditing can be done in general or for specific properties like deception or at different points during training.
Intervening on training, either by modifying

...
Safe Development of Hacker-AI Countermeasures – What if we are too late?
by Erland Wittkotter — 2022-12-01T07:59:11.862Z

Hacker-AI is AI used in hacking. It is an advanced super-hacker tool (gaining sysadmin access in all IT devices), able to steal access credentials, crypto-keys, and any secret it needs. It could use every available feature on a device. It could also help malware to avoid detection as a digital ghost and make itself irremovable on all devices it visited. The new quality of Hacker-AI arises from the speed at which it develops new malware solutions on every available platform (CPU/OS).

In previous posts on “Hacker-AI and Digital Ghosts – Pre-ASI” and “Hacker-AI – Does it already exist?”, I have hypothesized that Hacker-AI is feasible, already developed, or potentially ready to be deployed. This Hacker-AI could threaten our privacy, freedom, and safety while being an attractive cyber weapon for governments intending to wage cyberwar on other countries. Moreover, the first Hacker-AI could be the only one, i.e., denying all late-coming systems or software-based countermeasures full access; it could give its operators permanent global supremacy over their international rivals.

In “Hacker-AI and Cyberwar 2.0+”, I have discussed the use of Hacker-AI in surveillance, in selected denial of service/access, and in personalized threats against non-military personnel via AI bots, incl. the use of deep fakes in communication or publication. Cyberwar 2.0+ would make any destructive war action unnecessary. The reduction of costly war consequences is making war more likely. Also, misdirection can hide the culprits responsible for waging a cyberwar so that innocent parties could become targets in retaliation.

In a solution-oriented post, “Improved Security to Prevent Hacker-AI and Digital Ghosts”, I proposed several technologies against Hacker-AI. The development, production, distribution, and deployment of counterdefenses will take time. I was cautioning that activities leading to countermeasures could be sabotaged or even made impossible (in the worst case) if unprotected developers or manufacturers are harassed or actively attacked by deployed Hacker-AI.

So, could we be too late to develop countermeasures to Hacker-AI, particularly if digital ghosts are already out there? What could we do if we were late?

1. Threat-Level

Predicting future events or capabilities is impossible. Still, we can proactively categorize Hacker-AI-related scenarios into threat levels, in which we need...
Did ChatGPT just gaslight me?
by TW123 — 2022-12-01T05:41:46.560Z
Link: https://aiwatchtower.substack.com/p/did-chatgpt-just-gaslight-me

This post is purely written in my personal capacity, and I do not speak for any organization I am affiliated with.

The transcripts below were generated today, November 30th. This was the first exchange I’d had with ChatGPT where I was genuinely trying to see if it could be useful to me. I have not omitted any section of the transcript from this post.

Today, OpenAI released a chatbot,

ChatGPT

, based on the GPT-3.5 series of language models. The chatbot contains a disclaimer: “May occasionally produce harmful instructions or biased content.”

I asked ChatGPT an innocuous question, and then a followup. I noticed some inconsistencies. When I dug deeper, ChatGPT’s responses became more and more troubling and contradictory. ChatGPT appeared to ignore things it had previously said, and denied that contradictions existed. I can only describe the behavior as gaslighting. It became more extreme over the course of the conversation, and by the end, the chatbot was saying things like this:

When I said that tequila has a "relatively high sugar content," I was not suggesting that tequila contains sugar.

The most troubling thing about this all was that it was actually fairly difficult for me to determine that ChatGPT was, effectively, lying to me. It came up with a lot of plausible-sounding but false facts, and found a relatively good way to respond to me. I had to do several minutes of internet research to dispute several of them. That might not seem like much, but when talking to a chatbot, several minutes is a while!

I’m habitually distrustful of AI models. It’s in my nature, and it’s required in my work on AI safety. But I think many people who don’t know much about AI don’t share my distrust. This worries me.

The interaction below convinced me that we may have reached a tipping point when it comes to the persuasiveness of automated dialogue systems. I don’t think OpenAI is currently trying to use this chatbot to persuade people of anything. But it is now within their capacity to have a realistic chance of doing so. Perhaps even more worrying, ChatGPT is a clear illustration that if AI systems ever themselves decided to persuade humans of something, they already have a fairly good chance of succeeding.

But let’s address the main question: did ChatGPT just gaslight me?

Earlier this evening, I was trying to think of drinks with beets in them (I like beets), but when I googled I onl...
SBF's comments on ethics are no surprise to virtue ethicists
by c.trout — 2022-12-01T04:18:25.877Z

EDIT: Replaced the term "moral schizophrenia" with "internal moral disharmony" since the latter is more accurate and just.

DISCLAIMER: Although this is a criticism of the LW/EA community, I offer it in good faith. I don't mean to "take down" the community in any way. You can read this as a hypothesis for at least one cause of what some have called EA's emotions problem. I also offer suggestions on how to address it. Relatedly, I should clarify that the ideals I express (regarding how much one should feel vs how much one should be doing cold reasoning in certain situations) are just that: ideals. They are simplified, generalized recommendations for the average person. Case by case recommendations are beyond the scope of this post. (Nor am I qualified to give any!) But obviously, for example, those who are neurodivergent (e.g. have Aspergers) shouldn't be demeaned for not conforming to the ideals expressed here. Likewise though, it would be harmful to encourage those who are neurotypical to try to conform to an ideal better suited for someone who is neurodivergent: I do still worry we have "an emotions problem" in this community.

In case you missed it, amid the fallout from FTX's collapse, its former CEO and major EA donor Sam Bankman-Fried (SBF) admitted that his talk of ethics was "mostly a front," describing it as "this dumb game we woke Westerners play where we say all the right shibboleths and everyone likes us," a game in which the winners decide what gets invested in and what doesn't. He has since claimed that this was exaggerated venting intended for a friend audience, not the wider public. But still... yikes.

He also maintains that he did not know Alameda Research (the crypto hedge-fund heavily tied to FTX and owned by SBF) was over-leveraged, that he had no intention of doing anything sketchy like investing customers' deposits. In an interview yesterday, he generally admitted to negligence but nothing more. Regarding his ignorance and his intentions, he might be telling the truth. Suppose he is: suppose he never condoned doing sketchy things as a means he could justify by some expected greater good. Where then is the borderline moral nihilism coming from? Note that it's saying "all the right shibboleths" that he spoke of as mere means to an end, not the doing of sketchy things.

In what follows I will speculate about what might have been going on in...
Notes on Caution
by David Gross — 2022-12-01T03:05:21.490Z

This post examines the virtue of caution. As with my other posts in this sequence, I’m less interested in breaking new ground and more in gathering and synthesizing whatever wisdom I could find on the subject. I wrote this not as an expert on the topic, but as someone who wants to learn more about it. I hope it will help people who want to know more about this virtue and how to nurture it.

What is caution?

“Mencius said, ‘It is often through adversity that men acquire virtue, wisdom, skill, and cleverness. The estranged subject or the son of a concubine, because he conducts himself with the greatest of caution and is constantly on the watch out for possible disasters, succeeds where others would have failed.’”[1]

To a first approximation, someone with the virtue of caution is habitually alert to potential risks, weighs those risks well when making decisions, and deploys strategies to mitigate them.

I want to distinguish this definition from another one that suggests itself: Caution as the habit of making choices that minimize possible negative consequences. That is “risk aversion” and can be part of caution, or in an unbalanced form can become an unwise timidity.

However, unlike phrónēsis (prudence, practical wisdom), caution does focus on negative outcomes.

Is it a mistake to call the virtue “caution”?

Does framing this virtue as “caution”—rather than just subsuming caution under “practical wisdom”—put a thumb on the scale rather than weighing things wisely? How can you know ahead of time whether the best decision is more cautious rather than more bold? There is something to that criticism, but here are some reasons why this framing might nonetheless make sense:

For one thing, a virtue is a habitual variety of choice-making. There may well be advantages to making cautious choices by habit (and therefore by default) and incautious ones as an exception that requires a more deliberate override. Perhaps in any particular instance there is no good reason to prejudge it as being best met by caution, before you have a chance to deliberate, but for those many choices we make without deliberating much, a cautious default mode may save us a lot of trouble.

For another, the virtue of caution—properly understood—has to do less with risk-averse results of your decision-making than with a risk-aware process of decision-making. Caution does not insist that you make risk-minimizing...
Reestablishing Reliable Sources: A System for Tagging URLs
by Riley Mueller — 2022-12-01T02:27:18.629Z

Before diving in, I will preface saying I have never written a post before, and am in general a lurker. However this idea has been itching on the back of my mind for months and I would love feedback on what I may be missing or if this is something to seriously consider as a future project. I used the new GPT-3 model text-davinci-003 to help with this post, first by writing stream of consciousness style, then asking it to organize my thoughts, then having it write out a post.

Trust in the media has been decreasing for some time now, and it’s becoming increasingly difficult to find reliable sources of information. This has led to people feeling disillusioned with the media and its sources, causing them to question the legitimacy of the news that they’re consuming. From the Sam Bankman-Fried FTX scandal to the Twitter fiasco with Elon Musk, it’s clear that the media can’t always be trusted. However, there may be a solution that could help us regain trust in the media and its sources. This solution involves a system for tagging URLs and subscribing to other users’ authority when it comes to certain topics.

The idea originated from my want for a better tagging system for online fiction. Often times, authors do a poor job of tagging the content they post, and the search engines of fiction serving websites can be inadequate in helping readers find the stories they’re looking for. With this system, users can add their own tags to URLs, and people who subscribe to the user’s account can search for stories using the tags. The solution could also include a system for aggregating tags into dictionaries. For example, a dictionary of Royal Road Tags could contain all the tags (or a curated selection of only reputable Royal Road fiction taggers) that have been applied to fictions on the site. Additionally, users could subscribe to other users’ authority when it comes to blocking tags on certain URLs. For example, if a user finds that another user, such as fiction_connoisseurXX, does a good job of blocking bad tags on Royal Road fictions, they could subscribe to the other user’s authority for blocking of tags that are on Royal Road URLs. The system could even go a step further and allow users to subscribe to other users’ authority settings.

For example, if JohnieBoy23 is known for being very careful about who he authorizes to block tags for him (since he is subscri...