trentmkelly/LessWrong-43k huggingface dataset:
Universals of Morality: Toward Human-Centric Communication Platforms
by scafaria — 2022-04-26T21:15:50.520Z
Link: https://scafaria.com/universals-of-morality-toward-human-centric-communication-platforms-3899b36eecf9

Each person alone is powerless, a fleshy mammal that would typically die from the elements. Networked together into factions, we devise ways to benefit “us” or defeat “them”.

Social platforms like Facebook, Twitter, and Tik-Tok have brought this human story to unprecedented scale, connecting billions. Whatever the platforms might have done with this power, the chosen business model has been to monetize access to our minds. They sell the power to divide us into groups and to sell access to each group of “us” to the highest bidder, often pitting us against a chosen group of “them”. Dictators and demagogues have grown particularly adept at using these systems to bend our minds and fracture society. With 15 years of hindsight, it’s clear that great power should have come with greater responsibility.

Perhaps humanity’s communications will always be rooted in surveillance-based business models. But let’s indulge for the moment the ideal of a platform that networks people together for a novel purpose: not for monetizing access to our minds, but rather with the express purpose of elevating humanity.

Regardless of how it’s implemented (whether via an AI, an oversight board, or some other means), what moral framework would be “best” for humanity?

Can this question be answered? It is bold to try, but the status quo is clearly unacceptable. Let’s zoom out to find an anchor among universal truths.

The universe started as nothing. Then somehow, matter flashed into existence. Out of nothingness there was suddenly Creation.

Over time, the matter clumped together in stars and galaxies. Relative to the empty void of space, the mind perceives these more complex structures as “interesting”.

On Earth, matter formed ever more complex structures, eventually to include living creatures with minds, most notably humans. We formed complex societies, so much more interesting and dynamic than just the sum of the parts.

This has somehow happened despite the overwhelming tendency of the universe toward more dispersion, more randomness, more entropy as scientists would say. At a cosmic scale, “stuff” is moving apart and becoming less interesting over time. Eventually, atoms will just be randomly spread out, will no longer interact, and the universe will be dead.

Yet life, and particularly complex life, bucks this trend. If the universe were a TV screen, it went from all black nothingness (nihilism)...
[$20K in Prizes] AI Safety Arguments Competition
by Dan H — 2022-04-26T16:13:16.351Z

TL;DR—We’re distributing $20k in total as prizes for submissions that make effective arguments for the importance of AI safety. The goal is to generate short-form content for outreach to policymakers, management at tech companies, and ML researchers. This competition will be followed by another competition in around a month that focuses on long-form content.

This competition is for short-form arguments for the importance of AI safety. For the competition for distillations of posts, papers, and research agendas, see the

Distillation Contest

.

Objectives of the arguments

To mitigate AI risk, it’s essential that we convince relevant stakeholders sooner rather than later. To this end, we are initiating a pair of competitions to build effective arguments for a range of audiences. In particular, our audiences include policymakers, tech executives, and ML researchers.

Policymakers may be unfamiliar with the latest advances in machine learning, and may not have the technical background necessary to understand some/most of the details. Instead, they may focus on societal implications of AI as well as which policies are useful.
Tech executives are likely aware of the latest technology, but lack a mechanistic understanding. They may come from technical backgrounds and are likely highly educated. They will likely be reading with an eye towards how these arguments concretely affect which projects they fund and who they hire.
Machine learning researchers can be assumed to have high familiarity with the state of the art in deep learning. They may have previously encountered talk of x-risk but were not compelled to act. They may want to know how the arguments could affect what they should be researching.

We’d like arguments to be written for at least one of the three audiences listed above. Some arguments could speak to multiple audiences, but we expect that trying to speak to all at once could be difficult. After the competition ends, we will test arguments with each audience and collect feedback. We’ll also compile top submissions into a public repository for the benefit of the x-risk community.

Note that we are not interested in arguments for very specific technical strategies towards safety. We are simply looking for sound arguments that AI risk is real and important.

Competition details

The present competition addresses shorter arguments (paragraphs and one-liners) wit...
Continental Philosophy as Undergraduate Mathematics
by Jan — 2022-04-26T08:05:17.433Z
Link: https://universalprior.substack.com/p/continental-philosophy-as-undergraduate?s=r

TL;DR: Revisionist continental philosophy, starring Hegel, Heidegger, and Kant (!?). A fair bit of hand-waving and namedropping maths. Certainly on the weirder end of essays I've written.

Previously in this series:

Frankfurt Declaration on the Cambridge Declaration on Consciousness

,

Belief-conditional things - things that only exist when you believe in them

Not another declaration.

Looking back, I feel bad about how I wrote the

Frankfurt declaration

. I'm not sure I managed to convey that I am against both animal suffering and the Cambridge Declaration[1]. Also, mixing in that fictional narrative of intoxicated neuroscientists partying and signing something they don't understand might have taken things too far[2]. But, oh well,

live and learn

.

How can I avoid that failure mode? How can I clearly state that when I'm writing about consciousness, continental philosophy, and/or

ghosts

, I'm poking fun at something

I don't understand

? I am tempted to write another declaration but that would be a bit too on the nose. Instead, I'll just put a clear "

epistemic status

" warning at the top of each post in this series. Cool? Cool.

Epistemic status: Spurious fragments of sense-making, arranged to amuse, not to enlighten.

Words, not numbers

Does anyone remember

wordcels vs. shape rotators

?[3] No? Okay, good for you.

A rift divides philosophy, with analytic philosophy (

a broad and still ramifying movement in which various conceptions of analysis compete and pull in different directions

) on one side and continental philosophy (a term used by analytic philosophers to describe what they are not) on the other. Analytic philosophy gave us evergreen tautologies like

1+1=2

,

'snow is white' is true if and only if snow is white

, and

"Whereof one cannot speak, thereof one must be silent."

Continental philosophy, in contrast, provides rich head-scratchers to ponder, like

Transcendence constitutes selfhood

,

God is, as it were, the sewer into which all contradictions flow

, and "

It is hard enough to remember my opinions, without also remembering my reasons for them!

"

Analytic philosophy is sometimes

criticized for not engaging

with the deep questions, like "

Is true beauty subjective or objective?

", "

Is it easier to love or be loved?

" or, importantly, "

Would you ever eat insects in a restaurant?

"

Continental philosophy is sometimes criticized for... ahem. A lot of things.

Large swaths of philoso

...
dalle2 comments
by nostalgebraist — 2022-04-26T05:30:07.748Z

(i.)

On April 6, OpenAI announced “DALLE-2” to great fanfare.

There are two different ways that OpenAI talks to the public about its models: as research, and as marketable products.

These two ways of speaking are utterly distinct, and seemingly immiscible. The OpenAI blog contains various posts written in each of the two voices, the “research” voice and the “product” voice, but each post is wholly one or the other.

Here’s OpenAI introducing a model (the original DALLE) in the research voice:

DALL·E is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions, using a dataset of text–image pairs. […]

Like GPT-3, DALL·E is a transformer language model. It receives both the text and the image as a single stream of data containing up to 1280 tokens, and is trained using maximum likelihood to generate all of the tokens, one after another. This training procedure allows DALL·E to not only generate an image from scratch, but also to regenerate any rectangular region of an existing image that extends to the bottom-right corner, in a way that is consistent with the text prompt.

And here they are, introducing a model (Codex) in the product voice:

OpenAI Codex is a descendant of GPT-3; its training data contains both natural language and billions of lines of source code from publicly available sources, including code in public GitHub repositories. […]

GPT-3’s main skill is generating natural language in response to a natural language prompt, meaning the only way it affects the world is through the mind of the reader. OpenAI Codex has much of the natural language understanding of GPT-3, but it produces working code—meaning you can issue commands in English to any piece of software with an API. OpenAI Codex empowers computers to better understand people’s intent, which can empower everyone to do more with computers.

Interestingly, when OpenAI is planning to announce a model as a product, they tend not to publicize the research leading up to it.

They don’t write research posts as they go along, and then write a product post once they’re done. They do publish the research, in PDFs on the Arxiv. The researchers who were directly involved might tweet a link to it. And of course these papers are immediately noticed by ML geeks with RSS feeds, and then by their entire social networks. It’s not like OpenAI is trying to hide these publications. It’s just not...
Make a neural network in ~10 minutes
by Arjun Yadav — 2022-04-26T05:24:57.507Z
Link: https://arjunyadav.net/blog/make-a-neural-network-in-10-minutes

Prerequisites and Introduction

This post assumes you know Python decently well (though I do explain a good chunk of the code, so it shouldn't be a big issue if you don't) and that you know how neural networks work.

If you would like to learn how neural networks work, I highly recommend this roughly 20-minute video by 3Blue1Brown.

Most of my knowledge for this post came from the first two lessons of Practical Deep Learning for Coders by fast.ai and this post intends to be a gentle introduction to programming A.I. and is intentionally surface level. I hope it demystifies the process of creating simple models for projects for someone interested in A.I. safety.

Jupyter Notebook

Jupyter Notebook is an integrated development environment (IDE) that allows us to write Python code in smaller chunks (called cells) and see the output any time we run one of these cells.

Not only does this lead to regularly "sanity checking" our code (which is a good habit in programming as we catch silly errors quickly before they become a major issue), but we also regularly get the output, which is useful in conjunction with other packages, such as matplotlib.

Our notebooks can also turn into a simple app within the notebook using widgets (and you can even deploy them, but that isn't discussed in this post).

We'll be using Google Colab, an online Jupyter Notebook environment that's simple to use. Open up a fresh notebook via File > New Notebook and let's get started.

fast.ai

fast.ai is a deep learning library that provides high-level tools that make making models much simpler (as we cut through a lot of the boilerplate).

We'll be using the fastbook package (since it's bundled with fastai and a few extra tools, such as the previously mentioned widgets), copy and paste the code snippets (one cell for each snippet) and hit Ctrl/Cmd + Enter to run the cell.

!pip install fastbook==0.0.17
!pip uninstall tornado -y
!yes | pip install tornado==5.1.0

from fastbook import *
from fastai.vision.widgets import *

Getting our data

I recommend using the Bing Image Search API (it's free for 1,000 calls per month and you can use it later to make your own classifier for other types of images)[1] as fastai allows us to use it to easily get about 150 images of different types.

For this example, we're going to train our neural network to classify three types of bears: grizzly, black and teddy.

We provide our API ...
Framings of Deceptive Alignment
by peterbarnett — 2022-04-26T04:25:56.115Z

In this post I want to lay out some framings and thoughts about deception in misaligned AI systems.

Types of Deception

There seem to be two different things which people mean by deception which have different causes and likely different effects. Because these are both often called ‘deception’, they are often incorrectly equated. To reason clearly about the dangers of deceptive AI we should be clear about which one we are talking about.

Goodhart Deception

An AI system may learn a strategy which just tricks the evaluator into giving high reward during training, rather than actually doing well on the task. The AI is ‘Goodharting’ the reward by optimizing for a proxy rather than for what humans actually want.

As a specific example we might be training an AI system using reinforcement learning with human feedback. Here the AI takes an action, the human evaluates how good or bad the action was, and then this reward is used to reinforce the behavior which was evaluated as good. Because we are training on human feedback, any behavior which the human thinks is good will result in a positive reward. So AI could just choose actions which seem good to the human but aren’t actually good. This is an outer alignment problem, where the AI learns to do a bad thing because the humans are unable to adequately reward it for doing the correct thing.

This will become more of a problem as

The task becomes more difficult for the AI to actually do
The task becomes more difficult for the human to judge

These two things seem likely to increase in tandem; harder tasks will be harder to evaluate. If the task is very difficult, then it will instead just learn to do the simpler thing (trick the human) which achieves the same high reward.

It is important to note that this doesn’t require the AI to have any ‘agency’ or ‘objective’; the training process has just reinforced a behavior which leads to high reward. Once the AI starts to trick the human, this will lead to good reward, and so this deceptive behavior is in some sense an attractor state.

Notably this behavior was observed in the original “

Deep reinforcement learning from human preferences

” paper. Here an AI was being trained with human feedback to grasp a ball. Instead of actually grasping the ball, the hand was simply placed between the ball and the camera to give the impression of holding the ball (see the...
Why pessimism sounds smart
by jasoncrawford — 2022-04-25T20:10:31.344Z
Link: https://rootsofprogress.org/why-pessimism-sounds-smart

Pessimists sound smart. Optimists make money.
–Nat Friedman (quoted by Patrick)

I’ve realized a new reason why pessimism sounds smart: optimism often requires believing in unknown, unspecified future breakthroughs—which seems fanciful and naive. If you very soberly, wisely, prudently stick to the known and the proven, you will necessarily be pessimistic.

No proven resources or technologies can sustain economic growth. The status quo will plateau. To expect growth is to believe in future technologies. To expect very long-term growth is to believe in science fiction.

No known solutions can solve our hardest problems—that’s why they’re the hardest ones. And by the nature of problem-solving, we are aware of many problems before we are aware of their solutions. So there will always be a frontier of problems we don’t yet know how to solve.

Fears of Peak Oil and other resource shortages follow this pattern. Predictions of shortages are typically based on “proven reserves.” We are saved from shortage by the unproven and even the unknown reserves, and the new technologies that make them profitable to extract. Or, when certain resources really do run out, we are saved economically by new technologies that use different resources: Haber-Bosch saved us from the guano shortage; kerosene saved the sperm whales from extinction; plastic saved the elephants by replacing ivory.

In just the same way, it can seem that we’re running out of ideas—that all our technologies and industries are plateauing. Technologies do run a natural S-curve, just like oil fields. But when some breakthrough insight creates an entirely new field, it opens an entire new orchard of low-hanging fruit to pick. Focusing only on established sectors and proven fields thus naturally leads to pessimism. To be an optimist, you have to believe that at least some current wild-eyed speculation will come true.

Why is this style of pessimism repeatedly wrong? How can this optimism be justified? Not on the basis of specific future technologies—which, again, are unproven—but on the basis of philosophical premises about the nature of humans and of progress. The possibility of sustained progress is a consequence of the view of humans as “universal explainers” (cf. David Deutsch), and of progress as driven fundamentally by human choice and effort—that is, by human agency.

The opposite view is that progress is a matter of luck....
What is being improved in recursive self improvement?
by Lone Pine — 2022-04-25T18:30:47.848Z

When an AGI recursively self improves, is it improving just it's software or is it improving the hardware too? Is it acquiring more hardware (e.g. by creating a botnet on the internet)? Is it making algorithmic improvements? Which improvements are responsible for the biggest order-of-magnitude increases in the AI's total power?

I'm going to offer a four-factor model of software performance.
The reason why I bring this up is because I'm personally skeptical about the possibility of FOOM. Modern machine learning is just software, and a great deal of effort has already been applied to improve all four factors, so it's not obvious to me that there are still many orders of magnitude left that can be improved very quickly. Of course, it's possible that future AGI will be so exotic that this four-factor model doesn't apply. (Presumably such an AGI would run on application specific hardware, such as neuromorphic hardware.) You don't have to use this model in your answer.

My Four Factor Model of Software Performance

How performant are the most critical algorithms in the software, from a pure computer science perspective? (This is what big-O notation measures. "Critical" in this context refers to the part of the software most responsible for the slowdown.)

How well optimized is the software for the hardware? (This is usually about memory and cache performance. There is also an aspect of what CPU instructions to use, which modern programmers usually leave to the compiler to manage. In the old days, game devs would program the most critical parts in assembly to maximize performance. Vectorization also falls in this category.)

How well optimized is the hardware for single threaded performance? (Modern CPUs have already hit a limit here, although significant improvements can still be made for application specific hardware.)

How much parallel processing is possible and available? (This is limited by algorithms, software architecture and hardware. In practice, parallelism gives a fraction of the benefit that it should, due to the difficulty and complexity involved. Amdahl's Law puts a hard limit on the benefits of parallelism. There is also a speed-of-light limitation, but this only matters if the system is geographically distributed i.e. a botnet.)

Again, the question is, what is being improved in recursive self improvement?
21 on 21
by Amir Bolous — 2022-04-25T18:22:23.110Z

This is a linkpost for https://amirbolous.com/posts/21/

Today, I turn 21. Here are 21 reflections for 21 years.

The highest ROI thing you can invest in early on is learning to be an independent thinker. The world is full of noise. It's impossible to drown out it all out, but it's possible to pick out the tunes if you listen carefully. Learn to derive things for yourself. Read and listen to what other people say but always arrive at your own conclusions.

Corollary to this: there may be forces that drive you to take a certain path. A shocking number of new possibilities open up when you realize you don't have to walk on the path laid down by others for you, and you can in fact craft your own path. You're also probably underestimating yourself.

Who you surround yourself with is very important. Everyone talks about this but I still think we underestimate how large of an impact this has. Do your friends inspire you? Do they encourage you to become a better person? A good litmus test for what kind of person you are is are you a good influence on your friends' lives? Do you lift them up or bring them down?

Initially, the bottleneck to doing something is often knowing that it's even an option. After that, the bottleneck is courage not intelligence.

There's value in approaching different aspects of your life with more of a "scientist mindset." Many of us often spend too much time worrying about what to do as opposed to trying things to see what we actually like doing. Treat all aspects of your life as little experiments. Treat the project you've been to afraid to start as an experiment that may fail. Treat your exercise routine that you haven't gotten around to commit to as an experiment. Try, observe, and adapt. There's nothing to be afraid of if you fail because it's just an experiment.

Try not to judge others because you don't know what they care about or what their value function actually is. What makes sense for others might not make sense to you if you don't care about the same things.

Be competitive with yourself. Are you a better person than you were yesterday? Did you learn something new today? Try not to be competitive with others because it leads to some common failure modes like a zero-sum, scarcity mindset.

Being honest with yourself is hard, but your intuition is more telling than you think.

Be true to yourself is not great advice. Instead

...
Rationalist Inspired Coming-of-age Rituals
by iceplant — 2022-04-25T17:22:35.789Z

I was recently asked to lead part of my cousin's secular Bar Mitzvah inspired coming-of-age ceremony. Would love recommendations for rituals, quotes, or wisdom to use here. I've really enjoyed the rituals I've seen come out of the rationalist community, and I'm curious if anyone has done or has any ideas for something that could be fun for a nerdy soon to be teenager.