trentmkelly/LessWrong-43k huggingface dataset:
Why I'll Keep My Crummy Drawings—How Generative AI Art Won't Supplant... Art.
by James Stephen Brown — 2024-05-15T19:30:05.410Z

The other day the post below caught my attention. In its characteristic bad-grammar-ambiguous-origin-meme style, Neo’s sentiment is one I share.

As an artist of sorts (it’s the only thing I’m actually qualified for) I see art as more about transformation than creation. I find we (and yes, I include myself) tend to mythologise art, attributing to it an undeserved magical or spiritual significance, rather than recognising it as basic graft.

CREATIVE PLAGIARISM

In a way, art is creative plagiarism

*

; when creating an artwork, I can only combine my own experience with my understanding of what art is, which is an amalgamation of the art I have consumed. I literally have no other material to draw on. In fact, it turns out that too much originality in art can actually be counter-productive, as Derek Thompson in his fabulous TED Talk

The four-letter code to selling anything

states:

“To sell something surprising, you have to make it familiar.”

Software like

Stable Diffusion

and

Midjourney

took people by surprise because we imagined art and creativity as something magical, an echo of the human spirit, irreducible and therefore something AI would have trouble replicating. As late as 2020 in “Unlocking the Universe”, Beth West wrote this about ‘Cities of the Future’

†

“… lots of things can’t be done by machines: creative jobs such as writing books and creating art… These areas will continue to need people…”

We are all now aware that visual art in particular has turned out to be one of the lowest hanging fruits of all, and regarding writing… it seems that the writing may be on the wall (that was my own pun by the way, chat GPT isn’t the only author who will resort to the lowest form of wit, given the right conditions).

DON’T DESPAIR

My admittedly cynical view of art is informed by experiences at art school and in the gallery scene, where I’ve had some small opportunity to glimpse behind the smoke and mirrors used to multiply art’s perceived value and shroud it in illusory significance.

Don’t get me wrong, creativity is fun, and rewarding, and can be impressive—it is an essential part of life, but it’s not magic. Art is work. It’s no wonder that AI can replicate and even improve on this process, it’s put in the hours (which, for AI, take seconds). AI has absorbed more material than we can imagine (in part because our imagination is limited by what we have absorbed!).

BUT THAT’S NOT T

...
How is GPT-4o Related to GPT-4?
by Joel Burget — 2024-05-15T18:33:43.925Z

GPT-4o both has a new tokenizer and was trained directly on audio (whereas my understanding is that GPT-4 was trained only on text and images). Is there precedent for upgrading a model to a new tokenizer? It seems like it's probably better to think of it as an entirely new model. If that's the case, what actually makes it a GPT-4?
[Linkpost] Please don't take Lumina's anticavity probiotic
by ROM — 2024-05-15T18:03:20.336Z
Link: https://trevorklee.substack.com/p/please-dont-take-luminas-anticavity

Update:

Trevor Klee (author of the linked post) has published an update in which he (arguably) moderates his view (or at least that which he expresses publicly). Specifically, he states:

I believe (note the libel-friendly phrasing) that:

1. Lumina’s manufacturing process follows legally mandated GMP protocols, if not the probiotic trade association’s voluntary best practices.

2. It is weird to be secretive about your manufacturing until pressed on it, especially when you have made a point of trying to evade regulations. See Zbiotics for a great example of how to behave responsibly and communicate openly when selling genetically modified bacteria for human health issues. It’s especially weird to threaten lawsuits when people ask follow-up questions about your manufacturing.

3. Lumina’s product is a drug, not a cosmetic product. And, regardless of whether it is a cosmetic product, it has the potential to cause great harm. This means it needs extensive human safety testing. This can be under the FDA or not.

4. There are scientific reasons to believe that Lumina’s product can be unsafe and ineffective in humans, based on the reasoning in my previous posts. This uncertainty can and should be resolved by careful, well-designed human trials, not by releasing the product into the wild.

5. It was wrong for Lumina to take money for the product, like they did in Honduras and in pre-orders, without doing proper testing.

6. Threats of lawsuits have no place in open scientific debate.

This was prompted by Lumina founder Aaron Silverbook sending the following email to Klee:

Subject line: Defamation

From: Aaron Silverbrook aaron[at]lanternbioworks.com

To: trevor[at]highwaypharm.com

Hi Trevor;

I believe your post was made in good faith. Or rather—I didn’t, really, but after talking with Elizabeth, she vouched for your character and convinced me that it probably was, comments about my friends aside. So, I appreciate the efforts you've gone through out of a desire to keep people safe. As such, it’s probably for the best if we talk.

To speak to several of your concerns about our manufacturing processes—we are, actually, following Good Manufacturing Practices. We have, actually, sequenced the genome of the bacteria, and I posted that genome publicly on Manifold after "declaring mission success". We conduct batch testing through Eurofins. We have an experienced biomanufacturin

...
Was Partisanship Good for the Environmental Movement?
by Jeffrey Heninger — 2024-05-15T17:30:54.796Z
Link: https://blog.aiimpacts.org/p/was-partisanship-good-for-the-environmental

This is the third in a sequence of posts taken from my recent report: Why Did Environmentalism Become Partisan?

Summary

Rising partisanship did not make environmentalism more popular or politically effective. Instead, it saw flat or falling overall public opinion, fewer major legislative achievements, and fluctuating executive actions.

Public Opinion

One hypothesis is that partisanship was useful, or even necessary, for an issue to become popular. Maybe journalists never would have covered the story if it did not involve an exciting partisan contest. The public then might have never realized that this is a thing they could care about.

The polling data do not support this hypothesis.

The clearest data from McCright et al.[1] Over 70% of both parties, and both ideologies, supported more government spending on the environment in 1990. Then, over the next 20 years, Republicans’ support for environmental spending fell dramatically while Democrat support remained roughly constant. These polls show declining overall support as partisanship increased.

Figure 1: Percentages of Democrats and Republicans reporting that national spending on the environment is “Too Little,” 1974-2012. Reprinted from McCright et al. (2014).

The Gallup data is more ambiguous.[2] It starts later, so it cannot show what was happening in the 1990s. For four different questions about global warming, Gallup shows roughly flat support from 2001-2021, although there have been fluctuations in the level of support.

Figure 2: Percentage of Americans who agree with four different statements about global warming. Reprinted from Gallup (2021).

Public opinion is not quite the same thing as public attention. Maybe partisanship increases how much people are talking about an issue, even if it has little impact on the support as measured in polls. As a proxy for public attention, I will use Google Book’s Ngram, which shows how frequently phrases appear in a corpus of English language books published each year.

For many environmental issues, there was a peak in the early 1990s. Public attention was growing prior to the issue becoming partisan and declining once the issue had become partisan. This might be because the environmental movement itself shifted focus away from these issues. ‘Pollution’ shows a similar peak around 1990, but also a larger peak in the early 1970s. ‘Climate change’ shows a somewhat diff...
Calling all experts
by sleno — 2024-05-15T15:22:40.804Z

Hey everyone.

I'm in the process of building a hackernews-like website specifically oriented around discovering and discussing the state of the art in academic domains through public white papers; computer science, economics, physics, you get the idea.

If anyone is knowledgeable in any of these fields and familiar with the research landscape enough to help me curate a kind of 'front page' of white papers for the field, I'd very much appreciate it!

Also if anyone is interested in being a beta-tester and/or moderator please let me know.

Thanks!
Quantized vs. continuous nature of qualia
by Terence Coelho — 2024-05-15T12:52:07.633Z

This question is not very well-posed, but I've done my best to make it as well-posed as I can.

Suppose that humans with sufficiently functional brains are able have subjective experiences that transcend the "easy problems of consciousness".

I'm interested in understanding if this can be reasonably accepted without also concluding a theory of some sort of "panpsychism". For instance, taking a single conscious human and working backwards in time to conception: is it natural to believe a non-zero level of consciousness or subjective experience emerges at some time in this process, or is it more natural to believe the zygote has a level of consciousness?

From another perspective, start with the same human, and keep going up the ancestral tree (applying some system to pick a parent when there are multiple). If consciousness isn't quantized and keeps decreasing as we go back, and we further assume life emerged from inorganic processes, do we eventually arrive at some theory of panpsychism?

My intuition is that panpsychism seems false and I have genuine subjective experiences. To reconcile this, I think I would need consciousness to be quantized in some way. Is there a natural reason to believe consciousness is quantized? Is there any recommended reading on this?
How to be a messy thinker
by invertedpassion — 2024-05-15T11:57:31.215Z

Crossposted from my blog: https://invertedpassion.com/how-to-be-a-messy-thinker/

I love thinking about thinking. Give me a research paper on rationality, cognitive biases or mental models, and I’ll gobble it up. Given the amount of knowledge I’ve ingested on these topics, I had always assumed that I’m a clear thinker.

Recently, though, it hit me like a lightning strike that this belief is counter-productive. That’s because is you “know” that you’re a clear thinker, you’re less likely to suspect that you might be missing something big in your thought process. After all, if you are convinced that you think clearly by default, why would you put in any extra effort to scrutinize your thought process?

So, ironically, the belief that you’re a clear thinker creates a backdoor for cognitive biases to creep into your thinking because you’re less likely to suspect they might be there in the first place.

Once this realization dawned upon me, I resolved to lower the expectations from my thinking process. Instead of celebrating clever arguments and things making perfect sense, I now see clarity as signs of missing something.

The real world is often nuanced

and, hence, thinking about it should lead to messy details, not grand one-line aphorisms.

Clear thinking sets you up for unrealistic standards

As a clear thinker, you can make no mistakes. After all, you should have seen everything in advance, made perfect plans and thought of all contingencies. But that never happens – a moment of reflection is sufficient to convince anyone that such perfect foresight is impossible.

Clear thinking is perfectionism in disguise, and like all forms of perfectionism, it creates a persistent source of anxiety as you’re never good enough against your unrealistic standards.

So, abandoning the expectation of clear thinking is a huge relief because now you know you’ll always miss something. You are no longer in a state of persistent self-criticism because your standards are realistic.

Also, instead of beating yourself when inevitable surprises crop up, now you see them as a source of learning.

Abandon clear thinking, be a messy thinker instead

In nature, everything is continuous and interacts with one another. Unless you’re working with fundamental physics or mathematics, you shouldn’t expect simple, understandable models of reality that we crave so much.

A messy thinker acknowledges this incompatibility ...
Embedded Whistle Synth
by jefftk — 2024-05-15T02:50:02.284Z

A few years ago I ported my whistle synth system from my laptop

to a Raspberry Pi. This was a big
improvement, but I still wasn't that happy:

To get good quality audio in and out I was using a 2i2 audio
interface, which is expensive, bulky, and has a lot of buttons and
knobs that can be bumped.

To use a single mic for both whistle and talkbox I was using a
cheap
passive A/B switcher. Which feels fragile, causes pops when
phantom power is on, and is one more thing to plug in.

It's hard to get super low levels of latency with the Pi. It's
probably possible to get more performance out of my existing hardware
than I'm managing, but as it is I'm not happy with it.

The Pi's SD card gets corrupted every so often, so I
carry around a bunch of spares.

The Pi takes a little while to boot, which makes "no sound"
ambiguous between "because the SD card is corrupt or I've
plugged something in wrong" and "because it's still starting".

After my Electronic
Harp Non-Mandolin project I was feeling more at home with
electronics, and decided to build an embedded version of my whistle
synth.

It's built around a Teensy 4.0 with
the audio
shield. [1] This gives me stereo 44.1kHz in and out, though currently
I'm just using mono. It's plenty fast for the processing I'm doing.

I'm using a standard dynamic mic, which puts out a very weak signal.
Instead of connecting that directly to the line in on the Teensy, I
need a pre-amp. A normal pre-amp is fancy, but literally all I need
here is something that preserves zero crossings and gets amplitude
about right, so I made my own around an LM358P (github):

If you want any of these boards let me know; I have dozens of them
because the minimum order was five sheets and each sheet has six
boards.

I breadboarded
it to make sure it worked, and then mounted the electronics in a
project box:

This is an absurd number of lever nuts: once this isn't a prototype
anymore I'll take all of them off and solder the connections.

The box has XLR connections for input and output, because it
incorporates the A/B switching functionality. With the 3P3T switch on
the top I can send the XLR input to the whistle synth, or pass it
unmodified to the XLR output.

The whistle synth output is 1/4", for use with guitar pedals:

It gets power (and updated programming) over micro USB:

H...
Catastrophic Goodhart in RL with KL penalty
by Thomas Kwa — 2024-05-15T00:58:20.763Z

TLDR: In the last two posts, we showed that optimizing for a proxy can fail to increase true utility, but only when the error is heavy-tailed. We now show that this also happens in RLHF with a KL penalty.

This post builds on our earlier result with a more realistic setting and assumptions:

Rather than modeling optimization as conditioning on a minimum reward threshold, we study maximization of reward with a KL divergence penalty, as in RLHF.
We remove the assumption of independence between the error and utility distributions, which we think was the weakest part of the last post.
When the true utility V is light-tailed, the proxy can be maximized while keeping E[V]to the same level as the prior. We can't guarantee anything about E[V] when V is heavy tailed; it could even go to minus infinity.

Abstract

When applying KL regularization, the trained model is regularized towards some base policy π0. One would hope that a KL penalty can produce good outcomes even in the case of reward misspecification; that is, if the reward U is the sum of true utility V and an error term X, we would hope that optimal policies under a KL penalty achieve high V even if the magnitude of X is large. We show that this is not always the case: when X is heavy-tailed, there are arbitrarily well-performing policies π with Eπ[V]≈Eπ0[V]; that is, that get no higher true utility than the prior. However, when error is light-tailed and independent of V, the optimal policy under a KL penalty results in V>0, and V can be made arbitrarily large. Thus, the tails of the error distribution are crucial in determining how much utility will result from optimization towards an imperfect proxy.

Intuitive explanation of catastrophic Goodhart with a KL penalty

Recall that KL divergence between two distributions P and Q is defined as

DKL(P∥Q)=∑x∈XP(x)log(P(x)Q(x))

If we have two policies π,π0, we abuse notation to define DKL(π∥π0) as the KL divergence between the distributions of actions taken on the states in trajectories reached by π. That is, if Tr(π) is the distribution of trajectories taken by π, we penalize

DKL(π∥π0)≜Es∈T,T∼Tr(π)[DKL(π(s)∥π0(s))]

This strongly penalizes π taking actions the base policy never takes, but does not force the policy to take all actions the base policy takes.

If our reward mode...
Ilya Sutskever and Jan Leike resign from OpenAI [updated]
by Zach Stein-Perlman — 2024-05-15T00:45:02.436Z

Ilya Sutskever and Jan Leike have resigned. They led OpenAI's alignment work. Superalignment will now be led by John Schulman, it seems. Jakub Pachocki replaced Sutskever as Chief Scientist.

Reasons are unclear (as usual when safety people leave OpenAI).

The NYT piece (archive) and others I've seen don't really have details.

OpenAI announced Sutskever's departure in a blogpost.

Sutskever and Leike confirmed their departures in tweets.

Updates:

Friday May 17:

Superalignment dissolves.

Leike tweets, including:

I have been disagreeing with OpenAI leadership about the company's core priorities for quite some time, until we finally reached a breaking point.

I believe much more of our bandwidth should be spent getting ready for the next generations of models, on security, monitoring, preparedness, safety, adversarial robustness, (super)alignment, confidentiality, societal impact, and related topics.

These problems are quite hard to get right, and I am concerned we aren't on a trajectory to get there.

Over the past few months my team has been sailing against the wind. Sometimes we were struggling for compute and it was getting harder and harder to get this crucial research done.

Building smarter-than-human machines is an inherently dangerous endeavor. OpenAI is shouldering an enormous responsibility on behalf of all of humanity.

But over the past years, safety culture and processes have taken a backseat to shiny products.

Daniel Kokotajlo talks to Vox:

“I joined with substantial hope that OpenAI would rise to the occasion and behave more responsibly as they got closer to achieving AGI. It slowly became clear to many of us that this would not happen,” Kokotajlo told me. “I gradually lost trust in OpenAI leadership and their ability to responsibly handle AGI, so I quit.”

Kelsey Piper says:

I have seen the extremely restrictive off-boarding agreement that contains nondisclosure and non-disparagement provisions former OpenAI employees are subject to. It forbids them, for the rest of their lives, from criticizing their former employer. Even acknowledging that the NDA exists is a violation of it.

More.

TechCrunch says:

requests for . . . compute were often denied, blocking the [Superalignment] team from doing their work [according to someone on the team].

Piper is back:

OpenAI . . . says that going forward, they *won't* strip anyone of their equity for not signing the secret NDA.

(This is...
my note system
by bhauth — 2024-05-15T00:20:25.971Z
Link: https://www.bhauth.com/blog/thinking/note%20system.html

I've been told that my number of blog posts is impressive, but my personal notes are much larger than my blog, over a million words and with higher information density. Since I've had a bit of practice taking notes, I thought I'd describe the system I developed. It's more complex than some integrated solutions, but it's powerful, modular, free, and doesn't rely on any specific service.

is this necessary?

Most people don't take extensive notes and don't use git at all, so obviously a fancy note system isn't strictly necessary for most people. To some extent, you have to ask yourself: are your notes on some topic going to be more useful to you than a Wikipedia page or internet search or book? Do you need more records than your emails already provide? In my case, I think my blog posts show that I have some info that's hard to find, but I mostly just remember technical stuff instead of looking up notes, so many of them haven't been necessary.

But most people do take some notes, this is modular so you don't have to use every part, and the parts are useful for other things. I do think it's at least worth being aware of the kinds of tools described below.

my note format

My notes mainly go in text files in a folder.

When a text file gets large, replace it with a folder with multiple text files.

When pictures are needed, put the text file in a folder with pictures.

When a flowchart is needed, you can use yEd or the draw.io desktop app.

git

That notes folder goes in a git repository. If you install git, you can create that with:

cd [directory]

git init

git add .

git commit -m "initial commit"

If you're smart and don't know how to use git, you should learn how to use git.

viewing history

To look at the edit history of your notes, you'll probably want some GUI program, such as Github Desktop, GitKraken, or Gittyup.

markdown

If you want formatting or inline images, your text files can be Markdown. To view or edit the rendered Markdown, there are many options, such as:

VNote

Zettlr

MarkEdit (mac)

Typora (paid)

By the way, in Zettlr, if you want to remove the formatting bar when you highlight text, you can go to Appearance -> Custom CSS and add:

body .main-editor-wrapper .cm-editor .cm-tooltip.cm-formatting-bar { display: none; }

Markdown files can contain relative links to other notes, including to folders, and Markdown editors can follow those links. This is useful.

...