trentmkelly/LessWrong-43k huggingface dataset:
Referendum Mechanics in a Marketplace of Ideas
by Martin Sustrik — 2024-08-25T08:30:01.901Z

This is a cross post from https://250bpm.substack.com/p/referendum-mechanics-in-a-marketplace.

In a referendum, people vote either for or against the proposal and that's it. Right? Wrong!

One can definitely make referendums more complex by mixing in unrelated stuff, unnecessary restrictions, and complexities. But that's not the case with Swiss referendums. Quite the contrary, they seem to be designed for simplicity:

First: There is no quorum. Even if the turnout is low, the referendum is valid. If three people cast their votes in a national ballot and two of them vote yes, the proposal is accepted. It is then written into the constitution, so even the parliament cannot overrule it. No quorum means no strategic voting or, more precisely, no strategic withholding of votes to sabotage the referendum by making it invalid due to not meeting the quorum, as often happens elsewhere.

Second: The referendum process takes many years, often five or more, so it can't be used for short-term, tribal politics, like calling for premature elections. By the time the referendum reaches voters, five or six years after its initiation, other people are already in power, and the original reason for the referendum has long since become irrelevant. Hot-button issues of yesteryear are already blissfully forgotten.

Third: We know that defaults matter. If the referendum question is worded differently — when yes and no votes switch their meanings — it could lead to a different voting outcome. But in Switzerland, referendum questions are always worded the same way: It's a proposal to change the constitution. "Yes" always means a vote for change, while "No" always means to keep the status quo. And if the history of past referendums teaches us anything, it is that the default option is always "No." Only a few referendums in their 150-year history have been successful. If people don't understand or don't care about the proposal, they vote to keep the status quo by default.

Given all the above, why am I suggesting that the referendums can get complex? Read on and discover the fascinating world of referendum politics!

Imagine that the initiators of the referendum demand lowering a tax by 4%. (And yes, in Switzerland, any changes to taxes must be approved in a referendum.)

The government doesn't want to change the tax. If they did, they would have already put it on the ballot. So, the government commis...
Please stop using mediocre AI art in your posts
by Raemon — 2024-08-25T00:13:52.890Z

Epistemic Status: Old Man Blogs at Cloud

Lately there's been a wave of people on LessWrong (and maybe the whole internet) starting off their essays with some Dall-E3 art.

I don't object (and think it can be cool if done nicely) to get some good ML art to add some visual interest to your posts. But, the default thing I'm seeing is mediocre and making the web feel tacky.

I have an art background, which means I both have a lot of experience making/evaluating art and also probably have random-ass snooty connoisseurship syndrome. Also I was involved with the fleshing out of the current LessWrong Watercolor Aesthetic, and random clip-art looking images undercut it. (But, to be fair nobody actually voted for or opted into the LessWrong Watercolor Aesthetic, and maybe that's just my problem).

I think not all posts need art. But, if you do want art for your post, here's some hopefully slightly constructive advice/requests.

1. Generally, make landscape (widescreen) art.

Most image models output square images by default. This actually fits fairly awkwardly into blogposts – it either takes up a huge amount of vertical space, or you shrink it to fit and then it has weird padding. (The motivating instance for this blogpost was this post, which IMO would be greatly improved if the image was designed to take up more horizontal space).

Sometimes a post makes good use of a very tall art, to set some kind of mood, but it works better for more contemplative posts. (See On Green for an example).

2. Good AI art still takes a fair number of generations and effort.

I'm not sure how many generations people typically do that outputs that mediocre art, but I do want to note, for reference, when I'm making a quick piece of AI art for something (like a facebook event) it still usually involves at least like 5 generations (in Midjourney, where each generation includes 4 options), and often more like 20. And when I'm trying to make actually good art for something I want people to really appreciate (such as the Carving of Reality books), it might be hundreds of generations.

This was 15 generations in DallE3, plus several generations in Midjourney,
and then some post-processing in photoshop.

3. Think about the mood you want to convey, not just the posts' intellectual content.

Good art sets a tone and helps shift someone into a particular headspace. This doesn't just include the content ...
AXRP Episode 35 - Peter Hase on LLM Beliefs and Easy-to-Hard Generalization
by DanielFilan — 2024-08-24T22:30:02.039Z

YouTube link

How do we figure out what large language models believe? In fact, do they even have beliefs? Do those beliefs have locations, and if so, can we edit those locations to change the beliefs? Also, how are we going to get AI to perform tasks so hard that we can’t figure out if they succeeded at them? In this episode, I chat with Peter Hase about his research into these questions.

Topics we discuss:

NLP and interpretability

Interpretability lessons

Belief interpretability

Localizing and editing models’ beliefs

Beliefs beyond language models

Easy-to-hard generalization

What do easy-to-hard results tell us?

Easy-to-hard vs weak-to-strong

Different notions of hardness

Easy-to-hard vs weak-to-strong, round 2

Following Peter’s work

Daniel Filan (00:00:08):
Hello, everybody. This episode, I’ll be speaking with Peter Hase. Peter is an AI researcher who just finished his PhD at UNC Chapel Hill, where he specialized in natural language processing and interpretability research with a special interest in applications to AI safety. For links to what we’re discussing, you can check the description of the episode, and a transcript is available at AXRP.net. All right. Peter, welcome to AXRP.

Peter Hase (00:00:33):
Thanks so much, Daniel. I’m excited to be here.

NLP and interpretability

Daniel Filan (00:00:35):
I’m excited to have you on. So my understanding is that most of your work is in interpretability, roughly interpretability in language models. Is that fair to say?

Peter Hase (00:00:46):
Yeah. That’s right. I’ve been in an NLP [natural language processing] lab for my PhD, so we work mostly with language models, but a lot of it, in terms of methods, evals, has been focused on interpretability.

Daniel Filan (00:00:55):
Actually, maybe one thing I want to ask is: I have the impression that you were into language models even before they were cool, doing NLP before it was cool. Right? So just today, I looked at your Google Scholar and scrolled down to see the oldest paper you were a co-author on. It’s this 2018 paper on algorithmic sonnet generation. 2018, before the rest of us had caught up. What got you interested in NLP?

Peter Hase (00:01:23):
I remember the project you were talking about. That was when I was an undergrad. I feel so lucky to have had the opportunity to do a little special projects class, actually with Cynthia Rudin at Duke...
The top 30 books to expand the capabilities of AI: a biased reading list
by  — 2024-08-24T21:48:15.506Z

Large language models (LLMs) such as ChatGPT are surprisingly good, but they can’t expand human knowledge because they only interpolate existing information already on the internet. To create new knowledge and solve problems that no human has ever solved, AI needs to be adaptively embodied and be able to build causal models of the world. Embodiment entails that the AI has purpose, which enables it to be more flexible than stimulus-response agents, such as LLMs, by searching for ways to achieve its goals. Adaptive embodiment means that the AI can build meaning by learning representations and modifying how it maps sensory input to those representations. This learning and mapping gives the AI additional sensitivity to the environment. The final piece of embodiment is that the AI requires an executive system—a kind of proto-consciousness—that enables self-correction at various levels of operation. In addition to adaptive embodiment, the AI must be able to build causal models that enable it to know what to change to achieve its goals and to predict the future by simulating internal states forward.

I’ve read dozens of books since I started working in AI in 2002, and these are the ones that stand out and influenced my thinking. This list focuses on modern books, where I define “modern” as beginning in 1980, after the period of disembodied logic and search, when researchers realized that artificial intelligence needs to be embodied to achieve purpose and meaning. These 30 books presented in chronological order over the last 44 years each gets at a piece of the puzzle for what it will take to move beyond LLMs to expand the capabilities of AI.

Metaphors We Live By (1980) by George Lakoff and Mark Johnson

Modeling human thought through formal methods such as first-order logic seems like a natural choice, but these systems always fall short of capturing the richness of human cognition. Lakoff and Johnson argue for the necessity of less objective representations, proposing that much of human thought relies on understanding abstract concepts through metaphors rooted in lived experience. This metaphorical thinking enables us to grasp abstract concepts like “love” by relating them to more familiar ideas, such as a journey or a war (depending on your relationship). As another example, consider what an “argument” is. An argument can be understood as a co...
The Ap Distribution
by criticalpoints — 2024-08-24T21:45:35.029Z
Link: https://eregis.github.io/blog/2024/07/23/Ap-distribution.html

E.T. Jaynes introduces this really interesting concept called the Apdistribution in his book Probability Theory. Despite the book enjoying a cult following, the Ap distribution has failed to become widely-known among aspiring probability theorists. After finishing the relevant chapter in the book, I googled the phrase "Ap distribution" in an attempt to learn more, but I didn't get many search results. So here's an attempt to explain it for a wider audience.

The Ap distribution is a way to give a probability about a probability. It concerns a very special set of propositions {Ap} that serve as basis functions for our belief state about a proposition A—somewhat similarly to how the Dirac delta functions can serve as basis elements for measures over the real line. The proposition Ap says that "regardless of whatever additional evidence that I've observed, the probability that I will assign to proposition A is p". It's defined by the rule:

P(A|ApE)=p

This is a fairly strange proposition. But it's useful as it allows us to encode a belief about a belief. Here's why it's a useful conceptual tool:

If we say that there is a 1/2 probability of some proposition A being true, that can represent very different epistemic states. We can be more or less certain that the probability is "really" 1/2.

For example, imagine that you know for sure that a coin is fair: that in the long run, the number of heads flipped will equal the number of tails flipped. Let A be the proposition "the next time I flip this coin, it will come up heads." Then your best guess for A is 1/2. And importantly, no matter what sequence of coin flips you observe, you will always guess that there is a 50% chance that the next coin flip lands on heads.

But imagine a different scenario where you haven't seen a coin get flipped yet, but you are told that it's perfectly biased towards either heads or tails. Because of the principle of indifference, we would assign a probability of 1/2 to the next coin flip landing on heads—but it's a very different 1/2 from the case when we had a fair coin. Just one single flip of the coin is enough to collapse the probability of A to either p=0 or p=1 for all time.

There is a useful law in probability called the law of ...
What is it to solve the alignment problem? (Notes)
by Joe Carlsmith — 2024-08-24T21:19:34.280Z

(I originally wrote this post as some rough notes on defining the alignment problem, with the intention of turning them into something more polished later. I've now started doing that, as part of a broader series introduced here. In particular, the first post in that series covers some of the same ground as section 1 of this post. It also has the same title. And some of essays in the series will draw on these notes as well.)

People often talk about “solving the alignment problem.” But what is it to do such a thing? I wanted to clarify my thinking about this topic, so I wrote up some notes.

In brief, I’ll say that you’ve solved the alignment problem if you’ve:

avoided a bad form of AI takeover,
built the dangerous kind of superintelligent AI agents,
gained access to the main benefits of superintelligence, and

become able to elicit some significant portion of those benefits from some of the superintelligent AI agents at stake in (2).[1]

The post also discusses what it would take to do this. In particular:

I discuss various options for avoiding bad takeover, notably:

Avoiding what I call “vulnerability to alignment” conditions;
Ensuring that AIs don’t try to take over;
Preventing such attempts from succeeding;
Trying to ensure that AI takeover is somehow OK. (The alignment discourse has been surprisingly interested in this one; but I think it should be viewed as an extreme last resort.)

I discuss different things people can mean by the term “corrigibility”; I suggest that the best definition is something like “does not resist shut-down/values-modification”; and I suggest that we can basically just think about incentives for/against corrigibility in the same way we think about incentives for/against other types of problematic power-seeking, like actively seeking to gain resources.

I also don’t think you need corrigibility to avoid takeover; and I think avoiding takeover should be our focus.

I discuss the additional role of eliciting desired forms of task-performance, even once you’ve succeeded at avoiding takeover, and I modify the incentives framework I offered in a previous post to reflect the need for the AI to view desired task-performance as the best non-takeover option.
I examine the role of different types of “verification” in avoiding takeover and eliciting desired task-performance. In particular:

I distinguish between what I call “output-focused” v

...
Examine self modification as an intuition provider for the concept of consciousness 
by Canaletto — 2024-08-24T20:48:55.189Z

(epistemic status: this direction of thought is pretty strongly endorsed by me, but not very thoroughly expressed/developed, I'm working on it)

This post is kind of a mess, and made out of pieces of discussions I had in many places, but I decided to post it anyway, otherwise its editing stated to stretch out indefinitely.

Main motives of my approach

Start from normalcy and gradually poke it and look at at from weird angles, as opposed to doing a bold overstretching stab at the core of the problem, as many attempts at proposing theory of consciousness have done.
Consider things in terms of anticipated experiences. It's usually the most (empirically, as far I observed) fruitful approach for disentangling tricky philosophical stuff. Like, concretely and visibly productive.
I dislike the entire approach of "terms first, situations second". I absolutely love "situations first, terms second". Give me some thought experiments using arbitrary terms to convey them as situations accurately. I crave examples.
First person perspective is really important here. I should give some thought to how to communicate with people usefully, how to translate back and forth from communicating models and communicating direct observations, and how to make it uhhh sane for me and all other people involved.
Ask "How did you come to think of it this way?" obsessively. Like, jump straight to meta every time you have no object level feedback. How did you come to the conclusion that the word "consciousness" means what it means to you? How have other people come to their conclusions? In what alternative circumstances would they have come to the same or a different ontology?

(Expected) Self modification as intuition provider

I think in near future we will acquire very good instruments for observing processes in the brain and modifying them with great precision. I strongly expect it, and I decided to try to emulate what "intuitive" updates would I acquire from living in such a world.

And you can do that already, you can modify your experiences by modifying your physical body. I think it seems very uncontroversial? You can drink a cup of coffee that modifies your physical brain and feel it "from the inside", in the realm of feelings and "qualia".

I expect that in retrospect it would look very obvious, and hindsight bias will shine. "oh, it was obvious all along how I'm this piece of matter, I ...
Looking to interview AI Safety researchers for a book
by jeffreycaruso — 2024-08-24T19:57:33.119Z

If you are or have been employed as an AI Safety Researcher, I'd love to interview you as background for a novel that I'm writing. I'm interested in learning what a day in the life of an AI researcher typically looks like.

I'd be happy to include you in the acknowledgments or keep your participation completely confidential, whichever you prefer.

If you'd like to email me rather than respond in the comments, my email is jeff [AT] securityink [DOT] org.

Grazie!
Perplexity wins my AI race
by Elizabeth — 2024-08-24T19:20:10.859Z
Link: https://acesounderglass.com/2024/08/22/perplexity-wins-my-ai-race/

Perplexity is the first generalized AI chatbot I’ve found useful enough to integrate into any part of my daily workflow, much less across multiple domains. It speeds me up enough that I’m planning an increase in my freelancing rate.

Perplexity has three key advantages:

It provides citations, cleanly, in context

It has the persona of a sharp human instead of an intolerable customer service agent.

It is useful (and sufferable) across a wide variety of domains.

The citations are a bigger deal than they sound at first. I could of course google anything ChatGPT tells me and check the results, but that’s costly. I have to spin up a new mental thread (and browser tab), sort through Google’s increasingly shitty results, remember what claim I’m trying to check… the process eats up a lot of working memory at a time when it is scarce. If I wanted to trawl a sea of content farms I would have gone to Google in the first place.

Perplexity’s high quality inline citations solve that . Want to check if a claim is justified? Click the number next to it and look for the relevant quote. Not a perfect substitute for a 30 hour literature search, but a lot of power per minute. I don’t have to spin up a new mental thread, so the whole thing feels much lower friction. Also thus far I’ve never caught it in a hallucination on a question of fact, although if real answers are scarce it will sometimes do the thing where it draws increasingly tenuous connections instead of just telling me no.

I also found perplexity easier to talk to while trying to learn some math and math history. It felt about as useful as my human tutor- better at some things, worse at others.

Test Questions

The real test was which AI do I find myself using, and the answer to that is definitively Perplexity. But just for fun or in case I was wrong, I ran a few questions Perplexity through all major engines to see what they said.

In all cases I used the top free model available. I have Perplexity Pro thanks to a promo, but I haven’t ended up finding Pro to be much of an improvement. I do expect to use the pro-only PDF-analysis tool in the near future.

Comments from me are in italics and [brackets]

How do fractals relate to chaos theory?

I have a project investigating chaos theory as a case study in the development of scientific/mathematical paradigms....
Why should anyone boot *you* up?
by onur — 2024-08-24T17:51:21.608Z
Link: https://solmaz.io/why-should-anyone-boot-you-up

Imagine the following scenario:

We develop brain-scan technology today which can take a perfect snapshot of anyone's brain, down to the atomic level. You undergo this procedure after you die and your brain scan is kept in some fault-tolerant storage, along the lines of GitHub Arctic Code Vault.

But sufficiently cheap real-time brain emulation technology takes considerably longer to develop—say 1000 years in the future.

1000 years pass. Everyone that ever knew, loved or cared about you die.

Here is the crucial question:

Given that running a brain scan still costs money in 1000 years, why should anyone bring *you* back from the dead? Why should anyone boot *you* up?

Compute doesn't grow in trees. It might become very efficient, but it will never have zero cost, under physical laws.

In the 31st century, the economy, society, language, science and technology will all look different. Most likely, you will not only NOT be able to compete with your contemporaries due to lack of skill and knowledge, you will NOT even be able to speak their language. You will need to take a language course first, before you can start learning useful skills. And that assumes some future benefactor is willing to pay to keep you running before you can start making money, survive independently in the future society.

To give an example, I am a software developer who takes pride in his craft. But a lot of the skills I have today will most likely be obsolete by the 31st century. Try to imagine what an 11th century stonemason would need to learn to be able to survive in today's society.

1000 years into the future, you could be as helpless as a child. You could need somebody to adopt you, send you to school, and teach you how to live in the future. You---mentally an adult---could once again need a parent, a teacher.

(This is analogous to cryogenics or time-capsule sci-fi tropes. The further in the future you are unfrozen, the more irrelevant you become and the more help you will need to adapt.)

Patchy competence?

On the other hand, it would be a pity if a civilization which can emulate brain scans is unable to imbue them with relevant knowledge and skills, unable to update them.

For one second, let's assume that they could. Let's assume that they could inject your scan with 1000 years of knowledge, skills, language, ontology, history, culture and so on.

But then, would it still be you?

But then, why...