trentmkelly/LessWrong-43k huggingface dataset:
"What if we could redesign society from scratch? The promise of charter cities."  [Rational Animations video]
by Jackson Wagner — 2024-02-18T00:57:50.444Z
Link: https://www.youtube.com/watch?v=v-A1i2g9riU

In this video, we describe a speculative but promising idea to boost economic growth: giving cities the ability to write some of their own laws and design their own institutions, independently from their parent countries. Jackson Wagner, the first author of this post, has written the script. Writer, the second author, has given feedback.

Below, you can find the script of the video.

Introduction

When you think about ways to help people in developing countries, you probably think of international aid -- providing medical supplies, or coordinating disaster relief, or even just giving cash, like we discussed in our recent video about global poverty. These things are great, and this kind of charitable aid saves lives every day. But there’s something a little curious here, because these interventions AREN’T what helped the world’s most prosperous countries succeed in the first place.

Up until the 1700s, essentially the entire population of the planet lived in poverty. That started to change during the industrial revolution, when humanity developed more efficient, mechanized ways of producing goods and capturing energy. Hundreds of millions of people were able to live better lives thanks to economic growth -- learning to use technology to more efficiently grow food, make clothes, and get from place to place.

The overwhelming importance of economic development remains true in modern times -- by far the greatest ongoing reductions in poverty and suffering are coming not from international aid projects, but from development, as low-income countries find new ways to do things more efficiently and climb the ladder of technological advancement.

For example, in the year 1960, Singapore was a poor and undeveloped country, producing only $428 per citizen. Today, Singapore’s economy has grown by many times, to around $83,000 per person. Concurrently, Singaporeans’ life expectancy has climbed from 65 years to 85 years, and the literacy rate has jumped from 57% to 97%. That’s an incredible amount of progress within a single lifetime.

Almost every country on earth has made some economic progress over the past few decades, but unfortunately only a few have succeeded as spectacularly as economic-development superstars like Singapore, or South Korea. There are hundreds of millions of people still living on just a few doll...
Evaluating Solar
by jefftk — 2024-02-17T21:50:04.783Z

In 2018,

we
installed 14 solar panels on the Northwest roof of our house.
This is a little silly: wouldn't you want to install them facing the
sun? The problem was the enormous tree to our Southeast, twice the
height of our house, completely shading that area. It was a great
tree, but it was causing problems for our neighbor, so they took it
down a couple years ago. I looked at getting solar at the time and
got as far as signing a contract, but then it fell apart a week before
install when the company said they couldn't actually install the

sunlight
backup system we had planned.

Since then, electricity has gotten more expensive while panels have
gotten more efficient and cheaper, and I started thinking more after
seeing how heat
pumps don't work out with our electricity rates. Here's what
we're considering:

This is 22 QCell
425W panels initially producing 9,292 kWh/year. That would give
about ~$3.2k/y in electricity savings + credits. I got quotes from
three companies, and the best (after tax credits) was around $20k.

This is pretty good ROI, with payback in 5-6 years. But what I really
care about is how this compares to what else I would do with the
money. This isn't money we need short term, so I think the
opportunity cost is whatever the stock market would do with $20k. But
these are very different sorts of things: with stocks you put money in
and at some point in the future you take an, ideally larger, quantity
out. With solar you put money in and then each year for the
life of the system you pay less in electricity. I don't know the right
way to compare them in an open-ended way, but we can analyze the
decision over a specific number of years.

I'm going to model two scenarios. In the first we take $20k from
stocks to put in solar, and then invest returns back into stocks. In
the second, we leave the $20k in stocks. Results here also depend on
what happens to stocks and electricity rates. I'm going to keep
everything in 2024 dollars, and guess a real (inflation-adjusted)
price increases of 7% for stocks and 3% for electricity [1]. It got a
bit too complex for a spreadsheet, so I wrote a script [2]. It
looks like it takes a bit over seven years before solar comes out
ahead:

I'm pretty sure this is the right way to do the analysis, but it's not
obvious: the version solar installers give you ignores opportunity
...
Opinions survey 2 (with rationalism score at the end)
by tailcalled — 2024-02-17T12:03:02.410Z
Link: https://docs.google.com/forms/d/e/1FAIpQLSclPFQb2xUddV0opa8eY7Z1SA-yRCP0jVesUhr-TXt5_c8ehw/viewform

I've in light of the previous opinions survey, I've added some nuances to my model, which yields a new opinions survey: Opinions survey 2.

I'm mainly posting it here because I'm curious how well my factor model extrapolates. I want to have this data available when I do a more in-depth analysis of the results from the census. I don't necessarily think the rationalism scores will do great at characterizing rationalists, but they might.

I scored 12/18. If I add this to my score from my previous survey, I get 26/42.
Achieving AI Alignment through Deliberate Uncertainty in Multiagent Systems
by Florian_Dietz — 2024-02-17T08:45:37.538Z

An AI is different from a human in multiple ways. This is part of what makes AI alignment such a difficult problem, because our intuitions for how people act often do not apply to AI's. However, several of these differences between AI and humans actually work in our favor, and are essential for an approach I want to outline:

An AI can be copied. It is trivially easy to create an exactly identical copy of an AI.
An AI can be rolled back to an earlier state. It is easily possible to test what an AI will do in any given situation, then reset it to wipe its memory of the test.
It is possible to compose an AI out of a system of many independent AI agents. Through game theory, it is possible to incentivize independent AI agents to take on adversarial or supervisory roles towards other AI agents. This results in an evolutionary system. Not in the sense of genetic programming, since there is no need for an explicit algorithm for modifying the AI agents, but in the sense that only AI agents that follow the rules layed out for them get copied and thereby reproduce.
It is possible to inspect and modify every aspect of an AI while it is running. At least in theory it is possible to understand every aspect of an AI's thought process, and there is no way for the AI to hide anything about itself. This does not mean that it is easy to understand what an AI is thinking in practice. It only means that we can read an AI's mental state in theory, not that it will be easy. In particular, it is possible to train one AI to take on a supervisory role over another AI. The supervisor can read and alter everything about the supervised AI, and can be trained to understand how to enforce desired behaviors on the supervised AI.
It is possible for a supervising AI to make a supervised AI think anything it wants. The inspected AI may be tricked into believing that unusual situations have occurred by directly modifying its mental state at runtime.
It is not possible for an AI to tell the difference between a game and reality with certainty. If all you have known so far is Tic Tac Toe and someone shows you a complex videogame like Skyrim, you will be overwhelmed by the increase in complexity. If afterwards you get access to the real world for the first time, the increase in complexity from Skyrim to Real Life may actually be smaller than that from Tic Tac Toe to Skyrim. This means that it would se

...
Communication, consciousness, and belief strength measures
by Jakub Smékal — 2024-02-17T05:45:42.834Z

Communication is a fascinating subject. It’s a way of transferring information from one place to the next (or, to invert a quote by Chris Fields, communication is physics). Now you may have noticed that I used the word “communication” in two slightly different ways. In the first sentence you might have thought I’ll be talking about the way people communicate between each other, and in the next one I seemed to have abstracted away this base definition and applied it to a more abstract concept, physics (abstract in a very limited sense though). The funny thing is, even though I’m using up way more space to explain the first two sentences I wrote, there is still ambiguity in what I’m saying, there’s still information that’s being lost and transformed from me thinking it up, to writing it in particular words, to you reading them and interpreting them in your unique way. While the physics side of communication is certainly fascinating, I’ll just refer to this course for now and consider communication between people, and just ponder for a bit about the indirect costs of the loss of information in our daily communication (that kind of rhymes doesn’t it?). TL;DR*: I ended up talking about “consciousness”.*

There are a lot of angles to approach this topic from, but I’ll focus on one that seems more and more prevalent in both academic literature and daily conversation, namely the use of the word “consciousness” (and we’ll try to vaguely generalize from there). Consciousness comes up more and more as every new AI model seems to do more of the things we thought biological systems were unique at doing (#isSORAconscious?). What I always find thought-provoking is that a lot of people quite easily use the word “consciousness” in their work, and not only in random posts online, but also in academic papers, and I’m always left wondering: what do they mean by “consciousness”? I’m pretty sure they don’t mean the dictionary definition that pops up with a Google search: "the state of being awake and aware of one's surroundings”, because it often seems to be related to not only awareness of one’s surroundings, but also an awareness of the self, a quality that we perceive ourselves to perceive and that (some of us) assume other biological beings to also exhibit (e.g. looking in the mirror test with dolphins). But even this rather vague definition doesn’t seem to fully capture what a...
Self-Awareness: Taxonomy and eval suite proposal
by Daniel Kokotajlo — 2024-02-17T01:47:01.802Z

[This is a mildly-edited version of a google doc I wrote at OpenAI in July 2022. I had intended to get it published in some form, but never got around to it for various reasons. I have now received approval to put it up as a blog post. The main thing of interest here is the distinctions I make; particularly the concept of self-location. Also the examples in the appendix that illustrate the distinctions. I lump all three concepts (self-knowledge, self-location, introspection) together under the banner of Self-Awareness, but since that's a spicy term which may have other connotations, these days I'd probably use the more neutral term Situational Awareness.]

Summary

All three kinds of self-awareness come in degrees, but can be measured with appropriately designed evals/benchmarks. This doc explains how.
Self-knowledge: How much does the model know about [model_name]?
Introspection: Does the model know some things about [model_name] “directly,” or is its knowledge entirely inferred from training data, observations, etc.?
Self-location: When the model learns facts about what “[model_name]” is about to experience or should do to achieve its goals/reward/etc., does the model then make those predictions and take those actions? Or does it merely use that new knowledge to answer questions about what [model_name] should predict or do – as if it didn’t know “[model_name] is me!”
This doc also explains why this matters–why these three kinds of self-awareness are important and dangerous capabilities for powerful models to have. They also plausibly matter for the moral status/patienthood/personhood of the models.

Outline:

Self-knowledge

What it means
How to test for it

Introspection

What it means
How to test for it

Self-Location

What it means
How to test for it

Importance

Self-awareness → consciousness → moral patienthood
Self-awareness → Strategic awareness & Agency → APS-AI
Self-awareness → Situational awareness → Alignment failures

Recommendations
Appendix

Illustrative examples of hypothetical systems that have some kinds of self-awareness but not others.

Self Knowledge

What it means

Self-knowledge is knowledge about oneself. The model has self-knowledge to the extent that it knows relevant facts about [model_name], understands the circumstances of [model_name], etc. For example, does it know that [model_name] is an AI rather than a human? Does it know that [mo...
Opinions survey (with rationalism score at the end)
by tailcalled — 2024-02-17T00:41:20.188Z
Link: https://docs.google.com/forms/d/e/1FAIpQLSdSKvHo-6HyZqHprCDoBD-VjKxF-Rhp2qNhHV8d0SY40JJdjA/viewform

Based on the results from the recent LW census, I quickly threw together a test that measures how much of a rationalist you are.

I'm mainly posting it here because I'm curious how well my factor model extrapolates. I want to have this data available when I do a more in-depth analysis of the results from the census.

I scored 14/24.
Phallocentricity in GPT-J's bizarre stratified ontology
by mwatkins — 2024-02-17T00:16:15.649Z

TL;DR This relates to the findings reported in my posts Mapping the Semantic Void parts I and II. By creating a custom embedding at the token centroid (the mean vector of all 50,257 GPT-J token embeddings), prompting the model to define it and considering logits, it's possible to construct a "definition tree" which consists overwhelmingly of vague generalities. This is hardly surprising, as GPT-J is basically being challenged to define "the average thing". However, the most probable branch in the tree which gives a definition containing anything specific defines the "ghost token" at the centroid as "a man's penis". Lowering the cumulative probability cutoff to produce ever longer lists of possible definitions, we find that almost all the branches which provide definitions involving anything specific are of a sexual/procreative nature, with a few relating to status thrown in. As usual, I have no idea what any of this means, but I welcome suggestions!

Work supported by the Long Term Future Fund.

Defining the centroid

Using a custom embedding method, I applied the prompt

A typical definition of <token> would be '

(which I have been using as part of my "Mapping the Semantic Void" research) to the GPT-J token embedding centroid, which could perhaps be thought of as representing some kind of "mean concept" at the centre of the stratified ontology I've been exploring in recent posts.

The temperature 0 output is "A person who is not a member of a group", something we see for almost all points within distance 0.5 of the centroid (token embeddings typically live at distance 0.9–1.1 from the centroid, in what could be imagined as a 'fuzzy hyperspherical shell').

By considering top-5 logits, iteratively prompting and recording cumulative probabilities, it's possible to build a weighted "definition tree" for any token or non-token embedding.

definition tree for the ' broccoli' token
the root node corresponds to the prompt A typical definition of ' broccoli' would be '
branch thicknesses are scaled logarithmically so that differences are discernable

On initial inspection, the definition tree for the centroid was more or less what I would have expected: no mention of anything specific. It's like the model is trying to define "the average thing", and so it's "vaguing out" or else "playing for time", avoiding committing to anything specific for as many tok...
FUTARCHY NOW BABY
by sapphire — 2024-02-17T00:03:18.962Z

There is insane shit going on and I dont have time for an explainer but https://themetadao.org is POPPING OFF. If you like FUTURACHY join the discord or get involved.

TLDR is someone implemented hansionian futarchy. There is a 'selfish propossal' being decided over the next thirty hours. If you love this shit GET IN THERE BABY.
Making the "stance" explicit
by NicholasKees — 2024-02-16T23:57:11.265Z

Often, when people explain the behavior of an AI system, there is one of Daniel Dennett’s stances implicit in their explanation. I think that when this happens implicitly, it can often lead to misleading claims and sloppy reasoning. For this reason, I feel it can be really valuable to be deliberately explicit which “stance” one is using. Here are

Dennett’s three stances

(in my own words):

Physical/structural stance: Explaining a system’s behavior mechanically, as the interaction between its parts and with the outside environment. For example, we could explain the output of a neural net by looking at which combinations of neurons fired to bring that about.

Design/functional stance: Explaining a system by looking at what its function is, or what it was made to do. For example, we could explain the behavior of a thermometer by the fact that its function is to track the temperature.

Intentional stance: Explaining a system’s behavior as a product of beliefs, desires, or mental reasoning of the system. For example, my friend got up to get a cup of coffee because they believed there was coffee in the other room, wanted to have the coffee, and reasoned that to get the coffee they had to get up.

The idea is that all three stances can in principle be applied to any system, but that for different systems each stance offers a different amount of explanatory power. For example, if I tried to understand my friend in terms of their design (perhaps the evolutionary pressures placed upon humans, or the in-lifetime learning applied to my friend’s brain) this would get me a lot less bang for my buck than my much simpler model of them as wanting coffee. Likewise, thinking of a thermometer as “wanting” to track the temperature and “believing” that the temperature outside does not match what it is displaying is unnecessarily complicated.

Statements can sometimes make implicit assertions about which stance is the most useful. For example, consider: “GPT is just a text predictor.” This statement correctly identifies that from a design stance, GPT was made to predict next-tokens on a text corpus[1], but it goes a step further to implicitly claim that we should only understand GPT in this way (and that we shouldn’t be curious about explaining the behavior in other ways). I think it can be important to notice this kind of thing, and try to make these implicit cl...
2023 Survey Results
by Screwtape — 2024-02-16T22:24:28.132Z

The Data

0. Population

There were 558 responses over 32 days. The spacing and timing of the responses had hills and valleys because of an experiment I was performing where I'd get the survey advertised in a different place, then watch how many new responses happened in the day or two after that.

Previous surveys have been run over the last decade or so.

2009: 166
2011: 1090
2012: 1195
2013: 1636
2014: 1503
2016: 3083
2017: "About 300"
2020: 61
2022: 186
2023: 558

Last year when I got a hundred and eighty six responses, I said that the cheerfully optimistic interpretation was "cool! I got about as many as Scott did on his first try!" This time I got around half of what Scott did on his second try. A thousand responses feels pretty firmly achievable.

This is also the tenth such survey that’s been run. We missed a proper ten year anniversary in 2019, and in 2022 I was mostly focused on making the survey happen at all. Still, this is a cool milestone, and in celebration I’m going to be dipping into the datasets from previous years a lot. Unfortunately that doesn’t mean I have ten surveys worth of data; bit rot and the rotation of census runners means I only have access to about half of these.

I’ll talk about other surveys more later on. For the moment, let's talk about the basic breakdowns.

There’s two main formats I’m going to present information in.

One of them is where I'm not treating the answers as numbers. Here, I present

The option selected
How many people selected that option
The percentage of people who selected that option

Let's use Relationship Status as an example.

Relationship Status:
Single: 263, 48.3%
Relationship: 170, 31.3%
Married: 111, 20.4%

263 people said they were single. That's 48.3% of the answers.

The other format is where I have the mean and standard deviation. If you see a sequence of numbers like "30.1 + 8.9 (24, 28, 34) [n=186]" those numbers are "Mean + standard deviation (1st quartile, 2nd quartile, 3rd quartile) [n= number responding]." For an example, let's use Age.

Age: 30.5 + 9.2 (24, 29, 36) [n=552]

The mean is 30.5, the standard deviation is 9.2, the first quartile is 24, the second quartile (AKA the median) is 29, the third quartile is 34, and 552 people answered the question. If you're not familiar with statistics, the first number is basically an average and the last number is how many people answered the ...
Physics-based early warning signal shows that AMOC is on tipping course
by Annapurna — 2024-02-16T22:07:30.620Z
Link: https://www.science.org/doi/10.1126/sciadv.adk1189

Abstract

One of the most prominent climate tipping elements is the Atlantic meridional overturning circulation (AMOC), which can potentially collapse because of the input of fresh water in the North Atlantic. Although AMOC collapses have been induced in complex global climate models by strong freshwater forcing, the processes of an AMOC tipping event have so far not been investigated. Here, we show results of the first tipping event in the Community Earth System Model, including the large climate impacts of the collapse. Using these results, we develop a physics-based and observable early warning signal of AMOC tipping: the minimum of the AMOC-induced freshwater transport at the southern boundary of the Atlantic. Reanalysis products indicate that the present-day AMOC is on route to tipping. The early warning signal is a useful alternative to classical statistical ones, which, when applied to our simulated tipping event, turn out to be sensitive to the analyzed time interval before tipping.
Kingfisher Winter Tour 2024
by jefftk — 2024-02-16T21:40:15.784Z

Cecilia and I are going to be driving up and down the east coast
during winter break playing dances:

Wed Feb 21

Princeton NJ

Thr Feb 22

Philadelphia PA

Fri Feb 23

Glen Echo MD

Sat Feb 24

Bethlehem PA

Sun Feb 25

Brooklyn NY

Alex Deis-Lauby will be calling all of these. Come dance and/or say hi!

Comment via: facebook, mastodon