trentmkelly/LessWrong-43k huggingface dataset:
Technological solutions to the climate crisis
by dominicq — 2023-07-19T12:39:08.782Z
Link: https://sundaystopwatch.eu/technological-solutions-to-the-climate-crisis/

Climate and environmental activists frequently complain how the technocentric approach to solving the climate crisis isn't enough or isn't good in some other way than "enough". I agree that it is not enough. Policy probably plays an equally, if not more important role. Policy interventions are things you can do right away, while technological breakthroughs are not guaranteed. And while policy interventions may have unintended consequences, they are at least guaranteed in some way, while technology may - or may not - be developed.

Of course, the two are not mutually exclusive, and shouldn't be seen this way. However asks you to stop developing seeds that can withstand a wider spectrum of temperatures, or cheap anti-frost systems, or glasshouses that can withstand hail, and instead join their banner-holding protest in front of a regulator or a megacorp is probably uninformed, and likely stupid. (If they ask you to go join the protest in addition to your technological work, then I don't hold any judgment.) And naturally, the inverse is also true, but there's something to say about the historical effectiveness of technological interventions. Namely: they have the capacity to start, while regulation only has the capacity to stop. Ok, if you pass sensible laws, you could in theory also incite but you can never start something through a law, only stop it. Therefore, technology is inherently proactive, while regulation is inherently reactive (and often regressive, as shown by multiple outdated regulations). Therefore I place much more weight on technocentric solutions of physical problems (such as the climate crisis) - it's something that looks forwards. It's long term, while regulation is often a short term patch.

So to say it in the simplest terms:

Technology:

not guaranteed

takes a long time to create and adopt

could also be very harmful

solves the problem in a proactive, long-term way

Policy:

almost guaranteed

quick to implement on a local scale

ineffective when there's a coordination problem

can become outdated, but very difficult to get rid of at that point

serves at best as a quick patch

With those two out of the way, I want to make an additional case for technology in the context of "solving" the climate crisis.

The climate crisis is a set of physical problems: droughts, extreme weather, very high and very low temperatures, and more. These events then have s...
Critiques of prominent AI safety organizations: Introduction
by Omega. — 2023-07-19T06:54:20.986Z
Link: https://forum.effectivealtruism.org/posts/N4LKrktopDs5Qdqgn/critiques-of-prominent-ai-safety-organizations-introduction

Cross-posted from the EA Forum.

What is this series (and who are we)?

This is a series of evaluations of technical AI safety (TAIS) organizations. We evaluate organizations that have received more than $10 million per year in funding and that have had limited external evaluation.[1]

The primary authors of this series include one technical AI safety researcher (>4 years experience), and one non-technical person with experience in the EA community. Some posts also have contributions from others with experience in technical AI safety and/or the EA community.

This introduction was written after the first two posts in the series were published. Since we first started working on this series we have updated and refined our process for evaluating and publishing critiques, and this post reflects our present views.

Why are we writing this series?

Recently, there has been more attention on the field of technical AI safety (TAIS), meaning that many people are trying to get into TAIS roles. Without knowing significant context about different organizations, new entrants to the field will tend to apply to TAIS organizations based on their prominence, which is largely related to factors such as total funding, media coverage, volume of output, etc, rather than just the quality of their research or approach. Much of the discussion we have observed about TAIS organizations, especially criticisms of them, happens behind closed doors, in conversations that junior people are usually not privy to. We wish to help disseminate this information more broadly to enable individuals to make a better informed decision.

We focus on evaluating large organizations, defined as those with more than $10 million per year in funding. These organizations are amongst the most visible and tend to have a significant influence on the AI safety ecosystem by virtue of their size, making evaluation particularly important. Additionally, these organizations would only need to dedicate a small fraction of their resources to engaging with these criticisms.

How do we evaluate organizations?

We believe that an organization should be graded on multiple metrics. We consider:

Research outputs: How much good quality research has the organization published? This is the area where we put the most weight.
Research agenda: Does the organization’s research plan seem likely to bear fruit?
Research team: What proporti

...
House Grocery Spending
by jefftk — 2023-07-19T03:00:01.229Z

I was curious what our food spending looked like, so I tried to get a
picture of what this has looked like over the past year. We have a
good idea of our total spending: we buy most groceries on a

house credit card so we can split it
fairly, and if someone accidentally buys groceries on a personal card
(or vice versa) we are pretty good about fixing it in the spreadsheet.
This comes to $15,875 for 99 person months, or $160/person/month.
On the other hand, three of the people are children, currently 9y, 7y,
and 2y, so it's probably more useful to count them as two people,
which means 87 person months, and $182/person/month.

Of the 87 person months, 12 were vegan, 15 were vegetarian, and 18
were vegetarian-leaning. Since it's easier to cook a meal with fewer
dishes, however, vegan and vegetarian food was more common than you
would get by dividing person months.

These spending totals don't include any spending on restaurants, but with
the exception of Jeff's work providing lunch, and the older kids
getting school lunch, this isn't something anyone in the house does
much of.

The bulk of this, $140 of $182 (79%) is via grocery delivery, which
means it ought to be possible to see exactly how we spent the money.
Unfortunately, Instacart does not seem to have a nice export feature,
but I wrote some code to parse
manual copies from the order history. Here's our monthly per-person
grocery delivery spending by area with an attempt at a taxonomy:

Protein $51.39

Dairy $29.39

Cheese $10.63

Cheddar $3.96

Mozzarella $2.21

Parmesan $1.84

Fresh Mozzarella $1.32

Cream Cheese $0.57

Mexican Cheese $0.37

Feta $0.20

Ricotta $0.16

Milk $7.01

Butter $3.67

Fake Dairy $2.91

Soy Milk $2.04

Fake Cheese $0.59

Fake Mozzarella $0.42

Fake Mexican Cheese $0.09

Fake Cheddar $0.09

Fake Butter $0.19

Fake Sour Cream $0.09

Heavy Cream $2.49

Sour Cream $1.51

Greek Yoghurt $1.17

Meat $14.01

Fake Meat $8.53

Fake Sausages $5.07

Tofu $2.24

Fake Meatballs $0.57

Fake Burgers $0.26

Fake Chicken Nuggets $0.13

Fake Crab Cakes $0.10

Fake Chicken $0.09

...
A brief history of computers
by Adam Zerner — 2023-07-19T02:59:19.679Z

Recently I've been learning about the history of computers. I find it to be incredibly interesting. I'd like to write a post about it to summarize and comment on what I've learned.

I'm a little hesitant though. I'm no expert on this stuff. I'm largely learning about it all for the first time. So then, take all of this with a grain of salt.[1] It's more of a conversation starter than a finished product. If you want something authoritative, I'd recommend the Stanford Encyclopedia of Philosophy.

Logic

Let's start with logic. Computers are largely based on boolean logic. Y'know, 1s and 0s. AND, OR, NOT.

George Boole did a bunch of important work here in the mid 1800s, but let's try backing up even further. Was there anything important that came before Boolean logic?

Yeah, there was. It goes all the way back to Aristotle in ~350 BCE. Aristotle did a bunch of groundbreaking work in the field of logic.

Furthermore, after "breaking the ground", there weren't any significant developments until the mid 1800s. Wow! That's a long time. An unusually long time. In other fields like mathematics, natural sciences, literature and engineering, there were significant advances. I wonder why things in the field of logic were so quiet.

Anyway, let's talk about what exactly Aristotle did. In short, he looked at arguments in the abstract. It's one thing to say that:

All dogs have feet
Filo is a dog
Therefore, Filo has feet

It's another thing to say that:

All P have Q
R is a P
Therefore, R has Q

The former is concrete. It's talking about dogs, feet and Filo. The latter is abstract. It's talking about P's, Q's and R's. Do you see the difference?

Before Aristotle, people never[2] thought about this stuff in terms of P's and Q's. They just thought about dogs and feet. Thinking about P's and Q's totally opened things up. Pretty cool. Abstraction is powerful. I think this is very much worth noting as an important milestone in the history of computers.

Ok. So once Aristotle opened the flood gates with categorical logic, over time, people kinda piggybacked off of it and extended his work. For example, the Stoics did a bunch of work with propositional logic.

Propositional logic is different from categorical logic. Categorical logic is about what categories things belong to. For example, earlier we basically said that dogs belong to the category of "things with feet" and that Filo belongs to the...
Simple alignment plan that maybe works
by Iknownothing — 2023-07-18T22:48:36.771Z

I'll ask for feedback at the end of this post, please hold criticisms, judgements and criticisms for then.
Forget every single long complicated, theoretical mathsy alignment plan.
In my opinion, pretty much every single one of those is too complicated and isn't going to work.
Let's look at the one example we have of something dumb making something smart that isn't a complete disaster and at least try to emulate that first.
Evolution- again, hold judgements and criticisms until the end.
What if you trained a smart model on the level of say, GPT3 alongside a group of much dumber and slower models, in an environment like a game world or some virtual world?
Dumb models who, with the research in interpretability, you know what their utility function is.
The smart, fast model however, does not.
Every time the model does something that harms the utility function of the dumber models, it gets a loss function.
The smarter model will likely need to find a way to figure out the utility functions of the dumber models.
Eventually, you might have a model that's good at co-operating with a group of much dumber, slower models- which could be something like what we actually need!
Please feel free to now post any criticisms, comments, judgements, etc. All are welcome.
Tiny Mech Interp Projects: Emergent Positional Embeddings of Words
by Neel Nanda — 2023-07-18T21:24:41.990Z

This post was written in a rush and represents a few hours of research on a thing I was curious about, and is an exercise in being less of a perfectionist. I'd love to see someone build on this work! Thanks a lot to Wes Gurnee for pairing with me on this

Tokens are weird, man

Introduction

A particularly notable observation in interpretability in the wild is that part of the studied circuit moves around information about whether the indirect object of the sentence is the first or second name in the sentence. The natural guess is that heads are moving around the absolute position of the correct name. But even in prompt formats where the first and second names are in the different absolute positions, they find that the informations conveyed by these heads are exactly the same, and can be patched between prompt templates! (credit to Alexandre Variengien for making this point to me).

This raises the possibility that the model has learned what I call emergent positional embeddings - rather than representing "this is the token in position 5" it may represent "this token is the second name in the sentence" or "this token is the fourth word in the sentence" or "this is the third sentence in the paragraph" etc. Intuitively, models will often want to do things like attend to the previous word, or the corresponding word in the previous sentence, etc - there are lots of things it will plausibly want to do that are natural in some emergent coordinate scheme that are unnatural in the actual token coordinate scheme.

I was curious about this, and spent an afternoon poking around with Wes Gurnee at whether I could convince myself that these emergent positional embeddings were a thing. This post is an experiment: I'm speedrunning a rough write-up on a few hours of hacky experiments, because this seemed more interesting to write-up than not to, and I was never going to do the high-effort version. Please take all this with a mountain of salt, and I'd love to see anyone build on my incredibly rough results - code here.

Experiments

You can see some terrible code for these experiments here. See the Appendix for technical details

I wanted to come up with the dumbest experiment I could that could shed light on whether this was a thing. One thing that models should really care about is the ability to attend to tokens in the previous word. Words can commonly range from 1 to 3 tokens (and ...
Quick Thoughts on Language Models
by RohanS — 2023-07-18T20:38:57.866Z

Epistemic status: Somewhere between Exploratory and My Best Guess. Plenty of other people have written about similar or related ideas, and about alternative or conflicting views.

Epistemic effort: Ideas developed slowly over about 3 years, via learning ML, playing with language models, reading work by others, and discussing language model capabilities with various people. I've had a few recent discussions particularly related to the ideas below. About 1 hour of writing and editing this before posting.

Thoughtful feedback on this post is very welcome!

Language models trained on next-token prediction learn to model text-generating processes. The data that they are trained on - mainly massive internet text corpuses - encode a lot of information about the causal structure of the world.

For instance, there is a lot of text online about objects falling when people let go of them, which allows language models to learn the basics of how gravity works. Further, there is text about what happens when things are dropped on the Moon or the ISS, and about Newtonian mechanics, and about general relativity, etc., so language models can learn how gravity works in a much deeper and more general way than that too.

Since the processes that generated the text on the Internet are so diverse, language models have the potential to model a large amount of the causal structure of the world.
The limit of performance at next-token prediction, and text-generation process modeling, is far superhuman. The argument for this that I found most convincing was the following passage from GPTs are Predictors, not Imitators by Eliezer Yudkowsky:

Imagine somebody telling you to make up random words, and you say, "Morvelkainen bloombla ringa mongo."

Imagine a mind of a level - where, to be clear, I'm not saying GPTs are at this level yet -

Imagine a Mind of a level where it can hear you say 'morvelkainen blaambla ringa', and maybe also read your entire social media history, and then manage to assign 20% probability that your next utterance is 'mongo'.

The fact that this Mind could double as a really good actor playing your character, does not mean They are only exactly as smart as you.

When you're trying to be human-equivalent at writing text, you can just make up whatever output, and it's now a human output because you're human and you chose to output that.

GPT-4 is being asked to predict all that stuff

...
Still no Lie Detector for LLMs
by Daniel Herrmann — 2023-07-18T19:56:57.598Z

Background

This post is a short version of a paper we wrote that you can find here. You can read this post to get the core ideas. You can read the paper to go a little deeper.

The paper is about probing decoder-only LLMs for their beliefs, using either unsupervised methods (like CCS from Burns) or supervised methods. We give both philosophical/conceptual reasons we are pessimistic and demonstrate some empirical failings using LLaMA 30b. By way of background, we’re both philosophers, not ML people, but the paper is aimed at both audiences.

Introduction

One child says to the other “Wow! After reading some text, the AI understands what water is!”… The second child says “All it understands is relationships between words. None of the words connect to reality. It doesn’t have any internal concept of what water looks like or how it feels to be wet. …” …

Two angels are watching [some] chemists argue with each other. The first angel says “Wow! After seeing the relationship between the sensory and atomic-scale worlds, these chemists have realized that there are levels of understanding humans are incapable of accessing.” The second angel says “They haven’t truly realized it. They’re just abstracting over levels of relationship between the physical world and their internal thought-forms in a mechanical way. They have no concept of [$!&&!@] or [#@&#**]. You can’t even express it in their language!”

--- Scott Alexander, Meaningful

Do large language models (LLMs) have beliefs? And, if they do, how might we measure them?

These questions are relevant as one important problem that plagues current LLMs is their tendency to generate falsehoods with great conviction. This is sometimes called lying and sometimes called hallucinating. One strategy for addressing this problem is to find a way to read the beliefs of an LLM directly off its internal state. Such a strategy falls under the broad umbrella of model interpretability, but we can think of it as a form of mind-reading. Detecting lies in LLMs has many obvious applications, and is especially relevant for things like ELK.

We tackle the question about the status of beliefs in LLMs head-on. We proceed in two stages. First, we assume that LLMs do have beliefs, and consider two current approaches for how we might measure them, due to Azaria and Mitchell and to Burns et al. We provide empirical results from LLaMA 30b ...
Meta announces Llama 2; "open sources" it for commercial use
by LawrenceC — 2023-07-18T19:28:28.685Z
Link: https://about.fb.com/news/2023/07/llama-2/

See also their Llama 2 website here: https://ai.meta.com/llama, and their research paper here: https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/

From their blog post:

Takeaways

Today, we’re introducing the availability of Llama 2, the next generation of our open source large language model.
Llama 2 is free for research and commercial use.
Microsoft and Meta are expanding their longstanding partnership, with Microsoft as the preferred partner for Llama 2.
We’re opening access to Llama 2 with the support of a broad set of companies and people across tech, academia, and policy who also believe in an open innovation approach to today’s AI technologies.

Compared to the first Llama, LLama 2 is trained for 2T tokens instead of 1.4T, has 2x the context length (4096 instead of 2048), uses Grouped Query Attention, and performs better across the board, with performance generally exceeding code-davinci-002 on benchmarks:

They also release both a normal base model (Llama 2) and a RLHF'ed chat model (Llama 2-chat). Interestingly, they're only releasing the 7B/13B/70B models, and not the 34B model, "due to a lack of time to sufficiently red team".

More importantly, they're releasing it both on Microsoft Azure and also making it available for commercial use. The form for requesting access is very straightforward and does not require stating what you're using it for: (EDIT: they gave me access ~20 minutes after submitting the form, seems pretty straightforward.)

Note that their license is not technically free for commercial use always; it contains the following clauses:

[1.] v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof).

2. Additional Commercial Terms. If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights.

See also the Llama 2 Acceptable Use Policy (which seems pretty standard)....
AI Impacts Quarterly Newsletter, Apr-Jun 2023
by Harlan — 2023-07-18T17:14:09.774Z
Link: https://blog.aiimpacts.org/p/ai-impacts-quarterly-newsletter-apr

Every quarter, we have a newsletter with updates on what’s happening at AI Impacts, with an emphasis on what we’ve been working on. You can see past newsletters

here

and subscribe to receive more newsletters and other blogposts

here

.
During the past quarter, Katja wrote an article in TIME, we created and updated several wiki pages and blog posts, and we began working on several new research projects that are in progress.
We’re running a

reader survey

, which takes 2-5 minutes to complete. We appreciate your feedback!

If you’d like to donate to AI Impacts, you can do so

here

. Thank you!

News

Katja Grace’s TIME article

In May, TIME published Katja’s article “

AI Is Not an Arms Race

.” People sometimes say that the situation with AI is an arms race that rewards speeding forward to develop AI before anyone else. Katja argues that this is likely not the situation, and that if it is, we should try to get out of it.

References to AI Impacts Research

The

2022 Expert Survey on Progress in AI

was referenced in

an article in The Economist

, a

New York Times op-ed by Yuval Noah Harari

, a

Politico op-ed

that argues for a “Manhattan Project” for AI Safety, and a

report from Epoch AI’s Matthew Barnett and Tamay Besiroglu

about a method for forecasting the performance of AI models.

The Japanese news agency Kyodo News published an

article

about AI risks that referenced

Katja’s talk at EA Global

from earlier this year.

We also maintain an ongoing

list of citations of AI Impacts work

that we know of.

Research and writing highlights

Views on AI risks

Rick compiled a list of

quotes from prominent AI researchers and leaders

about their views on AI risks.
Jeffrey compiled a list of

quantitative estimates about the likelihood of AI risks

from people working in AI safety.
Zach compiled a list of

surveys that ask AI experts or AI safety/governance experts

for their views on AI risks.

Average given likelihoods of how good or bad human-level AI will be, from 559 machine learning experts in the

2022 Expert Survey on Progress in AI

.

The supply chain of AI development

Harlan wrote a page outlining some

factors that affect the price of AI hardware

Jeffrey wrote a

blogpost

arguing that slowing AI is easiest if AI companies are horizontally integrated, but not vertically integrated.

Ideas for internal and public policy about AI

Zach compiled reading lists of research and discussion about

safety-related ideas that AI

...