trentmkelly/LessWrong-43k huggingface dataset:
Principles Behind Bottlenecks
by curi — 2020-08-01T22:48:10.499Z

This post follows my Chains, Bottlenecks and Optimization (which has the followups Bottleneck Examples and Comment Replies for Chains, Bottlenecks and Optimization). This post expands on how to think about bottlenecks.

There are deeper concepts behind bottlenecks (aka constraints, limiting factors, or key factors).

First, different factors contribute different amounts to goal success. Second, there’s major variation in the amounts contributed.

E.g. I’m adding new features to my software. My goal is profit. Some new features will contribute way more to my profit than others There are lots of features my (potential) customers don’t care about. There are a few features that tons of customers would pay a bunch for.

A bottleneck is basically just a new feature that matters several orders of magnitude more than most others. So most features are approximately irrelevant if the bottleneck isn’t improved.

Put another way: improving the bottleneck translates fairly directly to more goal success, while improving non-bottlenecks translates poorly, e.g. only at 1/1000th effectiveness, or sometimes 0. (It’s possible, but I think uncommon, to have many factors that contribute similarly effectively to goal success. Designing stuff that way doesn’t work well. It’s the same issue as balanced production lines being bad, which Eli Goldratt explains in The Goal: A Process of Ongoing Improvement. It’s also similar to the Pareto Principle which says 80% of effects come from 20% of the causes – meaning most factors aren’t very important.)

What about the software not crashing, not corrupting saved data, and not phoning home with location tracking data? People want those things but I could have them and easily still make zero profit.

A good, typical model for viewing goal pursuit is:

There are many factors that would help, and just one or a few of them are the most important to focus on. This is because most factors have a significantly smaller impact. This is focusing on the key positives.

There are also many dealbreaker factors that cause failure if screwed up. This is avoiding major negatives.

People care about (1) conditional on (2) not being broken. Avoid anything awful, then optimize in the right places.

When buying a cat, I might try to optimize cuteness and cheapness, while also making sure the cat has 4 paws, a tail, no rabies, is tame, and isn’t too old. I want to do well ...
Agentic Language Model Memes
by FactorialCode — 2020-08-01T18:03:30.844Z

Related: What specific dangers arise when asking GPT-N to write an Alignment Forum post?

I've been thinking about the AI safety implications of ultra powerful language models. I think it makes more sense to think of language models trained with internet text as containing a collective of agents that can be queried with appropriate prompts. This is because language models are trained to mimic their training data. The best way to do this is for them to approximate the process generating the data. The data used to train modern language models is produced by millions of humans, each with their own goals and desires. We can expect the strongest of language models to emulate this, at least superficially, and the kind of agent that the language model emulates at any given point will be a function of the upstream prompt.

With a strong enough language model, it's not unthinkable that some of these agents will be smart enough and sufficiently unaligned to manipulate the humans that view the text they generate. Especially since this already happens between humans on the internet to some degree. This is especially true in environments like AI dungeon, where the agents can get feedback from interacting with the human.

More generally, from an AI safety perspective, I wouldn't be worried about language models per say, as much as I would be worried about what I'm going to call agentic language model memes. Prompts that get language models to emulate a specific, unaligned agent. The unaligned agent can then convince a human to spread the prompt that instanced it through social media or other means. Then, when a new language model arrives, the unaligned agent takes up a larger fraction of the networks probability density. Furthermore, prompts will experience variation and face evolutionary pressure, and more viral prompts will get more frequent. I hypothesize that this will create an environment that will lead to the emergence of prompts that are both agentic and effective at spreading themselves. With enough iterations, we could end up with a powerful self replicating memetic agent with arbitrary goals and desires coordinating with copies and variations of itself to manipulate humans and gain influence in the real world.

At the moment, I suspect GPT-3 isn't good enough to support powerful memetic agents, primarily because the 2048 BPE prompt length and architectural constraints...
What are you looking for in a Less Wrong post?
by adamShimi — 2020-08-01T18:00:04.738Z

My model of "What LW as a community considers a good post" is not accurate enough for my taste. Sometimes I don't understand the karma/number of comments (or lack thereof) of some posts, and it seems rude or arrogant to ask (whether it's someone else's post or one of mine). Among other things, this might help me decide whether to spend the hours necessary to write a post, or if it's not the kind of ideas people are interested in here.

So I'm asking you, yes you, what makes a LW post that you want to read. That you want to upvote or strong upvote. That you might comment. A good post, basically.
Food Spending During Covid
by jefftk — 2020-08-01T17:10:07.739Z

Our house has

shared groceries,
and in my most recent reconcile I saw that our spending is up quite a
bit relative to this time last year. In 2019, April through July, our
monthly spending per person was $172, while in 2020 it was
$303. In the big picture this is due to the pandemic,
but I'm not sure which factors are responsible for most of the
change. Some ideas:

Our house is not
going into stores, and we've been having groceries delivered. This
reduces our risk, but it's also somewhat better at limiting spread in
general. Grocery delivery is more expensive, especially when you
include the tips. I expect this is the largest component for us.

[EDIT: Julia points out that another reason Instacart is more
expensive is that we're generally choosing the cheapest option for
things, and so when they make substitutions they're picking more
expensive versions or generally not predicting our preferences very
well.]

Food has gotten more expensive, but only by ~3%: CPIFABSL
is up to 269 from 261 in February.

I'm not eating lunch at work, since the office is closed. I
also used to eat my fruits and vegetables there, and they're more
expensive than the mostly carbs I used to eat at home.

People in the house aren't eating out anymore. I don't think
any of us were eating that many other commercially prepared meals, but
it's a component. The house grocery budget won't count a restaurant
meal, but it will count the food you eat at home because you didn't go
out to eat.

I used to also buy things when they were on sale, and without
walking through the grocery store that doesn't really work
anymore.

We've been buying some things from a restaurant supply place,
and while for some products it is substantially cheaper, others are
fancier than what we would normally buy. I suspect this one about
cancels out.

It's very noisy, but here's a chart showing our historical food
spending. It's a 90 day running average, so the recent rising slope
is mostly the step-change of our newer more expensive situation
being smeared over a longer period.
Forecasting Newsletter: July 2020. 
by NunoSempere — 2020-08-01T17:08:15.401Z

Highlights

Social Science Prediction Platform launches.

Ioannidis and Taleb discuss optimal response to COVID-19.

Report tries to foresee the (potentially quite high) dividends of conflict prevention from 2020 to 2030.

Index

Highlights.

Prediction Markets & Forecasting Platforms.

New undertakings.

Negative Examples.

News & Hard to Categorize Content.

Long Content.

Sign up here, browse past newsletters here, or view it on the EA forum here.

Prediction Markets & Forecasting Platforms.

Ordered in subjective order of importance:

Metaculus continues hosting great discussion.

In particular, it has recently hosted some high-quality AI questions.

User @alexrjl, a moderator on the platform, offers on the EA forum to operationalize questions and post them on Metaculus, for free. This hasn't been picked up by the EA Forum algorithms, but the offer seems to me to be quite valuable. Some examples of things you might want to see operationalized and forecasted: the funding your organization will receive in 2020, whether any particularly key bills will become law, whether GiveWell will change their top charities, etc.

Foretell is a prediction market by the University of Georgetown's Center for Security and Emerging Technology, focused on questions relevant to technology-security policy, and on bringing those forecasts to policy-makers.

Some EAs, such as myself or a mysterious user named foretold, feature on the top spots of their (admittedly quite young) leaderboard.

I also have the opportunity to create a team on the site: if you have a proven track record and would be interested in joining such a team, get in touch before the 10th of August.

Replication Markets

published their first paper

had some difficulties with cheaters:

"The Team at Replication Markets is delaying announcing the Round 8 Survey winners because of an investigation into coordinated forecasting among a group of participants. As a result, eleven accounts have been suspended and their data has been excluded from the study. Scores are being recalculated and prize announcements will go out soon."

Because of how Replication Markets are structured, I'm betting the cheating was by manipulating the Keynesian beauty contest in a Predict-O-Matic fashion. That is, cheaters could have coordinated to output something surprising during the Keynesian Beauty Contest

...
Inner Alignment: Explain like I'm 12 Edition
by Rafael Harth — 2020-08-01T15:24:33.799Z

(This is an unofficial explanation of Inner Alignment based on the Miri paper Risks from Learned Optimization in Advanced Machine Learning Systems (which is almost identical to the LW sequence) and the Future of Life podcast with Evan Hubinger (Miri/LW). It's meant for anyone who found the sequence too long/challenging/technical to read.)

Note that bold and italics means "this is a new term I'm introducing," whereas underline and italics is used for emphasis.

What is Inner Alignment?

Let's start with an abridged guide to how Deep Learning works:

Choose a problem

Decide on a space of possible solutions

Find a good solution from that space

If the problem is "find a tool that can look at any image and decide whether or not it contains a cat," then each conceivable set of rules for answering this question (formally, each function from the set of all pixels to the set {yes,no}) defines one solution. We call each such solution a model. The space of possible models is depicted below.

Since that's all possible models, most of them are utter nonsense.

Pick a random one, and you're as likely to end up with a car-recognizer than a cat-recognizer – but far more likely with an algorithm that does nothing we can interpret. Note that even the examples I annotated aren't typical – most models would be more complex while still doing nothing related to cats. Nonetheless, somewhere in there is a model that would do a decent job on our problem. In the above, that's the one that says, "I look for cats."

How does ML find such a model? One way that does not work is trying out all of them. That's because the space is too large: it might contain over 101000000 candidates. Instead, there's this thing called Stochastic Gradient Descent (SGD). Here's how it works:

SGD begins with some (probably terrible) model and then proceeds in steps. In each step, it switches to another model that is "close" and hopefully a little better. Eventually, it stops and outputs the most recent model.[1] Note that, in the example above, we don't end up with the perfect cat-recognizer (the red box) but with something close to it – perhaps a model that looks for cats but has some unintended quirks. SGD does generally not guarantee optimality.

The speech bubbles where the models explain what they're doing are annotations for the reader. From the perspective of the programmer, it looks like this:

The programmer ha...
Online LessWrong Community Weekend,
September 11th-13th
by Linda Linsefors — 2020-08-01T14:55:38.986Z

From 11th-13th September aspiring rationalists will gather for 3 days of socialising, fun and intellectual exploration. There will be a few pre-planned activities, but most of the content will be unconference style and participant driven.

On Monday, September 6th, participants will be invited to the event Discord and to access the collaborative schedule, to start planning what shall happen the following weekend. Over the week, the schedule will be filled in with workshops, talks and activities.

This is the 7th year of the LessWrong Community Weekend. Those of you who have attended this event before know that the atmosphere and sense of community at these weekends is something really special. If this sounds like something you would enjoy and you have some exciting ideas or skills to contribute, do come along and get involved.

However, as you might have noticed, this year's event is significantly different in one regard. Normally the LessWrong Community Weekend is an in-person event held in Berlin. This is the first time we’re trying this online, which means new and interesting opportunities to explore. For example, who will program the

seal-of-approval

bot?

Applications

Apply

here

before September 4th.

Early applicants will be favoured in the application process.

Payment

This event is free. However I do encourage donations to keep the LessWrong Community Weekend (LWCW) alive for the future.

I believe that most of us wish to come back to the normal, in-person LWCW next year. However, it is unclear if this will happen since the usual organisers are tired of doing all that work for free. If you agree with me that organisers deserve to get paid, then I encourage you to make a donation to the Berlin LessWrong community (who are the ones normally organising the LWCW). This money will be split evenly between retrospectively paying past organisers and be put out as a bounty for future organisers.

Suggested amount: 10-50 Euro depending on your personal budget.

Payment recipient:
LessWrong Deutschland e.V.
IBAN: DE74 8306 5408 0004 8647 19
BIC: GENODEF1SLR
Comment: LWCW Organiser fund

If your bank asks you for the bank's name and address, this is it:
Deutsche Skatbank
Zweigniederlassung der VR-Bank Altenburger Land eG
Markt 10
04600 Altenburg (Thür.)

Some Platforms we will use

This list is preliminary. If you have better suggestions, please

let me know

!
(Don’t ...
On Procrastination - The art of shaping your future actions
by Neel Nanda — 2020-08-01T10:22:44.450Z
Link: https://www.neelnanda.io/blog/mini-blog-post-24-on-procrastination-the-art-of-shaping-your-future-actions

(This is a post from a daily blogging experiment I did at neelnanda.io, which I thought might also fit the tastes of LessWrong)

(Also relevant: A post I made on systems & willpower and notes for a workshop I taught on Murphyjitsu, a technique for better using your intuitions to plan)

Introduction

By default, I’m an excellent procrastinator. Worse, I’m a mathematician, and trained to be excited by existence proofs and theory, and to ignore the implementation details. As a result, by default, I am really bad at actually taking actions, or acting on my desires.

I often have cool ideas, but rarely act on them. I might hear about a cool book that sounds super worth reading, an article I want to check out, think about a friend I should message, or hear about an awesome idea to try solving one of my problems with. And this flashes in my mind in the moment, and it sounds worthwhile! It’s something I genuinely want to do. But, empirically, I often forget about these, and very, very rarely act upon them. And this is a really big problem! It’s so easy to fall into the classic trap of being academically minded - always thinking, never doing. Having a bunch of things I want to do, but never get round to. This post is about my tools for solving these.

And this is a problem I very commonly observe in others! If you resonate with this problem, I strongly urge you to actually try the ideas I’ll outline here, this is really common. To make it a bit more visceral, I recommend that you set a 5 minute timer right now, and list things you’re currently putting off, and never get round to - if you’re anything like me, that’s an uncomfortably long list.

Further, I think there’s a kind of person who’s been reading and enjoying my blog, who thinks I have some interesting ideas that might be worth acting upon. Yet, empirically, they have yet to do anything, and are unlikely to ever get round to it. This post is aimed at you.

The mindset of planning

I think the root cause of all these failure modes is a failure of planning. It’s easy to conceive of planning as sitting down and coming up with a careful, meticulous strategy for how to approach something. But I define planning as the art of shaping your future actions to achieve your goals. Making a careful, detailed strategy is just a means to an end.

I find this...
Generalized Efficient Markets in Political Power
by johnswentworth — 2020-08-01T04:49:32.240Z

Schelling Points

In Thomas Schelling’s

classic experiment

, we imagine trying to meet up with someone in New York City, but we haven’t specified a time or place in advance and have no way to communicate. Where do we go, and when, to maximize the chance of meeting? There are some “natural” choices - places and times which stand out, like the top of the Empire State Building at noon. These are called Schelling points.

More generally, Schelling points are relevant whenever two or more people need to make “matching” choices with limited ability to communicate in advance. For instance, certain markets, like Ebay or Uber, serve as “meeting points” for buyers and sellers. Schelling himself wrote a fair bit about negotiations, where people need to agree on how to divide some spoils, or where to draw a boundary, or ... They can talk to each other, but actually communicating is hard because both parties have no incentive to be honest - and therefore no reason to trust each other when e.g. when one person says “I just can’t afford to sell it below $10”. Schelling points become natural outcomes for the negotiations - e.g. split the spoils evenly, draw the boundary at the river, etc.

In practice, it’s often useful to create Schelling points. In the New York City experiment, one could put up a giant billboard that says “meeting point”, and place signs all over the city pointing toward the meeting point, making that point a natural place for people to meet. Some airports actually do this:

Ebay and Uber are of course also examples of purpose-built Schelling points.

One interesting feature of creating a Schelling point is that we may have some degrees of freedom available, and we can use those degrees of freedom to extract value.

In our meetup example, we could imagine putting the meetup point inside a building, and charging people to get in - much like Ebay or Uber charge fees for their services. Or, we could imagine local businesses wanting to put the meetup point nearby, in hopes of attracting business from meeters - one could imagine a gimicky airport restaurant with a bunch of “MEET HERE!” signs hoping to sell people overpriced nachos and drinks while they wait to meet up with friends or family. Alternatively, we could imagine users of the meetup point wanting it in locations convenient to them - e.g. in the New York City example, people in a particular neighborhood might camp...
The Era Of Unlimited Everything: Unlimited Materials & Unlimited Money
by jdcampolargo — 2020-08-01T03:01:57.489Z

This essay was originally published on my website.

Ask yourself these questions:

Sitting in your home, looking around, what do you see??
How did you buy those things?

Many of the things you’re surrounded by are probably made of plastic.

Many of those things were probably bought with credit.

Could you have bought them without credit? What if those things were not made out of plastic? Would have you been able to buy them?

Overview:

Before & After Plastic

Invention

Before & After Credit

Invention

A World without Credit & Plastic
Credit & Plastic: Economic Growth
A Product of Imagination
Unlimited Everything

Imagine a world without credit and a world without plastic.

Before & After Plastic

To illustrate this example, let’s use three examples of common things made out of plastic.

Plastic containers for food storage
Clothes
Electronic Devices (i.e. computers, phones)

Would you have stored your food as efficiently? Would you be able to afford clothes or electronic devices, considering many clothing and electronic devices are made of plastic?

A world without plastic is a world constrained by materials and their limitations. A world with plastic is a world constrained by...almost nothing.

Invention

Before plastic was created, we needed to use materials such as wood, animal fur, plants, or metals to make clothes and tools. Plastic was invented because we were running out of ivory as elephants were going extinct because of our insatiable human demand. Ivory was used for boxes, piano keys, combs, billiard balls, and many other things.

The legend says that a New York billiard supplier ran a newspaper ad offering “a handsome fortune,” ten thousand dollars in gold, to anyone who could invent an alternative to ivory.

An audacious entrepreneur, John Wesley Hyatt, a young printer read the ad and decided he could do it. There was a catch for him, though. He had no formal training in chemistry, but he was an inventor at heart, having patented a knife sharpener in his early 20s.

The entrepreneurial spirit ran through Hyatt and he started experimenting. He began with combinations of solvents and even mixtures of cotton. Little did he know that at his homemade lab, he was building one of the most important innovations in the history of humankind, creating a new era of unlimited materials, no longer would people h...