trentmkelly/LessWrong-43k huggingface dataset:
Three mental images from thinking about AGI debate & corrigibility
by Steven Byrnes — 2020-08-03T14:29:19.056Z

Here are three mental images I’ve used when sporadically struggling to understand the ideas and prospects for AI safety via debate, IDA, and related proposals. I have not been closely following the discussion, and may well be missing things, and I don’t know whether these mental images are helpful or misleading.

Reading this post over, I seem to come across as a big skeptic of these proposals. That's wrong: My actual opinion is not “skeptical” but rather “withholding judgment until I read more and think more”. Think of me as "newbie trying to learn", not "expert contributing to intellectual progress". Maybe writing this and getting feedback will help. :-)

1. AGI Debate as water-skiing behind a pair of nose-to-nose giant rocket engines

In AI safety via debate, we task two identical AGIs with arguing opposite sides of a question. That has always struck me as really weird, because one of them is advocating for a false conclusion—perhaps even knowingly! Why would we do that? Shouldn’t we program the AGIs to just figure out the right answer and explain it to us?

My understanding is that one aspect of it is that two equal-and-opposite AGIs (equal power, opposite goals) would keep each other in check, even if the AGIs were each very powerful.

So imagine you row to an island in the center of a little mountain lake, but then your boat gets eaten by beavers, and it's too far to swim to shore. What you do have on your little island is a giant, 100,000kg rocket engine with no throttle. Once you start it, it burns uncontrollably until it’s out of fuel, by which point it's typically way out in outer space! Oh, and the rocket also has a crappy steering system—coarse controls, laggy, poor feedback.

So what do you do? How do you cross the 300 meters of water to shore?

The answer is obvious: You do a copy-and-paste to make a second giant rocket engine, and build a frame that keeps the two pointed almost exactly nose-to-nose. Then you turn them both on simultaneously, so they just press on each other, and don't go anywhere. Then you use the steering mechanism to create a tiny imbalance in the direction you want to move, and you gently waterski to shore. Success!

This analogy naturally suggests a couple concerns. First, the rocket engines might not be pointed in exactly opposite directions. This was discussed in Vojtech Kovarik's recent post

AI Unsafety via Non-Zero-Sum Debate

and its ...
Unifying the Simulacra Definitions
by Zvi — 2020-08-03T12:10:01.076Z

Epistemic Status: Confident this is the perspective I find most useful. This is intended to both be a stand-alone post and to be the second post in the Simulacra sequence, with the first being Simulacra Levels and their Interactions. It should be readable on its own, but easier having read the previous post.

Simulacra levels are difficult to understand.

This is not without cause. This is complex and bizarre stuff.

Simulacra levels are a map of the metaphors we use to create metaphoric maps of both territory and the map itself.

The text that coined the term Simulacra levels does not help matters. The term was first referenced locally by Ben Hoffman in this post, but this was not the original source.

The original source of the term is a super-dense work of French philosophy. It requires the reader to pause after every sentence. It’s not clear that a proper review would be shorter than the book itself.

Thus, I’m still working through the book. The more I read Jean Baudrillard‘s further assertions, the less they seem deserving of engagement. He is opposed for nonsensical reasons not only to the concept of capitalism, but the concepts of money, value and trade, and even urbanization and mass production. He blames these for the rise of simulacra, whereas they are the primary forces opposed to simulacra.

Upon parsing many of his super-dense sentences, I find many of them to be outright false. I find many others to be based on models and frameworks very different from my own, and that are assumed rather than specified in the text. The idea that capitalism isn’t the cause of all the world’s problems (never mind whether it’s the solution) does not seem to parse in his mind. I find many others to be downright absurd, or to be carrying water for the agendas of History’s Greatest Villains.

This is a case where I strongly endorse taking the concepts that are useful and leaving the remaining giant mess behind.

Baudrillard’s definition will be kept. Beyond that, I’m tossing essentially everything else away.

The goal of this post is to reconcile Baudillard’s definition with the Lion definition used in my previous posts, integrating the relation of a simulacra to reality with the motivational explanatory framework. This deals with the dueling definition problem so it doesn’t get in the way down the line, and I hope it helps explain why higher level activity systematically has such ...
What are the most important papers/post/resources to read to understand more of GPT-3?
by adamShimi — 2020-08-02T20:53:30.913Z

I'm way more used to thinking about weird maths or distributed algorithms or abstract philosophical problems than about concrete machine learning architectures. But based on everything I see about GPT-3, it seems a nice idea to learn more about it, even if only for participating in the discussion without spouting non-sense.

So I'm asking for what you think are the must-reads on GPT-3 specifically, and maybe any requirement to understand them.
Irrational Resistance to Business Success
by curi — 2020-08-02T19:15:14.589Z

Below I quote from Process of On Going Improvement forum, letter 6. Eli Goldratt shares a letter he received. I added a few notes to help people follow acronyms.

My question is: Does anyone know of any applications of Less Wrong philosophy to a situation like this? How can LW ideas about rationality explain or fix this sort of problem? The scenario is that someone tried to use rational thinking to make business improvements, was highly successful (which was measured and isn't in dispute), but nevertheless has met so much ongoing resistance that he's at the point of giving up.

I am no expert in TOC but I believe my recent experiences have impact as to what you are writing about.

TOC = Theory Of Constraints. Summary.

About 2 years ago I started on my TOC adventure. Read everything I could get a hold off etc. Tried to get the company interested, etc. In fact, I finally got them interested enough that we had multiple locations participate in the satellite sessions and had enough for three facilitators (myself included). However, I could never get the company to spend for training at AGI. So, in the old air cav fashion, I felt it was up to me to make it happen.

Last year we had real problems with cost, service, high inventory, etc. My plant, I am the plant manager, was being analyzed for a possible shutdown or sell off. We were asking for 17 machines at about $300,000 each due to "lack of capacity" and we were being supplemented by outside producers.

Again, I am not a TOC expert. Basically my exposure has been reading and researching and building computer simulations to understand. But I put on TOC classes for all of my associates (200). I spent 8 hours with each of these associates in multiple classes. We talked about the goal, TIOE, we played the dice game (push, KanBan, DBR) with poker chips, paper clips, and different variations of multiple sided dice and talked about its impact, etc.

The Goal (summary) is a book by Eli Goldratt that has sold over 6 million copies.

TIOE = Throughput, Inventory and Operating Expense. These are the measurements Goldratt recommends.

The dice game is explained in The Goal. It's also now taught by e.g. MIT (section 3-2).

DBR = drum buffer rope. It's about coordinating activities around the bottleneck/constraint.

Last summer we started development on DBR and a new distribution strategy based on what I have read and researched on TOC

...
Why isn't there a LessWrong App? Is a "blog-app" sustainable/useful for a community?
by Osson4 — 2020-08-02T17:47:59.136Z

I am a newbie lesswronger, so it might be an already asked question, but I can't find an answer in the FAQs. I also don't have any idea of how many resources are needed to support a blog-app, given an already developed site/community. I would be glad if someone could explain the issues/difficulties, and why the community never exploited this channel to acquire new users/better accessibility (I would love to have an app instead of using the browser to browse the site).
Burley Bee kids bike trailer review
by jefftk — 2020-08-02T17:20:07.601Z

With the coronavirus we haven't been taking buses or taxis, which had
mostly limited us to walking distance with our kids. [1] Expanding to
bicycling distance would help a lot, so we decided to get a bicycle
trailer.

I did a bunch of reading on various forums, and it looked like the Burley Bee was the best
fit for us. It holds two children, with a combined weight of 100 lb,
while being relatively light and sturdy. It has a rain cover, in case
we get caught out in bad weather, and it doesn't convert into a
stroller. No suspension, aside from the tires and the give in the
fabric, but the kids haven't complained. There's a good amount of
storage space in the back of the trailer. The safety flag is
included.

It comes with one bicycle hitch, but it's a very simple part so I
bought additional
ones so that me, my wife, and our au pair could each have a hitch
on our bicycles. The hitch goes onto the rear axle, on the left hand
side, and is about 3/16" thick.

We've been using it for about 6 weeks now, and we haven't had any
problems. The kids are happy with it, it's easy to bike with, and it
generally seems well designed.

(When we decided we wanted to buy one, in late May I think, Amazon,
the other big online stores, and our local bike stores were sold
out. I did some looking, though, and found a non-local bike store that
had some in stock and was happy to ship. At this point, however, it
seems to be back in stock at the manufacturer and
on
Amazon.)

[1] We considered getting a car, but addition to being a lot more
expensive, we have two housemates with cars that we can occasionally borrow. With covid
their cars have mostly been sitting at home, so getting a third seemed
excessive. But for something within a ~30min bike ride of the house a
car seems like overkill, and it's not worth the coordination with the
car owner / using a favor.

Comment via: facebook
Engineering Experience Through Score
by ZorbaTHut — 2020-08-02T12:09:40.050Z

This post is about bowling and Super Mario Odyssey.
Bowling is a game about knocking down pins with a ball. There are ten pins, and you (sort of; keep reading) have to knock them all down ten times (each one called a "frame"). You get only two throws per frame; if you still haven't succeeded, the pins are reset and you go on to the next frame.
There was probably an early version of bowling where you got points based on how many pins you knocked down, for a maximum score of 100. That is, still, where the game starts. But it's gotten more complicated. Today, if you manage to knock all the pins down (a "spare"), you also get points for however many you knock down on your next throw. In fact, if you manage to knock all the pins down on your first attempt (a "strike"), you get points for however many you knock down on your next two throws.
So the best possible score here is now three hundred points; we get a strike on every single throw, we get a total of 17 ten-point bonuses thanks to our strikes, and then we get two bonus throws at the end just to finish scoring our earned bonuses (the first of which counts double because it's accumulating bonuses from frames 9 and 10, the second of which is just accumulating bonuses from frame 10.)
This has interesting interactions, however. Remember, we get two chances to knock down pins on every single pin reset. Imagine I bowl a perfect game, except that, on the fifth frame, I only knock down nine pins. On the next throw I get the next pin, then continue bowling perfectly from there on. First, my Frame 3 bonus will be only 19 instead of 20. Second, my Frame 4 bonus will be only 10 instead of 20 (I did get all the pins down, but it took two throws to do it.) Third, my Frame 5 bonus will be only 10 instead of 20 (because I got a spare instead of a strike, I only get a bonus for one throw, not two.) This single mistake, missing a single pin on a single throw, has cost me 21 points out of a possible total of 300.
The thing I find interesting about bowling's scoring is how hard it gets to pick up those last few points. Anyone can get fifty points, anyone moderately skilled can break 150, but 250 is quite hard and a perfect 300 is a thing of legend. Once you're at the point where you're racking up strike after strike, even a slight mistake is devastating to your score, wherea...
When Gears Go Wrong
by Matt Goldenberg — 2020-08-02T06:21:25.389Z

(This talk was given at the LessWrong curated talks event on July 19. mr-hire is responsible for the talk, David Lambert edited the transcript.

If you're a curated author and interested in giving a 5-min talk, which will then be transcribed and edited, sign up here.)

mr-hire: A question that I don’t think gets asked enough in the rationalist community is whether gears are a good investment. Typically, when it is asked, the answer is yes. Here's an example from John Wentworth talking about gears as a capital investment.

I'm not picking on John, and I mostly agree with him, but I think this is a question that should receive a little bit more scrutiny and nuance.

So, gears are good—that’s the typical response from most of the rationalist and Effective Altruism community.

Gears provide deeper understanding of a topic. They allow you to carve reality at the joints when you're using your models. They allow for true problem-solving, not just cargo culting around your ideas.

And I think that's fair. All those things are true.

But I personally mostly try to develop deep gears in two places.

One is my primary field. I think you should definitely develop gears around whatever your career is. Whatever you're really focused on right now.

For me, this is business and entrepreneurship. I try to develop deep gears around how businesses work: strategy, distribution, how they all fit together, etc.

Secondly, it is important to develop deep gears around the most important problems you're focused on right now, so that you can understand them from every angle and frame things differently. Again, for me right now, my business is focused on solving procrastination. So I really want nuanced and varied models of procrastination.

But for most other scenarios, I think that having a gears-level model is a waste.

In general, I think that we learn mental models to solve problems. (There are other reasons as well, like prediction and generation, but I will not focus on them here). And we do that using this thing called the TRIZ Prism:

This is how we solve problems that people in our field haven't been able to solve: by using models from outside our field. So the more mental models we have, the more reframes we can do, and the more problems we can solve.

&...
Tearing down the Chesterton's Fence principle
by DirectedEvolution — 2020-08-02T04:56:43.339Z

If you saw a fence in the middle of nowhere, and it didn't seem to be fencing anything in or out, would you feel OK about tearing it down?

I like Chesterton's Fence. I've seen how it can completely change somebody's thinking, even when they're angry, in one sentence. The answer to the fence question is obvious, and the link to the issue that they're railing against is clear. It seems like a sturdy and effective general principle of rationality.

Memeology

But how strong is this idea? It's worth discussing, and we might use it as a heuristic. But have we studied it? And if not, can we in good conscious recommend it to others? How do we know it's not just a powerful and potentially destructive intuition pump? How can we study a meme for its benefits and drawbacks? Is there a formal way to measure that evidence? When would we feel comfortable recommending a meme to a loved one when it might have harmful side-effects? How do we determine memetic benefit or harm in others?

Of course, we can test the meme out on ourselves or our friends. There, we have intuitions about what will sit well with their mind.

We can also try to anticipate a priori when the meme might turn destructive. It would be interesting to do some sort of an experiment on a meme. But it's easy to do the a priori examination.

How could Chesterton's Fence be harmful?

It could be used to bully someone into thinking that they have less knowledge about the thing in question than they really do. Or that they need perfect knowledge before tearing something down. Or that the thing in question is owned in a way that's akin to property rights, so that you feel not only that you need to know whether it’s a useful “fence,” but whether you have permission to tear it down.

Alternatively, it might divert their attention from an angry yet curious attitude - a frustration that might lead them to investigate the fence - to a calm and accepting, yet embarrassed and incurious attitude. "Oh yes, you're right, of course. There's probably a reason for this fence, no need to look into it any further."

Chesterton's Fence could also provoke a false sense of confidence. Someone thinks for five seconds, comes up with an easy explanation for why the "fence" is there and a justification for why that doesn't matter, and now feels like they've got permission to destroy the "fence."

This meme could also become a way to shut off empath...
Dissolving the "Is the Efficient Market Hypothesis Dead?" Question
by Alexei — 2020-08-02T04:46:39.481Z

(This talk was given at a public online event on Sunday July 12th. Alexei is responsible for the talk, David Lambert edited the transcript.

If you're a curated author and interested in giving a 5-min talk, which will then be transcribed and edited, sign up here.)

Alexei Andreev: Recently, it's been popular on LessWrong to claim that the EMH is dead. I want to talk about why hearing that annoys me, and then propose something that I think is better than EMH.

Quickly about me: I'm a co-founder of Temple Capital, one of the top five biggest cryptocurrency quant hedge funds. I spent the last three years developing quant strategies. Hence I have a bit of a practical experience. But I don't study theory that much, so take this with a grain of salt.

There's two things that annoy me when people say that EMH is dead.

First: what version of EMH are you talking about exactly? There's a lot of formulations: weak form, semi-strong form, strong form, and, of course, the final ultimate mega form:

I have some issues with those formulations.

The main one is that they claim that “information is priced in”. I think it's obvious that it's not priced in, because who is pricing it in? It's entities, agents, hedge funds and traders that are doing the work of pricing it in. And every day, they're improving their systems, which means that yesterday they weren't pricing in something that they're pricing in today.

If you believe that they're actually doing reasonable pricing, then they're also doing reasonable improvements. The system is continuously getting more efficient. It's not binary.

If you imagine a super intelligent AGI, and you give it just the most easily accessible, publicly available prices, I think it will still outperform most human traders.

EMH can't really be dead. It can only be more or less efficient than you thought.

The second thing that annoys me when people say EMH is dead is: Why exactly do you think it is dead?

You have to read between the lines, and I think people are usually saying one of these two things.

A: "Well, COVID-19 was obviously going to be bad. The prices should have gone down earlier, but they went down later. So obviously, the markets can't be that efficient."

B: "Well, I made a bet at some point on the market and I made money, therefore the market isn't efficient."

Let me address both of these p...
Would AGIs parent young AGIs?
by Vishrut Arya — 2020-08-02T00:57:36.242Z

Question. In the world in which multiple AGIs obtain (i.e., not a singleton), is it likely these AGIs will be parents/mentors/guardians of younger AGIs?

Context. I was wondering whether parenthood (or, more minimally, guardianship) is an intrinsic property of the evolution of sufficiently intelligent agents.

The answer seems to depend, at least partly, on the stability of the environment and the returns to learning. My off-the-cuff reasoning follows.

1. It seems that a species (or, perhaps more generically, sufficiently similar information processing systems), can persist over time either by increasing their own lifespan or by creating new instances of the species.

2.a. It seems that if the environment is highly stable, then increasing lifespan and/or creating clones is attractive because the agent has found agent-environment fit. Returns to learning are low because the environment is already known. Returns to guardianship are low because the usefulness of a young agent existing is itself low.

2.b. If, on the other hand, the environment is highly unstable, then shorter lives seems more adaptive because natural selection will ensure agent-environment fit. Returns to learning are again low but this time for the opposite reason: facts now will not be relevant in the future. Returns to guardianship are low because the probability of survival of any one agent is low which makes guardianship investment a low-return activity.

2.c. Finally, if the environment is moderately stable, then there seems to be an optimal lifespan: one that enables a relatively long learning phase and a relatively long exploit phase. Returns to learning seem high as do returns to guardianship.

3. If AGIs will need relatively long learning phases, and, further, if young AGIs benefit from parenting/mentorship/guardianship (perhaps because they can be hurt by their environment, including by other AGIs), then it seems there's a fitness advantage to AGIs which attract parents/mentors/guardians.

Open to any feedback. Thanks!