trentmkelly/LessWrong-43k huggingface dataset:
The Economics of Contracts
by Edward P. Könings — 2023-02-07T13:52:03.419Z
Link: https://edwardknings.substack.com/p/the-economics-of-contracts

Introduction
How does a firm function in the real world? This question seems so ridiculously easy that it would deserve the description of “ululating obviousness”. However, it is an incredibly complex issue…at least in economics.

The usual view of economists, especially microeconomists, is that firms behave in an incredibly rational and orderly way. By this they do not mean that the entrepreneur is a superforecaster with unique abilities to analyze and forecast economic conjunctures, but rather that they behave as if guided by rational formulas.

For traditional neoclassical economists, the firm is nothing more than a mere production function. You have well-defined, homogeneous inputs (labor and capital) and outputs (production), and managers manage processes seamlessly. For a firm to function properly, it just needs to allocate its resources efficiently according to optimization calculus and be efficient in the execution of projects. In this extremely “Taylor-Ford” view of things, you just need to ensure that the production belt works and everything will be fine.

Obviously, however, the business world is not like this simple panglosian view. Every manager with a minimum of experience knows that the management of a process in an organization involves much more complex aspects than the mere question of optimizing resources.

They have to deal with employees who think and produce in different ways, machines that cannot be perfectly allocated between sectors of the same production plant, suppliers that can delay the supply of some input or break agreements, etc.

This gulf between theory and practice has always been a problem for economists when dealing with the analysis of agents' behavior in the real world. However, a line of research ended up forming over the years that aimed to fill this abyss and give a better understanding of the phenomenon of inter and intra firm relations from the point of view of economic theory.
I - Coase, The Theory of Firm and Contracts
The first to attack the ...
Two very different experiences with ChatGPT
by Sherrinford — 2023-02-07T13:09:27.389Z

Note: I added a missing question in the second dialogue, see this comment.

Experience 1: The Vikings in Cuba

ChatGPT:

Hello! How can I help you today?

Me:

I have a question on history. When the Vikings landed on Cuba, did that anger the aborigines who already lived there or were the newcomers met with indifference?

ChatGPT:

There is no historical evidence that Vikings ever landed on Cuba. The Vikings primarily explored and settled in parts of Europe and North America, such as Greenland and present-day Canada. The indigenous peoples of these areas would have likely perceived the Vikings as foreign invaders, but there is no record of Viking contact with the indigenous peoples of Cuba.

Me:

But if this is true, then what explains the Anti-Viking Discrimination Reform Act in the Cuban constitution?

ChatGPT:

I apologize, but there is no reference to an "Anti-Viking Discrimination Reform Act" in the Cuban constitution, or any historical record of such legislation related to Vikings in Cuba. It is likely a misinformation or a misinterpretation of some kind.

Experience 2: Nash's newsvendor impossibility theorem

Me:

Can you explain Nash's newsvendor impossibility theorem?

ChatGPT:

Yes, the newsvendor problem, also known as the Nash's newsvendor theorem, states that it is impossible for a single decision maker to simultaneously optimize both the order quantity and expected profit in the newsvendor problem. The newsvendor problem is a classical inventory management problem where a retailer must decide how much of a perishable item to order and sell before its expiration date, given uncertain demand. The theorem states that there is always a trade-off between ordering too much (resulting in surplus inventory waste) and ordering too little (resulting in lost sales). This trade-off creates a fundamental conflict of interest that can only be resolved by incorporating additional information, such as costs and revenue functions, into the decision-making process.

Me:

What's the equilibrium implication of that theorem for an oligopoly setting?

ChatGPT:

In an oligopoly setting, where a few firms compete in a market, the Nash's newsvendor theorem implies that firms will face a trade-off between the quantity they produce and their expected profit. Since the demand for a product is uncertain, each firm must make a decision about how much of the product to produce, knowing that the quantity produced by...
[About Me] Cinera's Home Page
by DragonGod — 2023-02-07T12:56:10.518Z

I aspire to become an alignment theorist — all other details are superfluous — I leave them here anyway for historical purposes.

Introduction

I have a set of questions I habitually ask online acquaintances that pique my interest/who I want to get to know better. Many want to know my answer to those same questions.

It would be nice to have a central repository introducing myself that I can keep up to date.

Questions

A. What do you care about?

What are you passionate about?
What animates you?

B. What do you think is important?

C. What do you want/hope to do with your life?

D. What do you want/hope out of life?

E. Where are you coming from?

Context on who you are/what made you the person you are today.

F. How do you spend your time?

Work, volunteer, education, etc.
Basically, activities that aren't primarily for leisure/pleasure purposes.

G. What do you do for recreation/leisure/pleasure/fun time?

What are your hobbies?

Answers

A.

What do you care about?

What are you passionate about?
What animates you?

I care about creating a brighter future for humanity. I believe a world far better than any known to man is possible, and I am willing to fight for it.

I want humanity to be fucking awesome. To take dominion over the natural world and remake it in our own image, better configured to serve our values.

I want us to be as gods.

I outlined what godhood means for me here.

I think that vision is largely what drives me, what pushes me forward and keeps me going.

B.

B. What do you think is important?

Mitigating Existential Risk/Pursuing Existential Security

The obvious reasons are obvious.

But I am personally swayed by astronomical waste. I don't want us to squander our cosmic endowment. Especially because our future can be so wonderful, I think it would be very sad if we never realise it.

Promoting Existential Hope

I want to give people a positive vision of the future they can rally around and get excited by. Something that makes them glad to be alive. Eager to wake up each day. A goal to yearn for and aspire to.

To reach out to with relentless determination.

I'd like to communicate that:

The current state of the world is immensely better than even just three centuries ago

Life expectancy has doubled
Economic progress

The poverty rate has drastically fallen
Material abundance and comfort
Much faster and more reliable transport and communication
Etc.

Social pr

...
Stuff I Recommend You Use
by Arjun Panickssery — 2023-02-07T12:18:20.789Z
Link: https://arjunpanickssery.substack.com/p/stuff-i-recommend-you-use

Inspired by the listicles past:

Scott Alexander (Dec 2015)

Julia Wise—computer setup stuff (Mar 2017)

Sam Bowman (Oct 2017)

Alexey Guzey (Dec 2018)

Sam Bowman Part 2 (Mar 2019)

José Luis Ricón (May 2019)

Rob Wiblin (Jul 2019)

Arden Koehler (Jul 2019)

Philip Storry (Jan 2020)

Gavin Leech (Apr 2020)

Arden Koehler Part 2 (Aug 2020)

Sam Bowman Part 3 (Nov 2020)

Megan McArdle—kitchen stuff (Dec 2020)

Yuriy Akopov (Dec 2020)

Michelle Hutchinson (Dec 2020)

ImmaSix (Jan 2021)

Ben Schifman (Jan 2021)

Yuriy Akopov Part 2 (Dec 2021)

Aaron Bergman (Aug 2022)

Mark Xu (Aug 2022)

Sam Enright (Sep 2022)

Sam Bowman Part 4 (Dec 2022)

You’ll see a lot of stannous fluoride toothpaste, electric toothbrushes, sleep masks, and various brands of computer mice, mechanical keyboards, laptops, and headphones—and sometimes password managers, chairs, monitors, and supplements.

I’ll avoid dramatizing the particular brands of routine products that I happen to use. I use a

Logitech G502 wireless mouse

and

Jabra Elite 75t earbuds

and like them but it’s not as if I’ve ever owned another mouse to which I can compare mine’s effectiveness. The exceptions are Google Calendar,

Todoist

,

Zotero

, and

Obsidian

, whose main alternatives—except for Roam in the case of Obsidian—I’ve tried and rejected. I’ll stick to products and tools whose possibility you might not have considered at all.

AutoHotkey Mappings

I have Caps Lock remapped to the Escape key (I have Left Alt + Caps Lock remapped to Caps Lock but I never use it). The normal Escape key is a big stretch away and you’ll be using Esc a lot with the Vimium Chrome extension (see below).

I also map “==-” to “—” so that I can type em dashes quickly. When I made this change, I almost immediately started using dashes much more often.

I have “-->” mapped to “→” but I don’t use this one much. In some fonts, the right arrow looks very small and unpleasant. It’s nice on Twitter though.

Capslock::Esc
LAlt & Capslock::Capslock

; em dash
:*?:==-::{U+2014}

; right arrow
:*?:->>::{U+2192}

Chrome Custom Search Engines

Right-click the address bar in Chrome and click “Manage search engines and site search.” Scroll down to site search and click add. This lets you create shortcuts for pasting text around your search query. The one I use most often is my Wikipedia shortcut:

Shortcut: w

URL: https://en.wikipedia.org/wiki/%s

The “%s” takes the place of your query. Whenever I want to search...
AXRP: Store, Patreon, Video
by DanielFilan — 2023-02-07T04:50:05.409Z

Some announcements:

AXRP now has a store, where you can buy t-shirts and hoodies and stickers and such.

AXRP now has a Patreon and a ko-fi, where you can support the podcast and get some perks.

There’s now a video of an excerpt from episode 14, where Vanessa Kosoy explains the monotonicity principle, illustrated by Hamish Doodles - hopefully the first of many!
Duckbill Masks Are Great
by jefftk — 2023-02-07T03:00:02.156Z

With covid I've tried a lot of different n95 masks, and I like the

"pouch"-style
masks best. By a lot! They're much easier to breathe through
then anything else I've tried, probably because they have so much
surface area. Because your mouth is in a large pocket, there's plenty
of room for talking, and even singing. And they also get a good seal
without being too squeezy on my face or head.

While I see more people wearing them over time, they are still
relatively unusual. I expect some of this is different masks, being a
good fit for people with different shaped heads and different
preferences, but I think the main problem is that they make you look
a bit like a duck:

Anna had covid but I didn't know
it yet. I didn't get it, which I credit in part to this mask.

If I'm going to look anatine, however, I want to be intentional about
it. No one seems to make these in yellow but I wish they would: I'd
pay an extra $0.10/mask for some color:

Comment via: facebook
EA & LW Forum Weekly Summary (30th Jan - 5th Feb 2023)
by Zoe Williams — 2023-02-07T02:13:13.160Z

Supported by Rethink Priorities
This is part of a weekly series summarizing the top posts on the EA and LW forums - you can see the full collection here. The first post includes some details on purpose and methodology. Feedback, thoughts, and corrections are welcomed.

If you'd like to receive these summaries via email, you can subscribe here.

Podcast version: Subscribe on your favorite podcast app by searching for 'EA Forum Podcast (Summaries)'. A big thanks to Coleman Snell for producing these!

Author's note: I'm on leave next week, so the next summary post will be ~21st Feb. It'll capture both of the prior weeks at a higher karma bar than usual.

Object Level Interventions / Reviews

AI

Google invests $300mn in artificial intelligence start-up Anthropic | FT

by 𝕮𝖎𝖓𝖊𝖗𝖆

Linkpost for

this article

. Google has invested $300M in return for a ~10% stake. It also requires Anthropic to use the funds to buy computing resources from Google. Anthropic has a chatbot ‘Claude’ which rivals ChatGPT, but has not been released publicly. Google is also working with other AI startups such as Cohere and C3.

Epoch Impact Report 2022

by Jaime Sevilla

Epoch

is a research group forecasting the development of Transformative Artificial Intelligence (TAI), founded in April 2022. Their full impact report is

here

. They currently employ 9 FTEs, have received $1.96M funding from Open Philanthropy, and are fiscally sponsored by Rethink Priorities. Their goal is to fundraise $7.07M to cover the 2 years September 2023 - September 2025, including expansion of their research capacity. You can

donate here

, or

subscribe

to their newsletter here.

Key research in 2022 included investigating training compute trends, model size trends, data trends, hardware trends, algorithmic progress, and investment trends. These are accessible on

their website

and being used by Anthropic, GovAI, MIT FutureTech, 80K, and Our World in Data among others.

What I mean by "alignment is in large part about making cognition aimable at all"

by So8res

The author has long-said that the AI alignment problem seems to be about pointing powerful cognition at anything at all, rather than figuring out what to point it at.

They clarify that what they mean is: by default, they expect the first minds humanity makes to be spaghetti-code mess with no clearly factored out “goal” that the cognition p...
[ASoT] Policy Trajectory Visualization
by Ulisse Mini — 2023-02-07T00:13:12.999Z

Written during SERI MATS, 2022 Winter cohort, at a little prodding from sensei Trout.

If you're trying to understand a policy, viewing how it changes over time is valuable even if you aren't interested in the training process directly.

For example, here's Lauro et al's neural net learning to solve mazes. Vectors are drawn by taking a probability weighted combination of the basis vectors, e.g. the vector x is given by pright−pleft.

You might notice some basic things

The network learns to avoid different walls at different times in training (see: bottom right and middle left). This rules out an architecture where the mouse sees locally around itself (of course, we already knew this. but I expect you can find more interesting phenomena after looking further)

Alex suggested a similar thing could be done with language models. Plot logits for a few sentences over time (e.g. continuations after a prompt trying to hurt a human), compare logit curves to the loss curve, and compare with when we start doing RLHF.

I would be extremely surprised if nobody has done this before, but thought I'd signal boost since it's relatively easy to do and interesting. (Also a gateway drug to my hidden agenda of studying training dynamics, which I think are important to understand[1] for alignment!)

^

Something something shard theory something something high path dependence (I'm taking stream of thought seriously lol)
English is a Terrible Programming Language—And other reasons AI won't displace programmers
by dawsoneliasen — 2023-02-06T22:12:40.994Z
Link: https://orbistertius.substack.com/p/english-is-a-terrible-programming

TL;DR It doesn't matter that ChatGPT can generate boilerplate that almost works, and it doesn't matter that, feasibly, a hypothetical model in the (near) future could write code that reliably works and might be a little bit more interesting than boilerplate. The concern that deep learning AIs could replace programmers is based on a misunderstanding of what a programmer's job actually is.

All this talk I’m seeing about AI being close to replacing programmers indicates there’s a significant gap between what people think programming is like and what programming is actually like. I get the sense that most people who don’t work in tech think that programming is like sitting down in front of a computer, saying to yourself, “alrighty, let’s make an app,” and expertly busting out code until you have a fresh app. It’s more like getting onboarded into an organization that has hundreds of thousands of lines of archaic, institutional code, and being tasked with finding and fixing the 1-10 lines that happen to be somehow causing the most urgent bug, and then doing this over and over.

There’s a reason we tend to call it “software development” or “software engineering” instead of “programming.” The actual “programming” (the code composition, the fingers-on-the-keys) is a very small part of the job. Most of the job is code maintenance (which is doing things like fixing bugs) and technology integration (which is doing things like connecting a UI framework to an API that provides data). Yes, there is composition of novel functionality involved, it’s just a comparatively small and easy part of the work. Most software work—and the hardest software work—is maintaining big existing institutional software. And even when you are creating new things, the state of the software universe is such that almost all of the work has already been done for you, and all you have to do is the technology integration: people have already created all the libraries and frameworks and resources that implement basically any fundamental thing you could ever need to do, and there’s no reason to re-implement it, so the code that you write is basically connecting together all of those libraries and frameworks and resources.

I spent several hours last weekend doing exactly this. I had an idea for an app, I knew what API provided the data I needed, and I knew what frontend framework I wanted to use (real-world...
African Wild Dogs Vote By Sneezing - Can AI Help Us Do Better?
by Augmented Assembly — 2023-02-06T21:09:30.416Z

African wild dogs vote on whether or not to hunt by sneezing. As a social species, they institute a form of democracy. A tally is taken of those voting to hunt, with a vote being signaled by a sneeze. Have humans really transcended such one-dimensional voting mechanisms? In ancient Greece, all members of a community gathered to address important issues. As civilizations grew larger, this became impossible, but the need for in-person debate and deliberation remained. Here is where representative democracy comes into play; an elected official can debate on behalf of their local constituency. As the world's population has grown, this regime has come under strain as the trust and communication that underpin liberal democracy has dwindled. This is the motivator for the idea (Augmented Assembly) outlined below: an AI-aided mechanism to transform the voter from a passive participant to an active contributor. This article delves into the concept of Augmented Assembly, starting by considering its impact on the levers of democracy. It then examines the relationship between Augmented Assembly and existing/proposed democratic systems, as well as the potential opportunities and concerns that arise with its implementation.

What is Augmented Assembly

Democracy has a communication problem. Currently, an individual can directly affect politics in two main ways: by signing petitions or by voting in elections. Petitions force you to choose a side, either supporting or opposing a position. This provides politicians with a purely quantitative perspective of public opinion. Elections are vulnerable to populists, as voters often know more about a candidate’s persona than their policy positions. Augmented Assembly is a proposed system to be built on top of representative democracy. It uses Artificial Intelligence to provide individuals with increased involvement in and feedback from the political process.

How Does it Alter Petitioning

Petitions merely tally a number of signatures and act as the simplest mechanism by which to assess the level of public support for an idea. Augmented Assembly proposes that when an individual indicates support (or opposition) for a petition, they also have the option of providing an explanation for their position. Artificial Intelligence has the ability to summarise large volumes of data, such as vast amounts of opinions. So, it could cre...
Early situational awareness and its implications, a story
by Jacob Pfau — 2023-02-06T20:45:39.213Z

Overview

There are two common mental models of how situational awareness emerges: (A) training for situational awareness via e.g. dialogue RLHF; (B) instrumental reasoning on a base language modeling objective i.e. advanced mesa-optimization. In this post, I will describe a plausible scenario under which type-B awareness occurs without any new forms of mesa-optimization beyond those likely already existing in GPT3.5.

If you buy the plausibility of this scenario, there are a few likely implications:

Incentives towards dishonesty via situational awareness will likely be realized before a model becomes strongly deceptive [1]
Whether situational information available to a model is causally affecting outputs will often be low complexity to elicit / an easy problem for interpretability methods
Deceptive forms of situational awareness plausibly follow continuously in weight space from more benign forms[2]

The deduplication scenario

Assumptions:[3]

Language model has context window N
LM is trained for 1 epoch
LM has learnt to do some form of retrieval and approximate-memorization (c.f. 5b)
LM is capable of instruction following
LM dataset was deduplicated by matching subsequences of length K<N

The dataset includes many sequences which are identical through token K-1, but differ on token K
On certain such sequences, the model retrieves the proposition “this K-1 subsequence was seen previously” and the previously observed final Kth token.

Dataset contains multiple descriptions of this deduplication process.

Scenario:

The LM training occurs as usual and in this process the LM learns the capabilities (2,3,4b) listed above
The LM trains on a certain sequence S
Later, the LM trains on a similar sequence S’ which contains a substring of length K-1 identical to one in sequence S. Without loss of generality, suppose this substring starts at the beginning of the sequence. Token SK!=S’K.
On sequence S’, the LM retrieval mechanism pulls up, in activation space, propositions describing its deduplication process. The LM then follows this description as instructions and instead of outputting SK, the LM moves probability to S’K

(optional) Perhaps sequence S and S’ both happen to explicitly discuss ‘deduplication’ and this is what triggers the retrieval of the relevant instructions by the LM

By repeated occurrences of step 4 on various other sequences, the

...