trentmkelly/LessWrong-43k huggingface dataset:
The Defence production act and AI policy
by  — 2024-03-01T14:26:09.064Z

Quick Summary

Gives the President wide-ranging powers to strengthen the US industrial base
Has been around without changing that much since 1953
Has provisions which allow firms to make voluntary agreements that would normally be illegal under antitrust law
Provided the legal authority for many of the provisions in Biden’s recent Executive Order on AI

The

Defence Production Act

The Defence Production Act (DPA) has been reauthorised (and modified) by Congress since 1950, and in 1953 its powers very significantly reduced. I’m confident that it will continue to be passed - in a roughly similar form - for the foreseeable future. The current version was passed in 2019 under a Republican senate and is due for reauthorisation in 2025.

Since the Obama Presidency, there’s Republicans have begun to try to prevent bills proposed by Democrats from being passed by default. This is particularly easy for non-spending bills since for non-spending bills, 60 votes are needed to break the filibuster - a method used to prevent bills from being voted on - in the Senate. However, not only are defence bills consistently bipartisan, they have consistently high degrees of support from Republicans in particular. Therefore, I’m not concerned about the DPA not being passed by a Republican senate and a Democratic President when it's next due for reauthorisation.

The DPA gives the President very wide-ranging powers since the goal of the act of the act is to ensure that the US industrial base is strong enough to fight and win any war the US might need to undertake. Concretely, this allows the President to instruct firms to accept contracts; incentive expansion of the industrial base; and a grab bag of other specific provisions aimed at making sure that the US production base is strong enough to win a war.

Until 1953 the act was much more powerful and essentially allowed the President to take control of the US economy. The act now doesn’t give the President authority to set wage or price ceilings, control consumer credit or requisition stuff.

Antitrust provisions

Various AI governance proposals rely on explicit, voluntary agreements between leading AI labs. For instance,

this

paper proposes a scheme in which AI firms agree to pause the rollout out and training of large models if one doesn’t pass an evaluation which indicates that it could act dangerously....
Don't Endorse the Idea of Market Failure
by Maxwell Tabarrok — 2024-03-01T14:04:11.350Z
Link: https://www.maximum-progress.com/p/dont-endorse-the-idea-of-market-failure

In a fiery, though somewhat stilted

speech

with long pauses for translation, Javier Milei delivered this final message to a cheering crowd at the Conservative Political Action Conference last week:

Don't let socialism advance. Don't endorse regulations. Don't endorse the idea of market failure. Don't allow the advance of the murderous agenda. And don't let the siren calls of social justice woo you.

The

reactions on econ twitter

were unsurprisingly less positive than the CPAC crowd about calls to boycott market failure, one of the most well established facts in economics. James Medlock, for example,

begs libertarians to get a step past econ 101

.

The people cheering in the crowd and self-righteously quote tweeting on X are cheering for the wrong reasons. Medlock is correct that these credulous fans need to get a grip on the basics before they start denying established economic theories.

Milei, on the other hand, is an

accomplished academic economist

. Far from an infallible group, but we can be confident that he understands the concept of market failure well, and he’s right that we should treat it with suspicion.

What is the idea of market failure?

Market failures arise when a decision maker does not account for some costs or benefits that accrue to others. A West Virginia coal plant, for example, only considers the cost of fuel and electricity, not the cost of air pollution which accrues to people living in the surrounding mountain hollers.

In situations like this, West Virginians would be willing to pay coal mines to produce less and save them from the smog, but it is too costly to coordinate as a large group. These coordination frictions sully the conclusions of the

fundamental theorems of welfare economics

and drain value from the economy.

This is an uncontroversial example of the phenomenon of market failure, but the idea of market failure is more susceptible to motte-and-bailey strategies.

Here is the most common and easily defensible motte: Define “the idea of market failure” to be the claim that markets are not perfect; that there are > 0 market failures. This claim is true, and anyone who denies it plants themselves on the far left side of the mid-wit meme.

There are imperfections in every market and there are many markets with massive externalities. All the econ 101 examples are correct. Pollution, traffic, science and invention all impose costs and benefits...
Is it possible to make more specific bookmarks?
by numpyNaN — 2024-03-01T12:47:16.254Z

Pretty much what the title says, I've only seen a "general" bookmark option for posts (either it is bookmarked or it isn't) but even in my own bookmarks I don't seem to be able to group them in a way that might be useful i.e, can't separate "Things I want to have saved somewhere because they are really good/insightful" from "Things I want to read in more depth at a later date". There are more categories in which this might be handy, but these two seem pretty natural. Is there some way to do this I'm not seeing?
Wholesome Culture
by owencb — 2024-03-01T12:08:17.877Z

This is the third of a collection of three essays, ‘

On Wholesomeness

’.

Culture-consequentialism

What culture, if adopted, leads to the best consequences? This "culture-consequentialist" lens[1], applied to the idea of how cultures relate to wholesomeness, is the focus of this essay. The previous essays gave some reasons to like wholesomeness, and projected a vibe that it was good. But they didn't really consider tradeoffs.

This essay turns to that question. It is interested in whether it’s good at the individual level, and especially in whether it's good for communities to adopt a culture of acting wholesomely. Of course I can't think I'm offering the final word on this; instead I'll just give the cases I see on each side, and offer some concluding thoughts.

Non-fragility

So far, the main benefits of trying to act wholesomely that we’ve discussed relate to a kind of robustness. This is very stark in contrast with an all-in shut-up-and-multiply attitude. The problem there is that calculations are fragile. If you’re doing a lot of calculations, it’s pretty easy to accidentally lose or gain an order-of-magnitude in one of them — in the calculation itself, or in a bad parameter estimate, or in a confused assumption about the setup. And this can completely change your sense of what’s good or bad.

I think that that kind of fragility should be alarming when we’re just considering our own actions. If we consider choosing a culture for an entire community, it seems even more concerning. A community where everyone does their own calculations and acts on them is going to include some people haring off in wildly wrong directions — and some of these will probably be destructive.

By not relying on the calculations but on a more holistic sense of what’s good, we can keep things more robust. (This is blurring into a purely epistemic virtue; in doing any kind of calculation we may be more likely to catch errors if we keep in touch with a holistic sense of what the calculation is for and roughly how it should come out.)

The point about fragility is particularly stark for shut-up-and-multiply attitudes, but I think it applies much more broadly. People make mistakes, and then not-infrequently justify these mistakes to themselves — and to others. I’m sure that a culture focusing on wholesomeness wouldn’t be completely immune to this issue, but I do think that by construction ...
Adding Sensors to Mandolin?
by jefftk — 2024-03-01T02:10:06.930Z

I'd like to be able to play mandolin with my hands, drums with my
feet, and also choose the current bass note. Currently the closest I
can do is reach my left hand over to the piano to play a note, but
while this is possible with tunes that are not very notey and/or use a
lot of open strings it's pretty awkward:

Instead, I'd like to have some sort of sensors, buttons, or triggers
on the mandolin itself. That way it's right under my hands, and it
takes away much less from whatever else I'm doing. I think the main
candidate locations are:

Right hand, above the strings.

Right hand, below the strings.

Left hand, along the neck.

For (a) and (b), I could see either buttons that you tap with the pick
or a little stick you 'strum'. For (b) it could also work to tap
buttons with the fingers that aren't holding the pick. I'm
pessimistic about finding something that works for (c) without getting
in the way, but maybe tiny buttons on the top of the instrument would
be possible?

Someone made something like (b) for guitar (the Guitar
Wing) though I don't think the buttons are quite where I'd like
them:

I'm especially interested in the idea of something you interact with
using a pick, since that's what's in my hand already. I want to
mostly play mandolin, and then every so often send a MIDI signal.

Is this something anyone has been thinking about? Sensor suggestions?

(Another possibility would be to use the Yocto
3D V2 that I previously used for my pain-in-the-neck tilt hat,
mounting it on the headstock and have it sense changes in the
orientation of the instrument as a whole.)
The Parable Of The Fallen Pendulum - Part 1
by johnswentworth — 2024-03-01T00:25:00.111Z

One day a physics professor presents the standard physics 101 material on gravity and Newtonian mechanics: g = 9.8 m/s^2, sled on a ramp, pendulum, yada yada.

Later that week, the class has a lab session. Based on the standard physics 101 material, they calculate that a certain pendulum will have a period of approximately 3.6 seconds.

They then run the experiment: they set up the pendulum, draw it back to the appropriate starting position, and release. Result: the stand holding the pendulum tips over, and the whole thing falls on the floor. Stopwatch in hand, they watch the pendulum sit still on the floor, and time how often it returns to the same position. They conclude that the pendulum has a period of approximately 0.0 seconds.

Being avid LessWrong readers, the students reason: “This Newtonian mechanics theory predicted a period of approximately 3.6 seconds. Various factors we ignored (like e.g. friction) mean that we expect that estimate to be somewhat off, but the uncertainty is nowhere near large enough to predict a period of approximately 0.0 seconds. So this is a large Bayesian update against the Newtonian mechanics model. It is clearly flawed.”

The physics professor replies: “No no, Newtonian mechanics still works just fine! We just didn’t account for the possibility of the stand tipping over when predicting what would happen. If we go through the math again accounting for the geometry of the stand, we’ll see that Newtonian mechanics predicts it will tip over…” (At this point the professor begins to draw a diagram on the board.)

The students intervene: “Hindsight! Look, we all used this ‘Newtonian mechanics’ theory, and we predicted a period of 3.6 seconds. We did not predict 0.0 seconds, in advance. You did not predict 0.0 seconds, in advance. Theory is supposed to be validated by advance predictions! We’re not allowed to go back after-the-fact and revise the theory’s supposed prediction. Else how would the theory ever be falsifiable?”

The physics professor replies: “But Newtonian mechanics has been verified by massive numbers of experiments over the years! It’s enabled great works of engineering! And, while it does fail in some specific regimes, it consistently works on this kind of system - “

The students again intervene: “Apparently not. Unless you want to tell us that this pendulum on the floor is in fact moving back-and-forth with a period of approxi...
Gradations of moral weight
by MichaelStJules — 2024-02-29T23:08:58.271Z

Summary

Vagueness and gradations complicate the assignment of moral weights to potential moral patients and their interests.

I motivate and outline a general model of moral weights across possible moral patients to account for vagueness and gradations about what it means — the defining standards — to be a moral patient or realize a given kind of welfare on the given standard, like realizing certain functions, say self-awareness, to specific degrees of sophistication with a given precise method for calculating welfare (

more

).
I refine the model assuming maximizing expected choiceworthiness with intertheoretic comparisons of value and illustrate how subjective and poorly constrained expected moral weights can be on this model (

more

).
I both argue for and against the possibility that there are morally important standards humans don’t meet, and neither possibility seems extremely unlikely (

more

).

Acknowledgements

Thanks to Brian Tomasik and Bob Fischer for feedback. All errors are my own.

Gradations of consciousness

The Windows Task Manager may be a model of a computer’s own “attention”, and so an attention schema, and so be conscious to some degree, meeting a requirement of Attention Schema Theory (

Tomasik, 2014-2017

),[1] but you need to squint, and it seems at best only sort of true. Nonhuman animals fall somewhere between the Windows Task Manager and humans, having far more sophisticated versions of the capacities of Windows Task Manager and entire capacities it lacks entirely, but still, for most species, apparently lacking higher-order thoughts and self-narratives, which are typical of humans. We may also have different standards for what counts as a belief for the purpose of belief-like preferences or assign some special significance to the individual deciding how they judge their own life for global preferences (

Plant, 2020

).

Generated by DALL·E

There may be more than two degrees to which it can be the case that an animal or other system is conscious, has a (certain type of) welfare, or is otherwise a moral patient at all. There are multiple potentially defining features or functions — like higher-order thoughts, self-awareness of various kinds,[2] top-down attention control, beliefs, or effects on any of these — that may be independently present or absent or themselves realized to more than two degrees of sophistication or in different nu...
Approaching Human-Level Forecasting with Language Models
by Fred Zhang — 2024-02-29T22:36:34.012Z

TL;DR: We present a retrieval-augmented LM system that nears the human crowd performance on judgemental forecasting.

Paper: https://arxiv.org/abs/2402.18563 (Danny Halawi*, Fred Zhang*, Chen Yueh-Han*, and Jacob Steinhardt)

Twitter thread: https://twitter.com/JacobSteinhardt/status/1763243868353622089

Abstract

Forecasting future events is important for policy and decision-making. In this work, we study whether language models (LMs) can forecast at the level of competitive human forecasters. Towards this goal, we develop a retrieval-augmented LM system designed to automatically search for relevant information, generate forecasts, and aggregate predictions. To facilitate our study, we collect a large dataset of questions from competitive forecasting platforms. Under a test set published after the knowledge cut-offs of our LMs, we evaluate the end-to-end performance of our system against the aggregates of human forecasts. On average, the system nears the crowd aggregate of competitive forecasters and in some settings, surpasses it. Our work suggests that using LMs to forecast the future could provide accurate predictions at scale and help inform institutional decision-making.

For safety motivations on automated forecasting, see

Unsolved Problems in ML Safety (2021)

for discussions. In the following, we summarize our main research findings.

Current LMs are not naturally good at forecasting

First, we find that LMs are not naturally good at forecasting when evaluated zero-shot (with no fine-tuning and no retrieval). On 914 test questions that were opened after June 1, 2023 (post the knowledge cut-offs of these models), most LMs get near chance performance.

Here, all questions are binary, so random guessing gives a Brier score of 0.25. Averaging across all community predictions over time, the human crowd gets 0.149. We present the score of the best model of each series. Only GPT-4 and Claude-2 series beat random guessing (by a margin of >0.3), though still very far from human aggregates.

System building

Towards better automated forecasting, we build and optimize a retrieval-augmented LM pipeline for this task.

It functions in 3 steps, mimicking the traditional forecasting procedure:

Retrieval, which gathers relevant information from news sources. Here, we use LM to generate search queries given a question, use these queries to query a news corpus for articles, fi

...
Paper review: “The Unreasonable Effectiveness of Easy Training Data for Hard Tasks”
by Vassil Tashev — 2024-02-29T18:44:31.301Z

TL;DR: Scalable oversight seems easier based on experiments outlined in a recent paper; questions arise about the implications of these findings.

The following graciously provided feedback and advice on the draft, for which I am deeply grateful (in alphabetical order): Sawyer Bernath, Sam Bowman, Bogdan-Ionut Cirstea, Severin Field, Peter Hase, and Alfie Lamerton.

Introduction

In “The Unreasonable Effectiveness of Easy Training Data for Hard Tasks”,

Hase et al., (2024)

“study the problem of easy-to-hard generalization, which is relevant for determining how challenging the scalable oversight problem is in the first place, [and] present the surprising conclusion that current language models often generalize relatively well from easy to hard data, even performing as well as 'oracle' models trained on hard data.”

Methodology

The experiments aim to “elicit knowledge from models that we suspect they may know, using fundamentally weak supervision, [and feature] open models [Llama-2 base models, for sizes 7b, 13b, and 70b, Qwen-72b, and Mixtral-7x8b] and four publicly available question-answering datasets [ARC, MMLU, StrategyQA, GSM8k] with questions ranging in difficulty from 3rd grade science questions to college level STEM questions and general-knowledge trivia” and note that “if easy data is almost as good as hard data [and] if one cares most about model performance on hard data, [then] it can be better to collect and train on easy data rather than hard data, since hard data is generally noisier and costlier to collect.”

Findings

The authors “find that the Supervision Gap Recovered is usually between 70% and 100%, meaning that easy supervision is at least 70% as good as hard supervision for hard test performance. These results are robust across (1) model scale between 7b and 70b parameters, (2) six different human hardness measures and a model-based measure, (3) four datasets/tasks, and (4) several training methods including in-context learning with and without chain-of-thought reasoning

(Brown et al., 2020;

Wei et al., 2022)

, QLoRA

(Dettmers et al., 2023)

, and linear classifier heads

(Belinkov, 2022)

.”

Interestingly, unsupervised models do really well; for example, the Llama2-7b results show that the unsupervised model did 9% better than one trained o...
What’s in the box?! – Towards interpretability by distinguishing niches of value within neural networks.
by Joshua Clancy — 2024-02-29T18:33:42.811Z

Abstract

Mathematical models can describe neural network architectures and training environments, however the learned representations that emerge have remained difficult to model. Here we build a new theoretical model of internal representations. We do this via an economic and information theory framing. We distinguish niches of value that representations can fit within.

We utilize a pincer movement of theoretical deductions. First, we build a set of deductions from a top-down model of General Learning Networks. Second, we build a set of deductions from a model based on units of representation. In combination the deductions allow us to infer a “map” of valuable neural network niches.

Our theoretical model is based on measurable dimensions, albeit we simplify these dimensions greatly. It offers a prediction of the general role of a neuron based on its contextual placement and the neuron’s internal structure. Making these predictions practical and measurable requires software that has not yet been developed. Validating our predictions is beyond the scope of this paper.

Top-down direction:

We build the General Learning Network Model based on simple general principles.
We deduce a definition of internal value inherent to all learning networks.
We frame internal representations in information theory terms, which when combined with our general model, results in multiple abstraction metrics depending on the representation of interest.
This allows us to define the “signifier divide” which defines a boundary differentiating different representational domains.
These representational domains allow us to define “top-down” niches of value. Where neural structures within these different representational domains are specialized towards different roles.

Bottom-up direction:

We build the General Representative Inout Model based on simple general principles.
We isolate measurable dimensions in which we can compare the input-to-output mapping of different representational units.
We take these measurable dimensions and consider the extreme poles of these dimensions.
By combining the extreme poles of these measurable dimensions, we define polar archetypical neural structures. These define archetypal types of different neural input-to-output mappings. It is unlikely, your average neuron would display such extreme features, but together these arch

...
Short Post: Discerning Truth from Trash
by FinalFormal2 — 2024-02-29T18:09:42.987Z

I have read a lot of self-help in my time, which means I’ve also read a whole lot of bullshit. Worst than the actual process of ingesting and attempting to digest self-help bullshit was finding that my own writing started to produce features similar to self-help bullshit.

I would write advice that I had not attempted, which I knew would not work for me, and which I would never actually try myself.

I’m not unique in this respect- this type of writing infests the internet and literature at every level, but it’s not how it has to be.

I just wrote a post where I recommend hyperventilation in order to get in a better mindset for public speaking. This is something that I’ve actually done on multiple occasions and had results with. If you’re a nervous speaker or performer it’s something you should try at least once to see what you’re capable of- I expect you’ll be able to communicate at a much higher level than you previously thought possible.

This is the first mark of knowledge and the first heuristic for discerning truth from trash- Knowledge is power, therefore if you have no power you have no knowledge.

Archimedes said that if you gave him a lever long enough and a place to stand he could move the Earth, and that’s what knowledge is like- Knowledge is a lever to move the world.

Therefore I say to you my LessWrong followers- do not be like the Pharisees who read thick tomes in public and talk about obscure economic or metaphysical theories, but use your knowledge to gain mastery over yourself and your environment.

Evaluate all facts and models by their ability to help you achieve your goals, and you will never become so lost that you have only the image of intelligence to comfort you.