trentmkelly/LessWrong-43k huggingface dataset:
The predictive power of dissipative adaptation
by dr_s — 2023-12-17T14:01:31.568Z

There is an idea.

The idea, which has proven somewhat memetically sticky and is at the core of the so-called "effective accelerationist" (in short, e/acc) movement, is that there is a simple and powerful principle of non-equilibrium thermodynamics which favors the proliferation of ever more complex, intelligent and productive life (and life-like super-organisms, like countries or markets) over the alternative. It is phrased as follows in Beff Jezos "Notes on e/acc principles and tenets":

e/acc is about having faith in the dynamical adaptation process and aiming to accelerate the advent of its asymptotic limit; often reffered to as the technocapital singularity

Effective accelerationism aims to follow the “will of the universe”: leaning into the thermodynamic bias towards futures with greater and smarter civilizations that are more effective at finding/extracting free energy from the universe and converting it to utility at grander and grander scale

This idea originates in the work of physicist Jeremy England, and in particular in a paper which is explicitly cited in Jezos' post, "Statistical Physics of Adaptation". It's an open access paper so anyone with the necessary knowledge (the basics of statistical mechanics will do) and good will can easily go check it by themselves. I suggest using the linked final PRX version rather than the arXiv preprint, as the latter seems to have some significant difference that have been edited in the review process, and I found the final version much clearer (which suggests the editors and referees did their job well, so props to them).

Now, there are obvious philosophical criticisms one can move to this view, mainly that it seems to derive an ought from an is; even if Nature somehow preferred life that grows unchecked over every alternative, it does not mean we should stick to that preference any more than we did to its penchant for infant death; and if there's a choice to begin with for us to make, obviously it can't be that strong a preference. If you're thrown off an airplane in flight, you don't usually get to pick whether to obey or not the law of gravity. However, here I want to focus on the strength of the physical claim itself. I've read England's paper as well as some other related works and I want to go over its claims in detail, what they imply, to what kind of contexts they apply, and what they can't instead be used ...
Linkpost: Francesca v Harvard
by Linch — 2023-12-17T06:18:05.883Z
Link: https://www.francesca-v-harvard.org/

Just when I thought the Gino case came to a close, I started reading Francesca Gino's defense. It paints her as a belabored academic unfairly penalized by misleading and incompetent investigators, as well as a biased New Yorker reporter.

I'm reposting this here as I think it's useful to see both sides of every case, in an interesting parallel to recent, more local, events.

_____
Breaking My Silence

I have been sitting in near silence for over three months, since learning that HBS was putting me on unpaid administrative leave, and banning me from campus, teaching and research. It has been shattering to watch my career being decimated and my reputation completely destroyed. It has been hard to see how this situation impacted those around me - my family, my mentors, my collaborators and my students.

The information that has been available to the public, and the analysis posted by critics, may sound compelling. But the information is incomplete and misleading. The record needs to be corrected. This website is my attempt to do so.

Let that correction begin with this simple and unambiguous statement: I absolutely did not commit academic fraud.
____

In an earlier post, I refuted Data Colada’s critique of the PNAS paper by demonstrating that:

Data Colada cherry-picked the data it chose to include in its analysis,

Data Colada excluded a third condition in the study,

Data Colada excluded 2 of the 3 dependent variables in its analysis, and

Data Colada completely misrepresented the Excel calcChain function to buttress their claims.

Together, this allowed Data Colada to create an enormously misleading impression about my work. Importantly, I also demonstrated that if one were to exclude all of the data observations that Data Colada claimed should have been considered “suspicious,” the findings of the study still hold, suggesting there was no motivation for me to manipulate data.

In a more recent post, Data Colada doubled down on their claims, arguing that HBS’ investigation found additional evidence of data manipulation in this particular study.

What is this additional evidence?

In a nutshell, HBS claimed the following:

HBS claimed that it was able to obtain the original (unpublished) dataset for the study in question.

HBS claimed that, with the help of a forensics firm, it was able to compare this original dataset to the dataset published on OSF.

HBS claimed that this comparison yie...
Lessons from massaging myself, others, dogs, and cats

by Chris Lakin — 2023-12-17T04:28:40.080Z
Link: https://chipmonk.blog/p/lessons-from-massaging-myself-others?r=bgw61&utm_source=lesswrong&utm_medium=referral&utm_campaign=post

You can just massage yourself. Then once you learn to massage yourself massaging other people, dogs, and cats is almost the same.

I’m sure there are a bazillion youtube videos about how to massage yourself, but I haven’t watched any of them. I just started squeezing my muscles and did more of what feels nice.

In the last year I’ve taught ~50 people how to massage, too. More recently I’ve been teaching people how to massage others:

It’s super easy:

Find your muscles.
Squeeze them with more pressure than you think. I’m not going to specify precisely how to do this. **Just explore, be curious, and try random things.**
Follow your intuition. Do more of what feels nice.

Complexities:

Make sure that any muscle you massage is relaxed before you massage it. This is actually hard to do.

If massaging any area hurts, it’s nearly always because the muscle is tensed in some way. Usually this can be resolved by repositioning the body part in a way that it isn’t supporting itself. (E.g.: Don’t massage arms/legs while they are supporting themself in midair)

Squeezing fat doesn’t feel nice.

If you have a fat on your arm for example, you have to reach through the fat to the muscle to squeeze it.

Eventually you learn how much force to apply to a muscle in accordance to how large the muscle is.
Massage ability might be very dependent on grip strength. I’ve been told that I have hands of iron, so maybe I’m not skilled so much as just strong.
Massaging non-muscle body-parts sometimes also feels nice. Lightly rubbing the very thin amount of soft matter on top of bones can be quite nice.
There’s probably a bunch of other tacit knowledge I’m missing. Feel free to ask me to massage your arm hand etc. sometime.

Massaging others:

If you want to massage other people, my main advice is that you learn to massage yourself or at least that body part of yourself first. Then you understand how it works and can apply that to others. It’s like 70% the same.
When massaging other people, my rule of thumb is to increase the pressure until they say “ow”. Then decrease the pressure by 5%. Do this every time you go to a new area.

If they aren’t saying “ow” you aren’t being ambitious enough.

When first starting to massage a body part and they’re not used to you, start off with less pressure. Foreplay

Same thing also if they’re ticklish. Just rest your hands there with as much pressure as you can do without tickling them f

...
The Serendipity of Density
by jefftk — 2023-12-17T03:50:04.824Z

One thing I really appreciate about living in Somerville is how
density facilitates serendipity. Higher density means more of the
places we might want to go are within walking distance, and then
walking (and high density again) means we're more likely to run into
friends.

This afternoon we were walking Lily (9y) to a sleepover and we passed
one of the nearby playgrounds. Lily and Anna (7y) saw some school
friends and ran ahead to say hi. Anna wanted to stay and play, and I
asked one of the parents if they'd be up for having Anna while I
finished the dropoff. They were happy to (we take each other's kids a
decent amount) and Anna got to have a much more fun half hour. Then,
after I got back, we hung out at the playground with friends for a
while longer before heading home for dinner.

This certainly would have been possible to arrange intentionally, with
a bunch of communication, but if we had been driving I expect Anna
wouldn't have ended up spending this time with her friends.
I would guess that the majority of times we go out we run into
someone, though it's much more common that we stop and hang out
together than leave one of the kids and split up.

This sort of thing wasn't a common experience at all in the West
Medford neighborhood I grew up in. It's an area not far from here, but
probably only a third the density: the lots are about twice the size
and there are more single-family houses. Distances meant we would
usually drive, and even if our family had decided to primarily walk
you don't run into friends when out walking unless your friends also
tend to be out on foot. Another contribution is also that our kids'
school is very close and draws mostly from the immediate neighborhood,
which means their friends are almost all within easy walking
distance. Whereas my elementary school drew from all over Medford and
I only had one school friend whose house I could walk to, and only for
two of the six years.

Americans often worry that increasing density would lead to their
neighborhoods becoming impersonal. As someone who lives in one of the
densest municipalities in the country, I think this is backwards:
proximity fosters community.
Bounty: Diverse hard tasks for LLM agents
by Beth Barnes — 2023-12-17T01:04:05.460Z

Update 3/14/2024: This post is out of date. For current information on the task bounty, see our Task Development Guide.

Summary

METR (formerly ARC Evals) is looking for (1) ideas, (2) detailed specifications, and (3) well-tested implementations for tasks to measure performance of autonomous LLM agents.

Quick description of key desiderata for tasks:

Not too easy: would take a human professional >2 hours to complete, ideally some are >20 hours.
Easy to derisk and ensure that there aren't weird edge cases or bugs: we want to be able to trust that success or failure at the task is a real indication of ability, rather than a bug, loophole, or unexpected difficulty in the task
Plays to strengths of LLM agents: ideally, most of the tasks can be completed by writing code, using the command line, or other text-based interaction.
See desiderata section for more explanation of what makes an ideal task.

Important info

WE DON'T WANT TASKS OR SOLUTIONS TO END UP IN TRAINING DATA. PLEASE DON'T PUT TASK INSTRUCTIONS OR SOLUTIONS ON THE PUBLIC WEB
Also, while you're developing or testing tasks, turn off chatGPT history, make sure copilot settings don't allow training on your data, etc.

If you have questions about the desiderata, bounties, infra, etc, please post them in the comments section here so that other people can learn from the answers!
If your question requires sharing information about a task solution, please email

task-support@evals.alignment.org

instead.
If you're planning on submitting a task, would be great if you can let us know in the comments.

Exact bounties

1. Ideas
Existing list of ideas and more info

here

.
We will pay $20 (as 10% chance of $200) if we make the idea into a full specification.
Submit ideas here.

2. Specifications
See example specifications and description of requirements

here

.
We will pay $200 for specifications that meet the requirements and that meet the desiderata well enough that we feel excited about them.
Submit specifications here.

3. Task implementations

See guide to developing and submitting tasks

here

.
Update from 2/5/2024: For tasks that meet the core desiderata we’ll pay a base price of 3x the estimated $ cost of getting a human to complete the task, plus a %-based bonus based on how well the tasks performs on the other desiderata (e.g threat-model relevance, diversity, ease...
2022 (and All Time) Posts by Pingback Count
by Raemon — 2023-12-16T21:17:00.572Z

For the past couple years I've wished LessWrong had a "sort posts by number of pingbacks, or, ideally, by total karma of pingbacks". I particularly wished for this during the Annual Review, where "which posts got cited the most?" seemed like a useful thing to track for potential hidden gems.

We still haven't built a full-fledged feature for this, but I just ran a query against the database, and made it into a spreadsheet, which you can view here:

LessWrong 2022 Posts by Pingbacks

Here are the top 100 posts, sorted by Total Pingback Karma

Title/Link	Post Karma	Pingback Count	Total Pingback Karma	Avg Pingback Karma

AGI Ruin: A List of Lethalities

870	158	12,484	79

MIRI announces new "Death With Dignity" strategy

334	73	8,134	111

A central AI alignment problem: capabilities generalization, and the sharp left turn

273	96	7,704	80

Simulators

612	127	7,699	61

Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover

367	83	5,123	62

Reward is not the optimization target

341	62	4,493	72

A Mechanistic Interpretability Analysis of Grokking

367	48	3,450	72

How To Go From Interpretability To Alignment: Just Retarget The Search

167	45	3,374	75

On how various plans miss the hard bits of the alignment challenge

292	40	3,288	82

[Intro to brain-like-AGI safety] 3. Two subsystems: Learning & Steering

79	36	3,023	84

How likely is deceptive alignment?

101	47	2,907	62

The shard theory of human values

238	42	2,843	68

Mysteries of mode collapse

279	32	2,842	89

[Intro to brain-like-AGI safety] 2. “Learning from scratch” in the brain

57	30	2,731	91

Why Agent Foundations? An Overly Abstract Explanation

285	42	2,730	65

A Longlist of Theories of Impact for Interpretability

124	26	2,589	100

How might we align transformative AI if it’s developed very soon?

136	32	2,351	73

A transparency and interpretability tech tree

148	31	2,343	76

Discovering Language Model Behaviors with Model-Written Evaluations

100	19	2,336	123

A note about differential technological development

185	20	2,270	114

Causal Scrubbing: a method for rigorously testing interpretability hypotheses [Redwood Research]

195	35	2,267	65

Supervise Process, not Outcomes

132	25	2,262	90

Shard Theory: An Overview

157	28	2,019	72

Epistemological Vigilance for Alignment

61	21	2,008	96

A shot at the diamond-alignment problem

92	23	1,848	80

Where I agree and disagree with Eliezer

862	27	1,836	68

Brain Efficiency: Much More than You Wanted to Know

201	27	1,807	67

Refine: An Incubator for Conceptual Alignment Research Bets

143	21	1,793	85

E

...
"Humanity vs. AGI" Will Never Look Like "Humanity vs. AGI" to Humanity
by Thane Ruthenis — 2023-12-16T20:08:39.375Z

When discussing AGI Risk, people often talk about it in terms of a war between humanity and an AGI. Comparisons between the amounts of resources at both sides' disposal are brought up and factored in, big impressive nuclear stockpiles are sometimes waved around, etc.

I'm pretty sure it's not how that'd look like, on several levels.

1. Threat Ambiguity

I think what people imagine, when they imagine a war, is Terminator-style movie scenarios where the obviously evil AGI becomes obviously evil in a way that's obvious to everyone, and then it's a neatly arranged white-and-black humanity vs. machines all-out fight. Everyone sees the problem, and knows everyone else sees it too, the problem is common knowledge, and we can all decisively act against it.[1]

But in real life, such unambiguity is rare. The monsters don't look obviously evil, the signs of fatal issues are rarely blatant. Is this whiff of smoke a sign of fire, or just someone nearby being bad at cooking? Is this creepy guy actually planning to assault you, or you're just being paranoid? Is this weird feeling in your chest a sign of an impending heart attack, or just some biological noise? Is this epidemic truly following an exponential curve, or it's going to peter out somehow? Are you really, really sure the threat is real? So sure you'd actually take drastic actions — call emergency services, make a scene, declare a quarantine — and risk wasting resources and doing harm and looking foolish for overreacting?

And if you're not that sure, well...

Better not act up. Better not look like you're panicking. Act very concerned, sure, but in a calm, high-status manner. Provide a measured response. Definitely don't take any drastic, unilateral actions. After all, what if you do, but the threat turns out not to be real? Depending on what you've done, the punishment inflicted might range from embarrassment to complete social ostracization, and the fear of those is much more acute in our minds, compared to some vague concerns about death.

And the AGI, if it's worth the name, would not fail to exploit this. Even when it starts acting to amass power, there would always be a prosocial, plausible-sounding justification for why it's doing that. It'd never stop making pleasant noises about having people's best interests at heart. It'd never stop being genuinely useful to someone. It'd ensure that there's always clear, unambigu...
cold aluminum for medicine
by bhauth — 2023-12-16T14:38:03.260Z
Link: https://www.bhauth.com/blog/industrial%20design/cold%20aluminum.html

cold aluminum

Very pure aluminum at the boiling point of hydrogen is a very cool material. At 20K, 99.999% pure aluminum has 1000x the electrical conductivity it has at 0 C. At 4K, it has maybe 4000x.

At such low temperatures, electron free paths in aluminum become macroscopic, which is why even small amounts of impurities greatly increase resistance. Even wire diameter has a noticeable effect. Magnetic fields can also increase resistance, but this is also a purity-dependent effect: 99.999% aluminum might have 3x the resistance at 15T, but even purer aluminum is much less affected.

Yes, aluminum purification costs some money, but it's not particularly expensive. It might cost 3x as much as standard aluminum, but it's far cheaper than superconductors. Another point to note is that superconductors only have 0 resistance for constant current. Cryogenic aluminum isn't affected by current changes like superconductors are.

It seems like that sort of interesting effect that massively increases a figure of merit should have some sort of application, don't you think? Yet, while there are superconducting electric motors for ships, I'm not aware of cryogenic aluminum conductors being used for any commercial applications. What could it be used for?

power lines

An obvious application for low-resistance conductors is long-distance power transmission. My estimations indicated that using cryogenic aluminum for that is somewhat too expensive, because the (cryocooler cost)*(insulation cost) product is too high for reasonable line currents. Connecting it to ambient-temperature lines is also an issue, because cold pure aluminum also has high thermal conductivity.

As temperatures decrease, resistance decreases, but cryocoolers become more expensive and less efficient. In general, liquid hydrogen seems better than liquid helium or liquid nitrogen for cryogenic aluminum conductors.

At such temperatures, it's worth using multilayer vacuum insulation. That's far more effective than typical insulation like fiberglass or polyester, but it still doesn't seem good enough to make insulation + cryocoolers sufficiently cheap for large underground power lines.

While the economics don't work out, it is possible to use cryogenic aluminum for high-power electricity transmission. It's merely expensive, not unfeasible. Feel free to use that for flavor in hard SF stories.

What are some attributes of app...
Scalable Oversight and Weak-to-Strong Generalization: Compatible approaches to the same problem
by Ansh Radhakrishnan — 2023-12-16T05:49:23.672Z

Thanks to Roger Grosse, Cem Anil, Sam Bowman, Tamera Lanham, and Mrinank Sharma for helpful discussion and comments on drafts of this post.

Throughout this post, "I" refers to Ansh (Buck, Ryan, and Fabien helped substantially with the drafting and revising of this post, however).

Two approaches to addressing weak supervision

A key challenge for adequate supervision of future AI systems is the possibility that they’ll be more capable than their human overseers. Modern machine learning, particularly supervised learning, relies heavily on the labeler(s) being more capable than the model attempting to learn to predict labels. We shouldn’t expect this to always work well when the model is more capable than the labeler,[1] and this problem also gets worse with scale – as the AI systems being supervised become even more capable, naive supervision becomes even less effective.

One approach to solving this problem is to try to make the supervision signal stronger, such that we return to the “normal ML” regime. These

scalable

oversight

approaches aim to amplify the overseers of an AI system such that they are more capable than the system itself. It’s also crucial for this amplification to persist as the underlying system gets stronger. This is frequently accomplished by using the system being supervised as a part of a more complex oversight process, such as by

forcing it to argue against another instance of itself

, with the additional hope that verification is generally easier than generation.

Another approach is to make the strong student (the AI system) generalize correctly from the imperfect labels provided by the weak teacher. The hope for these

weak-to-strong generalization

techniques is that we can do better than naively relying on unreliable feedback from a weak overseer and instead access the latent, greater, capabilities that our AI system has, perhaps by a simple modification of the training objective.

So, I think of these as two orthogonal approaches to the same problem: improving how well we can train models to perform well in cases where we have trouble evaluating their labels. Scalable oversight just aims to increase the strength of the overseer, such that it becomes stronger than the system being overseen, whereas weak-to-strong generalization tries to ensure that the system generalizes appropriately from the sup...
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
by leogao — 2023-12-16T05:39:10.558Z

Links: Blog, Paper.

Abstract:

Widely used alignment techniques, such as reinforcement learning from human feedback (RLHF), rely on the ability of humans to supervise model behavior—for example, to evaluate whether a model faithfully followed instructions or generated safe outputs. However, future superhuman models will behave in complex ways too difficult for humans to reliably evaluate; humans will only be able to weakly supervise superhuman models. We study an analogy to this problem: can weak model supervision elicit the full capabilities of a much stronger model? We test this using a range of pretrained language models in the GPT-4 family on natural language processing (NLP), chess, and reward modeling tasks. We find that when we naively finetune strong pretrained models on labels generated by a weak model, they consistently perform better than their weak supervisors, a phenomenon we call weak-to-strong generalization. However, we are still far from recovering the full capabilities of strong models with naive finetuning alone, suggesting that techniques like RLHF may scale poorly to superhuman models without further work. We find that simple methods can often significantly improve weak-to-strong generalization: for example, when finetuning GPT-4 with a GPT-2-level supervisor and an auxiliary confidence loss, we can recover close to GPT-3.5-level performance on NLP tasks. Our results suggest that it is feasible to make empirical progress today on a fundamental challenge of aligning superhuman models.