trentmkelly/LessWrong-43k huggingface dataset:
How To Actually Succeed
by Jordan Arel — 2022-09-13T00:21:42.901Z

Epistemic Status:

Casual musings on Eliezer's LessWrong post "Trying to Try"

Slightly different though similar YouTube version:

Also viewable on my website:

https://jordanarel.com/2022/09/12/how-to-actually-succeed/

And The EA Forum

https://forum.effectivealtruism.org/posts/nbvcbB4iC8J29yjkx/how-to-actually-succeed

Summary:

Trying is not the same as doing something. It is easy to feel like you are trying to do something, without having any chance of actually succeeding. It is important to create a clear mental model for success, test the assumptions of that model, determing the absolute single most important next action each step along the way, break down the goal into manageable steps, recognize and avoid noble obstacles, be willing to fail and iterate, and be willing to fail at less important things. Maybe we could have a social exercise to make sure we are actually working to robustly achieve our goals, rather than just "trying."

Full Post:

In Eliezer Yudkowsky's post "Trying to Try", he reviews Yoda's phrase "Do or do not, there is no try," explaining that at first he found this highly dubious, sounding like what he calls "pretending to be wise", saying something enigmatic and profound sounding which is actually superficial or meaningless.

After more deeply considering the matter, he came to the conclusion that the phrase actually is quite meaningful, as typically when people say they are going to try to do something, or in fact even when they say they are going to do something, they actually really only mean that they are going to put in the minimal amount of effort to plausibly feel like they might achieve it, or to be able to convey to others that they are working to achieve it.

He explains that it is much easier to "try to make $1 million" versus actually making $1 million, and it is much easier to "try to launch a startup" then to actually launch a startup.

I was quite floored to read this and realize that this deeply applies to myself as I want to launch a Hybrid Market which I think could completely transform the trajectory of humanity if successful, but which is insanely hard to actually succeed at, and extremely easy to "try" to succeed at.

Eliezer points out that one of the main ways we get in our own way is that rather than asking "What needs to happen for this goal to actually be achieved?" we ask "What can I do to achieve this goal?" or "How c...
EA & LW Forums Weekly Summary (5 - 11 Sep 22')
by Zoe Williams — 2022-09-12T23:24:54.499Z

Supported by Rethink Priorities

Cross-posted on the EA forum here.

This is part of a weekly series - you can see the full collection here. The first post includes some details on purpose and methodology.

If you'd like to receive these summaries via email, you can subscribe here.

Top Readings / Curated

A new section, designed for those without the time to read all the summaries. Everything here is also within the relevant sections later on so feel free to skip if you’re planning to read it all.
Methodology - I’ve pulled out a few posts I think offer some particularly new and interesting ideas, or would be useful to have more people giving feedback on. This is purely based on my personal opinion. Keen on any thoughts in the comments if this section is useful or not!

A California Effect for Artificial Intelligence

By henryj

The California Effect occurs when companies adhere to California regulations even outside California’s borders. The author evaluates three types of AI regulation California could adopt, and finds two 80% likely to produce a california effect - regulating training data through data privacy, and risk-based regulation like that proposed by the EU. A full research paper is linked, the results of a summer research fellowship.

Developing Next Gen PPE to Reduce Biorisks

by AndyGraham, tsmilton

The authors are engineers planning to apply to LTTF for funding to run a feasibility study on improved PPE, particularly high protection versions (ie. suits). The current versions are bulky, expensive, hard to use, and have changed little since 1979. Feasibility will be primarily research to validate the need and potential impact on GCBRs. If that resolves favorably, they hope to finalize a product within 2-5 years.

They’re looking for advice and thoughts - particularly from those with bioexpertise or PPE expertise, or who can help with the grant application.

Cause Exploration Prizes: Announcing our prizes

Over 150 submissions. Top prize to

Organophosphate pesticides and other neurotoxicants

by Ben Stewart. Second prize to each of

Violence against women and girls

by Akhil,

Sickle Cell Disease

(anonymous) and

Shareholder activism

by sbehmer. 20 honourable mentions. Post on learnings may come in the future.

See section ‘Bonus: winners of Open Philanthropy’s cause exploration prizes’ section below for more detailed summaries.

Ann

...
Time is not the bottleneck (on making progress thinking about difficult things)
by kman — 2022-09-12T20:45:50.419Z

... at least, not for me. The bottleneck is something like mental energy. I can only make progress on a (subjectively) difficult topic for a handful of hours (at best) each day before hitting severe diminishing returns, and being forced to "relax" for the rest of the day.

In light of the above, the obvious thing to optimize for the sake of "making progress on difficult things" is the amount of "mental energy available per day", not "waking hours available per day". Sleep seems to serve the function of "restoring mental energy"; but I get diminishing returns on sleeping longer (in a single block) as well; for me, there doesn't seem to be a big difference in "available mental energy" during a day whether I'd slept 6 hours or 9 hours.

So, since I'm getting diminishing returns on both my waking and sleeping hours, the obvious thing to try is to split the day up into multiple "sub days", i.e. some form of polyphasic sleep. Ideally, I'd like to spend a larger total fraction of the day asleep; this contrasts things like "the Uberman's sleep schedule", where the goal is to have as much waking time as possible (which, through this lens, is pure folly).

Have any of you (or anyone you know) had any success with something like this?
[Linkpost] A survey on over 300 works about interpretability in deep networks
by scasper — 2022-09-12T19:07:09.156Z
Link: https://arxiv.org/abs/2207.13243

Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks

Tilman Räuker* (

traeuker@gmail.com

)
Anson Ho* (

anson@epochai.org

)

Stephen Casper* (

scasper@mit.edu

)
Dylan Hadfield-Menell

TL;DR: We wrote a survey paper on interpretability tools for deep networks. It was written for the general AI community but with AI safety as the key focus. We survey over 300 works and offer 15 discussion points for guiding future work. Here is

a link to a Twitter thread

about the paper.

Lately, there has been a growing interest in interpreting AI systems and a growing consensus that it will be key for building safer AI. There have been rapid recent developments in interpretability work, and the AI safety community will benefit from a better systemization of knowledge for it. There are also several epistemic and paradigmatic issues with much interpretability work today. In response to these challenges, we wrote a survey paper covering over 300 works and featuring 15 somewhat “hot takes” to guide future work.

Specifically, this survey focuses on “inner” interpretability methods that help explain internal parts of a network (i.e. not inputs, outputs, or the network as a whole). We do this because inner methods are popular and have some unique applications – not because we think that they are more valuable than other ones. The survey introduces a taxonomy of inner interpretability tools that organizes them by which part of the network’s computational graph they aim to explain: weights (S2), neurons (S3), subnetworks (S4), and latent representations (S5). Then we provide a discussion (S6) and propose directions for future work (S7).

Finally, here are a select few points that we would like to specifically highlight here.

Interpretability does not just mean circuits. In the survey sections of the paper (S2-S5), there are 21 subsections, and only one is about circuits. The circuits paradigm has received disproportionate attention in the AI safety community, partly due to

Distill’s

influential interpretability research in the past few years. But given how many other useful approaches there are, it would be myopic to focus too much on them.
Interpretability research has close connections to work in adversarial robustness, continual learning, modularity, network compression, and studying the human visual system. For example, adv

...
Why do People Think Intelligence Will be "Easy"?
by DragonGod — 2022-09-12T17:32:38.185Z

In many discussions I've hard with people around AI takeoff, I've frequently encountered a belief that intelligence is going to be "easy".
To quantify what I mean by "easy", many people seem to expect that the marginal returns on cognitive investment (whether via investment of computational resources, human intellectual labour, cognitive reinvestment or general economic resources) will not be diminishing, or will diminish gracefully (i.e. at a sublinear rate).

(Specifically marginal returns around and immediately beyond the human cognitive capability frontier.)

I find this a bit baffling/absurd honestly. My default intuitions lean towards marginal returns to cognitive reinvestment diminishing at a superlinear rate, and my arm chair philosophising so far (thought experiments and general thinking around the issue) seem to support this intuition.

A couple intuition pumps:

50%, 75%, 87.5%, 93.75%, ... are linear jumps in predictive accuracy (one bit each), but returns seem to diminish at an exponential rate

On the other hand 6.25%, 12.5%, 25%, 50% represent the same linear jumps, but this time with returns growing at an exponential returns
This suggests that the nature of returns to cognitive investment might exhibit differing behaviour depending on where in the cognitive capabilities curve you are

Though I've not yet thought about how this behaviour generalises to other aspects of cognition separate from predictive accuracy

Apriori, I'd expect that given how much human dominance is almost entirely dependent on our (collective) intelligence, our evolution would have selected strongly for intelligence until it met diminishing returns to higher intelligence

That is, we should expect that evolution made us as smart as was beneficial given its constraints:

Calorie requirements of larger brains
Energy budget
Size of the human birth canal
Slight variations on existing cognitive architecture
Etc.

I think the reported higher incidence of some neuronal issues/disorders among higher IQ human subpopulations (e.g. Ashkenazi Jews) may be an argument in favour of evolution having met said marginal returns
Perhaps the Flynn Effect is an argument against this

Another way to frame my question this is that there will obviously be diminishing marginal returns (perhaps superlinearly diminishing) at some point; why are we confident that point is far in front of the peak of hum...
Alignment via prosocial brain algorithms
by Cameron Berg — 2022-09-12T13:48:05.839Z

In this post, I want to briefly propose a semi-novel direction for alignment research that I'm excited about. Though some of these ideas are not brand new—they purposefully bear resemblance to recent (highly promising) work in

SHARD theory

and

Steve Byrnes’s approach

to brain-based AGI safety—I think my emphases are sufficiently different so as to justify a more thorough explanation.

Why are humans 'worthy' of being in the loop?

I think the following three claims help motivate the general research direction I have in mind.

1) Many of the most coherent AI safety strategies proposed to date (e.g., HCH, imitative and approval-based amplification, recursive reward modeling, and more) involve human decision-makers in some meaningful capacity. I claim, therefore, that these proposals implicitly presuppose that there are specific algorithmic properties of the human mind/brain that make us comfortable entrusting these ‘humans in the loop’ with the task of minimizing the likelihood of AI-induced bad outcomes. This idea is demonstrated especially clearly by ‘safety via debate,’ for instance:

Diagram from An overview of 11 proposals for building safe advanced AI, with my annotation in black.

2) I think the special brain algorithms in question—e.g., the ones that make us comfortable entrusting a neurotypical human to decide who won in the set-up above—are more familiarly thought of as prosocial or moral cognition. A claim like this would predict that we would be uncomfortable entrusting humans who lacked the relevant prosocial instincts (e.g., psychopaths) to oversee a safety-via-debate-type set-up, which seems correct. I think the reason that it is a very natural thought to want to incorporate neurotypical human decision-makers into alignment proposals is that we are confident (enough) that such decisions will be made carefully—or at least more carefully than if there were no humans involved. In other words, individual humans in the loop are entrusted-by-default to serve as competent advocates for the interests of society at large (and who are more than likely aware of the fact that they are serving this role), able to infer suspicious behavior, evaluate subtle short- and long-term predicted consequences, be humble about said evaluations, probably solicit second opinions, etc.—somehow!

3) Our understanding of human prosoci...
I’ve written a Fantasy Novel to Promote Effective Altruism
by Timothy Underwood — 2022-09-12T12:14:57.082Z

Back in February I got a grant through the extended set of ACX grants to write this. I now have a draft that is sufficiently polished that I’m comfortable showing to the community, though I suspect there will be a lot of things that I will want to change based on feedback here before I start publishing it in venues that aren’t part of the EA memespace.

I’m planning to post this one chapter at a time here for the next month or so, though depending on what sort of feedback I get, I might at some point stop or just make a post that has the rest of the book in one chapter. The whole thing is also available in a google doc which I strongly encourage anyone who wants to give me detailed feedback to go to and leave comments on.

I especially would like feedback on the philosophical arguments in the text, with an especial emphasis on criticisms of EA ideas that you think are important to be addressed, but that aren’t brought up by any of the characters in the text.

My plan is that after I’ve revised the book based on feedback received here, is that I will start publishing it serially on Royal Road and in the Spacebattles.com forums. After it is about half published on serial fiction websites, I plan to publish it on Amazon and make a website with the whole text posted. However this is a provisional plan — if anyone has good ideas on how I can help this novel find audiences that will like it and hopefully be influenced by it, I really want you to give me any advice or help that you can.

I did set up a

discord

to talk about the project, but I’m really not a discord person, and while I’ll participate in any conversations that happen there, I’m probably not going to start them. The best way to talk to me privately about the text is to send me an email at timunderwood9 at gmail. Or leave long comments on the google doc.

Anyways, everyone, tell me what you think, and I hope that this work is at least entertaining for some of you.

Prologue

“Even if we ignore the possibility of a glorious techno-utopian future, everybody alive today dying at the same time would be really, really bad for the people alive now as individuals.”

Isaac grinned at his new friend. “Sorry, but can we finish this argument later?” A quick pull of his phone from his pocket, and he checked the time. “I’m going to eat fish and chips while on this side of...
Ideological Inference Engines: Making Deontology Differentiable*
by Paul Bricman — 2022-09-12T12:00:43.836Z

* Rather, making deontology play well with differentiable systems trained end-to-end.

This post is part of my hypothesis subspace sequence, a living collection of proposals I'm exploring at Refine. Preceded by oversight leagues, and followed by representational tethers.

TL;DR: An ideological inference engine is a mechanism for automatically refining a given propositional representation of human values (e.g. a normative charter, a debate stance) in an attempt to disambiguate and generalize it to novel situations. While the inference algorithm and the seed representation form the crux of the system, a multi-modal entailment verifier is employed to order possible agent behaviors based on their compatibility with the estimated ideology. This proposal then describes a way of instilling deontological drives in prosaic systems while maintaining the appeal of end-to-end differentiation. Ideological inference engines draw on ideas from traditional expert systems, but replace much of the clunky symbolic manipulation with contemporary LLMs and NLI models.

Intro

Ideological inference engines are a slightly more general framework than oversight leagues, in the sense they rely on several global assumptions, but each more concrete instance of the proposal requires new assumptions when designing the seed representation, the inference algorithm, and the entailment verifier. Here's a non-exhaustive list of global assumptions:

Assumption 1, "Small Seed To Big Tree": Given a suitable inference algorithm and a finite propositional representation of human values, it is possible to estimate human values arbitrarily well given arbitrary amounts of compute. "Arbitrarily well" refers to there being an arbitrarily low error in the estimation. In the limit, the seed knowledge base would grow into the True Name of human values in propositional form.

Assumption 2, "Linear Capability Ordering": Similar to the assumption invoked in oversight leagues, this states that a system composed of a fixed knowledge base and a fixed entailment verifier would eventually be gamed by an agent whose capability is constantly increasing. This is due to the more complex agent becoming able to exploit the inaccuracies of the knowledge base with respect to actual human values.

Assumption 3, "Quantifiable Propositional-Behavioral Gap": The compatibility of a proposed behavioral sequence and a propositional representa...
Freeloading?
by jefftk — 2022-09-12T11:20:03.371Z

Selections from the category of "provider expects people in general
will make it worth their while, and you're behaving in a way where you
won't", roughly ordered from my impression of least to most acceptable:

Taking free newspapers from distribution boxes to use in craft projects or for heating
your house.

Accepting a swag t-shirt to get cotton to use for papermaking.

Going to a store without any intention of buying something,
just to eat the free samples.

Bringing outside food into a movie theater or amusement park.

Interviewing somewhere you would definitely not want to work,
just for the practice.

Using paywall circumvention software.

Independently decrypting
cable or satellite TV.

Using GNU
Parallel without citing it.

Running an ad blocker on
your computer.

Fast-forwarding through sponsored sections on YouTube or a podcast.

Stopping to watch a street performer or listen to busker
without paying.

Listening to NPR without contributing,
when you could afford to.

Accompanying friends to a restaurant and only ordering water.

Buying PS3 consoles and building a
supercomputer.

Buying a cheap printer and using third-party ink.

Fixing bugs in open source software without contributing your
fixes back upstream.

Using Wikipedia without fixing errors you find.

Leaving an amusement park for lunch.

Using a web browser but changing the default search engine.

Reading ad supported stuff but never buying anything from
the ads.

For example, Sony sold the PS3 below cost because
they expected people would make make up
for it through paying higher prices for games, but someone buying
thousands
of them to build a supercomputer breaks their business model. Or,
Firefox is funded by selling the right to be the default browser (in
the US they switched
from Google to Yahoo in 2014 and then back
to Google in 2017) and if you choose your own default search
engine you're slightly weakening Firefox's negotiating position.

In some cases we make this sort of behavior illegal: you're not
allowed to decrypt satellite TV, even though it's being transmitted
unasked right at your house. In other cases it's not illegal, but
people mostly don't do it through some combination of "I don't want to
be a jerk" and "it's not worth it": no one harvests freesheets for
the paper. And in other cases it's common and widely
acceptable, but not...
Can you force a neural network to keep generalizing?
by Q Home — 2022-09-12T10:14:27.181Z

I want to share an idea about how we could try to force a neural network to come with bigger and bigger generalizations. Potentially reaching abstractions similar to human abstractions.

Disclaimer: I'm not an expert.

In general

Here's the general idea of the required learning procedure:

Find a feature that classifies an object.
Generalize this feature so it fits all the objects.
Make different versions of this feature that classifies different objects. (1st iteration)
Generalize lower-level features from which the previous features were constructed.
Make different versions of those lower-level features for different higher-level features... (2nd iteration)

Example

Let's say our goal is to learn the visual difference between cars and poodles (a dog breed). We're looking only at pictures of cars and poodles, nothing else.

As I understand (judging by the definition and some illustrations I've seen), a typical deep learning model may approach the problem this way:

It learns what is a "wheel". (I know that the model is unlikely to learn a human concept.)
It learns that cars have wheels and poodles don't have wheels.
You show it a poodle on wheels and it needs to relearn everything from scratch. The new data completely destroyed the previous idea of what's a car and what's a poodle.

Note: of course, it doesn't have to learn about the "wheel", it can learn any other low-level feature (or combination of features). It's just an example.

But I thought about another idea, a different model:

It learns what is a "wheel".
It learns that cars have wheels and poodles don't have wheels.
It generalizes the definition of a "wheel" as "anything like a circle" so it can describe both cars and poodles. Cars have round wheels and poodles have round fur.
It learns to tell apart "car circles" from "poodle circles". It learns that car's circles are separate and poodle's circles are connected. (a poodle looks like many circles joined together)
You show it a poodle on wheels... and it doesn't fall for the trick. Or quickly adapts. The important point is that it still has all the tools to solve the problem. The ideas it learned didn't become meaningless.
And it can go deeper. The same way it can generalize the definition of "being connected" (learned on the step 4) and then create 2 versions of this definition for cars and poodles. It can learn that even if car's circles are connected, they're still di

...