trentmkelly/LessWrong-43k huggingface dataset:
The Friendly Drunk Fool Alignment Strategy
by JenniferRM — 2023-04-03T01:26:13.999Z

Let's set some context!

A Philosophic Frame (half ironic, or maybe level two sarcasm)

Systematicity isn't all that. I mean to say: formal systems only go so far.

Physicists gave up on having a single physics, so they just use two of them that contradict in general but that agree in "the middle" (between very large and very small things), and that's fine, right? And its kinda the same for everything at every level, if you think about it.

I'm not endorsing any of this. If this thesis is right, then endorsing things is for chumps, and if this thesis is wrong then I can just tell people I was joking later!

What this essay is for is kinda like... like what if we were all drunk, and we were all stupid, and we were all babies, and we couldn't fulfill the least of our paltry duties in any adequate way, and yet despite all of that: everything was OK anyway!

If that's just the way the structure of the universe works, then this essay might help us go faster towards the inevitably good outcome!

And if that's NOT the way the structure of the universe works... then this essay might cause some people to awaken from their sleep walking?

So... let's do it! Let's tickle the cosmos, and see if he smiles or frowns.

The Social Milieu (half ironic, or maybe level five sarcasm)

Up front we should notice that most scientists are so boring. They almost never have any fun. Engineers are the cool ones! They build neat things without stopping to worry if it is safe or not! Some of them do it with so much bravery that they test on themselves first. That's so rad ❤️

I'm not sure if I'd go so far as to say that "All Cops Are Categorically Bad" (because at BBQs I've met some who seemed as cool as anyone) but you know... you don't see a lot of movies about cops where they are the hero anymore, right?

Basically, when people propose any laws that will be enforced at gunpoint, they kinda seem like evil vibe-killers. I'm not saying they are killers. Just that they often kill the vibe, you know? And what is more important than vibing? Even the bouncers wear nice clothes, you know?

Maybe the only good laws are the ones that can be enforced by freezing the bank account of a corporation, right? And even those laws are pretty sus! So we could probably just defund the police (using laws that only touch bank accounts, see!) and then that would probably be fine!

I'm just say...
Slack Group: Rationalist Startup Founders
by Adam Zerner — 2023-04-03T00:44:54.791Z

Update: I decided to discontinue this group. Not enough people with concrete enough plans to start a startup joined, and there was a general lack of activity.

I have a friend who is a top poker player. In asking him about his experiences and how he got so good, one thing that stood out to me was community. He had a group of friends who were also top players. That group would text each other all the time about poker. Hands, theory, tilt, game selection, whatever. He said that having this group was absolutely huge.

Another thing that comes to my mind is Ray Dalio. I recall him talking a lot about how important it is to surround yourself with smart people who can "stress test" your ideas.

It's not just Dalio and my friend of course. I've heard this advice other times as well.

Personally, I am on a journey to start a successful startup. It is obvious to me that an important first step in this journey is to surround myself with people who I could bounce ideas around with. Preferably people who are smart, knowledgeable about startups, and current or prospective founders. To that end, I am starting a Slack group for rationalists who would like to participate in this. If you're interested, sign up via the link below:

https://join.slack.com/t/rationalist-startup/shared_invite/zt-1sf9qgbpf-RBXE~Raz9tMLxESzzWrtFQ

Tabooing "startup"

Different people mean different things when they use the term "startup". Sometimes they are only referring to companies that have the potential and desire to be huge. Worth at least $100M, let's say.

I'm not referring to that here. What I'm referring to also includes businesses that are just intended to bring in some side income.

The term "startup" can be further tabooed of course, but I think that the above clarification is probably sufficient for now.

Who exactly is this for?

Founder?

If you are either a founder of small startup or a prospective founder, you are a perfect fit.

If you plan to and are confident that you will start a startup at some point down the road, you are a pretty good fit. If you are a founder of a medium-or-larger-sized startup, you are also a pretty good fit.

If you are considering starting a startup one day, you are a decent fit. If you are an early employee at a startup and/or are just someone who is interested in startups, I guess those are also decent fits.

Ideally, everyone in the group would be founders. I think there is valu...
Orthogonality is Expensive
by DragonGod — 2023-04-03T00:43:34.566Z
Link: https://www.beren.io/2023-03-19-Orthogonality-is-expensive/

@beren discusses the assumption that intelligent systems would be well factored into a world model, objectives/values and a planning system.

He highlights that this factorisation doesn't describe intelligent agents created by ML systems (e.g. model free RL) well. Model free RL agents don't have cleanly factored architectures but tend to learn value functions/policies directly from the reward signal.

Such systems are much less general than their full model based counterpart as policies they learned that are optimal under one reward function may perform very poorly under another reward function.

Yet, contemporary ML favours such systems over their well factored counterparts because the are much more efficient:

Inference costs can be paid up front by learning a function approximator of the optimal policy and amortised over the agent's lifetime

A single inference step can be performed as a forward pass through the function approximator in a non factored system vs searching through a solution space to determine the optimal plan/strategy for well factored systems

The agent doesn't need to learn features of the environment that aren't relevant to their reward function
The agent can exploit the structure of the underlying problem domain

Specific recurring patterns can be better amortised

Beren attributes this tradeoff between specificity and generality to no free lunch theorems.

Attaining full generality is prohibitively expensive; as such full orthogonality is not the default or ideal case, but merely one end of a pareto tradeoff curve, with different architectures occupying various positions along it.

The future of AGI systems will be shaped by the slope of the pareto frontier across the range of general capabilities, determining whether we see fully general AGI singletons, multiple general systems, or a large number of highly specialised systems.
GTP4 capable of limited recursive improving?
by Boris Kashirin — 2023-04-02T21:38:22.959Z

Apparently with reflection technique (answer-critique-improve) GTP4 capable of giving much better answers. But that implies it should be capable of doing essentially Alpha Go Zero type of learning! It can't do complete self play from zero as there is no ground truth for it to learn from, but that basically burns all hopes of having further progress bottlenecked by data. Also, while still severely lacking, it constitutes limited self-improvement capability. Not only we have unclear distinction of GTP4 being an AGI or not, but it also have some slight self improvement capabilities! We really are boiling frog, aren't we?
Scared about the future of AI
by eitan weiss — 2023-04-02T20:37:46.414Z

The recent AI advancements have really driven home how close we are to Judgment Day and how much we need to work to prevent total destruction.
I am deeply scared for the future. I have trouble enjoying the things I normally enjoy. They only remind me of What's At Stake.

I am currently stuck in mandatory military service in Israel for the next two years. Very few people in my surroundings have ever heard of alignment, lesswrong, or anything relevant. I am alone in this regard. I add this to explain that I am currently stuck and cannot contribute to research in any meaningful way.

What can I do? How can I help? Who can I talk to that understands the situation who can explain either that there is a satisfying plan, we will not all die, or at least that I am not alone in my dread?

Editing to add that although I have read the sequences and lurked on LW for a few years, my actual technical knowledge on the topic of AI safety is sadly low. I never understood where and how to start. So, I am currently working on educating myself.
"a dialogue with myself concerning eliezer yudkowsky" (not author)
by the gears to ascension — 2023-04-02T20:12:32.584Z

This is a linkpost[1], but I don't think the author would be happy to get commentary from people here on their blog, so I've copied the post here. link is at the bottom, if you really really want to comment on the original; please don't, though.

red: i suppose i should at least give him credit for acting on his beliefs, but my god i am so tired of being sucked into the yudkowsky cinematic universe. no more of this shit for me. i am ready to break out of this stupid fucking simulation.

blue: what did he even think was going to happen after the time piece? this is the kind of shit that makes people either laugh at him or start hoarding GPUs. it’s not as if he’s been putting any skill points into being persuasive to normies and it shows. wasn’t he the one who taught us about consequentialism?

(i thought initially that blue was going to disagree with red but no, blue is just mad in a different way)

https://twitter.com/mattparlmer/status/1641232557374160897?s=20

red: it’s just insane to me in retrospect how much this one man’s paranoid fantasies have completely derailed the trajectory of my life. i came across his writing when i was in college. i was a child. this man is in some infuriating way my father and i don’t even have words for how badly he fucked that job up. my entire 20s spent in the rationality community was just an endless succession of believing in and then being disappointed by men who acted like they knew what they were doing and eliezer fucking yudkowsky was the final boss of that whole fucking gauntlet.

[Note: link intentionally non-clickable, and author intentionally cropped [edit: whoops guess it wasn't? uh sorry], to respect author's separation from this community. If you want to see the whole tweet thread, which is in fact very interesting, here's the link: https://twitter.com/QiaochuYuan/status/1542781304621518848?s=20]

blue: speaking of consequentialism the man dedicated his entire life to trying to warn people about the dangers of AI risk and, by his own admission, the main thing his efforts accomplished was get a ton of people interested in AI, help both openAI and deepmind come into existence, and overall make the AI situation dramatically worse by his own standards. what a fucking clown show. openAI is his torment nexus.

yellow: i just want to point out that none of this is actually a counterargument to -

red: yellow, shut the FUCK up -

ye...
Fine-insured bounties as AI deterrent
by Virtual Instinct — 2023-04-02T19:44:52.320Z

Epistemic status: Hare-brained scheme

Suppose the state of Arkansas decided to levy a fine of $1000 on anyone attempting to push the state of AI forward. Further, Arkansas decreed that this $1000 fine would be paid directly out of the coffers of those found guilty, and directly towards the people who turned them in - a 'fine bounty', if you will.

$1000 is not a lot of money in the grand scheme of things. However, like most people in Arkansas, I don't have $1000 to spare, so I would decide to just not risk working on advancing AI. Nor would I join an organization involved in such activities, lest I be caught in the crossfire. The more coworkers I had, the more likely it would become that one bad apple would turn everyone in at once to collect the $n000 reward. (turning themselves in would just mean paying $1000 to themselves, which makes 'getting away' with this a lot easier).

This is the essential thrust of Robin Hanson's Privately Enforced & Punished Crime; he calls this the fine-insured bounty (FIB) crime law system in a follow-up post. He is much more cogent in spelling out the edge cases of the system in general there, so I will focus on some details which I think make this system unusually well-suited to the problem of how one deters technological development, such as AI development.

It scales. It is very hard to develop new AI capabilities without someone suspecting what you're up to. Even if they cannot prove it, the insurance mechanism in the fine-insured bounty system means even a low-fidelity suspicion can carry with it considerable economic value. Much like how someone with a history of running red lights may have a higher car insurance premium, someone with a history of buying GPUs may have a higher AI bounty insurance premium. (Someone running a virus which collects a fingerprint of GPU activity at scale likely has a valuable source of information to sell to insurers and prospective bounty hunters alike.)
It chills. To use a leaky pipeline argument: Few people are smart enough to advance AI capabilities in the first place; of those, few possess the conscientiousness to build out such a project by themselves; of those, the world is already their oyster, and almost all of them have found better things to do with their time than labor in solitude. The chilling effect of knowing this will be your fate so long as you pursue this should not be un

...
What could EA's new name be?
by trevor — 2023-04-02T19:25:22.740Z

[Disclaimer: this was not an April fools joke, people are actually working on this right now and this is the only critical window to contribute]

Recent comments by Holden Karnofsky, the leader of Open Philanthropy (and influential enough to initiate a name-change for EA):

Having seen the EA brand under the spotlight, I now think it isn’t a great brand for wide public outreach. It throws together a lot of very different things (global health giving, global catastrophic risk reduction, longtermism) in a way that makes sense to me but seems highly confusing to many, and puts them all under a wrapper that seems self-righteous and, for lack of a better term, punchable? I still think of myself as an effective altruist and think we should continue to have an EA brand for attracting the sort of people (like myself) who want to put a lot of dedicated, intensive time into thinking about what issues they can work on to do the most good; but I’m not sure this is the brand that will or should attract most of the people who can be helpful on key causes. I think it’s probably good to focus more on building communities and professional networks around specific causes (e.g., AI risk, biorisk, animal welfare, global health) relative to building them around “EA.”

It would still be better if there was a parent organization based on mutual respect/pragmatism; EA is currently still the final frontier for every vegan everywhere, for example, including those in professional networks such as government policy.

It seems to me like if someone thinks of a good enough name for EA, that people at least accept as a parent organization and an umbrella of mutual respect/pragmatism/sanity, then that will substantially increase the odds that it gets adopted.

Yudkowsky proposed a decent name and gave a decent justification for it, but it's probably worthwhile to have some people put 5 minutes into a name that could work (and spending 5 minutes thinking has a significant chance of yielding a galaxy-brained solution)

This screenshot will be deleted upon request

"Quantifiable Altruism", for example, seems like it would attract lots of math people and altruistic people, rather than people really into moral philosophy.

That seems great; I'm not sure if EA is currently bottlenecked on quant folk, but if there were tons of quant folk, it would at least make for a substantially larger talent ...
Exposure to Lizardman is Lethal
by Duncan Sabien (Inactive) — 2023-04-02T18:57:43.750Z

This is a short, experimentally-off-the-cuff post about one way in which social groups and institutions are effectively destroyed, in practice. It's fairly straightforward, but I don't try to connect all the dots (or even name all of them).

In a recent FB post, I made the following (lightly edited) claims:

I think a pretty solid heuristic is to treat 5% as 0% on things involving large numbers of people.

This is often referred to as "lizardman's constant;" my own first exposure to the idea was the fact that there are surveys in which ~5% of Obama voters report thinking Obama is the antichrist.

What's that 5% made of? WHO KNOWS. Human error, lulzy trolls, actual crazies, people having a really bad day, people who have complex reasons for the choices they make—the list of possible explanations goes on and on and probably all of them are in the mix, to some extent.

But like. This is always the case. If you're dealing with substantially-more-than-Dunbar's-number of people, I think you need to expect around 5% of your data to just be frothing insanity, total contentless noise.

If you're studying some awful thing X, and you get your approval rating/advocates of X down to 5% of the population, this is effectively zero. It's not gonna go lower without some kind of filter; you're dealing with quantum chaos at that point.

One of the things that institutions and authority do is provide insulation from this frothing madness.

e.g. you have a deranged suburban woman wanting a black man in the neighborhood to be arrested, for no discernible reason besides that he is black and she is crazy. In the best cases, the police dispatcher who takes the call recognizes that there is no real situation, and doesn't send an officer; in the second-best cases, the officer arrives on scene, assesses the non-situation, and defuses things by informing the crazy person that they are being crazy and that Authority does not deign to take action.

(There are much worse cases, of course.)

Similarly, a deranged parent calls up a school superintendent wanting a principal to be fired because their child was exposed to Michaelangelo's David, and the superintendent laughs and gently communicates "No, we are not doing that."

A key feature of this kind of insulation is that the person (or group, or structure) under attack, and vulnerable to attack, is different from the person (or group, or structure) doing the def...
Talkbox Bagpipe Drones
by jefftk — 2023-04-02T18:50:04.921Z

A few weeks ago

I wrote:

There's also an instrument in this category that could have been
built any time in the last ~500y: talkbox bagpipes. While the
mouth-blown pipes are probably most familiar, all the mouth is doing is
putting air into a bag. Bellows pipes replace this with an
arm-operated pump. This opens up an opportunity to point the drones
into your mouth so you can manipulate the sound of your accompaniment,
turning the drones from a continuous, well, drone, into something
lively and danceable. Or, at least, it should. As far as I know no one
has built these, and my attempts to convince my bagpipe-maker cousin to
work on this haven't been successful.

I was finally successful in convincing my cousin to let me try this,
and it does work!

Probably would be slightly better with a louder drone, or by
amplifying the drone separately so it can be turned up relative to the
chanter.

You'd need to adjust the angles so it could fit in the player's own
mouth, and if you want multiple drones it's mildly tricky to get them
to terminate in the same place, but this seems very doable. You could
also use your tongue to tap drones on and off. Possibly this would
even let you switch between dissonant drones, like A and G for an Amix
tune?

If you want to play with this, though, be sure you're using bellows
pipes. With mouth-blown pipes you'd be risking bagpipe
lung.
Is there a LessWrong-adjacent place to hire freelancers/seek freelance work?
by nonzerosum — 2023-04-02T16:39:49.659Z

I've posted a similar question before: https://www.lesswrong.com/posts/YyEiPQH8snk4h8BfK/who-is-available-for-contract-work-a-la-the-hacker-news-who

I'm wondering if there's a place that has a decently high volume (10+ a month) of postings/applicants.

I'm hiring a few freelancers to do AI-future research forecasting the development of AI and impact, like the sort of stuff EpochAI does, and am wondering where would be best places to post those freelance roles would be.

Any suggestions?
AISC 2023, Progress Report for March: Team Interpretable Architectures
by Robert Kralisch — 2023-04-02T16:19:07.857Z

Hi everyone, we are working on a

project

for the AI Safety Camp (AISC) 2023, and this is the first progress report.

To summarize, the aim of the project is to begin exploring an alternative pathway to interpreting generally intelligent computational agents. We want to study neurosymbolic systems as an inherently more interpretable paradigm compared to the current deep learning architectures, and hope to resolve some of the limitations of neurosymbolic systems, develop new conceptual insights/frameworks in this context, as well as transport relevant insights from them back into the deep learning context. The project is very open-ended.

Neurosymbolic architectures can provide a rich research interface for studying various hypotheses about cognition, as well as alignment. The reason for this is generally that the effect of precise changes to the architecture’s modules or their content can be tracked much more clearly compared to within entirely subsymbolic systems. Learning occurs more locally, as opposed to a sweeping change through the entire architecture, and the pathways of decision making and modelling generally include multiple modules that can be studied in isolation. There is often greater clarity about which cognitive behavior is upstream of some given cognitive behavior, allowing a better grasp on the range of behavior for the given area.

Cognitive Architectures are a

diverse

field and it is difficult to make general statements about it. Our project is focusing on a novel cognitive architecture, the Prop-room and Stage Cognitive Architecture (PSCA), that was proposed by the research lead Robert Kralisch, to allow for a more open-ended and thoroughly understood framework in which up-to-date insights from neuroscience and cognitive science more generally can be integrated or discarded as the development progresses. An additional reason for proposing a novel architecture is to better construct and isolate particular dynamics that we believe may be important to understand for solving the Alignment Problem.

To be clear, this does not mean that we think that this approach will work for certain. However, it seems like an important thing to try.

In this Progress Report, we give a brief overview of what everyone has been working on over the course of the last month, and give some reflection on the pace and shape of the project progress. ...