trentmkelly/LessWrong-43k huggingface dataset:
Mosaic and Palimpsests: Two Shapes of Research
by adamShimi — 2022-07-12T09:05:28.984Z

(Minor update to change Steve's labelling following this comment, and also because I realized that I never added the footnotes...)

This post is part of the work done at Conjecture.

Introduction

In

Old Masters and Young Geniuses

, economist-turned-art-data-analyst David Galenson investigates a striking regularity in the careers of painters: art history and markets favors either their early pieces or the complete opposite — their last ones. From this pattern and additional data, Galenson extracts and defends a separation of creatives into two categories, two extremes of a spectrum: conceptual innovators (remembered as young geniuses) and experimental innovators (remembered as old masters).

Conceptual innovators, like painter

Pablo Picasso

, start with a crystal clear idea of their goal, and spend tremendous amounts of time in preliminary research, preparatory drawing and all-around planning. They then mostly stick to these extensive plans (or have them executed by others) when concretely creating the output. As such, their most impressive innovations generally come from their ground breaking ideas, when they know and have done so little that they can simplify and break all the rules they haven't yet learned — they're young geniuses. Galenson provides additional examples: artist

Andy Warhol

, novelist

Herman Melville

, movie director

Orson Welles

, scientist

Albert Einstein

...

On the other hand, experimental innovators, like painter

Paul Cézanne

, only figure out their aim by relentless trial-and-error, making something up and then iterating on it. Their intuitions start vague and their goals cloudy, leading to their perpetual uncertainty and doubt about having accomplished what they wanted. Yet because experimental innovators keep on refining their attempts, and because they build on all that happened before, their best output (measured by metrics like auction prices, bestselling list, mentions in textbooks) emerges towards the end of their lives — they're old masters. Some additional examples given by Galensen are: abstract painter

Jackson Pollock

, novelist

Virginia Woolf

, movie director

Alfred Hitchcock

, and scientist

Charles Darwin

.

Galenson's distinction jolted my mind and turned it racing with questions: in which box did I fall? And others in alignment? What were the consequences for accelerating science in gener...
How do you concisely communicate & navigate the politics / culture at your job working at a large corporation or institution?
by Willa — 2022-07-12T03:22:49.342Z

Previously I've worked at small companies and state government or educational institutions. Now I work at a ~30,000 person private corporation and need to better adapt to that cultural situation.

My biggest cultural struggles so far are communicating concisely to people at multiple levels in the hierarchy across multiple different teams and knowing how to navigate the politics of a large corporation (I'm not used to directors being meaningfully different from managers, or the massive differences between what's called "upper management", "management", and everyone else).

I work in a technical role and support customers across every rank & role in the enterprise.

What are your tips, tricks, and advice for handling concise communication & navigating politics / the culture at a large corporation or institution?
On how various plans miss the hard bits of the alignment challenge
by So8res — 2022-07-12T02:49:50.454Z

This post has been recorded as part of the LessWrong Curated Podcast, and can be listened to on Spotify, Apple Podcasts, and Libsyn.

(As usual, this post was written by Nate Soares with some help and editing from Rob Bensinger.)

In my

last post

, I described a “hard bit” of the challenge of aligning AGI—the sharp left turn that comes when your system slides into the “AGI” capabilities well, the fact that alignment doesn’t generalize similarly well at this turn, and the fact that this turn seems likely to break a bunch of your existing alignment properties.

Here, I want to briefly discuss a variety of current research proposals in the field, to explain why I think this problem is currently neglected.

I also want to mention research proposals that do strike me as having some promise, or that strike me as adjacent to promising approaches.

Before getting into that, let me be very explicit about three points:

On my model, solutions to how capabilities generalize further than alignment are necessary but not sufficient. There is dignity in attacking a variety of other real problems, and I endorse that practice.
The imaginary versions of people in the dialogs below are not the same as the people themselves. I'm probably misunderstanding the various proposals in important ways, and/or rounding them to stupider versions of themselves along some important dimensions.[1] If I've misrepresented your view, I apologize.
I do not subscribe to the

Copenhagen interpretation of ethics

wherein someone who takes a bad swing at the problem (or takes a swing at a different problem) is more culpable for civilization's failure than someone who never takes a swing at all. Everyone whose plans I discuss below is highly commendable, laudable, and virtuous by my accounting.

Also, many of the plans I touch upon below are not being given the depth of response that I'd ideally be able to give them, and I apologize for not engaging with their authors in significantly more depth first. I’ll be especially cursory in my discussion of some MIRI researchers and research associates like Vanessa Kosoy and Scott Garrabrant.[2]

In this document I'm attempting to summarize my high-level view of the approaches I know about; I'm not attempting to provide full arguments for why I think particular approaches are more or less promising.

Think of the below as a window into my thought pro...
Rainmaking
by WalterL — 2022-07-12T00:42:58.571Z
Link: https://www.youtube.com/watch?v=lxQw0VoT538

There is an old TV show called 'The Wire', and in it a con artist fleeces a mark.

The mark's friend tries to wake him up, whereupon the mark insists that the con artist has helped him. The mark's friend tells him to 'wake up, he rain-made you!'

"What?"

"A guy says if you pay him, he can make it rain. You pay him. If and when it rains, he takes the credit. If and when it doesn't, he finds reasons for you to pay him more."

This is a neat tv show moment, to be sure, but it is also a pattern that you will find everywhere in the corporate world.

A given product has a few devs, who do the work, a few bosses, who fire devs that suck, and a squad of rainmakers who mess around on their phones. They are called Security Consultants, Architects, Scrum Masters... but what they all have in common is the same thing that an old saying recites about political doctrines.

They can never fail, only be failed. Paying security guys doesn't stop you from getting hacked. Paying <insert political trend> trainers doesn't stop you from getting sued. There is, in fact, no world state that can ever indicate that paying one of these folks was a mistake. Either their flag is checked, in which case all is well, or it is not, in which case they need more resources.

Lesswrongers will be familiar with the old concept of 'make your beliefs pay rent'. It is terrifying how rare basic comprehension of this notion is in the organizations we depend upon. People will accept unfalsifiable axioms as doctrine without the slightest qualm, and then you are gritting your teeth through another round of <insert buzzword> exercises.

They trade upon the credit earned by the few legitimate practitioners of their occupations, who genuinely do consult on security or master scrums. It can be hard to tell the two apart, to grok whether your rainmaker is toting a cloud seeder or thoughts and prayers.

The only thing I've ever seen work was essentially formalizing the above. Have everyone give 'failure statements' of the form "I will have failed if at time X Y measurable statement is true" and look at the nature of the statements. Productive people's will be about their contribution, rainmakers will talk about the weather.
Book Review: Neal Stephenson’s “Termination Shock”
by Tyler Simmons — 2022-07-12T00:07:32.810Z
Link: http://www.words-and-dirt.com/dirt/double-take-the-edifying-ambiguity-of-neal-stephensons-termination-shock/

Cross posted from the Words&Dirt blog. I'm a longtime reader and fan of lesswrong, but not a regular poster. My best friend submitted this review of Neal Stephenson's 2021 book 'Termination Shock' into the ACX book review contest this year. It didn't make the finalists, but I thought the community here might get appreciate it anyway. The author (Miles) and I would love to here your perspectives on the book, review, and more generally, solar geoengineering.

Double Take: The Edifying Ambiguity of Neal Stephenson’s “Termination Shock”

I. Grabbing the Wheel

It’s like, we’ve been in a car with a brick on the gas pedal and no one at the wheel, careening down the road, running over people and crashing into things. We’re still in the car. We can’t get out of the car. But someone could at least grab the wheel. (655)

This passage doesn’t show up until the final act of Neal Stephenson‘s Termination Shock, but it’s a useful image for introducing the book’s premise. Imagine the late 2040s, just two and a half decades from now. Humanity has continued to do diddly-squat about climate change. Life is becoming increasingly difficult for people across the globe, primarily due to rising temperatures and sea levels. Frustrated with our collective complacency, a Texas billionaire decides to “grab the wheel” by building a giant gun in the ground, complete with six 240-meter barrels that point straight up toward the sky. Every eight minutes, this gun fires a “bullet” that is actually a miniature rocket. These rockets use molten sulfur as fuel, which converts into sulfur dioxide exhaust as they sail through the stratosphere. Once injected into the stratosphere, the sulfur dioxide reflects some portion of the sun’s light back into space, thereby reducing the amount of heat entering Earth’s atmosphere and bringing down global temperatures. From this starting point, Termination Shock introduces a host of characters whose lives intersect with this project, depicting the challenges they face amidst growing geopolitical tensions.

Depending on your familiarity with the climate literature, this scenario may strike you as completely bonkers, predictably inevitable, or somewhere in between. But for anyone unfamiliar with the idea, “solar geoengineering” (AKA “solar radiation modification” or “SRM”) is a very real technology with a theoretical hist...
Announcing Future Forum - Apply Now
by wANIEL — 2022-07-11T22:57:39.357Z
Link: https://forum.effectivealtruism.org/posts/szeE3je8MD4sZcevL/announcing-future-forum-apply-now

We are excited to announce Future Forum, an experimental 4-day conference taking place Aug 4-7 in San Francisco. We want to gather promising people from across communities interested in the long-term future of humanity, introduce them to one another, reflect on transformative technologies and their far-reaching consequences (upside and downside risks), and then generate new promising and thoughtful projects across this cross-community ecosystem. We expect around 250 attendees.

We’re putting special emphasis on mingling EA and Silicon Valley tech. We think that Silicon Valley contains promising talent that does or could work on pressing problems with agency and ambition, but that it could also benefit from EA-inspired thoughtfulness about impact and about downside risks of working on transformative technologies. On the flip side, we believe that EA could generally benefit from more agency and ambition (judicious ambition). We think agency and ambition are best fostered by curated exposure to high-agency individuals, ideally resulting in not just acute inspiration but also more long-term influences on personal trajectories, such as inspiring friends or mentors.

Other communities which will be represented or somewhat involved include Emergent Ventures, Rationality, Longevity, Crypto, and Space among others.

Generally, Future Forum will be an in-person-only event, with no virtual component and no talks being recorded.

The Event

While the vast majority of the event is explicitly focused on 1-1s and small group conversations, we have a few speakers who will hold brief talks (~30min max) about topics they care about and give participants common topics to talk about with others. We’ve gotten strong buy-in from many of the communities we hope to bring together for Future Forum, and our current speaker line-up is:

Holden Karnofsky from Open Philanthropy
Anders Sandberg from FHI
Daniela Amodei from Anthropic
Patrick Collison from Stripe
Ed Boyden from MIT/HHMI
Tamara Winter from Stripe Press
Jason Crawford from Roots of Progress
Grant Sanderson (3Blue1Brown)
Allison Duettmann from Foresight Institute
Sam Altman from OpenAI
Celine Halioua from Loyal

(Please be aware that this line-up is likely to evolve further, with more additions from across communities.)

Future Forum runs from August 4th-7th, which is the weekend after EAG San Francisco (EAG’s applications close on Jul...
Defining Optimization in a Deeper Way Part 2
by J Bostock — 2022-07-11T20:29:30.225Z

We have successfully eliminated the concepts of null actions and nonexistence from our definition of optimization. We have also eliminated the concept of repeated action. We are halfway there, and now have to eliminate uncertainty and absolute time. Then we will have achieved the goal of being able to wrap a 3D hyperplane boundary around a 4D chunk of relativistic spacetime and ask ourselves "Is this an optimizer?" in a meaningful way.

I'm going to tackle uncertainty next.

TL;DR I have allowed for a mor

We've already defined, for a deterministic system, that a joint probability distribution PABt(sA,sB) has a numerical optimizing-ness, in terms of entropy. Now I want to extend that to a non-joint probability distribution of the form PA(sA)PB(sB). We can do this by defining PAt−1(sA) and PBt−1(sB) for the previous timestep.

We can then define PABt(sA,sB) as by stepping forwards from t−1 to t as before, according to the dynamics of the system.

A question we might want to ask is, for a given PAt−1(sA) and PBt−1(sB), how "optimizing" is the distribution PABt(sA,sB)?

The Dumb Thermostat

Lets apply our new idea to the previous models, the two thermostats. Lets begin with uncorrelated, maximum entropy distributions.

For thermostat 1 we have the dynamic matrix:

hot	warm	cold
high	(hot,off)	(hot,low)	(warm,high)
low	(hot,off)	(warm,low)	(cold,high)
off	(warm,off)	(cold,low)	(cold,high)

(In this matrix, the entry for a cell represents the state at time=t+1 given the coordinates of that cell represent the state at time=t)

With the PRTt−1 distribution:

hot	warm	cold
high	1/9	1/9	1/9
low	1/9	1/9	1/9
off	1/9	1/9	1/9

As an aside this has 3.2 bits of entropy.

Leading to the PRTt

distribution:

hot	warm	cold
high	0	1/9	2/9
low	1/9	1/9	1/9
off	2/9	1/9	0

This gives us the "standard" PRTt+1 distribution of:

hot	warm	cold
high	0	2/9	1/9
low	1/9	1/9	1/9
off	1/9	2/9	0

And the "decorrelated" P′RTt+1 distribution is actually just the same as PRTt! When we decorrelate the probabilities for sR and sT we just get back to the maximum entropy distribution and so P′RTt+1=PRTt

It's clear by inspection that the distributions PRTt+1 and P′RTt+1 have the same entropy, so the decorrelated maximum entropy PRt−1PTt−1 does not produce an "optimizi...
Marriage, the Giving What We Can Pledge, and the damage caused by vague public commitments
by Jeffrey Ladish — 2022-07-11T19:38:42.468Z

I believe honesty is very important. I think most people agree that honesty is like, pretty important, but I think it’s a lot more important than that. I basically think that people will be dishonest in ways that hurt them and others by default, even when they’re trying to be pretty honest, because I think it’s just that hard. I think it’s hard because there are a lot of incentives that push away from honesty. E.g. You want the job so you’re tempted to overstate your experience or past performance. You said you wouldn’t tell anyone about your friend’s secret, but this seems like a situation where they wouldn’t mind, and it would be pretty awkward to say nothing…etc. There’s a huge variety of situations that incentivize small acts of dishonesty. And it’s not always clear whether something is a little dishonest or not - dishonesty can be quite a spectrum.

If I’m correct, and honesty is pretty hard by default, I think this is quite bad. Honesty is important because it greatly improves the ability of people to coordinate with each other. And it’s important because it allows people to reason better about themselves and about the world. Good coordination and good reasoning are things we badly need. Fortunately, I think many people could level up their honesty by putting in a reasonable amount of thinking and effort, and that a lot of the failure modes are caused by not paying attention to the incentives around them, or not thinking about how to structure their own lives and commitments to be more honest.

If you want to be honest, it’s important to think about how to structure your life so being honest isn’t extremely difficult. This is especially true when it comes to promises and commitments. It’s often easier to be honest about your current beliefs than it is to be honest about what you’re going to do in the future. After all, you don’t know what’s going to happen in the future. You can definitely influence it, and you can choose now to take particular actions in the future. But if you’re not careful, you might promise to do a thing in a week you think will be easy, and then find later the thing is extremely difficult or costly.

At present there is an understanding that some kinds of commitments are much stronger and more serious than other kinds. One particularly strong example of a commitment or promise is a commitment you make in publicly, with people witnessing...
Systemization
by CFAR!Duncan — 2022-07-11T18:39:04.750Z

Epistemic status: Anecdotally strong

Many of the flawed heuristics and biases that the Systemization unit seeks to address are well-known and well-researched (such as the planning fallacy and failure to account for switching costs). The underlying theory of attention draws on a combination of Daniel Kahneman’s prospect theory and anecdotal research from thinkers like Richard Thaler and David Allen. The combination of all of these into a generalized strategic intervention has been useful to large numbers of alumni, but has not been rigorously tested and may not be testable in the broadest sense.

The act of systemizing something has many advantages. You pay a (relatively) large cost up front, and save yourself from repeated future costs. You can lock in the best or most efficient version of a process, and then future training and ingraining only reinforces that best version. Often, the process of generating a system forces you to examine your goals and leads to greater insight into what you’re doing, and you can use the extra time and attention you’ve freed up either to do more of what you were already doing, or to add new things to your roster.

Not everything in your life needs to be systemized, of course. Some things deserve the “artisanal treatment”—if part of what you love about cooking is the exploration and improvisation, then there may be no need to systemize a list of meals and ingredients and recipes (though you might still benefit from tweaking the layout and contents of your kitchen). And there are many tasks and activities that are small enough, infrequent enough, or low-cost enough that it’s not worth the up-front investment of thinking through and building a robust system.

Most of our participants, though, find that there is a lot of opportunity for gain from systemization. Try mentally running through the following domains, looking for things that snag or require effort or are annoying or consume a lot of attention:

Common routines

Waking up: Dealing with the alarm, using the bathroom or shower, getting dressed, gathering the things you need for the day
Meals: Deciding what to eat (and when), shopping, budgeting, taking care of dishes and leftovers
Work: Commute, getting settled, parts of your job that tend to be the same week in and week out
Computer: Startup, tab and window management, sites that you rou- tinely visit and programs that you regula

...
How do AI timelines affect how you live your life? 
by Quadratic Reciprocity — 2022-07-11T13:54:12.961Z

This question is more about personal decision-making rather than for example deciding to work on AI safety for altruistic reasons. If I were thinking about this from a purely selfish perspective, it seems pretty likely that if I expect transformative AI to arrive in 20 years, I should live my life a bit differently than people who don't expect to get TAI in their lifetimes.
I'm curious about if your own beliefs on AI timelines have affected anything that you do in your personal life - perhaps decisions related to saving money, personal relationships, health etc.
Checksum Sensor Alignment
by lsusr — 2022-07-11T03:31:51.272Z

Problem

Suppose you have a powerful AGI protecting a diamond in a vault[1]. You have various sensors in the room, all monitoring the diamond. There is a camera, watching the diamond. There is a pressure plate under the diamond confirming it is still in its stand. There are a total of 10 sensors. The powerful AGI can hack all 10 of your sensors.

How do you align the AGI such that it does not hack your sensors? You cannot tell (directly) whether the AGI believes the diamond is in the vault. You must describe your instructions to the AGI in terms of the sensor readings.

Solution

Instruct the AGI to perform the simplest series of actions that cause 9 of your sensors to agree and 1 of your sensors to disagree. Suppose (without loss of generality) that the sensor which disagrees is your camera. Either the AGI hacked the camera or the AGI has hacked your other nine sensors. The AGI has probably just hacked your camera. Carefully examine what the AGI did to hack your camera. Fix the vulnerabilities.

Repeat the above process until the AGI can no longer hack your sensors.

This is a variant on ARC's ELK competition. ↩︎