trentmkelly/LessWrong-43k huggingface dataset:
Tactical Nuclear Weapons Aren't Cost-Effective Compared to Precision Artillery
by Lao Mein — 2022-10-31T04:33:36.855Z

Epistemic status: there is probably a US Army analysis that does this much better.

Tactical nuclear weapons (TNWs) are often times portrayed as being significantly more effective than conventional artillery. My research indicates that they are actually less effective than precision weapons of similar cost.

These have an average deviation of 2m with GPS guidance, far lower than their kill radius. I've seen people wondering why Russia wasn't using S-400 anti-air systems to intercept HIMARS strikes, with some reasoning that Ukraine was using them in conjunction with unguided rockets to make them difficult to identify. The real reason is economics. S-400 systems are something like $400 million (!!) per system (8 launchers + radar + control systems + 72 missile package). This comes to something like $5 million per interceptor launched. In contrast, HIMARS are $3.5 million per launcher + reloader system. The procurement cost for GMLRS systems, used by M270 and HIMARS, is around $150,000 per M30 rocket. M30s don't need decoys, since any interception system is going to be more expensive than it is.

Compare the cost of an M30 to a tactical nuclear warhead. The B61-7 warhead (300-400kt) costs around $28 million to build, and a lot to maintain. It also needs a delivery system, which is going to increase the price even more. Let's assume the most optimistic kill radius of 8km, which Nuke Map gives as the 100% 3rd degree burn distance for unprotected civilians. The kill-radius against armored vehicles is going to be far less (probably less than 2km for a ground-burst). In reality, troops don't neatly fit themselves into an 8km circle. They tend to spread themselves into a line. The vast majority of the killing power of a nuclear weapon is wasted against a line of troops. In order to kill a tank with a tactical nuke, it needs to either be destroyed by the air blast (it's a 60-ton tank, a detrack is probably the best you get), or literally boil off the metal to the point that the crew is exposed (the nuclear fireball for a 400kt groundburst only ha around a 800m radius for a groundburst). Metal is rather good at protecting vehicle crews from heat and radiation, and modern military vehicles are all fitted with air filters specifically for countering fallout during a nuclear war. In addition, warfare in Ukraine is centered around minimizing losses from artillery, which pe...
Gandalf or Saruman? A Soldier in Scout's Clothing
by DirectedEvolution — 2022-10-31T02:40:42.516Z

Sometimes, Scout Mindset can feel a lot like Soldier Mindset, and vice versa.[1]

When you've considered an issue with some care, formed an opinion, and somebody responds with raised voice about how awful that opinion is, it's hard not to get defensive or take it personally.

There are lots of ways you could respond. If your aim is to achieve a thoughtful and respectful conversation, then you have to do several things: control your emotional reaction, be polite, stay honest about the fact that you disagree, and choose a response that allows the discussion to return to civil discourse.

Going through all that feels like a Scout-y thing to do, but it isn't necessarily. Provoking a person with a strongly held (but possibly correct) view into offputting behavior, then assuming a posture of being the reasonable and civil side of the debate, is a wonderful rhetorical strategy. Loudly trying to shout down your opponent makes you look like a dumb Soldier. A smart Soldier tries to look like a Scout.

Likewise, if you're a Scout and your information is being ignored, you might need to raise your voice, fight your way through, and make them listen.

People who are into signaling talk a lot about virtue signaling and intelligence signaling. It might be helpful to talk about Scout signaling and Soldier signaling. Neither of these are the same as being a Scout or Soldier. They aren't reliable indicators in every context of where somebody is coming from. If a person is interested in developing Scout Mindset, it seems important to me for them to learn how to distinguish Scout Mindset from Scout signaling.

Back to our acrimonious debate, let's say the civil person in the debate sincerely wants to want to practice Scout Mindset. There are a couple of useful things they can do.

One is to stick to their honest beliefs. That is part of the territory they are honestly scouting out.
Another is to remember that their loud debate partner might be a Scout with a very important message. Or not! But if you haven't thoroughly explored their opinion and formed your own conclusions, it's often wise to reserve judgment. If you do so, and stick to your original view, then having fully internalized what your debate partner had to say will help you address that point the next time you're in debate.

I'll finish with an example from a debate I had today in which I was playing the "civil" role and my debate p...
Me (Steve Byrnes) on the “Brain Inspired” podcast
by Steven Byrnes — 2022-10-30T19:15:07.884Z
Link: https://braininspired.co/podcast/151/

Official episode page — Listen on your favorite podcasting app — Watch on YouTube

The discussion centered around my blog post series from earlier this year—“Intro to Brain-Like AGI Safety”—as well as AGI and safety and alignment more generally.

As context, the Brain Inspired podcast (which I have long listened to myself!) focuses on the intersection between neuroscience and AI. My assumption was that most listeners would be either unfamiliar with or skeptical about AGI in general and AGI x-risk in particular, so I tried my best to keep things simple and relatable, avoid jargon, and so on.

As is inevitable in a real-time format, there are lots of places where I misspoke[1] or could have said things better. But I’m generally pretty happy with how it turned out. :)

Thanks Paul Middlebrooks for having me on!

^

Next time I hear someone on a podcast say something that betrays a complete lack of understanding of some topic X, I will be more open to the possibility that they understand X perfectly well, and were just tripping over their words. Talking is hard, man!!
"Normal" is the equilibrium state of past optimization processes
by Alex_Altair — 2022-10-30T19:03:19.328Z

Orienting around the ideas and conclusions involved with AI x-risk can be very difficult. The future possibilities can feel extreme and far-mode, even when we whole-heartedly affirm their plausibility.

It helps me to remember that everything around me that feels normal and stable is itself the result of an optimization process that was, at the time, an outrageous black swan.

Modernity

If you were teleported into the body of a random human throughout history, then most likely, your life would look nothing like the present. You would likely be a hunter-gatherer, or perhaps a farmer. You would be poor by any reasonable standard. You would probably die as a child. You would have nothing resembling your current level of comfort, and your modern daily life would be utterly alien to most humans. What currently feels normal is a freeze-frame state of a shriekingly fast feedback loop involving knowledge, industry, and population. It is nowhere near normal, and it is nowhere near equilibrium.

The Anthropocene

This planet started glowing in the dark one day, and that is not normal.

For hundreds of millions of years, the earth was a wilderness, teeming with life. The teeming had ebbs and flow, evolutionary breakthroughs and power struggles, but essentially, it was a type of equilibrium.

For hundreds of thousands of years, humans were just another animal. We ran around the savanna, killed other animals and foraged for food, had sex, slept, and experienced joys and losses. If an alien were gazing at earth from afar, they would have had no reason to think humans were different from any other animal.

But all the while, humans were noticing things. They were curious, and desperate, and they had a capacity that the other animals didn't have. They got ideas about the world, and tested them with their hands, and gave this knowledge to their children with their words. Over time, the humans became conspicuous, and eventually, they transformed every inch of their surroundings.

For a while now, humans have been a threat to the species they live with. If you are a fish or a weed or a polar bear, your biggest problem is humans. But as John Green points out in The Anthropocene Reviewed, in the 21st century, if you are the atmosphere or a river or a desert, your biggest problem is humans. The rise of humanity is no longer just an ecological phenomenon; it has kicked off a new geological epoch. ...
AI as a Civilizational Risk Part 2/6: Behavioral Modification
by PashaKamyshev — 2022-10-30T16:57:51.046Z

Altering people for the worse using AI is easier than altering them for the better.

Redefining what “Capital” means

Narrow AIs and optimization algorithms of social media news feeds have already caused many adverse outcomes without being AGIs. When people talk about AI on Twitter or Less Wrong, they use manufacturing or job-replacement analogies as the core function of AI. This analogy imagines AI as a powerful extension of industrial capital, defined by its "ownership of the means of production." This 19/20th-century thought is becoming increasingly removed from the reality of capital today.

The core business of the largest tech companies, whether social media giants, advertising behemoths like Google, or middle-layer companies like Amazon, is not about production.

It is about behavioral modification.

This behavioral modification takes many forms: from digital addiction to engagement-driven news feed in the case of social media. Advertising funds much of the technology world and has become a hyper-optimized system for getting people to buy or engage with products. Usually, the original focus of behavioral modification is habit formation to use the particular company's site. Initial habit formation is innocent enough if done by cleaning up the UI and fixing issues. However, many of these goals eventually shift into the over-optimization territory.

In the 21st century, the term "Capital" has effectively changed meaning to become less about the means of production and more about the means of behavior modification. "Age of Surveillance Capitalism" has written about this change of meaning. Historically, behavioral modification has been mostly a function of governments, religions, or media. Large-scale involvement of capital in behavioral modification puts companies into traditionally occupied roles by those institutions, which creates predictable conflict and mutual attempts to control each other.

Companies frequently have governmental involvement

, while

corporate lobbying has also increased.

Viewing AI as an extension of capital is a good perspective. However, its manufacturing side is less dangerous and less impactful than the behavioral modification side of technocapital.

The leading carrier of behavioral modification problems is social media. Social med...
Instrumental ignoring AI, Dumb but not useless. 
by Donald Hobson — 2022-10-30T16:55:47.555Z

This post is yet another unworkable agent design that uses infinite compute. It outlines an AI that ignores all instrumental values, pursuing only it's terminal values.

What is does it look like to pursue an instrumental value. It means you can predict the AI is likely to do action X, even if you have no idea what it's utility function is.

This AI doesn't do that. It obeys the simple principle that, if you are clueless about it's utility function, then you are clueless about it's actions.

Let ΔX mean the probability distribution over the set X.

Lets have some set of Observations O, Actions A and (bounded? ) Utilities U.

Then Pick some probability distribution over utility functions U∈ΔU, such as uniform or complexity weighted. And some probability distribution A∈ΔA over the action space.

Then let the AI be a function f:(U,O)→ΔA

We can formalize our condition as ∀o∈O:∀a∈A:∑u∈UP(f(u,o)=a)U(u)=A(a).

(Note U(u)∈R refers to the probability that U assigns to u.)

Why might we expect such agents to be safe. Because if we picked a random utility function, we get a safely random output. So all humans need to do is be better than random in our choice of utility function.

Now let E(o,a,u) be the expected utility. Choose f to maximize ∑u∈U∑a∈AP(f(u,o)=a)U(u)E(u,o,a) subject to the condition. As the condition doesn't relate cases with different values of o, this optimization only needs to be computed for the particular o∈O observed.

Is this AI design safe when repeatedly called with the same utility function?

Imagine the action space just consists of outputting 0 or 1. And there are only 2 utility functions, paperclips and staples. We could imagine that when run millions of times with a utility of paperclips, this AI outputs code for a paperclip maximizer, and when run millions of times with a goal of staples, it outputs a bitwise negation of the paperclip maximizer. (Which could possibly be a staple maximizer, if the AI has done some op code magic)

This would be because the humans choice of the same utility function each time is worse than random. I think this is an actual problem. At least if the AI's have total knowledge of past and future, then this is one Schelling point to their coordination game. (The consider all possi...
Weekly Roundup #3
by Zvi — 2022-10-30T12:20:00.516Z

Another experiment for everyone (in addition to the one in this week’s Covid post) is that I want to better figure out when to split things off into their own posts. Is there anything here you would link to or want to later refer back to, and thus you think would benefit from standing on its own and/or is worth expanding out? Leave a comment to that effect, or like a comment already there, and good chance I’ll go ahead and do that. One of the problems with writing is the constant lack of good feedback.

[Note: This post was written on Friday and put on Substack, I simply forgot to post it to WordPress then]

Who To Blame

It is always important to have someone to blame. Does the person have to actually be responsible? Not strictly speaking, no, although it helps. There does need to be some symbolic link there, however flimsy.

For example: Thanks, Obama (or Biden, or the president of your choice).

Magic Online not working or give you a bad hand? Blame Worth Wollpert.

Skype not working or has bad call quality? Blame Jaan Tallinn.

Google Search getting worse every year? Blame, or complain to, Danny Sullivan.

(Also, yes. Yes it is.)

You heard him.

Sports Go Sports

NBA puts teams ‘on notice’ about tanking. This year’s upcoming draft class has a phenomenal top two picks, and there are six teams with expected win totals of under 27 out of 82 games (so less than a third). I expect quite a bit of tanking.

The commissioner has thought of the correct solution and it is amazing.

As the new season progresses, Silver and his constituents likely will continue to face questions about tanking as next year’s draft draws closer. In regard to a possible resolution to incentivize bad teams, Silver said the league has thought about relegation, but admitted it would be “destabilizing” to the NBA’s business model.

However, if this scenario were to come to fruition, Silver explained that relegation would potentially see one or two of the NBA’s worst teams demoted to the G League, while the one or two of the G League’s best squads would be promoted to the NBA.

“It would so disrupt our business model,” Silver said, per ESPN. “And even if you took two teams up from the G League, they wouldn’t be equipped to compete in the NBA.”

If they did this, I might actually start watching basketball. Relegation is awesome.

The ‘our franchises want to keep being worth billion...
Quickly refactoring the U.S. Constitution
by lc — 2022-10-30T07:17:50.229Z

Presented mostly without comment, but see the footnotes. The game is that it can't look out of place on Earth, so no futarchy. Bugfixes encouraged.

Legislative Branch

There is only one house of legislators, named The Senate, and it contains 100 senators.[1]

With a 50/100 senator majority, The Senate can repeal laws.
With a 55/100 senator majority, The Senate can pass or amend a new law, which will be enacted by the chief executive.[2]

Senators are voted in for eight year terms and cannot be re-elected.[3]
Senatorial elections are timed so that results are announced a month before the end of the calendar year. [4]

After election announcements, previously elected senators immediately abdicate their roles and take a ranked snap-vote for their "trainee", whom they will help transition and counsel into their new roles.[5] Incumbent senators will have the opportunity to learn about their new job and get national security briefings for about a month, and after the dawn of the new year senate elections will start.[6]

Senators are not voted in directly. Instead, during Voting Season, 160,000 "jurors" are lotted from the American populace. These jurors in turn do ranked voting to elect 4,000 "delegates", and those delegates do ranked voting to decide the 100 senators. [7]

All jurors and delegates vote for all positions at the same time[8]. Ranked voting systems determine the 4,000 and 100 that win, respectively.
Jurors and delegates must vote for at least 100 delegates and 20 senatorial candidates, respectively. There is no maximum.[9]
Jurors and delegates have two weeks and one month to vote for candidates, respectively.

Jurors cannot vote for themselves. They can however be voted into delegateship by other voters.
Delegates are sequestered from each other and direct contact from the public while they do research, and their communications during the month are monitored.
Delegates cannot vote for themselves. They can however be voted into senatorship by the remaining delegates.

Jurors and delegates can abstain, but must do so explicitly at the beginning of the process. If they don't abstain, they get grants pegged to national income from the government to cover living expenses and research while they take time off from work. The grants for delegates are 5x larger than the grants for jurors.[10]
Direct campaigning for senate or delegacy races is illegal.[11]

In par

...
«Boundaries», Part 3a: Defining boundaries as directed Markov blankets
by Andrew_Critch — 2022-10-30T06:31:00.277Z

This is Part 3a of my «Boundaries» Sequence on LessWrong.

Here I attempt to define (organismal) boundaries in a manner intended to apply to AI alignment and existential safety, in theory and in practice. A more detailed name for this concept might be an approximate directed (dynamic) Markov blanket.

Skip to the end if you're eager for a comparison to related work including Scott Garrabrant's Cartesian frames, Karl Friston's active inference, and Eliezer Yudkowsky's functional decision theory; these are not prerequisites.

Motivation

In Part 3b, I'm hoping to survey a list of problems that I believe are related, insofar as they would all benefit from a better notion of what constitutes the boundary of a living system and a better normative theory for interfacing with those boundaries. Here are the problems:

AI boxing / Containment — the method and challenge of confining an AI system to a "box", i.e., preventing the system from interacting with the external world except through specific restricted output channels (Bostrom, 2014, p.129).
Corrigibility — the problem of constructing a mind that will cooperate with what its creators regard as a corrective intervention (Soares et al, 2015).
Mild Optimization — the problem of designing AI systems and objective functions that, in an intuitive sense, don’t optimize more than they have to (Taylor et al, 2016).
Impact Regularization — the problem of formalizing "change to the environment" in a way that can be effectively used as a regularizer penalizing negative side effects from AI systems (Amodei et al, 2016).
Counterfactuals in Decision Theory — the problem of defining what would have happened if an AI system had made a different choice, such as in the Twin Prisoner's Dilemma (Yudkowsky & Soares, 2017).
Mesa-optimizers — instances of learned models that are themselves optimizers, which give rise to the so called inner alignment problem (Hubinger et al, 2019).
Preference Plasticity — the possibility of changes to the preferences of human preferences over time, and the challenge of defining alignment in light of time-varying preferences (Russell, 2019, p.263).
(Unscoped) Consequentialism — the problem that an AI system engaging in consequentialist reasoning, for many objectives, is at odds with corrigibility and containment (Yudkowsky, 2022, no. 23).

Also, in the comments after Part 1 of this sequence, I asked commenters t...
Am I secretly excited for AI getting weird?
by porby — 2022-10-29T22:16:52.592Z

This post is arguably darker than my other one. I don't make any persuasive arguments about AI forecasting here; if you don't feel like looking at doominess, feel free to skip this.

I've noticed a few instances of what look like people assuming that those who are visibly concerned about AI risk don't really buy into the full weight of what they're saying.

Recently, I came across this (hi, niknoble!):

As a specific example of what I suspect is a bit of cognitive dissonance, look at the recent post on AGI by porby, which predicts AGI by 2030. I loved reading that post because it promises that the future is going to be wild. If porby is right, we're all in for an adventure. Based on the breathless tone of the post, I would surmise that porby is as excited by his conclusion as I am. For example, we have this excerpt:

This is crazy! I'm raising my eyebrows right now to emphasize it! Consider also doing so! This is weird enough to warrant it!

Would you have predicted this in 2016? I don't think I would have!

Does this strike you as someone who dreads the arrival of AGI? It seems to me like he is awaiting it with great anticipation.

But then in the comments on the post, he says that he hopes he's wrong about AGI! If you're reading this porby, do you really want to be wrong?

This is an excellent example of the kind of thing I'm talking about, so I'm going to use it.

I think my writing and speaking style defaults to a kind of lightness that can be misleading. So let me try to write something a little darker.

Well, do you?

Because I don't think P(doom | AGI) is anywhere close to 0, especially for AGI developed on very short timescales:
YES, I DO WANT TO BE WRONG.

The kind of "excitement" I feel about near-term AGI is adjacent to hearing the tornado siren, looking at the radar, seeing the warned cell moving straight east, walking out on my porch to look at a black wall of rain a mile or two away, and seeing the power flashes straight west of me as the tornado rips lives apart. While grabbing a mattress to throw over a tub, I'm doing some quick mental calculations- the statistical rarity of EF-3 or stronger tornadoes, will it stay on the ground, how large is it (glance at the hook on the reflectivity map), how sturdy is this house (the feeling of the entire house shunting to one side during an earlier storm's 120 mph winds wasn't promising), how much damage would ...