trentmkelly/LessWrong-43k huggingface dataset:
Helio-Selenic Laser Telescope (in SPACE!?)
by Alexander Gietelink Oldenziel — 2023-05-26T11:24:26.504Z

The use of drones in modern warfare has become increasingly prevalent in recent years. Drone swarms can be difficult to track and defend against using anti-air defense.

One potential solution to this problem is the use of mobile ground-based laser system to combat drones.

The biggest disadvantage is the cost of implementing such a system. Drones are cheap. Lasers are expensive.
Another disadvantage is the potential for bad weather to disrupt its effectiveness. In poor weather conditions such as heavy rain or fog, the laser beam may be diffracted or scattered, reducing its accuracy and effectiveness.
The phenomenon of thermal blooming means that laser weapons bleed off energy when penetrating the atmosphere, limiting their effective range, even in good weather conditions.
Laser weapons are limited by their need for a direct line-of-sight.
Laser weapons are limited by the diffraction limit (see below).

These are severe limitations that plausibly will prevent laser weapons from being deployed on the battlefield.

A potential way to get around the last two limitations is by deploying a (mobile) laser system together with an interlinked network of "mirror drones" or "selenic drones". These 'selenic' drones would carry mirrors that are able to redirect the initial laser weapon, allowing for much longer range and the ability to fire beyond direct-line-of-sight. This is similar to how moonlight is a reflection of the fact that the Moon acts as a giant mirror for the sun. Let us therefore call this system a helioselenic telescope or helioselenic hive.

The use of mirror-equipped selenic drones allows for a significantly increased range for laser weapons. By using mirrors to redirect the laser beam, it is possible to effectively fire around corners and over obstacles, greatly increasing the range of these weapons.

Another advantage is the ability to rapidly reposition the mirror drones as needed. In the event that an enemy drone swarm is moving or changing position, the mirror drones can be quickly repositioned to maintain the laser's line-of-sight, allowing for continuous engagement.

In SPACE!

Even if helioselenic hives are not cost-effective against drone swarms in the near future similar systems may still appear in the Far Future.

At the ranges and velocities that future space warfare is likely fought guns are mostly useless as an o...
Where do you lie on two axes of world manipulability?
by Max H — 2023-05-26T03:04:17.780Z

In a recent tweet, Eliezer gave a long list of examples of what he thought a superintelligence could and couldn't do:

Full text here.

The claims and probability estimates in this tweet appear to imply a world model in which:

In principle, most matter and energy in the universe can be re-arranged almost arbitrarily.
Any particular re-arrangement would be relatively easy for a superintelligence, given modest amounts of starting resources (e.g. access to current human tech) and time.
Such a superintelligence could be built by humans in the near future, perhaps through fast recursive self-improvement (RSI) or perhaps through other means (e.g. humans scaling up and innovating on current ML techniques).

I think there are two main axes on which people commonly disagree about the first two claims above:

How possible something is in principle, according to all known and unknown laws of physics, given unlimited (except for physical laws) amounts of time, resources, intelligence, etc. Interesting questions here include:

is FTL travel possible?
is FTL communication possible?
is "hacking" the human brain, using only normal-range inputs (e.g. regular video, audio), possible, for various definitions of hacking and bounds on time and prior knowledge?
is Drexler-style nanotech possible?
Are Dyson spheres and other galactic-scale terraforming possible?

How "easy" something is, in the sense of how much resources / time / data / experimentation / iteration it would take even a superintelligent system to accomplish it.
Reasonable starting conditions might include: access to an internet-connected computer on Earth, with existing human-level tech around. Another assumption is that there are no other superintelligent agents around to interfere (though humans might be around and try to interfere, including by creating other superintelligences).
Questions here include:

Given that some class of technology is possible (e.g. nanotech), what is the minimum amount of experimentation / iteration / time required to build it?
Given that some class of exploit (e.g. "hacking" a human brain) is possible in principle, what are the minimal conditions and information bandwidth under which it is feasible and reliable for a near-future realizable superintelligence?

These two axes might not be perfectly orthogonal, but one could probably still come up with a Political Compass-style quiz that asks the user ...
Some thoughts on automating alignment research
by Lukas Finnveden — 2023-05-26T01:50:20.099Z

As AI systems get more capable, they may at some point be able to help us with alignment research. This increases the chance that things turn out ok.[1] Right now, we don’t have any particularly scalable or competitive alignment solutions. But the methods we do have might let us use AI to vastly increase the amount of labor spent on the problem before AI has the capability and motivation to take over. In particular, if we’re only trying to make progress on alignment, the outer alignment problem is reduced to (i) recognising progress on sub-problems of alignment (potentially by imitating productive human researchers), and (ii) recognising dangerous actions like e.g. attempts at hacking the servers.[2]

But worlds in which we’re hoping for significant automated progress on alignment are fundamentally scary. For one, we don’t know in what order we’ll get capabilities that help with alignment vs. dangerous capabilities.[3] But even putting that aside, AIs will probably become helpful for alignment research around the same time as AIs become better at capabilities research. Once AI systems can significantly contribute to alignment (say, speed up research by >3x), superintelligence will be years or months away.[4] (Though intentional, collective slow-downs could potentially buy us more time. Increasing the probability that such slow downs happen at key moments seems hugely important.)

In such a situation, we should be very uncertain about how things will go.

To illustrate one end of the spectrum: It’s possible that automated alignment research could save the day even in an extremely tense situation, where multiple actors (whether companies or nations) were racing towards superintelligence. I analyze this in some detail here. To briefly summarize:

If a cautious coalition were to (i) competitively advance capabilities (without compromising safety) for long enough that their AI systems became really productive, and (ii) pause dangerous capabilities research at the right time — then even if they only had a fairly small initial lead, that could be enough to do a lot of alignment research.

How could we get AI systems that significantly accelerates alignment research without themselves posing an unacceptable level of risk? It’s not clear that we will, but one possible story is motivated in this post: It’s easier to align subhuman models than broadly superhuman models, an

...
What's your viewpoint on the likelihood of GPT-5 being able to autonomously create, train, and implement an AI superior to GPT-5?
by Super AGI — 2023-05-26T01:43:47.845Z

If this seems unlikely, could you elaborate on the reasons? What essential capabilities would it lack in order to accomplish self-improvement? For instance:
- Enhanced Short-Term Memory: An extended token window
- Improved Long-Term Memory: The competence to modify or augment its own weights and training dataset
- Specific Resources: Access to a DGX data center for testing and training purposes
- Any other abilities?

In my perspective, GPT-4 already demonstrates respectable proficiency in code composition. However, it seems to fall short in short-term memory capacity, which is a crucial element for planning and testing associated with larger code blocks and projects, such as developing a new GPT. Thoughts?
Before smart AI, there will be many mediocre or specialized AIs
by Lukas Finnveden — 2023-05-26T01:38:41.562Z

Summary:

In the current paradigm, training is much more expensive than inference. So whenever we finish end-to-end training a language model, we can run a lot of them in parallel.

If a language model was trained with Chinchilla scaling laws on the FLOP-equivalent of a large fraction of the world’s current GPU and TPUs: I estimate that the training budget could produce at least ~20 million tokens per second.

Larger models trained on more data would support more tokens per second.

Language models can also run faster than humans. Current models generate 10-100 tokens per second. It’s unclear whether future models will be slower or faster.

This suggests that, before AI changes the world via being broadly superior to human experts, it will change the world via providing a lot of either mediocre (by the standard of human experts) or specialized thinking.

This might make the early alignment problem easier. But the full alignment problem will come soon thereafter, in calendar-time, so this mainly matters if we can use the weaker AI to buy time or make progress on alignment.

More expensive AI → you can run more AIs with your training budget

The more expensive it is to train AI an AI, the more copies of that AI system can be run in parallel using your training budget. At least, that's the case if we’re making them more expensive by increasing parameter-count and training data.

We’re currently in a paradigm where:

Training isn’t very sample-efficient.

When increasing capabilities, training costs increase faster (~squared) than inference costs.

Training is massively parallelizable.[1]
While this paradigm holds, it implies that the most capable models will be trained using massively parallelized training schemes, equivalent to running a large number of models in parallel. The larger the model, the more data it needs, and so more copies of them will have to be run in parallel during training, in order to finish within a reasonable time-frame.[2]

This means that, once you have trained a highly capable model, you are guaranteed to have the resources to run a huge number of them in parallel. And the bigger and more expensive the model was — the more of them can run in parallel on your training cluster.

Here’s a rough calculation of how many language models you can run in parallel using just your training cluster:

Let’s say you use p parameters.

Running the model for on

...
how humans are aligned
by bhauth — 2023-05-26T00:09:20.626Z

This is a description of how I think humans are kept somewhat aligned towards genetically-specified goals. It's not an argument, just my views.

It works well enough, but not perfectly. South Korea isn't having kids anymore. Sometimes you get serial killers or Dick Cheney. So, anything less seems likely to be inadequate.

previously: AI self-improvement is possible

limited self-modification

Don't allow systems to modify lower-level systems, and strongly limit self-modification at the same system level. When it's done at all, reduce the amount allowed after an initial learning period, so that children have more flexibility than old people.

limited length

Don't have long chains of systems generating systems. Limit things to 2 steps of systems generating higher-level systems.

lifespan

Even if you try to restrict self-modification, some "leakage" will happen anyway. A physical time limit that can't be extended past a maximum keeps that from becoming excessive. (Perhaps that's why some octopus species are "hardcoded" to die at specific points in their reproductive cycle.)

old age

More-capable misaligned systems are more dangerous, so degrading capabilities of systems as they approach their time limit makes them safer. I don't actually think this is the limiting factor for human senility; I think that's mainly due to uncontrolled covalent modification of DNA. What I do think is that humans have some low-level system that, when they get old, reduces influence of or shuts down certain mid-level systems.

monitoring

Humans have monitoring systems with a lesser degree of self-modification than what they monitor. They have access to the internal opinions of what they monitor, but can be deceived. The net benefit is actually somewhat questionable.

democracy

If alignment drift is somewhat random, then making many separate agents that act according to consensus reduces the net drift. Hermits and dictators do weird stuff.

internal democracy

Like democracy, but for multiple agents inside a single individual. Obviously there has to be some way to prevent agents from coalescing into a single blob, but that could be managed by lower-level hardcoded systems that blindly bottleneck the bandwidth of some connection patterns and force some of that limited bandwidth through low-level systems.
What vegan food resources have you found useful? 
by Elizabeth — 2023-05-25T22:46:24.994Z

My vegan nutrition project has focused on supplements, but there have been several requests for resources on how to eat better. I'm not well equipped to answer this- I'm not vegan, and my stomach only got even passably good at digestion a year ago. So I'd like to crowdsource this question: what resources have you found useful in eating well while eliminating or reducing animal products?

My existing list focuses on overviews and onboarding guides, but I think things like recipe books would also be useful.

Veganhealth.org is the number one resource I see cited, by both people I know and other vegan websites.
TheVeganRD.com guide is far from comprehensive but seems extremely approachable.
A reader pointed me to spreadsheet, which links to a number of resources.
Lincoln Quirk's Practical Plant-Based Meal Planning for Groups

Have people found these helpful? Are there other things that are better, or fill in a gap? How did you learn to cook well?

Diet and supplements overlap but there is already a supplements post and I don't want to split the information, so please lean food-based here.
Mob and Bailey
by Screwtape — 2023-05-25T22:14:52.377Z

Epistemological status: Moderately confident that this is a more useful way to use a concept that has been expanded upon by others.

Previous building blocks: See Logical Rudeness and All Another Brick in the Motte and for the foundations, as well as Against Accusing People of Motte and Bailey for the direct predecessor.

If you haven’t read the previous building blocks, the core idea is called the Motte and Bailey. A Motte and Bailey argument is what you call it when someone makes a clearly supported and uncontested claim, then makes an outrageous but advantageous claim, then swaps between these two claims whenever it's useful to them. It draws from the medieval tactic of having an easily farmable bailey right next to a heavily fortified motte, then moving your peasants and troops back and forth between them whenever raiders come or leave.

I

Amy and Bob would like to have a civil discussion about a philosophical difference they have. Their conversation goes something like this:

Amy: I don't understand why you think tautologies are important. I mean, you can't get any extra information out of them, right?
Bob: There are actually a number of different kinds of tautologies. For example, a logical tautology might say "either X equals Y or X does not equal Y" and while you might be correct that no new information is gained from this, I find it helps me organize my thoughts.
A: Ah, I didn't know that. I've mostly seen them used as rhetorical devices.
B: They can be used that way, but it's far from the most interesting thing about them for me.
A: As long as people are going to keep using tautologies to win arguments though, how do we help those who don’t understand them well enough to defend against tautology based arguments?
B: Oh go soak your head. I think if you learned more about them you’d be able to actually counter them when people did use them in arguments.
A: Even if I studied tautologies enough to do so, I worry that making a general rule of needing to study all potential rhetorical devices to be able to defend against them might be prohibitively difficult.
B: As much as I love tautologies, I do think tautology proponents should be more careful in their usage.
B: At least as long as we have to deal with idiots who try to ban anything they don’t understand.

This conversation disintegrated quickly. Bob seems to be moving between the position th...
Look At What's In Front Of You (Conclusion to The Nuts and Bolts of Naturalism)
by LoganStrohl — 2023-05-25T19:00:44.835Z

If naturalism has been the subject of this sequence, then what is naturalism? Now might be a good time to try stating your own answer, in your own words, before reading more of mine.

Here is what I have to say about it.

Naturalism is an investigative method that focuses attention on the points in daily life where subjective experience intersects with crucial information.

It brings reflective awareness to experiences that were always available, but that our preconceptions inclined us to discard; it thereby grants us the opportunity to fold those observations into our stories about the world. It is a gradual process of original seeing, clarification, and deconfusion.

At its best, naturalism results in a greater ability to interact agentically with the world as it is, rather than fumbling haphazardly through a facade of misapprehensions.

Despite belonging to a whole different phylum, the white-lined sphinx moth is easily mistaken for a hummingbird when seen briefly or from a distance.

This sequence was my attempt to outline the curriculum I guide people through when helping them learn the practice of naturalism—when helping them learn to behave as though knowing the territory takes patient and direct observation.

The curriculum is not the practice itself. Pedagogical considerations require that the curriculum be relatively linear and self-contained. The actual practice of naturalist investigation tends to draw as needed from all phases at once, interweaves with other methodologies as appropriate, and sometimes requires the invention of novel techniques on the spot.

Furthermore, my curriculum does not teach The One True Way of Patient and Direct Observation. It is only my way, tailored to and limited by the idiosyncratic shape of my particular mind, and adapted somewhat for the unusual class of people willing to work with me.

This is not an easy practice, and it certainly is not fast. It is a lot like lifting weights to get stronger. When easy answers are readily available through expert opinion; when problems are technical rather than adaptive; when you are certain that all you need is a “yes” or a “no”, and not to build a whole new way of understanding—in these cases, naturalism is often the wrong toolset for the job.

Nevertheless, I find that even when naturalism is the wrong toolset overall for the particular question I'm deali...
[Market] Will AI xrisk seem to be handled seriously by the end of 2026?
by tailcalled — 2023-05-25T18:51:49.184Z
Link: https://manifold.markets/tailcalled/will-ai-xrisk-seem-to-be-handled-se?r=dGFpbGNhbGxlZA

https://manifold.markets/tailcalled/will-ai-xrisk-seem-to-be-handled-se?r=dGFpbGNhbGxlZA

Lately it seems like there's been a bunch of people taking big AI risks seriously, from OpenAI's Governance of superintelligence to Deepmind's Model evaluation for extreme risks.

We're not quite there yet in terms of my standards for safety, and even less there in terms of e.g. Eliezer Yudkowsky's. However I wonder if this marks a turning point.

Obviously this is a very subjective question, so I am warning you ahead of time that it is going to resolve in opinionated ways. Trying to anchor the discussion, I expect the following to be necessary for a YES resolution:

Major leading AI companies openly acknowledge that existential risk is a possibility, not just in a marginal sense (e.g. internal discussion of it by employees, rare cases where the leaders begrudgingly admit it) but also in a central sense (e.g. openly having research teams working on it, having long sections of documents for politicians discussing it).
Something is done to handle unilateral actors, e.g. there is active progress made in creating an international organization which can prevent unilateral actors from creating unfriendly AI, or maybe somehow the only plausible creators all take AI xrisk seriously.
Yann LeCun changes his mind to take AI xrisk seriously or no longer holds much sway about it at Facebook.
The lessons of Worlds Where Iterative Design Fails are taken seriously by the above systems.

Please ask me more questions in the comments to help cement the resolution criteria. If my opinion on the inherent danger of AI xrisk changes during the resolution period, I will try to respond based on the level of risk implied by my criteria, not based on my later evaluation of things.

However, if it turns out that there is a similarly powerful way of handling AI xrisk which is qualitatively different, which gets implemented in practice, I will also resolve this question to YES.
What should my college major be if I want to do AI alignment research?
by metachirality — 2023-05-25T18:23:43.442Z

And what college degree level? (associate, bachelor's, master's, or doctorate)
It is also possible that I could just not go to college at all.
For context, I'm already very interested in pure math and have been self-learning other subjects such as probability theory and AI for the purposes of doing AI alignment research at some point.