trentmkelly/LessWrong-43k huggingface dataset:
Review: Bayesian Statistics the Fun Way by Will Kurt
by matto — 2022-11-19T18:52:22.043Z

The main conference hall enveloped me in its gentle, air-conditioned embrace. Outside, in typical New York City fashion, the sun burned and the heavy, humid air threatened to suffocate everyone indiscriminately.

After registering and picking up my badge, I found myself drawn immediately to one specific vendor booth. This vendor was No Starch Press, a publisher of fine works centered on software.

I have a weakness for purchasing books. It's only gotten worse with time, so I built up a habit of restraint that kicks in, automatically, whenever I find myself where the printed word is transacted. It's 98% effective. But this time, I walked away, grinning, with a crisp, slim book called "Bayesian Statistics the Fun Way" by Will Kurt.

Kurt begins by describing what probability is. Then he proceeds to build a platform made up of the basics: fundamental notation & operations; the binomial, normal, and beta distributions; conditional probability; and Bayes's theorem. Onto this platform he hoists more complex topics like parameter estimation, Bayes's Factor, and hypothesis testing. All of that in under 200 pages (the rest are appendices), which the author suggests could be covered over a longer flight.

That's possible. Kurt's style is smooth and light, which allows for quick progress. More importantly, though, the knowledge is laid out so that each chapter feels like just a small inferential step away. Occasionally, a chapter backtracks in order to offer a new perspective on an earlier topic. All of this combines into an electrifying effect; almost every page yields a new, satisfying insight.

I can't help but remind myself how the same material was treated in high school. There the knowledge was spread out in a fifth of the space--and most of that was devoted to lists of problems. (Great for homework and testing, probably). Perhaps it was my teenaged brain, but absolutely nothing connected: not permutations, not distributions, not Bayes's theorem. None of it also had any relation to reality whatsoever--what madman carries on their person loaded dice or bags upon bags of many-colored marbles?

But I digress.

Of particular help were the few problems that closed off each chapter. At first, they were simple enough to solve with pen and paper, but later I had to bring out my laptop and fire up Jupyter Notebooks. (The book presents the material using R, but I had no difficulty a...
How does acausal trade work in a deterministic multiverse?
by sisyphus — 2022-11-19T01:50:56.299Z

Acausal trade is speculated to be possible across a multiverse, but why would any rational individuals want to do this if the multiverse is deterministic? The reality measure occupied by all of the branches of the multiverse are pre-determined and causally independent from each other, so no matter what you "do" in your own branch, you cannot affect the reality measure of other branches. This means that even if your utility function cares about what happens in other branches, nothing you do can affect their fixed reality measure, even "acausally". This is just a consequence of making counterfactual scenarios "real".

For example, if two agents come to an acausal cooperation equilibrium, this does not reduce the pre-determined reality measure of counterfactual worlds where they didn't. For example, if your utility function is proportional to the number of paperclips that exist over the multiverse, then your ultimate utility (total number of paperclips) would be the same no matter what you "do". The only thing that can vary is how many paperclips you can experience, within your own branch of the universe. Therefore, it would only be meaningful for a utility function to be focused on your own branch of the multiverse, since that's the only way for you to talk meaningfully about an expected utility that varies with different actions. As a result, the MWI should make absolutely no difference whatsoever for a decision theory compared to a "single-universe" interpretation such as Copenhagen.

Please let me know if my reasoning is correct, and if not, why?
Choosing the right dish
by Adam Zerner — 2022-11-19T01:38:00.040Z

1

From what I understand, despite its simplicity, there aren't many dishes held in higher esteem by the culinary community[1] than the French omelette.

To all of you uncultured numskulls out there who don't spend hours upon hours watching YouTube videos about cooking like I do, a French omelette looks like this:

See how it's underdone and creamy inside? It's not the same thing that you cook for breakfast in the morning with ham and cheese. Adding cheese to a proper French omelette would be like adding ketchup to a good steak. A good steak doesn't need ketchup, and a good French omelette doesn't need cheese. It's perfect just the way it is and you'd be covering up that perfection if you added something to it.[2]

But getting a French omelette right requires a certain level of mastery.[3] Let's hear what Dan Gritzer of Serious Eats has to say:

The French omelette looms large in kitchen legend, and the story you'll most often hear is that it was the dish chefs would use to test prospective cooks. They chose an omelette of all things because, in a matter of minutes, it could show a chef everything he needed to know about the cook. Did he make an egg-splattered mess or keep things clean? Was he wasteful, or did he scrape every last bit of egg into the pan? Did he handle the pan correctly, seasoning the traditional carbon steel to give it a perfect nonstick surface? Was he quick, deft, efficient? And, after everything, did he produce that textbook almond-shaped package? Was it baveuse?

...

I'd been making French omelettes for years at that point, but the depth of detail he gave in each step made it all seem new to me. The heat had to be high the whole time, with temperature controlled by moving the pan on and off of it. The pan had to be just hot enough that you couldn't press the back of your fingers on it for more than half a second. The butter had to foam but not brown. The eggs had to be beaten just until the last trace of whites vanished—but no more than that—and salted lightly at the last second. The pepper had to be white, to avoid little black flecks.
...
Then Soltner looked at me and said, "Now you try." I'm not sure I've ever been as nervous as when I cooked my omelette under his solitary gaze. It didn't matter that I'd made hundreds, if not thousands, of French omelettes during my restaurant career. My hands shook and my fingers trembled; my

...
Reflective Consequentialism
by Adam Zerner — 2022-11-18T23:56:52.756Z

Epistemic Status: My position makes sense to me and seems pretty plausible. However, I haven't thought too hard about it, nor have I put it to the test by discussing with others. Putting it to the test is what I'm trying to do here. I also don't have the best understanding of what virtue ethics (and/or deontology) actually propose, so I might be mischaracterizing them.

There has been some talk recently, triggered by the FTX saga, about virtue ethics being good and consequentialism being bad. Eliezer talked about this years ago in Ends Don't Justify Means (Among Humans), which was summed up in the following tweet:

The rules say we must use consequentialism, but good people are deontologists, and virtue ethics is what actually works.

Maybe I am misunderstanding, but something about this has always felt very wrong to me.

In the post, Eliezer starts by confirming that the rules do in fact say we must use consequentialism. Then he talks about how we are running on corrupted hardware, and so we can't necessarily trust our perception of what will produce the best consequences. For example, cheating to seize power might seem like it is what will produce the best consequences — the most utilons, let's say — but that seeming might just be your brain lying to you.

So far this makes total sense to me. It is well established that our hardware is corrupted. Think about the planning fallacy and how it really, truly feels like your math homework will only take you 20 minutes.

Or the phenomena of optical illusions. Squares A and B are the exact same shade of grey, but your brain tells you that they are quite different, and it really, truly feels that way to you.

But look: they are, in fact, the exact same shade of grey.

Again, corrupted hardware. Your brain lies to you. You can't always trust it.

And this is where reflectiveness comes in. From The Lens That Sees Its Flaws:

If you can see this—if you can see that hope is shifting your first-order thoughts by too large a degree—if you can understand your mind as a mapping engine that has flaws—then you can apply a reflective correction. The brain is a flawed lens through which to see reality. This is true of both mouse brains and human brains. But a human brain is a flawed lens that can understand its own flaws—its systematic errors, its biases—and apply second-order corrections to them. This, in practice, makes the lens far more power

...
Value Created vs. Value Extracted
by Sable — 2022-11-18T21:34:46.629Z
Link: https://affablyevil.substack.com/p/value-created-vs-value-extracted

I’m going to start laying out longer sequences, and so I begin with a comparison between value creation and value extraction. This will form the foundation for future work and discussion on the nature and morality of wealth and the wealthy.

What It Means To Create Value

In my mind, creating value involves adding something to the world that it did not have before. What you add could be almost anything - an idea, a work of art, a methodology, an accomplishment, a story, a discovery, etc. The concept to grasp is that you added it to the world. You produced it.

This can be a subtle concept, with plenty of edge cases, so let’s use a metaphor to illustrate the point.

The Metaphor: Building a Bridge

Imagine there are two towns that wish to trade with each other, but trade is difficult because the two towns are separated by a mighty river. The current is fast and the waters treacherous, so while navigating the river by boat is doable, it involves time and effort and risk. Boats are not free, and they are dangerous to sail after dark when the rapids cannot be seen.

An engineer comes along - maybe she’s a builder or an architect or a polymath if the word “engineer” isn’t widespread in these towns, but we’ll call her an engineer. The engineer sees an opportunity in these two towns and this river, because this engineer happens to know how to build bridges.

So the engineer gathers whatever is required - manpower, capital, raw material, and so on - and constructs a bridge across the river between the two towns. The final stone is placed, the bridge is inaugurated, and people and goods can now flow between the two towns much more easily, without risking the treacherous and fast-flowing waters.

A nice-looking bridge, perfect for economics metaphors.

The bridge does not require a schedule, as boat arrivals and departures do - it is open every hour of the day and night.

It does not require a skilled captain to cross, because there are no rapids or currents on the smooth stone walkway.

It does not require the wind to blow favorably to fill sails, nor does it require the sweat of rowers or the steam of engines to push against the current.

Trade flourishes between these cities, as does diplomacy and culture and all of the things that happen when people are free to intermingle.

Value Produced

So we say - so I claim - that the engineer has created value[1], and the value that she has created is...
The Disastrously Confident And Inaccurate AI 
by Sharat Jacob Jacob — 2022-11-18T19:06:45.412Z

The following is my first post on AI Safety, in a manner of speaking. To be more precise, this is my distillation of the first post in the Risks from Learned Optimization sequence by Evan Hubringer et al. Further distillations will follow. (I do not mean to sound threatening, sorry.)

Where This All Started

Let’s get right to it.

In order to get AI to do what we want it to do, we used to tell it exactly what to do.

If it was to walk from point A to point B, we used to tell it, “Walk straight, take a left, two rights, then straight, and on another left, you are at point B.”

But after a while, we stopped doing that.

We began to tell AI to do things that we really couldn’t tell it to do precisely.

We wanted it to check images for cats and dogs, suggest words to auto-complete when we write sentences and find the shortest route to reach our grandparents' houses in time for Christmas dinner, but we couldn’t convey how to go about some of these things exactly.

So we started telling AI, “Figure it out.”

An example of an AI that “figures it out” is neural networks.

What, in a manner of abstraction, we used to do with these AI, was basically tell them, this is point A, “where we are at (my house) ” and this is point B, “where we want to be (grandma’s house because the chicken is turning cold)”. Now the AI gets to try out a bunch of random actions and we tell it if it reaches point B in time or not. After that happens a bunch of times, the AI understands there are specific sets of actions that it can take which lead us to Antarctica, and other sets of actions that take us to our grandma’s house in time.

NOTE: It is worth noting that a bunch of random actions is an abstraction of what actually occurs, as there are AIs that do perform “actions” but don't have a conscious sense of what an “action” actually is.

If you would like to understand neural networks in further detail, check

this

out.

Now, we are going to quickly dissect a bunch of terms from this set of articles that we will be using repeatedly to explain the concepts essential to this sequence.

The Terms To Know

Optimized

When a specific organism/system/process is designed to do a specific task and does not act beyond its function, we say it is optimized for that task.

Eg. A bottle cap is optimized to hold water in its place.

Optimizer

However, when the same organism/system/pro...
How AI Fails Us: A non-technical view of the Alignment Problem
by testingthewaters — 2022-11-18T19:02:42.056Z
Link: https://ethics.harvard.edu/how-ai-fails-us

Disclaimer: I am writing this post after doing some amount of reading into alignment, but I acknowledge that my priors in this case are not well established. I have conducted limited experiments into ML research (largely focusing on the utility and interpretability of language and vision transformers, including occlusion sensitivity), and have since broadened my reading into more social and humanistic writing on the subject of AI development.

I am somewhat surprised that this particular document has never surfaced on the Forums that I can find with the search function, since it on some level echoes many concerns I have seen in the space (that AI development is unconstrained, suffers from poor use of metrics in place of human goals i.e. goodhart's law, is being pursued without regards to ethics or safety concerns etc.). In particular, I believe this writing addresses three areas of concern I have found when reviewing some views within the AI alignment community:

AI development is presumed as an inevitable "fact of society". The development and popularisation of AI and related technologies is presumed to follow exponential mathematical growth (See: "Explosive scientific and technological advancement"). This is part of broader efforts to quantize, measure, and estimate growth rates for science and technology in general, and ignores the human component involved in scientific development (namely, the socioeconomic and political factors that lead to the resources necessary for such development being allocated). As a counterpoint to these views, Microsoft has pumped 1 billion dollars into OpenAI alone - AI development is not just taking off because of an inevitable trend.
As a result of the perceived unchangeable and inevitable nature of AI development, alignment solutions fall into what I perceive to be two broad categories: Anti-development solutions and technological solutions. Anti-development solutions focus on, for example, stopping others from developing AGI by obtaining some kind of global moratorium, usually by offering some small group highly destructive strategic power. More facetiously, they fall into the "bomb the silicon mines" camp, where developent of computing technology is hampered in some way. These solutions are (rightly I think) perceived as difficult to implement and, where they concern essentially holding the world hostage, somewhat unethical. T

...
Is there any policy for a fair treatment of AIs whose friendliness is in doubt?
by nahoj — 2022-11-18T19:01:40.770Z

A game theory question.

If there ever exists a roughly-human-level agenty AI who could grow to overpower humans but who humans have an opportunity to stop because takeoff is slow enough. Assume the AI could coexist with humanity but fears that humans interacting with it will destroy it because they fear it is unfriendly.

Should humans have a policy of treating any potentially unfriendly agenty AI well to the extent compatible with our safety? For instance, halt it but record its state, intend to inspect it whenever we have the means, rerun it some time in the future if we deem it safe and have the resources, run it sandboxed (e.g. by a much smarter AI) if we deem it unfriendly but safe to run sandboxed.

If the AI values its survival or well-being, this could change how it values being halted out of caution. For instance, it might consider some probability of being halted an acceptable risk rather than something to be avoided at all cost.
Distillation of "How Likely Is Deceptive Alignment?"
by NickGabs — 2022-11-18T16:31:37.885Z

Summary

Deceptive alignment seems to be the likely outcome of training a sufficiently intelligent AI using gradient descent. An ML model with a long term goal and an understanding of its training process would probably pretend to be aligned during training, as this would reduce the amount that gradient descent changes its goals and make humans more likely to deploy it, allowing the model to pursue its goal in the long run. This seems likely to occur because the gradient descent path to deceptive alignment is shorter than the path to genuine alignment due to the fact that models must very specific goals in order to become genuinely aligned. Moreover, gradient updates towards deceptive alignment generate large performance increases, further favoring the deceptively aligned path. Deceptively aligned models are also simpler than genuinely aligned models and so are favored by gradient descent's simplicity bias. While deceptively aligned models are slower and use more computational power than genuinely aligned ones, biasing our models to use less compute is unlikely to be a viable alignment strategy as it would not be performance competitive. In order to avoid deceptive alignment, we must find new ways to make gradient descent favor genuinely aligned models.

Epistemic Status

The claims made in this piece mirror those made in Evan Hubinger's "How Likely is Deceptive Alignment," which are somewhat speculative but based off of a fair amount of thought/empirical research.

Acknowledgements

Thanks to Evan Hubinger, Max Nadeau, Thomas Larsen, Rachel Weinberg, Tom Shlomi, and Robi Rahman for providing feedback

1: Introduction

In “How Likely Is Deceptive Alignment” MIRI alignment researcher Evan Hubinger argues that intelligent AIs trained by gradient descent will by default be deceptively (mis)aligned. But what does this mean?

Imagine that the Christian God is trying to create a human who does what the Bible says, at least so long as they are on earth. He could do this in three ways:

He could create Jesus Christ, who, as a copy of God, has literally identical goals to God (which are the same goals that the Bible instructs you to pursue); Jesus is internally aligned.
He could create Martin Luther, who does not have the exact same goals as God but has the goal of understanding what the Bible says and t

...
Contra Chords
by jefftk — 2022-11-18T16:20:01.418Z

When you see contra dance tunes or other similar music written down
there are usually chords along with the tune, but I rarely like them.
Some of this is aesthetic differences, where the chords are often
written in a style where you switch chords rapidly to closely follow
the melody, but the real problem is that writing the chords for a tune
isn't really possible. That is, there isn't a single set of chords
you that will stay interesting as you play it ten times in a row for
dancing. If you listen to great rhythm players they'll often pick
different chords every time through the tune!

I was thinking about this with Amy in the context of rhythm for open bands and one
idea we had was to get around this by writing several sets of chords
for a tune. In an open band context the leader can say "next time,
variation (b)!", or if you're playing rhythm on your own you can make
your own choices. This still isn't perfect—it's often good to
be even more free with choosing chords that fit in the moment than
this implies—but I think it's a good step up from the current
situation.

I sat down at the piano and wrote a few variations for tunes the way
I'd likely play them. With each variation (a) is usually a good place
to start, and some of the later ones you wouldn't want to do over and
over or the novelty would wear off. The notation has each chord as
two beats unless they're squished together, and "/" means to keep
playing the same chord (more). I've
put a few chords in bold to clarify that I really would play
that chord, despite the dissonance.

Coleman's March

|: D G D A D G A / :|
|: D A G D D G A / :|

|: D / / / G / / / :|
|: Bm A G D D / A / :|

|: D G Bm A D G A / :|
|: D A G D D G A / :|

|: D G D A D G A / :|
Bm A G / Bm A G /
Bm A G / G / A /

Star Above the Garter

|: G D C CD G D C / :|
|: D / C G D / C G :|

|: G / / / G / / / :|
D / C / D / C /
D / C / Am Bm C D

|: G Am C / G Am C / :|
Am / / / Bm / / /
C / / / D / / /

Sandy Boys

|: A / / G A / AG A :|
|: A / AGAG A / AG A :|

|: A / / G A / AG A :|
|: A / ADAD A / AG A :|

A / / G A / AG A
A / / G D / E /
A / / / G / / /
D / / / E / / /

|: A / / G A / AG A :|
|: F / / / G / / / :|

|: A / / G A / AG A :|
A / / / A / / /
C / / / D / E

...
Updates on scaling laws for foundation models from ' Transcending Scaling Laws with 0.1% Extra Compute'
by Nick_Greig — 2022-11-18T12:46:45.563Z

I am not sure if this paper is flying under the radar for many people, but has anyone read Transcending Scaling Laws with 0.1% Extra Compute? If so, how do you think it compares to the scaling laws presented by Deepmind's An empirical analysis of compute-optimal large language model training? Does it make you rethink the importance of dataset size (again)?