trentmkelly/LessWrong-43k huggingface dataset:
Aligning my web server with devops practices: part 1 (backups)
by VipulNaik — 2022-11-27T18:38:38.244Z

UPDATE: Part 2 is now published
here.

I have a web server that serves a double-digit number of different
domains and subdomains, such as
contractwork.vipulnaik.com and
groupprops.subwiki.org. I
originally set it up in 2013, and more recently upgraded it a few
years ago. Back when I did the original setup, I had no knowledge or
experience of the principles of devops (I didn't even have any
experience with software engineering, though I had done some
programming for academic and personal purposes).

Over time, as part of my job as a data scientist and machine learning
engineer, I acquired deeper familiarity with software engineering and
devops. However, for the most part, I didn't have the time or energy
to apply this knowledge to my personal web server setup. As a result,
my web server continued to basically be a
snowflake
-- a special hodgepodge of stuff that would be hard to regenerate from
scratch -- and the thought of regenerating it from scratch was
terrifying. The time commitment to do so was big enough that I didn't
have enough time away from my job to do so.

In the past year, I've finally started work on desnowflaking my web
server and formulating a set of recipes that could be used to build it
from scratch, assembling all the websites and turning them on. This
work only happens in the spare time from my day job, and competes with
other personal projects, so progress is slow, but I've made a lot of
it.

This series of posts talks about various choices I made in setting up
various pieces of the system. Many of these pieces already existed
prior to my systematic efforts this year, but I had to add several
other pieces.

This particular post focuses on backups.

Recommendations for others (transferable learnings)

These are some of the key learnings that I expect to transfer to other
contexts, and that are sufficiently non-obvious and that may not come
to your attention until it is too late:

Establish monitoring of the backups so that you can be confident
that they're happening as expected, and get notified if they
aren't. If using a managed service, make sure you are subscribed to
notifications for the status page of the service so that you get
notified if there are any issues with backups.

Establish a well-defined process to restore from backups, and
fire-drill it so that you know it works.

This may seem irrelevant to you if your budget is way higher than

...
Some thoughts about natural computation and interactions
by Adam Shai — 2022-11-27T18:15:24.724Z

Epistemic Status: Ramblings of my current thoughts on computation.

I have been wondering about the nature of computation for some time now. For instance, what do we mean when we say the brain computes? I think the traditional answers are unsatisfactory. Chief amongst these issues is the role of observers in computation.

When an electronic calculator performs addition, it outputs the result by controlling pixels on a screen. Photons bounce off the screen and hit our retina and the brain performs a dizzying array of computational work in order to make sense of that retinal input. However, when discussing the computation the calculator performs, we talk about automata and locations in Chomsky hierarchies and the like, but we do not even consider the computational work the brain is doing in order to compute addition.

The issue is that while the dynamics of the calculator are well defined and independent of any observer, both the inputs and the outputs only make sense by virtue of an external entity doing some work. What kind of work is this observer doing?

Ultimately I think the observer must be doing the same kind of work the calculator is doing. Everything is (open) dynamical systems changing states, interacting with eachother.

When I think of a brain computing, I imagine a dynamical system taking inputs.

What are the "states" of the computation the brain is performing, given we think of it as a dynamical system? Who is the observer of our brain dynamics? In order to not have a homonculus, I think the meaningful computational states must be defined internally, without reference to external systems. Consider two distinct sets of neurons firing. Are these different computational states or not? The proposed answer here is that they are different only insofar as they constrain the future evolution of the brain in the different ways. If they constrain the dynamics similarly, then they are similar computational states.

There are many details here about what it means exactly to constrain future states in similar ways, but I want to keep things high level in this post.

One issue with this framing is that we seem to have drawn an arbitrary box around the neurons in the brain, and have been considering it "the system of interest", but is there any reason we couldn't add in arbitrary sets of particles outside of the brain? What about the brain plus the calculator? Couldn't we dr...
Review: LOVE in a simbox
by PeterMcCluskey — 2022-11-27T17:41:26.067Z
Link: https://bayesianinvestor.com/blog/index.php/2022/11/26/review-love-in-a-simbox/

Blog post review: LOVE in a simbox.

Jake Cannell has a very interesting post on LessWrong called LOVE in a
simbox is all you
need,
with potentially important implications for AGI alignment. (LOVE stands
for Learning Other's Values or Empowerment.)

Alas, he organized it so that the most alignment-relevant ideas are near
the end of a long-winded discussion of topics whose alignment relevance
seems somewhat marginal. I suspect many people gave up before reaching
the best sections.

I will summarize and review the post in roughly the opposite order, in
hopes of appealing to a different audience. I'll likely create a
different set of misunderstandings from what Jake's post has created.
Hopefully this different perspective will help readers triangulate on
some hypotheses that are worth further analysis.

How to Box an AGI

I've been pessimistic about containing an AGI of unknown alignment.

Many discussions of keeping an AGI in a box focus on how to use the AGI
as an oracle (possibly due to Eliezer's mysterious
experiments being more
colorful than the best proposals for testing an AGI in simulations?).
(Is there a general tendency to focus too much on oracles? I'm reminded
that OpenAI seems to have underestimated
GPT-3 by
thinking of it as an oracle.)

I've been downplaying the value of testing AGIs in simulations, maybe
because I over-generalized Eliezer's arguments.

Jake proposes using simulation sandboxes (simboxes) to train AGIs and to
test their alignment. The simboxes should be designed so that the AGIs
have little chance of detecting that they're in a simulation, and
little evidence about what kind of a world might be running the
simulation.

The AGI grows up in a toy world that contains other intelligent (but not
exactly human?) agents. This world bears enough resemblance to ours that
the AGI will face some test(s) of whether it's aligned with those
agents. The main communication with the real world happens when the
AGI's developers shut down the simbox and evaluate the results.

How well did the AGI cooperate with other agents? A properly designed
simbox will generate clear evidence about whether an AGI will help other
agents, or whether it will engage in world conquest. (The AGI's
intelligence will be evaluated at the same time.)

I'll ask you to assume for now that the AGI has been designed so that
there's a decent chance that it can be aligned with humans. Then we can
...
Mastodon's Dubious Crawler Exemption
by jefftk — 2022-11-27T14:20:03.508Z

When you share a link on social media the platform fetches the page
and includes a preview with your post:

Even though this happens automatically the system doesn't obey the
robots.txt exclusion rules (RFC 3909): I have
/test/no-robots excluded in my robots.txt but that doesn't stop it:

I don't even see a request for robots.txt in my logs.
But that's actually ok! The robots exclusion standard is for
"crawlers", or automated agents. When I share a link I want the
preview to be included, and so the bot fetching the page is acting as
my agent. This is the same reason why it's fine that WebPageTest and PageSpeed Insights also ignore
robots.txt: they're fetching specific pages at the user
s request so they can measure performance.

This puts Mastodon in an awkward situation. They do want to include
previews, because they're trying to do all the standard social network
things, and if they respected robots.txt many sites
you'd want to be able to preview won't work. They also don't want
the originating instance to generate the preview and include it in the
post, because it's open to abuse:

You can trust mastodon.mit.edu about what
@jefftk@mastodon.mit.edu says, but not about what
newyorktimes.com says.

The approach Mastodon has gone with is to have each instance generate
link previews for each incoming link. This means that while having a
link shared on Twitter or Facebook might give you one automated
pageview for the preview, on Mastodon it gives you one for each
instance the link is federated to. For a link shared by a
widely-followed account this might mean thousands of link-preview
requests hitting a server, and I'd also now consider this traffic to
be from an automated agent.

I'm not sure what the right approach is for Mastodon. Making the link
preview fetcher respect robots.txt would be a good
start. [1]
Longer term I think including link previews when composing a post and
handling abuse with defederation seems like it should work as well as
the rest of Mastodon's abuse handling.

(context)

[1] Wouldn't that just add an extra request and increase the load on
sites even more? No: everyone builds sites to make serving
robots.txt very cheaply. Serving HTML, however, often
involves scripting languages, DB lookups, and other slow operations.
Reward Is Not Necessary: How To Create A Compositional Self-Preserving Agent For Life-Long Learning
by Capybasilisk — 2022-11-27T14:05:16.933Z
Link: https://arxiv.org/abs/2211.10851

We introduce a physiological model-based agent as proof-of-principle that it is possible to define a flexible self-preserving system that does not use a reward signal or reward-maximization as an objective. We achieve this by introducing the Self-Preserving Agent (SPA) with a physiological structure where the system can get trapped in an absorbing state if the agent does not solve and execute goal-directed polices. Our agent is defined using new class of Bellman equations called Operator Bellman Equations (OBEs), for encoding jointly non-stationary non-Markovian tasks formalized as a Temporal Goal Markov Decision Process (TGMDP). OBEs produce optimal goal-conditioned spatiotemporal transition operators that map an initial state-time to the final state-times of a policy used to complete a goal, and can also be used to forecast future states in multiple dynamic physiological state-spaces. SPA is equipped with an intrinsic motivation function called the valence function, which quantifies the changes in empowerment (the channel capacity of a transition operator) after following a policy. Because empowerment is a function of a transition operator, there is a natural synergism between empowerment and OBEs: the OBEs create hierarchical transition operators, and the valence function can evaluate hierarchical empowerment change defined on these operators. The valence function can then be used for goal selection, wherein the agent chooses a policy sequence that realizes goal states which produce maximum empowerment gain. In doing so, the agent will seek freedom and avoid internal death-states that undermine its ability to control both external and internal states in the future, thereby exhibiting the capacity of predictive and anticipatory self-preservation. We also compare SPA to Multi-objective RL, and discuss its capacity for symbolic reasoning and life-long learning.

FYI: Just linking the paper. Wasn't involved in it.
About probabilities
by mikbp — 2022-11-27T08:19:47.950Z

There is one thing I don't understand about probabilities:

If we toss a coin, there is a 50% chance that it shows heads or tails. If we do it 20 times and all of them showed heads, there is still a 50% chance that the next one shows heads, since the tosses are independent. However, we also know that series of X tosses showing heads are increasingly improbable when X grows. So, although there is a 50% chance that the toss shows heads again, at the same time the probability that it shows heads again are lower.

Why do we have to take into account one piece of information and not the other one when finding the probability that the next toss will show heads or tails? Are there 2 (or more) types of probabilities and I am just mixing them up (I'm thinking on things like the reported "probabilities" that polls show about one party or another getting elected in an election, for example)? Is the difference related to ergodicity (time vs ensemble averages)?
Always know where your abstractions break
by lsusr — 2022-11-27T06:32:09.643Z

General relativity plus quantum field theory can describe almost everything in the universe. There are a few exceptions like cosmic expansion and black holes but to human beings confined to a single solar system, fundamental physics is (for all practical purposes) a solved problem. Yet there are many things we don't know. It's as if the universe were a game of chess; we've learned the basic rules but are still figuring out the strategies.

All the universe cares about is fundamental physics. The universe always obeys the small-scale fundamental laws of physics. The universe never does anything else. It doesn't care about evolution or chemistry or orbital mechanics or beauty. All of those things are high-level abstractions we (or evolution) invented to make sense of the world. Most of the time we think about how the world works we don't think about fundamental physics. We use our higher-level abstractions instead.

Which is fine…most of the time. The problem is that any theory other than "the universe always obeys the fundamental laws of physics" is wrong in the sense that it is not perfectly generalizable.

There are many ways abstractions can malfunction when misapplied.

All abstractions have limited domains of applicability. Modern political ideologies—Marxist revolutionary theory, libertarianism, feminism—were invented in the context of an industrial civilization. Try too hard to apply these ideas to New Guinean hunter-gatherers or to medieval Japan and they'll cloud your ability to understand what's actually going on.

All high-level abstractions are, ultimately, probabilistic. Statistical mechanics almost always works. Almost.

Perhaps most importantly, "the concepts we use in everyday life are fuzzy, and break down if pushed too hard". Even Newton's Laws of Motion break when you apply them to too small of a scale.

Even general relativity and quantum field theory are not not universally generalizable. General relativity breaks on small-scale phenomena. Quantum field theory breaks on large-scale phenomena.

Postmodernists use the idea of leaky abstractions to dismiss the idea of objective truth entirely. That's like driving your car into the ocean and then declaring that cars don't work. Cars do work, but you need to take care of yours and drive it only on the terrain it functions on.

The most dangerous philosophers aren't the postmodernists who believe nothing i...
Science and Math
by lsusr — 2022-11-27T04:05:25.977Z

When I was in college, it blew my mind to discover that there are exactly two effective systems for distilling legible[1] truth. Science starts with observations and then summarizes them into theories. Math starts with axioms and then generates theorems via a series of proofs. Most of the classes I took were science and math classes.

I took two philosophy classes. One of them was taught by an anthropologist who lost his sanity in Sierra Leone. His class was highly entertaining but not-at-all "legible". The other one was taught by a traditional professor of philosophy. Her class covered the Greek classics: Socrates and Aristotle. I don't feel like I learned anything from Socrates and Aristotle. The problem with philosophy is there's no system for resolving disputes. Any two scientists who disagree can (in theory) conduct an experiment to determine who is correct. Any two mathematicians can (in practice) resolve any mathematical dispute from inside a locked room with nothing but paper and pencils. Philosophers don't discuss things which can be falsified.

So what have philosophers been arguing about for thousands of years? Not talking precisely about observable reality. Consider the Arian controversy, a disagreement about whether the God the Father (God) was composed of the same substance as Jesus Christ. What does that even mean? How would you measure it? When philosophers lack dispute-resolution mechanism they can get away with saying anything, as long as its sufficiently meaningless.

I'm not saying philosophy isn't useful. Philosophy is useful for lawyers, politicians and anyone else in the business of persuasion who needs to be good at rhetoric. But persuasion and truth-finding are different things.

So are science and math. How can science and math—two very different epistemologies—both lead us to truth? The trick is to notice that the word "truth" is pointing to two slightly different ideas. Scientific truth and mathematical truth aren't the same thing. Science cares whether a fact is observable. Math cares whether a fact is tautological.

For example, consider the Banach–Tarski paradox. Scientifically, the Banach–Tarski paradox is nonsense. You can't disassemble a ball and then reassemble the pieces into two balls identical to the original. Mathematically, you can (if you take the axiom of choice, that is).

Does this mean math is better than science? Of course n...
Microstartup Stories: Initial Thoughts
by Adam Zerner — 2022-11-27T01:22:49.401Z

A while back I read this post saying that you should write a business plan. Previously I thought business plans were dumb. I recall that being the conventional wisdom in the tech world and hearing people like Paul Graham and David Heinemeier Hansson say so. But that blog post changed my mind. And it opened my eyes to the idea that business plans don't have to look like stereotypical business plans, just like how essays don't have to look like the things you wrote in English class.

Basically, writing is a great way to think. I've found this to be true myself, and I've heard various smart people who I trust say it as well. So then, before pursuing some sort of business, why not write about it? I think it makes sense, so that's what I am going to do here.

I am going to write about a new podcast I am launching: Microstartup Stories. The idea is to interview founders of companies that are making less than $5k/month. The interviews will be very long and wide ranging. 2-4 hours probably. And they will loosely follow an outline.

Costs

Overall, the costs of pursuing this seem almost negligibly low.

Money

This shouldn't really require me to spend any money. I'm using Anchor which is free. I already bought a nice microphone a few years ago.

Time

A few days ago I listened to this interview of Courtland Allen of Indie Hackers. Courtland talked about a lot of great things in this episode, but one thing in particular that stuck with me was this:

Maybe two or three months into it, I was like, this is an endless slog of coding. This is going to take me forever. I don't see like the end of the rainbow. I need to just call it quits right now. At the time where I was working on that, I saw a couple stories on Hacker News that people who would come up with new ideas and we're already making five or ten grand a month, and I hadn't even launched my thing yet.

So I was like, okay, this is like a dud. I should scrap this and go back to the drawing board and figure out what are all the different lessons that I've learned after years of doing this? Let me make sure I'm systematic in how I approach this so I'm not just going to make the same mistakes again.

I had repeatedly made this mistake that a lot of software engineers make where we just spend way too much time coding. I think the biggest lesson at the top of the list was only work on things and you can build in a few weeks. You know

...
Don't align agents to evaluations of plans
by TurnTrout — 2022-11-26T21:16:23.425Z

Another stab at explaining Don't design agents which exploit adversarial inputs. This is not the follow-up post mentioned therein. That post will come next.

More precise title: "Don't try directing a superintelligence to maximize your valuations of their plans using a consequentialist procedure."

After asking several readers for their understandings, I think that I didn't successfully communicate my points to many readers. I'm now trying again, because I think these points are deeply important. In particular, I think that my arguments rule out many target AI motivational structures, including approval-directed agents (over a rich action space), approval-based amplification (if the trained agent is supposed to be terminally motivated by the amplified overseer's ratings), and some kinds of indirect normativity.

Background material

One motif in some AI alignment proposals is:

An actor which proposes plans, and
A grader which evaluates them.

For simplicity, imagine we want the AI to find a plan where it makes an enormous number of diamonds. We train an actor to propose plans which the grading procedure predicts lead to lots of diamonds.

In this setting, here's one way of slicing up the problem:

Outer alignment: Find a sufficiently good grader.

Inner alignment: Train the actor to propose plans which the grader rates as highly possible (ideally argmaxing on grader output, but possibly just intent alignment with high grader output).

This "grader optimization" paradigm ordains that the AI find plans which make the grader output good evaluations. An inner-aligned actor is singlemindedly motivated to find plans which are graded maximally well by the grader. Therefore, for any goal by which the grader may grade, an inner-aligned actor is positively searching for adversarial inputs which fool the grader into spitting out a high number!

In the diamond case, if the actor is inner-aligned to the grading procedure, then the actor isn't actually aligned towards diamond-production. The actor is aligned towards diamond-production as quoted via the grader's evaluations. In the end, the actor is aligned to the evaluations.

Clarifications

Grader-optimization is about the intended agent motivational structure. It's about a trained agent which is trying to find plans which grade highly according to some criterion.

Grader-optimization is not about grading agents when

...
What videos should Rational Animations make?
by Writer — 2022-11-26T20:28:28.603Z

I want to know what you would be excited to see covered on the channel.

I'd especially like to know what videos you think would be optimal to make according to an optimization target you specify. It's more effortful, but more useful, to answer that question instead of just "what would be good video topics?".

Answers that go the extra mile and explain their reasoning in detail will be especially appreciated.

That said, I also welcome answers to the easier question "what would be good video topics?" and without much argumentation.