[tab] index.html (html)
<html>
<head>
  <style>
    body {
      margin: 20px;
      margin-left: 12%;
      margin-right:12%;
      a {
        color: green;
        text-decoration-style: dotted;
      }
    }
  </style>
</head>
<p>
  Franz Srambical
  <br>
  ============
  <br>
Hi! I'm Franz. I started and scaled <a href="https://pdoom.org">p(doom)</a>, a Discord-based research community, from zero >200 members. We work on addressing core blockers towards general intelligence that cannot be solved by scaling up compute. We work on everything from kernel-level optimizations to large-scale distributed systems for pre-training and reinforcement learning. A lot of our current work involves finding, investigating and exploiting novel data troves <a href="https://pdoom.org/crowd_code.html">[1]</a> <a href="https://pdoom.org/jasmine.html">[2]</a>, and building open infrastructure/codebases.
I am ex-distributed systems software engineer at <a href="https://celonis.com">Celonis</a>, dropped out of uni (Informatics at <a href="https://tum.de">TUM</a>), minored in Computational Neuroscience (courses at <a href="https://www.gsn.uni-muenchen.de">LMU Graduate School of Systemic Neurosciences</a>), <a href="https://www.youtube.com/watch?v=N5nVSXV9Hbk&t=21971s">ex-linux kernel developer</a>, ex-<a href="public/vwa.pdf">'alignment researcher'</a> in my high school years, ex-RA at <a href="https://aidos.group">Bastian Rieck's lab</a>, ex-'BCI researcher' & founding member at neuroTUM.
<br>
<br>
I want to create AGI (i.e. design architectures that push the pareto-frontier of intelligent systems).
<br>
<br>
If any of our research work interests you, check out the <a href="https://pdoom.org/blog.html">p(doom) blog</a>. I also have <a href="https://www.linkedin.com/in/franz-srambical-418630178/" >linkedin</a>, <a href="https://twitter.com/lemergenz" >twitter</a> and <a href="https://scholar.google.com/citations?user=W26dT4EAAAAJ&hl=en&oi=ao" >google scholar</a>.
<br>
<br>
List of preprints accumulated throughout my education that are not worthy of publication (& thus not on arxiv), but valuable to some nonetheless:
<ul>
  <li>
    <a href="https://pdoom.org/jax_assert.html">
      A blog post on performance-degradation free value assertions in JAX using a very recent and still private JAX API.
    </a>
  </li>
  <li>
    <a href="https://pdoom.org/gae_rlax.html">
      A blog post on how a lot of PPO implementations are technically wrong (including Deepmind's reference implementation in RLax).
    </a>
  </li>
  <li>
    <a href="https://pdoom.org/ppo.html">
        A blog post on how practically everyone in LLM post-training is currently implicitly using REINFORCE with baseline, clipping and a likelihood ratio, and not PPO (in the classical sense).
    </a>
  </li>
  <li>
    <a href="https://pdoom.org/thesis.html">
        A blog post on a simple mental model that permits straightforward contextualization of the current research frontier and extrapolation of what the most important future research directions are going to be.
    </a>
  </li>
  <li>
    <a href="https://pdoom.org/causal_mask.html">
      A blog post on why the Transformer's causal mask is its ultimate feature, not a bug
    </a>
  </li>
  <li>
    <a href="public/vwa.pdf">
      A 30-page paper on AGI alignment written in 2018/19 during my high school years (unfortunately in German)
    </a>
  </li>
  <li>
    <a href="public/cfr.pdf">
      A review paper on superhuman poker bots
    </a>
  </li>
  <li>
    <a href="public/mup-lr-warmup.pdf">
      A 2-page writeup on the necessity of learning rate warmup under muParametrization
    </a> (with <a href="https://github.com/emergenz/mup-lr-warmup">code</a>)
  </li>
  <li>
    <a href="public/causal_mask_poster.pdf">
      A poster on why the Transformer's causal mask is its ultimate feature, not a bug
    </a>
  </li>
  <li>
    <a href="public/wenn_besitzen_unfair_ist.pdf">
      A poster on the 99-year leasehold system in Singapore as a means to mitigate generational wealth (unfortunately in German)
    </a>
  </li>
  <li>
    <a href="public/panoptic-3d-reconstruction.pdf">
      A crappy 'it's just x but with y'-type computer vision paper on 2% better (and 3% worse) panoptic 3D reconstruction (which included one-shot finetuning a 1B parameter diffusion model hours before the deadline)
    </a>
  </li>
  <li>
    <a href="https://docs.google.com/document/d/14xx883ywhbJeaPz13S2lu9NY5RBwUzbNnh6K-o-Y06I/edit?usp=sharing">
      A 2-page doc outlining my thoughts on whether machines can think 
    </a>
  </li>
</ul>

Incomplete list of talks I have given:
<ul>
  <li>
    <a href="https://docs.google.com/presentation/d/1fq_JiOP9zZS0w_fi9sZxMb-9TmKnleugl9ik3sJK_sc/edit?usp=sharing">
      Talk on the necessity of learning rate warmup under muParametrization (and a real-time case study on the absurd speed of modern AI research)
    </a>
  </li>
  <li>
    <a href="https://docs.google.com/presentation/d/1V-lfRj54czkQiZw9ziEnFPdByIuM6lkuSF5gKmHvbQQ/edit?usp=sharing">
      Talk on AlphaFold 3, motivating its architectural design through the lens of the Transformer architecture and its modern variants
    </a>    
  </li>
  <li>
    <a href="https://docs.google.com/presentation/d/1jxDhbyrCtgme_ebzchh8qIIlr8vriu4H7FClAphuDpU/edit?usp=sharing">
      Talk at MunichNLP on p(doom), adaptive compute at inference time and predicting text-based protein function descriptions directly from sequence, bypassing structure
    </a>    
  </li>
  <li>
    <a href="https://docs.google.com/presentation/d/1JVqy-0HdfE7POWw5LWiD02fYMa7VKgi2fFkYaxibJKI/edit?usp=sharing">
      Talk on the 'translation gap' between core and applied machine learning research, scaling protein function prediction as neural machine translation, ESM-3, AlphaFold 3, the causal mask, muTransfer and ARC-AGI
    </a>    
  </li>
  <li>
    <a href="https://docs.google.com/presentation/d/1KXu-bkNRr_0VLh1KdZazt3u3IFlFPtfkAu4AeYFqnAw/edit?usp=sharing">
      Talk on beating modern dynamic graph classification baselines using a 1-layer LSTM
    </a>    
  </li>
  <li>
    <a href="public/counting_neurons_and_ultrasonic_communication.pdf">
      On neurons that count and ultrasonic communication in frogs (I have catastrophically forgotten the contents of this talk)
    </a>
  </li>
</ul>
</p>
<br>
<br>
PS: no free lunch is a myth
</html>