Active learning for program induction pdf version of this post Timothy Hanson & Justin Jung May 10 2024 Abstract This post goes into more detail on what is meant by active learning and how it relates to program induction. We discuss the use of a simulator for running a program (\(\sim \) compressed model), and…

This post discusses an older paper that shows how a clever idea (butterfly factorizations) can be learned in a typical deep-learning pipeline. It concludes with some conjecture how these fast algorithms can be applied to other slow, data-intensive algorithms: attention. Learning fast algorithms for linear transforms with butterfly factorizations. Tri Dao (author of FlashAttention), Albert…

Are Transformers all you need? pdf version of this post Timothy Hanson – November 8, 2023 Abstract This post discusses active learning and reasoning, and the strengths and limitations of using transformers for it. After setting up the problem context, we conclude that for transformers to serve as world-models to these purposes, they will need…

Hello hello — this is a blog where we’ll post research progress, thoughts, things too small to be preprints, company news, etc. Looking forward to sharing 🙂