Springtail develops AI agents that learn to program from experimentation and experience -- the same way that humans learn.
We leverage advanced machine learning + biological inspiration to drive programming mastery beyond LLMs.
To create AI programming agents, we research the following areas:
We train agents primarily from scratch: if you can go from 0 to 1, then by induction it can go to N, where N is > human.
Humans and animals are notably more data-efficient than deep learning models; they also actively interpolate between memorization and generalization. We're working on bio-inspired, transductive, & novel mechanisms to improve data efficiency.
With novel architectural changes, transformers can be adapted to solve not just prediction problems, but also constraint satisfaction problems (CSP), essential for real-world programming.
Complementary to CSP is the induction of constraints from observation and active intervention. Sensory-motor learning also grounds the model - it understands the effects of edits, and how a program internally functions.
In the process of learning what does have an effect, agents must learn what doesn't (invariance) and what are independent factors of variation (equivariance). This allows the search space to be dramatically compressed.
Following the pioneering work of DreamCoder, we use code library generation to further reduce the search space depth, as well as to induce imtermediate representations for problem solving.
Work is led by Tim Hanson, with many collaborators! (My primary claim to fame is having invented the sewing machine
, which became Neuralink. Now onto even more ambitious things :)
We are actively seeking collaborators, so if any of the above seems interesting, please: