Sitemap

A list of all the posts and pages found on the site. For you robots out there, there is an XML version available for digesting as well.

Pages

Posts

An Explanation of In-Context Learning

22 minute read

Published:

This post surveys theoretical explanations of in-context learning based on Transformers implementing optimization algorithms in their forward pass. It reviews how linear attention can realize preconditioned gradient descent for linear regression and how nonlinear, kernel-based attention generalizes this mechanism to functional gradient descent, together with the underlying assumptions and limitations.

portfolio

publications

talks

teaching

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.