Week#1: Feeling It Out
Things done:
PMPP book ch1
Used cursor + sonnet 3.5 to download and work with various codebases. Its awesome. Like fr fr.
Tinygrad codebase: had sonnet tell me the important files in codebase. Had it write a simple nn using tinygrad. Understood it and its conversion to apple's metal(working on a m1 mac).
pytorch/ao: downloaded codebase. again had sonnet walk me through the major parts.
FP8 Formats for Deep Learning: read this paper, realised the use while going through pytorch codebase.
Karpathy sensei tutorial on micrograd. Halfway done. This man is a treasure.
Lurked around some discord groups(tinygrad and cudamode)
Review: First time diving into ml systems, goal was to get a good feel for the field. Ml systems is about the frameworks and systems that make ml possible(i.e make it go brrr on a gpu). Happy with the progress so far.
Resources:
PMPP book : Programming Massively Parallel Processors