DDDI26

DDDI - Training Very Small Language Models On-Device

nanoGPT:

• Step-by-step instructions for Apple Silicon: https://til.simonwillison.net/llms/nanogpt-shakespeare-m2

  • You will need pip (https://pip.pypa.io/en/stable/installation/) and homebrew (https://brew.sh/)

• github repo: https://github.com/karpathy/nanoGPT

Other resources:

• Little Language Machine https://lm.gen-ai.fi/workspace/home

• nanoChat: https://github.com/karpathy/nanochat

• transformers.js examples: https://github.com/huggingface/transformers.js-examples

• transformers.js docs: https://huggingface.co/docs/transformers.js/index

Research papers:

• Morales-Navarro, L., Noh, D. J., Servat, L., Netting, C., Kafai, Y. B., & Metaxa, D. (2026). Building to Understand: Examining Teens' Technical and Socio-Ethical Pieces of Understandings in the Construction of Small Generative Language Models. Proceedings of the 25th ACM Interaction Design and Children Conference (IDC ’26).

• Morales-Navarro, L., Noh, D. J., & Kafai, Y. B. (2025). High school students building babyGPTs: Engaging in data practices and addressing ethical issues through the construction of generative language models. International Journal of Child-Computer Interaction, 100769.

If you have further questions, email me at luismn@upenn.edu.