Source: https://mayphus.org/tinystories-rp2040/ Title: A TinyStories model on my RP2040 board Metadata: {"category":"artifact","date":"2026-08-13T00:00:00Z","id":"tinystories-rp2040","kind":"article","language":"en","locale":"en","route":"/tinystories-rp2040/","slug":"tinystories-rp2040","tags":["ai","rp2040","raspberry pi pico","embedded","tinystories"],"type":"article"} A TinyStories model on my RP2040 board Training · quantization · C inference · physical hardware I trained a very small language model, converted it into an embedded C runtime, and ran it locally on my custom Raspberry Pi Pico display board. I did not solve the individual technical problems myself. I described what I wanted and what I could see; the AI agent translated that into code, measurements, firmware, and the next test. The result The final model is a 98,944-parameter byte-level Transformer with a 256-byte context. It was trained on short setup-and-conclusion stories, quantized to INT8 weights, checked against its Python reference through a portable C implementation, and then built into native RP2040 firmware. Model weights 98,944 bytes INT8 weights, plus 80 bytes of scales Static SRAM 143,504 bytes within the RP2040's 264 KiB budget Firmware image 145,980 bytes native RP2040 flash and code image Physical generation 10.394 seconds sentence-aware output on the connected board From intention to a physical result - Describe. I said what I wanted: a TinyStories-style model made specifically for the RP2040, not a larger computer hidden behind it. - Inspect reality. I connected the board and described what I saw on its screen, buttons, and USB connection. The agent inspected the software and measurements. - Implement. The agent built the Python model, training path, quantization export, numerically checked C runtime, display driver, and firmware. - Verify. I supplied the physical observations the computer could not infer: whether the display was blank, whether text appeared, and what happened after I pressed BOOTSEL. - Use. The board now generates a fresh bounded micro-story locally at startup or after a button press. My contribution was not pretending to be the firmware engineer. It was holding the intention, making the decisions, connecting the physical world, and describing reality accurately enough for the next step. The working loop was: describe, inspect reality, implement, verify, use. What ran where Training ran on my Apple M1 Mac, because training a Transformer is not the job of this microcontroller. The completed quantized model and inference runtime run on the RP2040 itself. Story generation does not call a cloud model or another computer. The 256-byte model trained for 10,000 steps on 18,522 derived micro-stories. Its final validation loss was 0.7502 and perplexity was 2.1175. A scaled INT16 key-value cache made the longer context fit in SRAM, while the display driver draws glyphs directly over SPI instead of reserving a framebuffer. Captured from the physical board TINYSTORIES_RP2040_READY model_state_bytes=138244 DISPLAY_READY STORY_BEGIN Once upon a time, there was a big treest of animals. STORY_END bytes=52 generation_ms=10394 The language is imperfect, and I am keeping that visible. This is a deployment proof for a genuinely tiny model, not a claim that a 99K-parameter storyteller matches a large language model. What this proves The physical board executed the quantized Transformer, initialized its ST7789 display, generated a learned story ending locally, and displayed the result. Earlier tests also verified that pressing and releasing BOOTSEL triggered a different generation without rebooting. It does not prove that the model was trained on the Pico, that its writing is generally coherent, or that an AI can observe physical hardware without a person. The achievement is the complete collaboration: a person can direct a real embedded-AI project through intention and observation while an agent handles the technical path and preserves an evidence trail.