Functiongemma is very handy if you scope the task.  I only used it on phone but want to play with it on more private devices

I've been studying `functiongemma` for a while now but it's still too big for ESP32-S3. Would be useful in on-edge "decision making", but on the other hand, offloading to a closeby Pi+HAT will probably work better (and provides way more compute.)

optimism

This is a 28.9 million parameter language model that generates text on an ESP32-S3 microcontroller. It runs on the chip itself, with nothing sent to a server, and it displays generated text at 9.88 tokens per second on a small screen wired to the chip. It fits because most of the model lives in flash instead of RAM, using Per-Layer Embeddings, an idea from Google's Gemma 3n.

> ![](https://m.stacker.news/150897)
>
> This is a 28.9 million parameter language model that generates text on an ESP32-S3 microcontroller. It runs on the chip itself, with nothing sent to a server, and it displays generated text at 9.88 tokens per second on a small screen wired to the chip. It fits because most of the model lives in flash instead of RAM, using Per-Layer Embeddings, an idea from Google's Gemma 3n.