pull down to refresh
the ability to let stuff loop overnight without worrying about costs.
I love this part the most. I don't really like the small models for the workloads that I generally use LLMs for (code tracing, impact analysis) but whenever I have a pipeline that needs smoltelligence I can just run it on my old M1 and at 30W or so it can sit there forever for all I care.
If/when hardware becomes more reasonable
There is no per-Mtoken pricing that beats per-kWh pricing with old, fully depreciated hardware. On new hardware at current prices, that is a huge challenge. I was looking at a higher-end M5 macbook the other day and at 10 grand it means I can have plenty subscriptions for the depreciation rate of that hardware.
Flock type cameras for my neighborhood
If you DIY you get to actually be able to have real features, not the bullshit stuff you get in exchange for sharing footage. Facial rec, crowd and concealed weapon detection and things like that are very useful if there's actual lives at stake. And most of it is available as open weights on YoLo based models.
One of my friends is a property manager including in some rather bad neighborhoods (think North Philly-like) and they complained they couldn't get any of these features - it's apparently all LE-gated - even though they have personnel on the ground that could really use some automated alerting. Should check in how they fare.
There is no per-Mtoken pricing that beats per-kWh pricing with old, fully depreciated hardware.
Not sure about this necessarily, esp with token prices having tanked the last few months. I live in an area with some of the highest electric rates in the country, even sticking an Avalon or something in my office as a space heater would lose money on-net if it was free.
Problem with older hardware is what it can actually do, useful old hardware is even overpriced... I just sold some RAM sticks on eBay that I found in a drawer for probably more than I paid for them new years ago.
I probably should investigate more with smoltelligence pipelines, but then again smoltelligence is practically free... or literally free via API as long as you don't care if it gets trained on. That 6GB of nVidia VRAM I have does nothing right now, can't think of anything useful to do with it except the voice model/flock stuff.
complained they couldn't get any of these features
Yea there's definitely demand, it didn't come out of nowhere, they started in gated communities iirc. First company through the door gets shot though, Flock didn't stand a chance. Options were either partner with politically aligned VC, which just draws suppressing fire from the opposite political side, or stay bootstrapped and draw suppressing fire from all sides until you pick one or die.
My neighborhood use-case is mostly unfamiliar cars, the nature of my road makes them immediately sus. It'd be trivial to let neighbors tag legit cars via a webapp, get push notifications, watch their houses when their away, pull data from public (or even paid) sources, but never egress data to big brother.
Only one neighbor balked at the idea of me rolling one for us, the female politician ofc.
Most did, but that's ultimately on me since I'm not invested in hardware and was only interested in reducing subscription costs for coding and Hermes/OpenClaw like use.
Open models that I could run were generally not good enough to replace what I could get via API.
Next foray into the Cactus Needle class would be a different expectation/use-case.
If/when hardware becomes more reasonable, I'll probably try again for the IP/privacy factor (as I think most companies will) and the ability to let stuff loop overnight without worrying about costs.
On the privacy factor, another thing I've considered tinkering with is a local "Alexa" type system for the house with open voice models, and building my own Flock type cameras for my neighborhood... actually started looking into the latter and discussed with neighbors long before the Flock psyop.
I'm not worried about the boot coming down, we're only 10 or so years post regime change, will take another 20 before enough pieces can be installed for it to happen again. To the extent the shadow regime is effective, model distribution I don't think will be the bottleneck either, we can already see they're attacking at the training level- ensuring that no one but the IC-linked labs can get enough compute to actually train anything useful.