pull down to refresh
Yeah the problem with the recent flash models is that they're huge so you got me puzzled there for a moment. I wish I were able to run Kimi or even GLM-5.2 locally. I'd deal with living in the noise if that were an option available to me.
The torrent based clone is largely unseeded. And it removes the git metadata in the torrent. So needs some work. But by itself the torrent idea is already cool, if anyone seeds it. I've put down a todo item to play a bit with it and see if I can integrate it with git+LFS or git-annex. The fallback URL isn't decentralized so you can't depend on that if it ever comes to legislative bullcrap (I really don't know how deep the fear factor will be milked, it's hard to say), but we can do something with it in this state too and harden it further.
Not sure what you mean by quant in this context, but I don't run local LLM much.
The best I can run on my hardware is deepseek coder or Qwen 3.8 Q4... And I just realized quant might mean quantization here lol. So Q4.
When I finally get around to dropping a bunch of money and make my setup, it would be for more than LLM. Nodes, torrent seedbox, etc. so it's not a rabbit hole I've gone deep into yet.
I am keeping an eye on the open weights and huggingface scene though. Apparently there's a torrent based clone out (pirate face) to be censorship resistant of they ever crack down on models but I'm not sure how reputed it is atm. Just glad to see torrent still being used.