pull down to refresh

115 sats \ 4 replies \ @SimpleStacker 21 Feb \ parent \ on: Has anyone had success with AI agents? AI

Are most people you know running frontier models on the local machine? Or are they plugging into an online API service?

write
preview
147 sats \ 2 replies \ @k00b 21 Feb

Plugging into model providers. Running large frontier models locally requires a machine with a few GPUs and 1TB of RAM afaik.

At least in the office, everyone is using Codex-5.3 with a bit of Opus and Sonnet 4.6.

reply
36 sats \ 1 reply \ @SimpleStacker 21 Feb

That's what I thought but you mentioned RAM so I wasn't sure. Coz if you're just running text based feeds to and from the model provider I didn't think hardware would matter much. But maybe there's a lot of additional orchestration that happens locally

reply
147 sats \ 0 replies \ @k00b 21 Feb

OpenClaw is pretty bloated. If you're having it run web browsers and stuff, memory can get tight.

reply
15 sats \ 0 replies \ @optimism 21 Feb

That's what you would take the Mac Mini M3 Ultra w/ 512GB RAM [1] for, or 4x M4 Pro with 128GB in a cluster, see #1360715 for the latter, which is perhaps the better setup (because you can add mac minis to it)

You'll run quantized GLM-5 (or Kimi K2.5 on a cluster of 8). Then you run your agent on a much lower spec box.

I'm still looking for a clone of openclaw that I can actually compile - maybe nullclaw, because with less sloploc the chance of it being unable to compile is lower 😂 Going to be "fun" diving into Zig tho, ugh.

  1. 🥺 I remember when my new computer (in the late 80s iirc) had 512kB RAM and that was a beast. ↩

reply