pull down to refresh

140 sats \ 0 replies \ @optimism OP 16 Sep \ on: What open weights models do you use? AI

Here's my list. Almost all are gguf, and this is what I currently have on disk across 2 machines. There are things I deleted (mostly: reapers/dolphins) that I don't remember and also was mostly disappointed by.

Chat models:

  • Qwen/Qwen3.8-27B <--- in use
  • Qwen/Qwen3.6-27B
  • Qwen/Qwen3.5-27B
  • Qwen/Qwen3.5-4B
  • Qwen/Qwen3-Coder-Next-Q4_K_M
  • OpenGVLab/InternVL3_5-8B
  • OpenGVLab/InternVL3_5-4B-Pretrained
  • google/gemma-4-31B-it-qat-q4_0-gguf <--- in use
  • ggml-org/gemma-4-E4B-it-GGUF
  • mradermacher/gemma-4-E2B-GGUF
  • google/gemma-3-4b-it-qat-q4_0-gguf
  • meta-llama/Meta-Llama-3-8B-Instruct
  • meta-llama/Llama-3.1-8B-Instruct
  • meta-llama/Llama-3.2-3B
  • mxmcc/xLAM-2-32b-fc-r-mlx-8Bit
  • Menlo/Jan-nano-gguf
  • janhq/Jan-v3-4B-base-instruct-gguf
  • janhq/Jan-v3.5-4B-gguf <--- need to eval still
  • bartowski/dolphin-2.9.4-llama3.1-8b-GGUF
  • bartowski/nvidia_Orchestrator-8B-GGUF

Non-chat models:

  • opendatalab/MinerU2.5-2509-1.2B <--- in use
  • handy-computer/nemotron-3.5-asr-streaming-0.6b-gguf
  • handy-computer/parakeet-unified-en-0.6b-gguf <--- in use
  • ds4sd/SmolDocling-256M-preview-mlx-bf16
  • Qwen/Qwen3-Embedding-4B <--- in use
  • microsoft/VibeVoice-1.5B
  • mlx-community/whisper-large-v3-mlx
  • mlx-community/whisper-large-v3-turbo <--- in use
  • sentence-transformers/all-MiniLM-L6-v2 <--- in use
write
compose