ibm-granite/granite-switch-4.1-30b
main boardmodel
Granite Switch 4.1 30B is a modular LLM that composes a base model, multiple adapters and an encoder together so that a single hosted endpoint can move between natural-language responses, general and specialized decisions, and speech-to-text. Dense, 30B params. Context: 128K tokens (config). License: apache-2.0.
- Org
- ibm-granite
- Released
- 2026-10-14
- Status
- main board
- License
- apache-2.0 (permissive)
- Parameters
- 30.0B
- Architecture
- dense · GraniteSwitchForCausalLM
- Context
- 131,072 tokens
- Task
- text-generation · text
- Signal
- 29 HF likes · 1788 downloads
Runs on
real quant files; need = weights + FP16 KV cache for 8K tokens + 1GB. A sizing calculation, not tested.
- Q4 ~24.2 GB: 32GB GPU (5090), Mac 36GB, Mac 48GB, CPU only, 32GB RAM (slow)
- Q8 ~37.2 GB: 48GB GPU (RTX 6000 Ada / A6000), Mac 64GB (M1 Max), CPU only, 64GB RAM (slow)
- FP16 ~64.3 GB: 96GB GPU (RTX Pro 6000 Blackwell), 1× DGX Spark (128GB), 2× DGX Spark (256GB), Strix Halo 128GB unified (~96GB usable), Mac 96GB, Mac 128GB, Mac 192GB, Mac 256GB, Mac 512GB, CPU only, 128GB RAM (slow), CPU only, 256GB RAM (slow)
Links
First published here 2026-10-14 · updated 2026-10-14 · markdown · JSON