---
title: dmis-lab/UltraBERT-380M
url: https://recentmodel.com/models/dmis-lab/UltraBERT-380M/
status: main board
published: 2026-10-10T10:53:18Z
updated: 2026-10-10T10:53:18Z
source: https://huggingface.co/dmis-lab/UltraBERT-380M
---

# dmis-lab/UltraBERT-380M

UltraBERT is a family of English bidirectional encoders with a native 32K context that reach on both classification and retrieval, two task families on which previous encoders had to choose a side. Dense, 430M params. License: apache-2.0. Vendor-reported: "UltraBERT is a family of English bidirectional encoders with a native 32K context that reach state of the art on both classification and retrieval, two task families on which previous encoders had to choose a side."

- Vendor-reported: UltraBERT is a family of English bidirectional encoders with a native 32K context that reach state of the art on both classification and retrieval, two task families on which previous encoders had to choose a side.

- **Org:** dmis-lab
- **Released:** 2026-10-10
- **Status:** main board
- **License:** apache-2.0 (permissive)
- **Parameters:** 430M
- **Architecture:** dense · UltrabertForMaskedLM
- **Task:** fill-mask · other
- **Signal:** 73 HF likes · 9977 downloads

## Runs on

estimate from params; sizing calculation, not tested.

- Q4 ~2.1 GB (estimated): no listed device
- Q8 ~2.3 GB (estimated): no listed device
- FP16 ~2.7 GB (estimated): 8GB GPU (4060 / 3070), 12GB GPU (RTX 3060), 16GB GPU (5060 Ti / 4060 Ti / 4080 / 5080), 24GB GPU (3090 / 4090), 32GB GPU (5090), 48GB GPU (RTX 6000 Ada / A6000), 96GB GPU (RTX Pro 6000 Blackwell), 1× DGX Spark (128GB), 2× DGX Spark (256GB), 16GB GPU (7800 XT / 9070 XT), 24GB GPU (7900 XTX), Strix Halo 128GB unified (~96GB usable), Arc B60 24GB, Arc B70 32GB, Mac 16GB, Mac 24GB, Mac 36GB, Mac 48GB, Mac 64GB (M1 Max), Mac 96GB, Mac 128GB, Mac 192GB, Mac 256GB, Mac 512GB, CPU only, 32GB RAM (slow), CPU only, 64GB RAM (slow), CPU only, 128GB RAM (slow), CPU only, 256GB RAM (slow)

## Links

- [Hugging Face](https://huggingface.co/dmis-lab/UltraBERT-380M)