Local GGUF Model Selection and Inference
Find llama.cpp-compatible GGUF models, compare quantization choices and launch local inference with llama-cli or llama-server after checking…
hf_mem estimates the required memory for inference, including model weights and an optional KV cache, for Safetensors and GGUF for models on the Hugging Face Hub using HTTP Range requests i.e., without downloading or loading any weights locally.
uv installed (for uvx)HF_TOKEN env var or --hf-token flag (for gated or private models only)Run with --model-id pointing to the Hugging Face Hub repository which will check that it either contains Safetensors (via model.safetensors, model.safetensors.index.json if sharded, or model_index.json for Diffusers) or GGUF model weights within.
uvx hf-mem --model-id <model-id> --json-output
If the repository contains GGUF model weights in multiple precisions / quantizations, the estimations will be on a per-file basis, whereas for inference you won't load all of those but rather only a single precision. This being said, for GGUF you might as well need to provide --gguf-file to target the specific file (or path if sharded) you want to run.
uvx hf-mem --model-id <model-id> --gguf-file <file-or-path> --json-output
Additionally, hf-mem comes with an --experimental flag that will also calculate the KV cache memory requirements too, useful for large-language models, meaning it applies to LLMs (...ForCausalLM), VLMs (...ForConditionalGeneration), and GGUF models.
As per the context window, it will be read from the default or overridden with --max-model-len a la vLLM. And, same goes for the KV cache precision, which will default to the model precision unless manually set via --kv-cache-dtype a la vLLM too.
For Safetensors use as:
uvx hf-mem --model-id <model-id> --experimental [--max-model-len N] [--batch-size N] [--kv-cache-dtype auto|bfloat16|fp8|fp8_ds_mla|fp8_e4m3|fp8_e5m2|fp8_inc] --json-output
And, for GGUF use as:
uvx hf-mem --model-id <model-id> --gguf-file <file-or-path> --experimental [--max-model-len N] [--batch-size N] [--kv-cache-dtype auto|F32|F16|Q4_0|Q4_1|Q5_0|Q5_1|Q8_0|Q8_1|Q2_K|Q3_K|Q4_K|Q5_K|Q6_K|Q8_K|IQ2_XXS|IQ2_XS|IQ3_XXS|IQ1_S|IQ4_NL|IQ3_S|IQ2_S|IQ4_XS|I8|I16|I32|I64|F64|IQ1_M|BF16|TQ1_0|TQ2_0|MXFP4] --json-output
For Transformers with Safetensors weights:
uvx hf-mem --model-id MiniMaxAI/MiniMax-M2 --json-output
For Diffusers with Safetensors weights:
uvx hf-mem --model-id Qwen/Qwen-Image --json-output
For Sentence Transformers with Safetensors weights:
uvx hf-mem --model-id google/embeddinggemma-300m --json-output
With --experimental to include the KV cache estimation for LLMs and VLMs:
uvx hf-mem --model-id mistralai/Mistral-7B-v0.1 --experimental --json-output
And, for LLMs or VLMs with GGUF weights:
uvx hf-mem --model-id unsloth/Qwen3.5-397B-A17B-GGUF --gguf-file Q4_K_M --experimental --json-output
Source: Hugging Face · Apache-2.0 · SHA-256 shown alongside the download.
License file included. A license and checksum are not a security certification. Review package instructions and scripts before running them.
huggingface-hf-mem/BB-SKILLS-REVIEW.json1544 byteshuggingface-hf-mem/LICENSE.upstream.txt11357 byteshuggingface-hf-mem/SKILL.md3445 byteshuggingface-hf-mem/SOURCE.txt182 byteshuggingface-hf-mem/upstream-notices/LICENSE11357 bytesAn adaptation record is bundled. Inspect the declared changes and archived original before use. Review adaptation and original-file hashes →
Machine-readable installation guide →Source and packaging checks recorded on 2026-10-03. These notes are not safety certification or measured task performance.
uv/uvx and network access; HF_TOKEN only for gated or private repositories.
Estimates are not measured runtime benchmarks; inference overhead and hardware support vary. Core frontmatter was adapted by BB Skills for the existing namespace directory; unchanged original SKILL.md is archived. This does not imply client execution was tested.
Upstream commit: ca0325bb20b2d0a1b2efa893670c4c72f79e707b
Runtime status: not tested by this catalog. Configure your client and test the skill in your own environment.
Records are supplied by the site administrator and bound to a specific package. They are not third-party safety certification. This page does not execute skills.
No published scenario records yet. Resource availability and download counts do not imply measured task performance.
Be the first to share your experience.
Sign in to leave a review →Links use published guide associations and catalog labels. Inspect each resource’s requirements and license.
Find llama.cpp-compatible GGUF models, compare quantization choices and launch local inference with llama-cli or llama-server after checking…
Organize training metrics, dashboards, alerts and experiment queries with Trackio, checking storage and reporting requirements for the inten…
Inspect dataset splits and rows, apply supported filters and locate Parquet downloads through the Dataset Viewer API before choosing data to…
1 current outcome record · Read scope
Turn recorded speech into text, with optional speaker labels and known-speaker hints. Inspect the transcription script and API requirements …