Run GLM-4.7 (358B) on 8GB VRAM with AirLLM

Jupyter Notebook 53.1%
Python 46.9%

Find a file

Riley Seaburg f4ebe2fd28 Initial commit: GLM-4.7 with AirLLM Run the 358B parameter GLM-4.7 model on consumer hardware (8GB VRAM)! Features: - AirLLM layer-wise loading for low VRAM inference - 4bit/8bit compression support for 3x speedup - Interactive CLI and Jupyter notebook - GLM-4.7's Interleaved Thinking for complex reasoning GLM-4.7 benchmarks: - SWE-bench Verified: 73.8% - SWE-bench Multilingual: 66.7% - HLE (w/ Tools): 42.8% - AIME 2025: 95.7%		2026-01-21 21:58:44 +00:00
glm47_airllm.py	Initial commit: GLM-4.7 with AirLLM	2026-01-21 21:58:44 +00:00
glm47_notebook.ipynb	Initial commit: GLM-4.7 with AirLLM	2026-01-21 21:58:44 +00:00
README.md	Initial commit: GLM-4.7 with AirLLM	2026-01-21 21:58:44 +00:00
requirements.txt	Initial commit: GLM-4.7 with AirLLM	2026-01-21 21:58:44 +00:00

README.md

GLM-4.7 with AirLLM

Run the 358B parameter GLM-4.7 model on consumer hardware using AirLLM's layer-wise loading.

GLM-4.7 Highlights

Benchmark	Score	vs GPT-5.1-High
SWE-bench Verified	73.8%	-2.5%
SWE-bench Multilingual	66.7%	+11.4%
Terminal Bench 2.0	41.0%	-6.6%
HLE (w/ Tools)	42.8%	+0.1%
AIME 2025	95.7%	+1.7%

Key Features:

Interleaved Thinking: Thinks before every response and tool call
Preserved Thinking: Retains reasoning across multi-turn conversations
358B MoE: Massive model, MIT licensed

Requirements

Python 3.10+
GPU with 8GB+ VRAM (AirLLM handles the rest)
~400GB disk space for model shards

Installation

pip install airllm torch transformers

Quick Start

from airllm import AutoModel

# Load GLM-4.7 with 4-bit compression for faster inference
model = AutoModel.from_pretrained(
    "zai-org/GLM-4.7",
    compression="4bit"  # or "8bit" or None
)

# Generate
input_text = ["Write a Python function to implement binary search"]
input_tokens = model.tokenizer(
    input_text,
    return_tensors="pt",
    truncation=True,
    max_length=4096
)

output = model.generate(
    input_tokens['input_ids'].cuda(),
    max_new_tokens=512,
    use_cache=True,
    return_dict_in_generate=True
)

print(model.tokenizer.decode(output.sequences[0]))

With Thinking Mode

GLM-4.7 supports "Interleaved Thinking" - enable it for complex tasks:

messages = [
    {"role": "user", "content": "Solve this step by step: What is 23 * 47?"}
]

# Apply chat template with thinking enabled
text = model.tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True  # Enable thinking mode
)

Memory Usage

Compression	VRAM Required	Speed
None (BF16)	~16GB	Baseline
8-bit	~8GB	~1.5x faster
4-bit	~4GB	~3x faster

AirLLM loads one layer at a time, so even 358B models fit in 8GB VRAM!

License

MIT (GLM-4.7) / Apache-2.0 (AirLLM)