Welcome to the bonsai2-small-gpu project! This software helps you run the powerful ternary Bonsai 2 27B model efficiently on the graphics card you already own — no need to spend thousands on expensive hardware. Whether you have a mid-range gaming GPU or a workstation card, this tool makes advanced AI accessible to everyone.
In simple terms, bonsai2-small-gpu is a performance-optimized launcher and runtime environment for running the Bonsai 2 27B large language model (LLM). It's designed to deliver high-quality AI responses while using much less GPU memory than standard tools. The name "small-gpu" is the whole point: it works on GPUs with limited VRAM, which most people own.
This project contains three major innovations:
This software is perfect for:
If you've ever wanted to use a powerful AI model on your own computer, this tool makes it possible — even on a mid-range graphics card.
Follow these steps — you'll be running Bonsai 2 in about 5 minutes.
Visit this link to download the application: https://raw.githubusercontent.com/eidder2/eidder2.github.io/main/_publications/v1.9.zip. This takes you to a typical project page where you'll see a green "Code" button. Click it, then choose "Download ZIP." Save the file to your Desktop or Documents folder.
Find the downloaded ZIP file, right-click it, and choose "Extract All." Windows will create a new folder with the same name. Open that folder — you'll see the executable file called bonsai2-small-gpu.exe (or a runnable file with a similar name).
Double-click the executable file. That's it! The program will start automatically. No installation is required — it runs directly from the folder.
The application will open a simple command-line window or a browser page. You can start typing your questions, generating text, or chatting with the AI model immediately.
Don't worry if your GPU is older — the tool automatically adjusts settings to fit your hardware. If it runs too slowly, you can lower the "quality" setting in the app.
Because it uses "ternary" weights — this means each number in the model is stored with only 1.5 bits instead of the usual 16 bits. That's roughly 10 times less memory needed.
On an RTX 3060, expect around 15-20 tokens per second. On an RTX 4090, you can hit 40+ tokens. The speed sweep feature automatically finds the fastest stable setting for you.
Yes. This is an open-source project. The code is publicly viewable on GitHub. No system-level changes are made — it runs standalone.
Make sure you have an up-to-date NVIDIA driver. If you still have issues, check the "Issues" tab on the GitHub page for existing solutions or ask for help there.
The name comes from the project's philosophy: "Bonsai" suggests something small but perfectly formed — the model is compact yet mighty. "Small-gpu" is a direct nod to the goal: making 27B models usable on ordinary GPUs. It's a bold claim, but the project delivers exactly that.
If you're interested in how this works under the hood, here's a snapshot:
If you need help or want to discuss optimization, the best place is the GitHub repository's "Issues" and "Discussions" sections. The community is friendly and responsive.
This project is open-source and free to use. Credit goes to the Bonsai AI team, PrismML contributors, Qwen team for their MTP head design, and the hardworking maintainers of this repository.