3DWebAI: Fine-Tuning Llama 3 to Write 3D Scenes

3DWebAI started from a simple curiosity: could a small language model learn to write React Three Fiber (R3F) scenes from a plain-English description? R3F is a fairly narrow domain — it has its own conventions and component patterns — which made it a good testbed for studying how well a model follows instructions inside a specific code ecosystem.

So I fine-tuned and aligned Meta-Llama-3 (~1.8B) to generate 3D scene code from natural-language prompts.

The setup

The whole project was built to be reproducible on modest hardware, so the tooling choices mattered:

• Unsloth for efficient fine-tuning of the small Llama 3 model.
• TRL for the alignment step.
• A curated dataset drawn from ~100 repos of real R3F code, so the model learned from genuine usage rather than synthetic examples.

Keeping the base model small (~1.8B) meant I could iterate on the dataset and training loop quickly instead of waiting on huge runs.

Why a narrow domain is a good experiment

Domain-specific code generation is a cleaner lens on instruction-following than general coding, because the "correct" behavior is more constrained:

• The vocabulary of components and patterns is limited and well-defined.
• It's easier to tell whether the model understood the request versus pattern-matched.
• Curating from real repositories means the target style is consistent, so the model has a coherent thing to learn.

That constraint is what makes the study interesting — you can actually see whether fine-tuning taught the model the domain's idioms.

What I learned

Building 3DWebAI reinforced a few things for me:

• Dataset curation is most of the work. Pulling from ~100 real repos shaped the model's behavior far more than any hyperparameter.
• Small models go far in a narrow domain. A ~1.8B model can pick up a specific code style when the target is focused.
• Unsloth plus TRL is a practical loop. Efficient fine-tuning and alignment on accessible hardware made rapid iteration possible.

This was a research-minded project about instruction-following in domain-specific code generation — a way to probe how a compact model handles a constrained, real-world coding task.

If you want to check out the project directly, here it is: 3DWebAI on Hugging Face.