Fine-Tuning (Experimental)
SmartCursorX includes an experimental fine-tuning pipeline that lets you train custom adapter models using local hardware. This is part of the "100% democratized IDE" vision — anyone can fine-tune models on their own machine.
Overview
The fine-tuning pipeline supports:
- LoRA adapters: Lightweight fine-tuning using Low-Rank Adaptation (PEFT)
- QLoRA: Quantized LoRA for reduced memory usage
- Multi-GPU: DDP, FSDP, and DeepSpeed configurations
- Micro-benchmarking: 10-step benchmark to estimate training time before committing
Hardware Requirements
| Component | Minimum | Recommended |
|---|---|---|
| RAM | 16 GB | 32 GB |
| VRAM | 4 GB | 8 GB (NVIDIA) |
| Storage | 10 GB free | 50 GB free |
| Python | 3.10+ | 3.12 |
| PyTorch | 2.0+ | 2.4+ |
Low-VRAM Models
For consumer GPUs with 4-6 GB VRAM:
| Model | Parameters | VRAM (4-bit) |
|---|---|---|
| Qwen2.5-Coder-1.5B | 1.5B | ~1.2 GB |
| DeepSeek-Coder-1.3B | 1.3B | ~1.0 GB |
| Stable-Code-3B | 3B | ~2.4 GB |
| CodeGemma-1.1B | 1.1B | ~0.9 GB |
Pipeline
Hardware Detection
The system detects your hardware capabilities via nvidia-smi or WMI, identifying GPU, VRAM, RAM, and CPU cores. Results are cached for 30 days with an available force refresh option.
Model Recommendation
Based on detected hardware, the system recommends suitable base models. A primary best-fit model is shown along with alternatives categorized by use case (Python, JavaScript, General). VRAM-aware filtering removes models that exceed available memory.
Dataset Preparation
Training data is prepared in JSONL format with instruction/response pairs. Built-in datasets include CodeAlpaca and CodeFeedback. Custom datasets can be imported via file picker, and prepared datasets can be exported for external use.
Training
Training configuration includes 1-10 epochs (default: 3), auto-tuned learning rate, VRAM-computed batch size, LoRA rank 8-64 (default: 16), and configurable target modules (Q, K, V, O projections).
Micro-Benchmark
Before full training, a 10-step micro-benchmark runs to measure actual tokens/second on your hardware and estimate total training duration. Results are shown in a highlighted panel and the process takes approximately 30-60 seconds.
Registration
After training completes, the adapter is registered in the database and becomes available in Settings → Fine-Tuned Models. It can be used for inference via the finetuned provider through llama.cpp or Python backends.
Troubleshooting
| Issue | Solution |
|---|---|
import transformers timeout | First import takes ~10s — timeout is 30s |
| Out of memory | Reduce batch size, use lower rank, or try a smaller model |
| Missing PEFT | Run pip install peft bitsandbytes |
| No GPU detected | Check nvidia-smi driver, or fall back to CPU training |
| Watchdog warning | If no progress for 60s, a warning is shown; suggest stopping at 5 min |