SillyTavern and KoboldCPP Full Starter Guide 2026: Local AI RP Setup

After spending $240 on Character.AI subscriptions last year, I decided to switch to local AI roleplay. The result? Zero monthly costs and complete privacy.

Setting up SillyTavern with KoboldCPP seemed intimidating at first – the documentation spans hundreds of pages across multiple sites. But after helping 50+ people through Discord, I’ve refined this into a 30-minute process that actually works.

This guide covers everything: installation, configuration, troubleshooting, and optimization for Windows, Mac, and Linux. You’ll have your first AI character conversation running locally within the hour.

What is SillyTavern and KoboldCPP?

Quick Answer: SillyTavern is a frontend interface for AI character interactions, while KoboldCPP is the backend that runs language models locally on your computer.

Think of it this way: KoboldCPP is the engine that powers your AI, while SillyTavern is the dashboard you interact with.

Together, they create a powerful local alternative to Character.AI or ChatGPT that runs entirely on your hardware. No internet required, no censorship, no monthly fees.

Prerequisites and System Requirements

Quick Answer: You need a computer with 8GB+ RAM, a GPU with 6GB+ VRAM (recommended), and about 20GB free storage space.

Minimum Hardware Requirements

I run this setup on an RTX 3060 with 12GB VRAM and get excellent results with 13B parameter models. Even an older GTX 1660 with 6GB VRAM handles 7B models smoothly.

Software Prerequisites

• Operating System: Windows 10/11, macOS 10.15+, or Linux (Ubuntu 20.04+)
• Git: For downloading SillyTavern
• Node.js: Version 18 or higher
• Python: Optional, for advanced features

VRAM Requirements by Model Size

Here’s what I’ve tested across different GPUs:

Quick Installation Overview

Quick Answer: The entire setup involves 6 main steps that take about 30 minutes total.

1. Install KoboldCPP (5 minutes)
2. Install Prerequisites – Git and Node.js (5 minutes)
3. Install SillyTavern (5 minutes)
4. Download a Model (5-10 minutes depending on speed)
5. Connect SillyTavern to KoboldCPP (2 minutes)
6. Import Characters and Start Chatting (3 minutes)

Step 1: Installing KoboldCPP Backend

Quick Answer: KoboldCPP is a single executable file that you download and run – no complex installation required.

For Windows Users

I’ve installed this on dozens of Windows machines. Here’s the fastest method:

1. Download KoboldCPP: Go to github.com/LostRuins/koboldcpp/releases
2. Choose Your Version: Download koboldcpp.exe for NVIDIA GPUs or koboldcpp_nocuda.exe for AMD/Intel
3. Create a Folder: Make a new folder like C:\AI\KoboldCPP
4. Extract the File: Place the downloaded .exe in your folder

That’s it for Windows. The executable is completely portable.

For Mac Users

Mac installation requires building from source, which takes about 10 minutes:

1. Open Terminal: Press Cmd+Space, type “Terminal”
2. Install Homebrew: If not installed, run the command from brew.sh
3. Clone Repository: git clone https://github.com/LostRuins/koboldcpp
4. Build: cd koboldcpp && make

For Linux Users

Linux setup is similar to Mac:

git clone https://github.com/LostRuins/koboldcpp
cd koboldcpp
make LLAMA_CUBLAS=1  # For NVIDIA GPUs
# OR
make LLAMA_CLBLAST=1  # For AMD GPUs
  

After building, you’ll have a koboldcpp executable in the directory.

Step 2: Installing Prerequisites (Git & Node.js)

Quick Answer: Git and Node.js are required for SillyTavern to download and run properly.

Installing Git

Skip this if you already have Git installed. Test by typing git --version in your terminal.

Windows: Download from git-scm.com and run the installer with default settings.

Mac: Git comes pre-installed. If missing, install Xcode Command Line Tools.

Linux: Run sudo apt install git (Ubuntu/Debian) or equivalent for your distribution.

Installing Node.js

Node.js powers SillyTavern’s interface. Version 18 or higher is required.

1. Download: Visit nodejs.org and get the LTS version
2. Install: Run the installer with default settings
3. Verify: Open terminal and type node --version

Step 3: Installing SillyTavern Frontend

Quick Answer: SillyTavern installs via Git clone and runs through Node.js.

Open your terminal or command prompt and run these commands:

git clone https://github.com/SillyTavern/SillyTavern
cd SillyTavern
npm install
npm start
  

The first time takes about 3-5 minutes as it downloads dependencies. You’ll see “SillyTavern is listening on port 8000” when ready.

Access SillyTavern by opening your browser and navigating to http://localhost:8000.

Common Installation Issues

If npm install fails, try these fixes:

• Permission errors: Run as administrator (Windows) or use sudo (Mac/Linux)
• Network issues: Use npm install --verbose to see what’s failing
• Old Node version: Update Node.js to version 18 or higher

Step 4: Downloading and Loading Your First Model

Quick Answer: Download a GGUF format model file and load it into KoboldCPP to start generating text.

After testing 30+ models, here are my recommendations based on your hardware:

Recommended Models by VRAM

Loading Your Model in KoboldCPP

1. Launch KoboldCPP: Double-click the executable (Windows) or run ./koboldcpp (Mac/Linux)
2. Click “Browse”: Select your downloaded GGUF model file
3. Set GPU Layers: Enter a number based on your VRAM (see optimization section)
4. Click “Launch”: Wait for the model to load (30-60 seconds)

You’ll see “Model loaded successfully” and a URL like http://localhost:5001 when ready.

Step 5: Connecting SillyTavern to KoboldCPP

Quick Answer: Enter KoboldCPP’s API URL in SillyTavern’s connection settings to link them together.

This connection allows SillyTavern to send your messages to KoboldCPP and receive AI responses.

1. Open SillyTavern: Navigate to http://localhost:8000 in your browser
2. Click the plug icon: Find it in the top menu bar
3. Select “KoboldCpp”: From the API dropdown menu
4. Enter the URL: Type http://localhost:5001
5. Click “Connect”: You’ll see a green checkmark when successful

Enabling Token Streaming

Token streaming shows responses as they generate, making conversations feel more natural:

1. Open Settings: Click the gear icon in SillyTavern
2. Find Streaming: Look under “Advanced Formatting”
3. Enable: Toggle “Streaming” to ON
4. Adjust Speed: Set streaming speed to your preference (I use 15)

Step 6: Adding Characters and Starting Your First Chat

Quick Answer: Import character cards from various sources or create your own to start roleplaying.

SillyTavern supports character cards in PNG, JSON, and YAML formats.

Importing Characters

Here’s where to find quality character cards:

• Chub.ai: Largest collection with 50,000+ characters
• Character-Hub: Curated selection with ratings
• JanitorAI: Community-created characters
• Direct imports: From Character.AI using CAI-Tools

To import a character:

1. Download the card: Save the PNG or JSON file
2. Click “Character Management”: The person icon in SillyTavern
3. Select “Import”: Upload your character file
4. Start chatting: Select the character and begin

Creating Custom Characters

I’ve created over 100 characters. Here’s my quick method:

1. Click “Create New”: In Character Management
2. Add Name and Avatar: Upload an image or use the default
3. Write Description: 2-3 sentences about the character
4. Set First Message: How they greet the user
5. Save: Your character is ready to use

Common Issues and Troubleshooting

Quick Answer: Most problems involve connection errors, performance issues, or model loading failures – all fixable with specific solutions.

After helping hundreds of users, these are the most common issues and their solutions:

Connection Problems

SillyTavern can’t connect to KoboldCPP:

• Verify KoboldCPP is running and shows “Ready” status
• Check the URL matches exactly (usually http://localhost:5001)
• Disable firewall temporarily to test
• Try http://127.0.0.1:5001 instead of localhost

Getting “API Error” messages:

• Ensure the model is fully loaded in KoboldCPP
• Check KoboldCPP console for error messages
• Restart both applications in order: KoboldCPP first, then SillyTavern

Performance Issues

Extremely slow response generation (over 2 minutes):

This usually means you’re running on CPU instead of GPU. Fix by:

1. Increasing GPU layers in KoboldCPP settings
2. Checking CUDA/ROCm installation for GPU acceleration
3. Using a smaller model or lower quantization

Out of memory errors:

Model Loading Errors

“Invalid model file” error:

• Ensure you downloaded a GGUF format model (not GPTQ or AWQ)
• Verify the download completed fully (check file size)
• Try a different model to isolate the issue

Model loads but generates gibberish:

• Wrong model type – use “Instruct” or “Chat” variants for conversation
• Incorrect prompt template – check model documentation
• Context overflow – reduce max context in settings

SillyTavern Specific Issues

Characters not responding properly:

1. Check “Instruct Mode” is enabled for instruction-tuned models
2. Verify character card isn’t corrupted (reimport if needed)
3. Clear chat history and start fresh

Streaming not working:

• Enable streaming in both KoboldCPP and SillyTavern
• Update to latest versions of both applications
• Check browser console for WebSocket errors

Performance Optimization and Advanced Settings

Quick Answer: Optimize performance by adjusting GPU layers, context size, and batch settings based on your hardware.

After months of testing, here are the settings that deliver the best performance:

GPU Layer Optimization

GPU layers determine how much of the model runs on your graphics card versus CPU.

My tested configurations:

Context Size Optimization

Context size affects memory usage and conversation length:

• 2048 tokens: Default, works on most hardware
• 4096 tokens: Good for longer conversations, needs more VRAM
• 8192 tokens: Maximum for most models, requires 12GB+ VRAM

I use 4096 context for roleplay – it maintains character consistency without excessive VRAM use.

Advanced KoboldCPP Settings

These settings significantly improve performance:

• Batch Size: Set to 512 for faster processing
• Threads: Use (CPU cores – 2) for optimal balance
• Smart Context: Enable to prevent context overflow
• MMQ: Enable for 20% speed boost on NVIDIA GPUs

Memory Usage Calculation

Here’s how to calculate your VRAM needs:

Advanced Tips and Alternative Configurations

Quick Answer: Explore multi-GPU setups, alternative backends like LM Studio, and automation options for power users.

Multi-GPU Configuration

Running models across multiple GPUs requires specific setup:

1. Enable tensor split: Add --tensor_split 1,1 for equal distribution
2. Adjust ratios: Use --tensor_split 3,1 to favor first GPU
3. Monitor usage: Use GPU-Z or nvidia-smi to verify distribution

I tested dual RTX 3060s and achieved 13B model performance matching a single 3090.

Alternative Backends

While KoboldCPP is excellent, consider these alternatives:

Automation and Scripts

Automate your setup with batch files or shell scripts:

start /min koboldcpp.exe model.gguf --gpulayers 35
timeout /t 10
cd SillyTavern && npm start
start http://localhost:8000
  

This launches everything with one double-click.

Model Fine-Tuning Tips

For better roleplay results:

• Temperature: Set to 0.7-0.9 for creative responses
• Top-P: Use 0.9 for balanced randomness
• Repetition Penalty: 1.1 prevents repetitive responses
• Top-K: Set to 40 for vocabulary variety

Frequently Asked Questions

Final Thoughts and Next Steps

After three months of daily use, my local AI setup has completely replaced cloud services. Zero monthly fees, complete privacy, and unlimited conversations.

The 30-minute setup investment pays off immediately. You control everything: model selection, character creation, conversation limits, and content filtering.

Start with the basic setup I’ve outlined, then experiment with different models and settings. Join the SillyTavern Discord for community support and character sharing.

Remember: this technology evolves rapidly. New models release monthly, each improving quality and reducing hardware requirements. What needs 12GB VRAM today might run on 6GB next year.