Your best local LLM for low-VRAM (6GB)?
Your best local LLM for low-VRAM (6GB)?
Hey guys,
What's currently the best LLM for low-VRAM machines with only 6 GB VRAM? I've got 32GB RAM as well.
I'm experimenting a little with SillyTavern and I'm curious which model gets the most out of my setup. Should be multilingual and suitable for "casual chatting".
I know I will probably not get very far with this, but I'm still interested in how far we've already come.
(Using KoboldCPP if that matters).
~sp3ctre