Np. I actually use Newelle with an fork of llama.cpp that introduces the ability to use ternary models - these work a lot better under tight memory constraints, and achieve performance similar to models with 2-3 times the parameters. Was as easy as installing the fork, starting it and pointing Newelle there instead of the integrated llama.cpp. You can do that with Unsloth too of course, but it’s not that smooth.

(if you are interested: https://huggingface.co/prism-ml/collections for the ternary models, https://github.com/PrismML-Eng/llama.cpp for the build that allows for ternary models.)