llama.cpp under review - R.I.P. LMStudio

Aug 16 (edited) in Tech-Talk

llama.cpp is a standalone server that can run in Self-Host Studio and replace OpenAI for generating stories and image prompts and any other ChatBot/LLM related tasks. Stay tuned for my results and hopefully, a new release of Studio right behind it.

Backstory. I use LMStudio for benchmarking and results comparisons of various LLMs and it's a great tool but it only runs as a desktop app. That's too cumbersome and required some human interaction to set it up so, I won't be adding LMStudio into Self-Host Studio.

Cheers

3 comments