Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • PetteriPano@lemmy.world
    cake
    link
    fedilink
    English
    arrow-up
    0
    ·
    2 months ago

    Running qwen3.6 27b through llama.cpp.

    It’s about as capable as sonnet 3.5.

    I use it for light scripting, but real coding is done by cloud models.

    I’m also using it as the brain for my Hermes agent. It sends me digests of news, subreddits, chats that I’d like to read but don’t have time for. It does a great job researching things on the web for me, too.

  • PapaSkwat@lemmy.wtf
    link
    fedilink
    English
    arrow-up
    0
    arrow-down
    1
    ·
    2 months ago

    I host my own AI, mostly for testing and because I wanted something that was mine and mine alone. I use Ollama and run models like Llama, Mistral, and Qwen. I honestly don’t use it much, but I wanted to have my own setup just in case online services go down or become less available. It’s part of my whole “own everything I use” mantra that I’ve been on lately.