openmic.social is an uncensored community. You may encounter strong language, controversial opinions, and mature or NSFW material. You must be 18+ to browse. Illegal content is prohibited and removed on sight — please report it. By continuing, you accept that you may see content you personally disagree with.
You can host a small LLM model with a phone, or a whisper or a TTS, maybe some light diffusion.
You cannot do tool use, reasonable coding, get non-hallucination answers on even moderate questions. That stuff starts to get viable around 24G and really needs closer to 200G to be able to do most of that they’re doing.
the reason they bought up all the ram is to keep someone from buying 200G of vram and diying cards putting them out of business.
Frankly, you can self-host most of these services on an old phone. Setting it up is getting easier than ever with cheap agents flying around.
You can host a small LLM model with a phone, or a whisper or a TTS, maybe some light diffusion.
You cannot do tool use, reasonable coding, get non-hallucination answers on even moderate questions. That stuff starts to get viable around 24G and really needs closer to 200G to be able to do most of that they’re doing.
the reason they bought up all the ram is to keep someone from buying 200G of vram and diying cards putting them out of business.