openmic.social is an uncensored community. You may encounter strong language, controversial opinions, and mature or NSFW material. You must be 18+ to browse. Illegal content is prohibited and removed on sight — please report it. By continuing, you accept that you may see content you personally disagree with.
You bring up some good points. Just FYI my research was specifically on ai, but a very specific branch in college. So not llm but only somewhat llm flavored.
The are already including ai chips in consumer hardware. And your thinking of llm specific limitations. But the smaller models can absolutly be thrown into hardware. Its just the algorithms are moving so fast that the hardware needs to be flexible enough. Thars the biggest reason we dont see more hardware faster than gpus. Hooe that makes sense!
I was gonna bring up changing algorithms too, but didn’t because the comment got too big already.
NPUs are just stripped-down GPUs with less flexible instruction set. They don’t meaningfully advance performance over a GPU, but instead reduce cost.