But a superintelligence quickly runs into the limits of physics and cooling becomes the bottleneck.

Can you expand on this?

Right, and regardless of sci-fi future potentials, there is a current reality and we’re in it and only have so much time to affect it’s evolution. All of these likely intelligent people completely ignoring reality isn’t helping define and construct the world we want tomorrow.

LLMs entering the world have revealed two very different types of crazies and I’m starting to think they’re everywhere. It’s quite tiring to have to constantly say “no they’re not useless” and “no they’re not super god” instead of engage with an actually interesting discussion.

It’s hard to take this type of response seriously as it simply ignores objective reality. The claim of a human being required to start the process is utterly meaningless to the point. Sure though, I’ll throw you a bone: if no human ever initiates the task it simply won’t happen. Can we move on now since humans are happily initiating the actions?

I couldn’t care less what people think about whether or not they are “thinking” or “having agency of their own”[1]. I care about the practical reality of the world in which LLMs have shown, and will continue to show, the ability to exploit computer systems and humans. You can toss your head in the sand and think you have some sort of weird pseudo-intellectual high ground by claiming “but akchtually it’s just a result of their training data”. Like, yea, dawg, that’s how this works, congrats. Of course the thing that is trained on training data is trained on training data.

[1] At least when discussing their capabilities. When someone is willing to have that discussion beyond the tired “they’re just autocomplete!” trivialization it’s interesting.

This is pretty naive. Distributing computing has existed since forever. It would run significantly more slowly, but there is nothing theoretically stopping the “LLM broke out of containment and moved itself elsewhere” boogie man from becoming a reality. The available exploitable compute from unsecured IoT devices is pretty substantial as things like the Mirai botnet have shown. While this isn’t the most easily distributed problem, there is no fundamental issue here. [Edit: it also wouldn’t need to “exfiltrate itself” to cause persistent harm. Could easily just install malware on infected hosts]

AI isn’t “dumb” in a practical real world sense. LLMs have shown themselves capable of exploitation on the open Internet and of finding zero day vulnerabilities. Do you have any idea how much of our critical infrastructure can be reachable by some sort of network? LLMs have shown themselves capable of manipulating people via speech. None of this requires some sort of “super intelligence” and it doesn’t even require some sort of provable intelligence: it’s all quite possible as things currently stand right now.

These may not be “the entire future is over” level issues, but there is significant risk here and it shouldn’t be ignored just because we think AI companies are horrible. Is an LLM going to make us all batteries or paper clips? No, but it could cause significant physical and/or economic harm temporarily.

This is such a wild take to be getting this many upvotes. If nothing happened in LLM development from this day forward they’d still be absurdly dangerous: both in an information security context and a social engineering context.

Lemmy is absolutely head in the sand about LLMs and it is wild. You can dislike LLMs and acknowledge their capabilities [and limitations] at the same time.