squishy is a better term than “hard”

Eh, agree to disagree. My use of the phrase might be easier to parse if you think about the “hard problem of consciousness” as opposed to the “easy problem of consciousness”. Also, more remotely, the notion of NP-hard vs NP-easy in computational complexity. I guess we are arguing about semantics aren’t we ?

I’m fairly certain that’s not a strict definition - LLMs also seem very heavily influencable via their context window inputs, though those are more limited and transient than the training set.

While LLMs don’t simply “statistically predict the next token” (common over-simplification which bears no relation to what is actually going on), the output they produce does exhibit the same statistical distribution as their training data. That’s how you get intelligible language, code that compiles, chains of thought that makes sense etc… The context input will steer the output towards a certain subset of the corpus (highly optimized C functions have slightly different distribution than throw-away Python scripts), but it can never direct the model fully out of distribution as those out of distribution vectors cannot be expressed within the model’s embedding space.

she was a commodity - as talented as you could ask, more talented than many national acts, but not unique

Yes exactly, if music was solvable then the most skilled people would systematically outclass the least skilled ones which is obviously not the case.