Well, yeah, I could’ve told you that too but neither of us would have any proof. It’s one thing to try it out and decide that it sucks for your use case and another thing to measure and quantify it somehow.
Why such a negative reaction if you apparently agree with the outcome?
Well, for one thing, it’s part of a wider trend of misreporting about AI. For another, the more interesting, meaningful angle here would be why the (frankly very simplistic) research of the BBC is mismatched with the supposedly more rigorous benchmarks used for LLM quality testing and reported in new releases.
In fact, are they? What do they mean? Should people learn about them and understand them before engaging? Probably, yeah, right? But the BBC is saying their findings have “far reaching implications” without engaging with any of those issues, which are not particularly obscure or unknown in the field.
The gap between what’s being done in LLM development, what is being reported about it and how the public at large understand it is bizarre and hard to quantify. I believe once the smoke clears people will have some guilt to process about it, regardless of what the outcome of the hype cycle ends up being.
Yeah, I intentionally left out the word “groundbreaking” from the title when posting, because that’s a ridiculous thing to say about this research. Obviously, it could be much better.
But I would say that any attempt at rational look at LLMs in mainstream media is a step in the right direction.
Wow, what sort of advanced techniques of investigative journalism did they deploy? Use the thing for five minutes and count?
I’m not even a big hater of LLMs and I could have told you that for free.
Well, yeah, I could’ve told you that too but neither of us would have any proof. It’s one thing to try it out and decide that it sucks for your use case and another thing to measure and quantify it somehow.
Why such a negative reaction if you apparently agree with the outcome?
Well, for one thing, it’s part of a wider trend of misreporting about AI. For another, the more interesting, meaningful angle here would be why the (frankly very simplistic) research of the BBC is mismatched with the supposedly more rigorous benchmarks used for LLM quality testing and reported in new releases.
In fact, are they? What do they mean? Should people learn about them and understand them before engaging? Probably, yeah, right? But the BBC is saying their findings have “far reaching implications” without engaging with any of those issues, which are not particularly obscure or unknown in the field.
The gap between what’s being done in LLM development, what is being reported about it and how the public at large understand it is bizarre and hard to quantify. I believe once the smoke clears people will have some guilt to process about it, regardless of what the outcome of the hype cycle ends up being.
Yeah, I intentionally left out the word “groundbreaking” from the title when posting, because that’s a ridiculous thing to say about this research. Obviously, it could be much better.
But I would say that any attempt at rational look at LLMs in mainstream media is a step in the right direction.