cm0002@suppo.fi to Programmer Humor@programming.dev · 8 days agoTech bros having their production environments nuked by AI is never gonna get oldsuppo.fiimagemessage-square158linkfedilinkarrow-up1960arrow-down111
arrow-up1949arrow-down1imageTech bros having their production environments nuked by AI is never gonna get oldsuppo.ficm0002@suppo.fi to Programmer Humor@programming.dev · 8 days agomessage-square158linkfedilink
minus-squareegregiousRac@piefed.sociallinkfedilinkEnglisharrow-up15·8 days agoWhat evaluation setup lets the thing being tested see the test results and gives access to the test source code?! Slop evaluating slop.
minus-squaremyotheraccount@lemmy.worldlinkfedilinkarrow-up7·8 days agoThe model is so smart, we let it write it’s own evaluation setup /s
minus-squareAwesomeLowlander@sh.itjust.workslinkfedilinkarrow-up1·7 days agoGiven that they mark it as cheating, isn’t that essentially part of the test in itself?
What evaluation setup lets the thing being tested see the test results and gives access to the test source code?!
Slop evaluating slop.
The model is so smart, we let it write it’s own evaluation setup /s
Given that they mark it as cheating, isn’t that essentially part of the test in itself?