evolves robots.txt instructions by adding an automated licensing layer that’s designed to block bots that don’t fairly compensate creators for content
robots.txt - the well known technology to block bad-intention bots /s
What’s automated about the licensing layer? At some point, I started skimming the article. They didn’t seem clear about it. The AI can “automatically” parse it?
# NOTICE: all crawlers and bots are strictly prohibited from using this # content for AI training without complying with the terms of the RSL # Collective AI royalty license. Any use of this content for AI training # without a license is a violation of our intellectual property rights.License:https://rslcollective.org/royalty.xml
Yeah, this is as useless as I thought it would be. Nothing here is actively blocking.
I love that the XML then points to a text/html content website. I guess nothing for machine parsing, maybe for AI parsing.
I don’t remember which AI company, but they argued they’re not crawlers but agents acting on the users behalf for their specific request/action, ignoring robots.txt. Who knows how they will react. But their incentives and history is ignoring robots.txt.
robots.txt - the well known technology to block bad-intention bots /s
What’s automated about the licensing layer? At some point, I started skimming the article. They didn’t seem clear about it. The AI can “automatically” parse it?
# NOTICE: all crawlers and bots are strictly prohibited from using this # content for AI training without complying with the terms of the RSL # Collective AI royalty license. Any use of this content for AI training # without a license is a violation of our intellectual property rights. License: https://rslcollective.org/royalty.xmlYeah, this is as useless as I thought it would be. Nothing here is actively blocking.
I love that the XML then points to a text/html content website. I guess nothing for machine parsing, maybe for AI parsing.
I don’t remember which AI company, but they argued they’re not crawlers but agents acting on the users behalf for their specific request/action, ignoring robots.txt. Who knows how they will react. But their incentives and history is ignoring robots.txt.
Why
am Iis this comment so negative. Oh well.