• 2 Posts
  • 25 Comments
Joined 1 year ago
cake
Cake day: August 2nd, 2023

help-circle



  • IMO copying communities from Reddit as-is was a mistake long-term, but was maybe necessary short-term so that people wouldn’t be confused. If I had my druthers, I’d make a new system where communities are uniquely identified purely as !UUID@lemmy.instance (though still with a human-friendly display name). You don’t get to create a community that namesquats something like !gaming@lemmy.world. All posts would be made with hashtags like Mastodon, and then each community would just configure “Include all posts with this tag in our community”. The big issue then is who moderates tags? I think a system like Bluesky has would work well, as you mention. People can moderate tags and other people can follow their work, or not.

    If that was combined with seamless account/community migration, that would solve a lot of moderation issues. If you mod a community and the admins suck, just move it to a new instance. If the mods of a particular community suck, start your own. They won’t be able to monopolize a common name, so it’s much easier to get traction.

    On the long-ago internet, there were many, many different software options that supported the same protocols, and they were also a lot more configurable generally speaking

    Lemmy is pretty good about that, actually. It’s interoperable with Mastodon via ActivityPub, and there’s other projects like MBin that work nicely with Lemmy.











  • It would be nice if there was a way to handle instance/user migrations. If an instance gets their domain name taken away, there’s no way AFAIK for the admin to say “Here’s our new location, with a verifiable signature”. Likewise there’s no way for a user AFAIK to move their account with a verifiable signature that the new one is still them. Ideally this could all happen automatically with signatures getting synced automatically and all that.

    I’m sure it would be a lot of work and no idea if ActivityPub would get in the way, but it would give people a lot more assurance that they didn’t pick a server that will screw them over by going down.


  • The whole “it’s just autocomplete” is just a comforting mantra. A sufficiently advanced autocomplete is indistinguishable from intelligence. LLMs provably have a world model, just like humans do. They build that model by experiencing the universe via the medium of human-generated text, which is much more limited than human sensory input, but has allowed for some very surprising behavior already.

    We’re not seeing diminishing returns yet, and in fact we’re going to see some interesting stuff happen as we start hooking up sensors and cameras as direct input, instead of these models building their world model indirectly through purely text. Let’s see what happens in 5 years or so before saying that there’s any diminishing returns.


  • Gary Marcus should be disregarded because he’s emotionally invested in The Bitter Lesson being wrong. He really wants LLMs to not be as good as they already are. He’ll find some interesting research about “here’s a limitation that we found” and turn that into “LLMS BTFO IT’S SO OVER”.

    The research is interesting for helping improve LLMs, but that’s the extent of it. I would not be worried about the limitations the paper found for a number of reasons:

    • There doesn’t seem to be any reason to believe that there’s a ceiling on scaling up
    • LLM’s reasoning abilities improve with scale (notice that the example they use for kiwis they included the answers from o1-mini and llama3-8B, which are much smaller models with much more limited capabilities. GPT-4o got the problem correct when I tested it, without any special prompting techniques or anything)
    • Techniques such as RAG and Chain of Thought help immensely on many problems
    • Basic prompting techniques help, like “Make sure you evaluate the question to ignore extraneous information, and make sure it’s not a trick question”
    • LLMs are smart enough to use tools. They can go “Hey, this looks like a math problem, I’ll use a calculator”, just like a human would
    • There’s a lot of research happening very quickly here. For example, LLMs improve at math when you use a different tokenization method, because it changes how the model “sees” the problem

    Until we hit a wall and really can’t find a way around it for several years, this sort of research falls into the “huh, interesting” territory for anybody that isn’t a researcher.






  • I read that and was prepared to have my mind blown. Not really impressed, though. That article says this:

    And you can literally say anything about North Korea, the most absurd thing you could imagine, and people would believe it.

    That links to this article, which says:

    The country has been in the news of late, as ongoing negotiations between the Trump and Kim Jong-un administrations appear to have soured. The chief casualty of this diplomatic failure, the New York Times (5/31/19) breathlessly reported, was Kim Jong-un’s negotiating team, with the vice chair of the North Korean Workers’ Party, Kim Yong-chol, being sent to a forced labor camp in “the latest example of how a senior North Korean official’s political fortune is made or broken at the whims of Kim Jong-un.”

    The linked NYT article says this:

    Now, he has suddenly become the latest example of how a senior North Korean official’s political fortune is made or broken at the whims of Kim Jong-un. This week, leading South Korean newspapers reported Kim Yong-chol’s fall from grace. One of them, the conservative daily Chosun Ilbo, went so far as to report that Mr. Kim had been banished to forced labor, with many of his negotiating team members either executed or sent to prison camps.

    South Korean officials and analysts cautioned that it was too early to say with precision what was happening inside Kim Jong-un’s opaque regime. South Korean news media offered differing conjectures, including whether Kim Hyok-chol, the North’s special nuclear envoy to the United States, had been executed by firing squad in March, as the Chosun Ilbo reported, or was still under interrogation.

    But they all agree on one thing: Kim Yong-chol and his negotiating team, which had driven Kim Jong-un’s diplomatic outreach toward Washington, have been sidelined, as the North Korean leader sought a scapegoat to blame for his disastrous second summit meeting with Mr. Trump, held in Hanoi, Vietnam, in February.

    That seems pretty reasonable? It says that the official has found disfavor, says what one other paper reported with language of “went so far as to report”, and also notes that it’s hard to say for sure because North Korea is very opaque.

    The FAIR article then says:

    There was one problem: Kim Yong-chol appeared only a few days later at a high profile art performance alongside Kim Jong-un.

    Yeah, that’s hard evidence he wasn’t executed, but that’s about it. Situations like this can change on a whim in a dictatorship. Maybe Kim Jong-un had a good breakfast and decided that the official’s forced labor could be done.

    FAIR also says this in that article:

    North Korea is also a favorite location for wacky and easily disprovable stories. The BBC (3/28/14) originally reported that all men were required to wear their hair like Kim Jong-un, with other haircuts banned.

    The BBC article has a correction that it’s university students and not all men (which is missing from the FAIR article), so is that true? And it’s weird to say that stuff like that is wacky when stuff like this apparently happens:

    A second, and unprecedented, TV series this winter showed hidden-camera style video of “long-haired” men in various locations throughout Pyongyang.

    In a break with North Korean TV’s usual approach, the programme gave their names and addresses, and challenged the fashion victims directly over their appearance.

    That looks legit, with footage on youtube. Is there any reason to think that’s fake? That certainly confirms my mental model of North Korea as a wacky dictatorship if it’s true.

    EDIT: FAIR’s other statements in that article are dunking on the worst possible interpretations of what people say, which just makes FAIR seem like it has a chip on its shoulder about North Korea for some reason. I’d take what they say about North Korea with a grain of salt.