• 34 Posts
  • 2.56K Comments
Joined 1 year ago
cake
Cake day: March 22nd, 2024

help-circle
  • I elaborated below, but basically Musk has no idea WTF he’s talking about.

    If I had his “f you” money, I’d at least try a diffusion or bitnet model (and open the weights for others to improve on), and probably 100 other papers I consider low hanging fruit, before this absolutely dumb boomer take.

    He’s such an idiot know it all. It’s so painful whenever he ventures into a field you sorta know.

    But he might just be shouting nonsense on Twitter while X employees actually do something different. Because if they take his orders verbatim they’re going to get crap models, even with all the stupid brute force they have.


  • There’s some nuance.

    Using LLMs to augment data, especially for fine tuning (not training the base model), is a sound method. The Deepseek paper using, for instance, generated reasoning traces is famous for it.

    Another is using LLMs to generate logprobs of text, and train not just on the text itself but on the *probability a frontier LLM sees in every ‘word.’ This is called distillation, though there’s some variation and complication. This is also great because it’s more power/time efficient. Look up Arcee models and their distillation training kit for more on this, and code to see how it works.

    There are some papers on “self play” that can indeed help LLMs.

    But yes, the “dumb” way, aka putting data into a text box and asking an LLM to correct it, is dumb and dumber, because:

    • You introduce some combination of sampling errors and repetition/overused word issues, depending on the sampling settings. There’s no way around this with old autoregressive LLMs.

    • You possibly pollute your dataset with “filler”

    • In Musk’s specific proposition, it doesn’t even fill knowledge gaps the old Grok has.

    In other words, Musk has no idea WTF he’s talking about. It’s the most boomer, AI Bro, not techy ChatGPT user thing he could propose.




  • On the two subs I frequented:

    • /r/thelastairbender is just cultish and shallow now. I abandoned it. But it’s painful for me, as this is like the only sane place left the fandom has any critical mass. /c/thelastairbender is nice, but very quiet.

    • /r/localllama Has… lost its intelligence? Like no one seems to experiment or talk technically anymore, good talk seems to be on github, or shattered across Discords, while the ‘critical mass’ is in the AI Bro black hole of Twitter and Linkedin. I read it, but never post anymore. localllama here is better, but smaller and downvoted to hell.

    Also, I’ve been shadowbanned on like 4 accounts in 3 different IPs/machines, no explanation, no recourse. I never post anything political or even remotely provocative (unless links to Lemmy count) and only visit those two subs, so… Yeah, kinda sick of that.




  • I hate to sound preachy, but this is a good example of “rivals” peacefully meeting.

    So many people I meet IRL seem conditioned to think this person they hate on the internet would be someone they’d shout at like they’re an axe murderer, in the middle of a murder. It’s the example they see. Death threats are, like, normal on Facebook or TV News or whatever they’re into, apparently.

    Again at risk of reaching… this feels like positive masculinity to me.

    And leaders acting like adults.



  • Only because the sound card is exclusively designed for windows.

    It’s not that way anymore. I actually can’t configure gain (and some other features) for my Fiio KA3 on Windows. Now Android (and iOS) are their main priority.

    Which does give the useful quirk of allowing me to configure it in desktop linux…

    This is going to be a pattern though. It won’t necessarily get better for the Linux desktop, but Windows is going to increasingly feel the pain of being a “lower priority” OS for hardware.


  • Yes, and that was a cruel, stupid move on the US’s part.

    …But even if cooperation continued, it still would have given Iran expertise. Further enrichment is not a huge step, especially behind the cover of real civilian power programs, and given the rhetoric the state broadcasts and their neighbor’s hostility, it seems likely.

    And that’s fine IMO.

    I’m hugely afraid of proliferation, but going to these lengths to worry about it while the rest of the world burns seems ridiculous.


  • To be fair, Iran wants a nuke down the line, and civilian uranium enrichment is a huge stepping stone. There’s lots of technical alternatives they could pursue if they really just want civilian power.

    …And that’s kinda understandable. They have a neighbor that randomly bombs their civilians.

    Fuck it, let them have one.

    Heck, they should get a tiny bit of old Soviet+US stock in some kind of international deal, so they have credible deterrence with the guaranteed stability+security mechanisms (and oversight?) of their weapons.

    (To be more specific, Cold War nukes typically have elaborate tiggers and failsafes meant to stop unauthorized parties from detonating them with any nuclear yield, and the old school Soviet and US systems are pretty good. Better for them to have that than an “insecure” home cooked design they waste money on, like the North Koreans allegedly have, IMO. On top of that, they’d have “known” detonation signatures, so if they ever go off everyone would know it’s Iran (defeating the fear of them “losing” a nuke to another party, or a false flag op against Iran)).


  • IMO that reaction is healthy, as long as it isn’t a hostile “you’re holding it wrong” (which was not my intent, and is very much a community problem). Communal troubleshooting is the nature of the Linux desktop.

    If you don’t want advice, that’s fine, probably reasonable based on what you described. But I have had some similar (but not so severe) issues with Fioo and Xonar cards that got fixed with some low level configs I had no idea existed.


  • I’m on the millennial train here, and am fully onboard with the monopolization angle, but this is taking it a bit far? Chromebooks aren’t that bad.

    Stepping back and maybe over generalizing again, I think the problem might be… attention spans? Like kids are so bombarded with feeds and notification spam that, on average, there’s less patience to sit down, look stuff up, and neurotically tinker (which was still the vast minority in my generation). Its the same problem leading to less interest in literature, TV, anything long form.

    Learning the bare minimum to function in Windows is not exactly “tech literate” to me, it just happens to be the system so many businesses are stuck with, and some generations were forced to learn by coincidence. Looking back, modern Android and iOS are really accessible by comparison, though of course they have enshittification issues.






  • Whether World ID’s system catches on and succeeds will come down to trust.

    That’s ironic, because the core concept is that you don’t need to “trust” Tools for Humanity with your personal data. You can read the company’s white papers and understand exactly how it all works.

    But humans aren’t wired to think that way. Most people will want to trust the company scanning their irises. That’s a challenge for a company that takes a lot of licks in the press for its futuristic introduction of blockchain into the real world in the form of the metallic orbs.

    Oh, I understand how it works, I understand I don’t want that shit going in and out of some public ledger on their stack.

    I understand I don’t want to boost another crypto pyramid scheme.

    I understand this is associated with Sam Altman, I understand I work with open weights models and (for Lemmy) am a gigantic ML enthusiast, and I understand that, even then, I would not wish one bit of crap that con man sells on my worst enemy, and will shout it to anyone who will listen.



  • Hear me out, theoretically this is actually cool?

    Like, theoretically, take away the Fox propaganda and shameless lack of disclosure. Picture this guy presenting himself as a CI, like a convicted hacker, former cartel, defector or something. He has real experience with Iran, missiles, the whole Contra affair. He knows what they’d do with them, how they get them, what the buyers are like because he shamelessly sold them.

    That’d be a cool perspective.

    It’s of course not reality, but still.