[Bug] Downloading revisions not working

I made a similar post about this one (https://lemmy.world/post/11134086) and it happened again, and hopefully the Dev can notice. (ping please)

Here’s the error that Perchance throws when I try to download revisions from https://perchance.org/power-generator-manager-10k-milestone:

Uncaught URIError: URI malformed
at decodeURI (see image)

EDIT: After some later investigation, I’ve found out that this does NOT affect earlier revisions (see this comment: https://lemmy.world/comment/8770044)

@perchance

  • BluePower@sh.itjust.worksEnglish
    3·
    6 months ago

    Thanks! I will try to find more generators affected by this bug and maybe start messing around with that blank page (revisions worked on 2 of my easter egg generators so far). But for now, here’s a little snippet that I would use to check if a generator’s revision is affected by the bug:

    for (let i = 0; i < diffStuff.modelTextPatches.length; i++) {
  decodeURI(diffStuff._patchesToRevision(diffStuff.modelTextPatches, i));
  // when executed, the console tries to evaluate that so that's why it'll get an error if the revision's corrupted
  console.log("modelText: " + i);
  // count COUNT COUNT
}
for (let i = 0; i < diffStuff.outputTemplatePatches.length; i++) {
  decodeURI(diffStuff._patchesToRevision(diffStuff.outputTemplatePatches, i));
  // same with this too ^^
  console.log("outputTemp: " + i);
}
// you'd have to load the revisions in order for it to work, and for generators with lots of revisions, it'll probably take longer to finish the check

    Which in this case, I just ran on my milestone announcement page, and it returned this:

    (Also @VioneT@lemmy.world, @Alllo@lemmy.world, @eatham@aussie.zone and others, if you wanted to contribute by finding some buggy revisions in your generators by running that in the console and found errors like that, that’ll be very helpful.)

    Why not fork it and start with a fresh? That will reset your revisions.

    Seems like you have a good point. 🙂 I will try this with my milestone announcement page after my preparation with it is fully done.

    • allo@lemmy.worldEnglish
      3·
      6 months ago

      i basically have no idea any of this but will look at what all this is and do it tonight because you say it will be helpful (tonight here is like 5 hours from now)

    • VioneT@lemmy.worldMEnglish
      1·
      6 months ago

      Here is probably a smaller code to check, also it wouldn’t stop at the first malformed:

      let text;
let x = diffStuff.modelTextPatches; // Change to `outputTemplatePatches`
x.forEach((a,i) => {
  try {
    text = decodeURI(diffStuff._patchesToRevision(x, i));
    console.log('Checked: ', i);
    text = '';
  } catch (err) {
    console.error('Error: ', i, err);
  }
})
console.log('Check Complete');

      You can also just copy and paste it on the console directly. Just need to load revisions first.

      It seems the /hub has the last ‘101’ revisions on modelTextPatches to be malformed. The outputTemplatePatches didn’t throw any errors. Looking at the other pages that I have, mostly previous revisions are malformed. The date it started appearing (at least on my revisions on ai-generated-realistic-portraits) is Jan 8, 2024 about 10:38:43 AM UTC, but about 10:38:34 AM the revisions are still okay. On the hub it is also Jan 8, 2024 about 11:30:42 AM UTC.

      EDIT: It seems to be only generators that were updated on Jan 8 before 12:03:58 PM and after 10:38:34 AM seems to have problems on my end.

      • BluePower@sh.itjust.worksEnglish
        1·
        6 months ago

        That’s interesting. I didn’t know the hub also has the same revisions problem. I’ve been tinkering with that code for hours and this is what I made:

        function checkForMalformedRevs(patch, loglabel, startFrom) {
  let text;
  let dateRegex = [/^.* ([0-9][0-9]:[0-9][0-9]:[0-9][0-9]) GMT+.*/g, "$1"];
  console.log('[' + loglabel + ']', Date().replace(/ \(GMT+.*\)/g, ""), 'Check Started.');
  patch.forEach((a,i) => {
    if (i < startFrom) return;
    let pi = diffStuff._patchesToRevision(patch, i);
    let x = 0;
    let offset = 0;
    try {
      for (x = 0; x < pi.length; x += 100) {
        offset = (!!pi.substring(x, x + 100).match(/%.$/g) * 1) + (!!pi.substring(x, x + 100).match(/%$/g) * 2); // checks if there's any "chopped" URI characters at the end of the chunk to prevent `URI malformed` errors just because of that
        text = decodeURI(pi.substring(x, x + 100 + offset));
        text = '';
        x += offset;
        offset = 0;
      }
      console.log('[' + loglabel + ']', Date().replace(...dateRegex), 'Checked:', i);
      x = 0;
    } catch (err) {
      console.error('[' + loglabel + ']', Date().replace(...dateRegex), 'Error:', i, 'at chunk', x + '-' + (x + 100 + offset), 'of', pi.length, err);
    }
  })
  console.log('[' + loglabel + ']', Date().replace(/ \(GMT+.*\)/g, ""), 'Check Complete.');
}
checkForMalformedRevs(diffStuff.modelTextPatches, "modelTextPatches");
checkForMalformedRevs(diffStuff.outputTemplatePatches, "outputTemplatePatches");

        There we go! It’s a greatly enhanced version of the checker tool. To reduce all the wait, the checker first processes a chunk of data, then if that chunk of data has been checked successfully, then the next chunk will be checked. (But still, if the revision data has around tens of thousands of characters it’s going to take seconds to process)

        The process roughly looks like this:

        "generatorStats%20=%20%7Bimport:generator-stats-plugin%7D%0%0Atitle%0A%20%20Power%20Generator%20Manager%0A%0A%0A%0A\n\n"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ < checking

        The checker will take a part of the data and then decodeURI’s them. Then if it’s successful, another chunk is also checked:

        "generatorStats%20=%20%7Bimport:generator-stats-plugin%7D%0%0Atitle%0A%20%20Power%20Generator%20Manager%0A%0A%0A%0A\n\n"
                                                  ~~~~~~~~!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ < checking

        And there, once it’s URI malformed time, the checker stops for checking further chunks. Additionally, the logger not only throws the error, but also locates the chunk location where the error is throwed and shows when the check happened in time. You can also set on which index the checker should start checking with startFrom for easier debugging.

        Additional note: While testing, I also found that other “suspicious” (invalid) characters (including %9F) and the single percentage % are also affecting this whole revisions problem.

        (Also I might create a whole “utility” generator out of this so more people can contribute to this 😊)

        • VioneT@lemmy.worldMEnglish
          1·
          6 months ago

          On that note, I duplicated the hub and deleted the old one, it did resolve the revisions problem (though the view count reset).

            • VioneT@lemmy.worldMEnglish
              2·
              6 months ago
              function checkForMalformedRevs(patch, loglabel, startFrom) {
  let text;
  let errors = []
  let dateRegex = [/^.* ([0-9][0-9]:[0-9][0-9]:[0-9][0-9]) GMT+.*/g, "$1"];
  console.log('[' + loglabel + ']', Date().replace(/ \(GMT+.*\)/g, ""), 'Check Started.');
  patch.forEach((a,i) => {
    if (i < startFrom) return;
    let pi = diffStuff._patchesToRevision(patch, i);
    let x = 0;
    let offset = 0;
    try {
      for (x = 0; x < pi.length; x += 100) {
        offset = (!!pi.substring(x, x + 100).match(/%.$/g) * 1) + (!!pi.substring(x, x + 100).match(/%$/g) * 2); // checks if there's any "chopped" URI characters at the end of the chunk to prevent `URI malformed` errors just because of that
        text = pi.substring(x, x + 100 + offset)
        let decoded = decodeURI(text);
        decoded = '';
        text = '';
        x += offset;
        offset = 0;
      }
      console.log('[' + loglabel + ']', Date().replace(...dateRegex), 'Checked:', i);
      x = 0;
    } catch (err) {
      console.error('[' + loglabel + ']', Date().replace(...dateRegex), 'Error:', i, 'at chunk', x + '-' + (x + 100 + offset), 'of', pi.length, err, 'Chunk Text:', text);
      errors.push({date: new Date(diffStuff.patches[i].creationTime), log: loglabel, revisionId: i, chunkRange: `${x}-${x + 100 + offset}`, revisionLength: pi.length, text: text});
      
    }
  })
  console.log('[' + loglabel + ']', Date().replace(/ \(GMT+.*\)/g, ""), 'Check Complete.');
  console.table(errors, ['date', 'text']);
}
checkForMalformedRevs(diffStuff.modelTextPatches, 'modelTextPatches')
// checkForMalformedRevs(diffStuff.outputTemplatePatches, 'outputTemplatePatches')

              Here is a code that also outputs the Date of the Revision as well as the Text that was throwing the error neatly in a table. It seems ONLY the modelTextPatches are throwing the malformeds.