I do a backup and full factory reset once every major OS version releases, but only then.
I have never had any issues, bugs nor anything similar that’s widely reported when new versions of iOS are released, and I believe that’s because upgrading always leave remnants of lingering trash. But that’s me.
To be completely honest, there’s barely any difference ever since the X. Incremental upgrades will get you a better camera, but the XR had such a great battery life and remains super snappy, so if you aren’t really a camera guy there won’t be a lot new besides ProMotion and Dynamic Island.