Why cant you download a 4.1 ipsw, go into iTunes, right-click your device and hold alt while clicking restore backup?
Do most people opt out of this option because its a complete factory reset? Personally I don't care, I can get my contacts restored and re-download everything. I just want to know if its a feasible option because I can't stand not having a jailbroken phone.
To make it quick: SHSH blobs prevent users from doing just that.
Secure/Signature HaSHes are provided by Apple per-device and per-firmware and are required for any restore of firmware. They are unique and can only be obtained from Apple (even Cydia has to originally get the blobs from Apple).
Apple only gives out the latest firmware's SHSH blobs, thereby preventing downgrading firmware versions for anybody who didn't save the blobs with Cydia/TinyUmbrella in time.
No blobs, no downgrade (at the moment). It's that simple.