Major, major dilemma

[iYeow]

Correct, you don't need to install Rocky Racoon. If your iphone is tethered at 5.1.1, to go untethered you run latest redsn0w without ticking any option to go untethered. Install Ultrasn0w to unlock your unlockable baseband.
If you are now stuck at DFU or in recovery mode, you will need a restore to custom firmware of 5.1.1 either created by Snowbreeze or use Redsn0w 0.9.14b2 to create a N0_BB ipsw of 5.1.1. Use redsn0w to put iphone into pwned dfu mode prior to restore to this CFW.

[GottaLoveRice]

My restores dont work anymore... I restored to stock, which at first worked. Then I did what you said to try and restore a custom firmware. Now, when I restore, no more apple logo. Only a screen lit up. And I've been getting I think it's error 26 or 28.

[DC_Dave]

Hi, have you tried turning off any antivirus software or Firewall on your PC?
Apple has the following link on error 26 problem
iTunes: Troubleshooting security software issues

[DC_Dave]

Hi, did you try spraying your home button with some WD40 or Alchohol based electrical cleaner? Or are there any repair shops you could use to get a new home button fitted?

[GottaLoveRice]

No, I didn't. I don't have any. However, I once again jailbroke using ENTER_DFU ipsw to 5.1.1. Problem is, whenever I lock my phone and its unplugged, or turn it off, it still goes into DFU mode. All I want is for that to not happen...
I don't care about the home button because I have accessibility assistive touch. I just want to use the phone and being able to at least lock the screen..

[DC_Dave]

Hi, so you used ENTER_DFU with stock 5.1.1 ipsw and did a normal iTunes restore to 5.1.1 first, next you did ENTER_DFU again and a Redsnow jailbreak (only ticking install Cydia). But you still get stuck as if you are using a tethered jailbreak.
Might be worth reinstalling the latest Redsnow and iTunes. I remember reading on this forum, someone else was having major problems jailbreaking a 3GS with Redsnow and after pages of posts, he tried Snowbreeze and it jailbroke first time?! This might be worth giving a go, restore back to stock 5.1.1 first though.
Also...
I would download a fresh copy of iPhone 3GS 5.1.1 ipsw
By the way...
When you were on iOS5.1.1 restored with iTunes (unjaibroken), did your normal power off and back on work OK?