This is a tricky job!!
This can certainly be done from under the truck without removing the exhaust.
I didn't even jack the truck up, just layed on my back under it. But i suggest putting the truck on jacks if you are interested in any sort of comfort.
First, Remove the battery or disconnect it.
Next, You'll want to unbolt the fly wheel cover, if you have transmission cooler lines in the area, it will not be able to completely be out of the way but will fall down about 2 inches and is enough.
You will need to put a jack under the oil pan (use a piece of wood as not to pierce the pan) and jack the engine up slightly. This will give you a little more wiggle room.
(The outer most bolt (of 2) that holds the starter in would not fully unscrew without hitting part of the frame for the upper control arm. This is why I suggest jacking the engine, but nothing too crazy.)
Now you can take out the 2 bolts that hold the starter in. Once the bolts are out, you will need to slide the starter towards the front of the vehicle and angle it slightly upwards. It should now be at an angle where you will be able to slide the starter down towards the rear of the vehicle and out through the whole between the manifold and the block. Now once it is out of its hole, you can take a wrench and disconnect the wires from the starter.
Putting a new starter back in is quite tricky as well. First, connect the wires to the starter. next, Take note of how you were able to slide the starter out from behind the manifold when you are removing it or else you will be there trying to put it back in like its a puzzle piece that just does not want to fit. Trust me...
Once its back in place and bolted up, bolt the fly wheel cover back on.
Lastly, connect up the battery then try firing it up , fingers crossed

haha
It is quite the job and does not help if you are impatient, but all in all, from start to finish, it took me 45 minutes and a handful of swear words.
Good Luck and happy wrenching
--OnlySonoma