In a single-shot where rounds are fed one-at-a-time, cartridge shape is immaterial!
Go for MAXIMUM performance and versatility, whether or not you reload, or do not..
Also, I have to disagree with Stimpy (whom I respect). By having an '06 in the single-stuffer you only need to stock one flavor of ammo - which I find easier "logistically" in many ways.
Shooting the '06 offers far more advantages I think - and I own a .308 Handi. Although there are good arguments for both.
Were you buying a tactical bolt gun or a quick handling hunting repeater, the .308 often shines over the 30/06 (not always).
Cartridge length and compactness is moot in a single-shot, IMO.