i hate that you weren't able to get a deer this season.

i'm not an expert on 30 cal handi's by any means, but you said that you clean religiously...and you've mentioned scrubbing the bore many, many times. maybe you're cleaning too much? like you said, you need fouling shots for the barrel to "set" and start grouping consistently, but maybe your rifle requires more fouling shots than you think.
like i said, i'm unfamiliar with .308 handi's, but i do own an ultra varmint chambered in .223. it's darn accurate; i've shot 0.6" @ 100 yds and i know it's capable of better. i definitely don't clean it after every range session; i generally only clean it after i've shot 80-100 rounds. i do, however, run a lightly oiled patch through it every time i use it to protect the bore from rust.
also, i've never had to rezero it between range sessions. if you're curious, i use a swift premier 4-12x40 scope with cheap aluminum rings (leupold, i think...$15 from walmart), and i never removed and remounted the weaver rail. i also shoot off a bipod, and i've never had any issue with it flexing my (synthetic) stock and barrel.
hope you have a bit more luck with your handi in the future. i've been thinking about buying one in .308 or 30-06 for some time now because i love my g-hog rifle so much.