I bought my handi in 243 and should have never traded that barrel... 65gr v-max handloads and 100gr winchester pp ammo both shot 5/8" to 3/4". When I hear accuracy problems from others, I really regret trading it...
I currently own a 270 ultra comp barrel and get the same accuracy - all with handloads however (130/140gr). Kicks less than my 243 did with 100gr loads.
Now I know, you all will say, but it's loud. Yes it is. But, I NEVER pull the trigger without my ear plugs in... I put them it as soon game is spotted and have never missed an shot opportunity as a result (takes 3-5 seconds tops -send me a pm and I can explain how... ). So noise is a mute point. I use them for my non-ported guns too.
Anyway, you need to find the load that works for your gun.
Don't be afraid to try a 270, 25-06, or a 7mm-08. Half the fun is finding that gem of a load (or brand of ammo) that works for your particular rifle.
If I had to do it all over again, I would probably lean towards the 7mm-08. Lot's of great posts on this site.
Good luck,
Mike