Buy the one you have always wanted... I, 90% of the time, load my 30 caliber rounds with some type of 180 grainer, and they both handle the 180 grainer pretty good. the 30-06 can push them a little faster, maybe 100 fps faster. the only time I load 165 grainers is when I use the Barnes TSX, those bullets, the faster you push them, the better they perform, they were designed to be pushed at super velocities of the big magnums.
a long time ago, I had a choice on my first rifle, and it was a 30-06, I used to own a 308, but after seeing how close they both performed, I decided to keep only 1, so I kept the one that has about 5% more power, '06