i can't say for sure but i believe that a .308 would need a lighter bullet than you're using if the deer are relatively small.    in other words, a 150 grain bullet is marginal,  i'd guess,  for opening-up on a small whitetail like i'd expect to see in MOST places so far south as georgia.   i think in a .30 caliber you ought to go with a 130 grain bullet to get more velocity and to open up the bullet better on a smaller critter.
the 180 grain bullet, in my humble opinion, is NEVER a good bullet on a whitetail or mulie unless it's the only option for miles around.   i wouldn't use a 180 gr' projectile out of a .30 caliber rifle on the large deer we have here in ohio, nor up there in the area where milo hansen took his monster.   it's relatively slow, relatively heavy-jacketed, and is too much bullet for anything less than a very large black bear.   it's not really needed on a large hog, in my opinion.    naturally if you're using a .30 caliber MEGA-MAG' like the "gee-whiz" group uses out of some of those bambi-blasters (that are stronger than a .300 Win' Mag') then maybe the velocity would open up the bullet on a chihuahua !   but i wouldn't go that route either !    
i think you're choosing the wrong projectile, and can easily resolve that problem.
good hunting to you,
ss'