Being only 21, it was only about 6 years ago that I received a Savage 110 in .243 for Christmas. It was one of those Rifle/scope/case combos from Dick's Sporting goods. I think it only cost about $350 or so, but I'm not sure.
In my limited experience deer hunting (I say limited because I cannot attest to possesing wisdom beyond my years) I've never felt uncomfortable taking a shot a NY whitetail...and that's because I've never taken a shot at one that I thought I couldn't hit well enough to kill.
If there was one thing that my father did right, when buying this rifle for me, it was that he gave it to me when I was 15, a year before I was old enough to hunt with it. He also took me shooting during that year and I practiced shooting from various positions. I used to head into our back field and try to demolish apples with that rifle at about 80 yards from a standing position.
Now I don't claim to be a great marksman, but I can say that the confidence one gets from knowing he/she can hit their target is much greater than the false confidence provided by a more powerful rifle. Know your limitations and take all your shots within those limits, and it really doesn't matter how big your gun is.