I don't know much about the Kimber Varmint rifle, or your shooting/cleaning procedures but I think Don's question is right on. What do you think, and how well can you shoot?
I have many rifles, none of which (to my knowledge) shoots everything well, so your variances with different ammo is to be expected, and your varied results seem pretty typical. With factory ammo, you seem to be doing pretty darn well. Factory ammo + factory rifle, 3 shot groups sub .5moa, then a "flier" opens the groups to 5/8", or 3/4", or 7/8"?
Are you letting the barrel cool adequately? Seems that your first 3 shots are nice and tight and then things start to open up. What kind of rest are you using? Wind?
Just my opinion - but from what you've related, I think your results are very acceptable. You haven't shot that rig a whole lot yet, and I think as you and the bore "settle in", and identify the best ammo (there's a lot more ammo to try also), maybe start handloading, you are likely to see better consistency and improved results. So far - so good, I think.