Theoretically, a 1 11/16" hole would just take a golf ball, but you would have to clean after every shot and might have a problem with hot golf balls, which I have heard expand from the heat a significant amount. I suppose you could keep them in an ice chest until they were loaded.
I use a 1 23/32" bit (1.719") bit which allows me to use 1 lb spherical lead sinkers also.
On the other hand, you could drill 1.6875" and bore the extra 1/32. You should learn how to bore.