That front sight is usually stuck in there really tight. Some have reported it being glued it. Mine wasn't but you have to pound on it pretty hard with a brass punch to get it to move. If you have the adjustment in your Williams, I'd leave the front sight alone until I got my Smith sight so I only had to adjust it once.