Regarding peeps, many make the mistake of trying to use too small an aperture thinking it will tighten up the groups; not so!
I usually wear bifocals, and have a couple of older outdated sgl. vision glasses that work pretty well for most shooting (try them all to 'see'). I had to relearn keeping both eyes open and my groups are better (Ive tested this with side by side targets; both eyes open then closing the left). I do this with open, peep, or scoped guns.
I have shot a lot of target matches and hardly anyone uses bead sights, and when they do it is for certain specific situations. It can get difficult to define the very top of a blade, so we carry a little tube of 'white out'(remember typewriters?) and a black MagicMarker to have 'tune-up' options.
What almost all target shooters agree on is that you focus on the front sight and let the rear and the target blur. I tend to re-focus between the front and rear to get my initial sight picture, then focus on the front and its placement as regards the somewhat blurred target, and let fly. When the magic works, I still often place or (occasionally) win.
I also have scopes on some, and with my eyes do appreciate them! But I keep at shooting the old open sights on some sporting rifles and peeps on several vintage target rifles just to keep the discipline up, and it helps at lot. I surprise myself with some challenging hits and it is very satisfying (BIG surprise!! hits are more fun than misses....

).
Specifically to your elevation variations vs windage: Is this off the bench? It is so easy to not get the buttstock quite into the same position each time, or your body leaned in the same, and it changes the recoil impulse enough to do what you say. Some days are worse than others......Try it again, paying more attention to your positioning and see what you get. Also work on keeping that front sight top right in line with the open rear top, centered in the notch, then line it up with a bullseye sitting right on top of the blade (I cannot hold center on a bullseye and get tight groups compared to using a 6:00 hold). If you are using a peep (I know, they say ignore it) do make an initial reference of the front being somewhat in the middle of that rear hole, then settle in and go.
Hope some of this helps!