They've done something pretty similar in a research study before. Took a guy, make-uped him to look like a Neanderthal (for which the Geico cavemen are pretty good approximations of), but then just dressed him in a wind breaker and jeans. Everyday clothes. He was able to walk all around at least a few cities without anyone paying any mind.
Kinda makes me wonder. Just for the sake of matching I will do little things, like never wear a black belt with brown shoes (or vice versa), but if they don't notice a caveman, it certainly would seem that almost no one would notice even more minor things that were "off" about a person.