I like the 260 Remington as a first medium game gun for a young hunter. It's available in the "compact" and "youth" sizes from several companies, as well as full size models. If you reload, this cartridge is even better, but there's enough factory stuff to handle most needs. If you or a friend reload, you might start out with the 100 grain Nosler partitions, which will work fine on deer and kick just like a 243. Unlike the 243, the 260 can handle bullets up to 160 grains, so as your son grows, he can switch to harder-hitting loads for bigger game.
My dad also started me hunting deer when I was 12, and I started with my dad's 308, the first gun I ever shot. I was taller than my dad, though, and weighed around 175, so he never really spent any time looking for a smaller gun for me. At any rate, I'll be starting my girl and her little brother with the 260 when they get old enough to pack their own rifles. Have fun!