I have been searching for such a Cartridge / Caliber for years. Like Ernie says it probably aint gonan happen with ONE BARREL / ONE CARTRIDGE, and anything that would work well for one animal would be a compamise for the other.
With that said the closest I have came is with the 6mm T/CU in a Contender, and no it wasn't perfect. I used 55gr. Nosler Ballistic Tips for Varmints & Coyotes at 3000 FPS. For Varmints this bullet was excellent and super explosive. I shot 2 Coyotes one morning at the same stand with this Gun-n-Load. The first I hit a little too low and too far back and it really messed things up (my fault not the gun not cartridge nor bullet's fault). As the first Coyote was yipping it brought back the 2nd Coyote. When my barking finally stopped the 2nd Coyote I was ready and put the 55gr. Nosler Ballistic Tip where it needed to go, right behind the shoulder. There was a tiny entrance hole but no exit hole. The distance was about 200 yards. I then turned my attention to the wounded Coyote that was just standing out about 100 yards all hunched up. This time I hit where I was supposed to and like the other Coyote there was a tiny entrance but no exit from this last shot and the Coyote dropped in it's tracks.
For Whitetail Deer I have always used the 80gr. Speer Hot Core Spitzer in the 6mm T/CU. This bullet is a little tough for the velocities possible with this cartridge when fired in a Handgun, but every single Whitetail I have ever shot with it resulted in a one shot kill (even when I had to penetrate bone). I have always been able to get good accuracy out of this bullet as well, and have shot Whitetails at distances greater than I should have with this bullet and carttridge with good results.
On a different note I used the 80gr. Speer Hot Core Spitzer in a 15" T/C Encore chambered in .243 Winchester this past season. I took 2 Whitetail Does New Years Day with this bullet and cartridge. One was a one shot kill with perfect bullet performance at 200 yards. The other one I messed up the shot, hit too far back and had to use a second shot to finish off the Deer.
Yes I am a fan of 6mm Cartridges. I have also had good luck with 6mm Cartridges over the years. However they are not for everyone and require extra care in choosing the right bullet and shot placement.
Good luck
Larry