Timing has a lot to do with it. If I see the deer far enough away to cock it without the buck hearing it, single action. But if it's up close and personal before I see it, I go ahead and fire double action to not risk spooking it.
Now that said, I usually hunt with a Bisley SA these days, but my Anaconda and 57 get mad if I don't take them out sometimes.