I had that trouble till I found out it was ME!

The Handi likes to have the trigger fully engaged, (squeezed), to work properly. Something about the transfer bar dropping. They are a little different than many other style trigger's.
Next time out shooting, concentrate on a full-slow trigger squeeze, and hold the trigger back all the way. I always know I'm doing the proper trigger thing when I hear "bang", then the "klatch" of the trigger being released after.