If you are not trying to have period implements, you should consider a safety rammer which is shaped like a cane. You insert the long leg into the bore and hold the short leg. If you have a premature discharge, (a) your hand is not in front of the muzzle and the fire coming out of it, and (b) the rammer leaves your hand nearly immediately with no head or other enlargements to tear your fingers off.
If you need a period rammer, I would consider using steel tubing for the shaft with a soft or collapsible head. In case of premature discharge, the head would break up into small pieces and/or blow down the middle of the tube, the tube would not pick up much velocity because it presents a small cross-section to the expanding gas, and most of the gas would be vented down the middle of the tube. You may get some complaints that the steel shaft could cause sparks but I think that unlikely.