Ok. I may have mischaracterized the non-hunting landowners as bunny huggers.
But my point is not to say that we should eliminate landowner rights. However, a landowners rights stop where his property line ends and the next landowner's begins. So if one landowner wants lots of deer, I think it is only just that he pay for the damage those deer cause when this is a result of his own land use practices. The hunters are not the one's causing the deer damage.
The reason I think this might be a part of the solution is because it would potentially cause the sanctuary owners to harvest more deer or put more pressure on their property. This would result in the DNR reducing the quota for the entire management unit.
Say you have a management unit that is half public land and half private land. The DNR sets the population goal based on the land use. If the private land has a lot of agriculture and those farmers complain of a lot of damage, the DNR will set a goal accordingly. If the unit as a whole is over populated, the DNR will issue limitless tags.
What ends up happening is those tags get filled on the public land, and the deer in the sanctuaries survive. Farmers continue to complain, and the DNR issues another season's-worth of unlimited antlerless tags. And the deer on the public land go through another slaughter. This has continued for at least two decades until we are now in a situation where hunters hardly see any deer on public land.
If, on the other hand, the sanctuary owners faced the reality that giving safe-haven to all those crop pests could cost them money they may decide the safe-haven isn't worth it (Or they could decide it's worth it). If they can avoid paying money by harvesting more deer on their property they might. This will cause the population of the unit as a whole to go down. Once it meets the goal the DNR will reign back on the antlerless permits resulting in better survival on the public properties.
If the alternative is to let deer go unmanaged in agricultural areas, I think that the sanctuary owners, be they bunny-huggers or other hunters, would realize it is in their best interest to put some pressure on their properties rather than let the farmers, shoot, trap, and poison deer to extinction.