I'm with beemanbeme, get a new gunsmith. Had he checked the squareness ot the crown, I'd buy that but not adding Mag-na-port. Try tighting the guard screws then loosing the front screw slowly while watching the barrel at the front of the stock; it should not move at all. If it does there's abedding problem. Next try running a dollar bill between the barrel and the stock with the stock snugged down. Your feeling for high spots or tight spots in the barrel channel. You might have up pressure built into the fore end, put the dollar bill behind that first when you reinstall the barrel/action. Then it might be bedded solid full length, I'm not sure any are any more, but maybe. If thats the case, try sanding out the barrel channel to create a gap between the barrel and the barrel channel in the stock. I like to bed solid for about 2" in front of the action but my mof 70 is free floated full length and shoots great. If you need to take out some barrel channel, first stop about 2" short of the action, easier to leave it than to replace it later and you can always remove it later. Also make sure that where you stop is straight across the stock and not at an angle. If it's solid but at an angle, the barrel will bounce off the long end and cause acccuracy problems.
It might be the action and barrel are out of square but unless your a machinest, you'll have to have a gunsmith fix it, a good gunsmith. I doubt it's that far out that it would make that much difference. It's also possible to have a bad chamber but again not likely that bad. My guess is bedding. It's the easiest to correct and that's where I would start.
Now one more question. I assume you have other rifles, how well do you shoot them?