If you walk into a bar and someone you have never met provokes you by making false accusations and trying to shit stir then you are fully entitled IMO to say Look, lets go outside and settle this like men.
Let's look at the facts, the vicar has made a string of allegations with absolutely no fact or evidence. He has pushed and pushed until het got a response he doesn't like.
Perhaps now would be the time for the vicar to admit his.mistakes. If he does that then I will admit to mine.