Just a thought HC, but it might be an idea to go on the attack now? Along the lines of " I'm really hurt that you thought I've been sleeping around" etc., Saying nothing might just appear suspicious and if you reinforce your hurt feeling now then that should give you a free pass that you haven't and won't be up to anything!