How would they ever find you anyway? Ignore them! They will go away.
Using the everyday phone can give a huge amount of info depending on whether he has linked it to social media or if self employed has put it on a website etc. Maybe his voicemail message gives away where he works.
If I google my work phone number, my name and details appear No1 on Google.
If the pro$$ie B&S she is probably used to walkers, and probably forget about you in a day or so, but if this is just a personal use phone, I would ditch the sim immediately and try and forget about it.