I actually have an Ashley Madison account from a while back when I signed up to it to see what it was all about. I didn't get anywhere with it, but I still have the account and get some spam emails from them. I'm not married or even in a relationship, was just looking for NSA hookups.
Anyway, that means my details were probably in that list of details that got leaked, but it's not really an issue for me as I don't think anyone's going to go searching for it. The same is pretty much true of AW. If there were a big leak, I doubt anyone who knows me would think to see if I've got an account.
Remember that there were millions of accounts, so it's not like your name will be splashed across the front page (unless you are someone famous perhaps). It will most likely be lost in the sea of names unless someone might decide to search for you - whether or not that's a possibility is for you to decide.