For me the issue of sex came about because he didn't do anything to help out not even a little bit, so by the time we were child free I just wanted to go to bed. Once this became a pattern I started to see sex as another thing he wanted me to do for him, not a mutual thing where we were both horny and up for it. And as Dani rightly says the baby/toddler strange is god awful and draining.
I stopped finding him sexually attractive due to the underlying little digs about our sex life, and when I knew he was at it with someone else I decided to too. Aafter the first one night stand I felt alive and not just some skivvy housewife covered in baby snot. He changed after we had kids, well his attitude to me did.
This is a common problem between my friends, most of them married guys in the forces so I know if that brings along different issues. We all married and had kids young, moved away from our families and found it hard to find work. Would this have happened in another relationship, I don't know. All I know is that my friend (who is not with a forces guy) was in tears the other day over the same issue (tired, depressed, at home with a small baby all day and constant rows).