The truth? Gay men that focus on MARRIED men either have complex committment issues, low self esteem in general, or are transexed.
Emotional abandonment can create committment issues. The fear that you might be abandoned again can sometimes be so strong that you then fear getting to attached to anyone, ever! So a married man that can maintain a comfortable distance seems to fulfill a need of emotional distance. The resulting detached relationship creates a comfort zone, though, you will never be fulfilled. Also, emotional abandonment can take many different forms that can be very subtle, so if you notice that you fear committment, go easy on yourself, you are not alone
But, the good news is recognizing that fear of committment exists can put you on the road to "Loving like you've never been hurt and dancing like no one's watching"
Alternatively, if you ARE or are willing to be emotionally invested and want something bigger (in contrast to the issue above) then you have probably have low self esteem if you continue in a relationship or seek out a relationship with a married man. You simply don't think you deserve any more, and that is just not true. If you give all of yourself, then you can (and should) expect the same in return. If not, then you should move on with your life.
Lastly, if it is the fact that the man is sleeping with a woman that turns you on, you might just be a tranny. How about that?
You could also be a tranny with low self-esteem and committment issues. Those who repeatedly fetishize and or focus heavily on the "score" of a married man for that sake alone probably have a few of the issues above going on.
I realize that there may be someone out in the middle of nowhere whose only gay opportunity might be with a married man because "he is the only gay in the village," but I still think there are self-esteem issues if you accept less than what you think you deserve.