As far as the title
of your post goes, I think that most of us here would agree that it's not a moral problem to have an fb. YMMV on other boards!
In terms of the fb being married/partnered... It seems to me that your question should be answered by your own beliefs/morality, not ours!
My guess is that your fb wants to be responsible for his own marriage, and would not appreciate your concern in that area. However, you know that he is married and, almost certainly, he is breaking a promise when he has sex with you. How much, if anything, does that mean to you?(For the record, I've had married/partnered fbs... I'm not trying to be preachy.)