That's a very good question. Years ago, I was working with several European teams; when the Clinton/Lawinski scandal broke, their reaction was "So what? You elected him to do a job - is he doing it? What does his private life matter?"
Maybe if we had a king/queen to worship, and a prime minister to get mad at?
But seriously, it seems to me that the conservative psychology is to want authority and a father-figure, and that the larger amount of "disillusionment" with presidents comes from the right, not the left; and that when a conservative president is elected, the urge from conservatives is not to question, but almost to worship... This will of course land me in endless trouble!
Frankly, if I look for a doctor, or a mechanic, I want one who knows his job, not one who wants to ban disease, or thinks Jesus doesn't like bad motors - and his private life is of no concern to me. Guess I'm weird...