I can imagine doing it, or some sort of little tribal arrangement. Not the coercive old-guy-gets-all-the girls-kick-out-all-the-boys sort of thing that gives polygamy a bad name.
But finding one partner is hard enough. It might be that finding two is exponentially harder. Or it might be that there is some sort of energy barrier where four or five is actually more stable than two? Dunno.
I know a couple of guys my age that came out of the old beat/hippie free love colonies. They have just as many hang-ups as the rest of us.