You know, I can't speak for anyone else, but after the first time, I don't really notice anymore. Naked Men's Yoga is not a sensual activity so getting aroused doesn't happen. And there aren't any women present - although, the local place in SF now offers Naked Women's Yoga.
And when you're doing downward facing dog (for example), if I were clothed, my shirt would have slid down my back and half over my head. That seems pointless if not uncomfortable. Therefore, it seems more natural to be sans clothing.
Honestly, you should try it. Or try it with clothes on; I've done that, too.