The button is to keep his necktie from slipping out in the back. Some shirts still have it.
In the past people didn't have closets with rods and hangers, all clothes were stored on pegs or hooks and shirts were hung on the pegs or hooks by the loop. There is one reference to it being used to hold the shirt close to jacket when the jacket had a button in the center back. The loop would go over the button to keep the shirt from creeping up over the shoulders. Today it's just done out of tradition and decoration.