He continued: “I don’t think that the government ought to ever have the right to tell a church to marry people if the church doesn’t want to."
In this he's quite right.
Problem with that is, no one is pushing to force churches to do anything. That's virtually a straw man argument.
Because legal marriage in the US is a CIVIL function of the State, The religious add-ons the churches perform are window dressing, of an underlying CIVIL ceremony. Which a church can decline to perform, and a civil marriage performed by a judge or other official still has all the force of the law behind it.
That's the myth of the church wedding Christian fundamentalists try to promote. To claim legal marriage, and its definition, is their exclusive domain. And would change the law to make religious weddings the only legal ones, which by themselves they are not.
So what's being argued is civil gay marriages by the invidual States. Nothing to do with churches at all. Carter may indeed be revealing some mental decline.