Wrong. In Tennessee, the state constitution prohibits anyone from holding public office that "denies the being of God, or a future state of rewards and punishments". This doesn't just discriminate against atheists. Also many eastern religions are banned from holding public offices.
And it doesn't stop there. It also dicriminates against clergy even in the religions it would allow:
Whereas ministers of the Gospels are by their profession, dedicated to the care of souls, and ought not to be diverted from the great duties of their functions; therefore, no minister of the Gospels or priest of any denomination whatever shall be eligible to a seat in either House of the Legislature.
Thus, to hold office in Tennessee, you must be a Christian or Muslim and not be a member of the clergy. I would say I expect a constitutional challange at some point, but given America's religiophobic attitude towards non-Christians, it's so rare that they would even get nominated by their parties that no challange to the law could foreseeably occur.
However, were there ever to be a constiutional challange of this law, I have a good feeling it would be found unconstitutional under the precedent set forth in Torcaso v. Watkins.