Even before the wars of religion the city was Protestant and the Catholic clergy had only a minor role to play The bishop, the abbot of Montauriol, had seen his abbey destroyed in 1563. When he was not expelled by the Protestants, he lived in a town house. Anne de Murviel, bishop from 1598 to 1652, was not combative and had to bear many humiliations until the Peace of Alès in 1626.

Pierre de Bertier, coadjutor from 1636 to 1652, took advantage of the policy of systematic growth of the Catholic (the arrival of religious orders and other office holders which did justice to his importance). In 1664 he had Campmartin, an architect from Toulouse, built a building that was both a mansion and a castle. The wing located along the bridge rests on the remains of a castle built in the time of the English (1360-1369).