The college was funded by Chełmno voivode Jan Działyński. The building was erected in 1647–1725 on a rectangular floor plan, featuring two wings from the north. 18th century polychrome with illusionistic painting of a dome is seen on the vault in the entrance hall.
On the first floor in the western wing there is a refectory with a cavetto vault and lunettes, featuring stucco decoration dating from 1720–1730.
When the Jesuit Order was disbanded in 1781 the building was housing a middle school, and starting from 1816 – a teachers’ training college. In 1897, after the Town Hall in the Grudziądz Market Square was consumed by fire, the Municipal Office was moved here, operating in this place to this day. Starting from July 22, 1956, the Grudziądz bugle call composed by Cpt. Stanisław Szpulecki has been played in the Baroque tower.