Infrastructure building can mean one of two things, depending on context.
When a medium island is being colonized, an infrastructure building is any building that does not count towards the building limit. These are the inn, the commodities market, the bank, the estate agent, and the fort (or palace), plus one attraction. Any of these buildings can be built regardless of the number of non-infrastructure buildings, space permitting.
The other use of the term refers to the buildings whose deeds change hands whenever the governorship changes hands. This includes all the buildings listed above except the inn, but also includes all the bazaars.
The deedholder has no ability to change anything in the market. Most of the buildings can be painted and/or have furniture added, and the estate agent can be run like a normal shoppe.