Any Spanish/Italian style buildings, Mexican Adobes, or more Northern European half-timbered styles would do. Just get a load of different architectural styles and mix n match them.
A Southern USA style 18thC plantation house would be ideal as an Island Governor's mansion etc. Even wooden ACW styles would fit with no problems.
Any Medieval fortifications would suit for town defences.
Basically, unless you're looking for a hard-core historical look, just go with what you like. Its Pirates, after all, so the more "Hollwood" the better, at least as far as I'm concerned
