On the coast they do, yes. And it is... well. I believe they did fight, but the Concord's forces were fairly strong, so eventually it just made more sense for the Empire to make trade deals rather than waste money and people trying to control them directly.