I have conquered four territories in addition to the starting territory in the lower left but can only build in that one and the two immediately adjacent to it.