Your fort is the safe, reclaimed portion of a city that you've walled off to keep out the zombies. Expanding your fort is the main way to gain more housing space and farm land to house more survivors. But defending a larger fort is also harder, especially against raiders who can steal resources from any block in your fort.
You may only reclaim buildings adjacent to your fort, but it is possible to have independent sections of fort not adjacent to each other (by losing a building to zombies or gaining a independent building through events). Independent forts function no different in all regards.
Survivors will return to the nearest building in your fort when they finish a mission. The early goal of the game is to expand your fort until you've made a large portion of the city safe again. (Exact number required depends on map size.)
Other factions may also have forts in the city that will grow as yours grows.(Except for the Rotten, who doesn't normally expand, and Gustav, who doesn't have a fort.) If the player successfully attacks an enemy building adjacent to your own fort, the building will be merged into your own fort. If not adjacent, the building will simply be lost to the zombies. If an enemy soldier unit successfully attacks your fort, it is always lost to the zombies, whether it be adjacent to enemy fort or not.