Using Computer Vision to Assess Building Damage from Satellite Imagery

By Raj Saha, PhD

Disclaimer: This site is a technical demonstration only. The analysis, data, and visualizations presented are for illustrative purposes and should not be used as a source of news or information.

Access to satellite imagery and building footprint data can be limited, particularly in geopolitically sensitive areas. This project examines one such conflict zone with before-after satellite imagery to identify damaged buildings and bomb craters. An analysis like this requires high resolution (0.5m) historical satellite imagery, which in this particular case is not readily available for download. However, there are viewable data sources like Google Earth Web with sufficient spatial and historical resolution, just that the imagery is not directly usable for analysis. So this project builds an array of tools to systematically gather the viewable imagery and post-process the imagery for geospatial analysis.

Map showing locations of damaged buildings based on manual annotations and model predictions. Also overlaid are locations of bomb craters. At high zoom, the satellite imagery is visible. All three layers can be toggled on and off in the legend.

Data Sources

2022 2023
2022
2023

Before and after satellite imagery comparison with overlaid building footprints.

Methodology

  1. Data Collection
    • Before and after images, from 2022 and 2023 respectively, were obtained from Google Earth's web interface with automated screenshots tiled over a specified region. Each image tile covered roughly 590m x 290m ~ 0.17 sq. km and had slight overlap with adjoining image tiles for image alignment and fine tuning.
    • Building footprint data from Open Street Maps was obtained through https://overpass-turbo.eu/.
  2. Data Processing
    • Image alignment: The image tiles were geo-referenced and aligned. Additional corrective adjustments were made to take into account the loss of precision due to the interface.
    • Extraction of individual building images: Overlaying the building footprint polygons on the imagery allowed for the extraction of individual images of buildings in both before and after sets. These image pairs, with the help of manual annotations, would later be used to train a machine learning model to identify damaged buildings.

This is a demo of the annotation tool over a single image tile. The drawing tool can be used to select multiple building polygons and toggle their damage status. The crater tool can be used to mark the position and size of bomb craters.

  1. Annotation tool
    • Given the importance of making correct positive identifications, it is important to have a tool that allows for efficient manual annotations of the imagery.
    • The web-based tool developed here randomly selects image tiles for annotation, has drawing tools to select and tag multiple building polygons, as well as to mark the locations and sizes of bomb craters to be used in a subsequent analysis to identify them in new imagery.
  2. Model Training
    • Roughly 50% of the image tiles were manually annotated to mark damaged buildings.
    • This dataset was split into training (70%), validation (15%), and test (15%) sets to develop a Siamese Twin Network comparing before/after satellite images.
    • The model achieved 0.79 recall on the test set (identifying 79% of damaged buildings).
  3. Predictions and Refinement
    • After the initial predictions, the annotation tool was used again to review and refine the prediction labels.
  4. Output and Visualization
    • The final set of annotated and predicted labels were put together in an interactive map.

Process Overview

Here is a schematic overview of the process and workflow.

Flowchart of end-to-end process flow. Drag sideways to view connections.

Uncertainties and Limitations

The dataset used in this analysis presents several constraints – first that any manual or model based predictions can only take into account how buildings and structures look from above. This means that the damage labels are likely to be undercounts.

Several image tiles, as obtained through Google Earth, were composites of imagery from different time periods as can be seen with seams through the images. This also results in potential undercounts.

The downward viewing angle and shadows present another challenge for both models and human annotators to correctly identify damaged structures.

Given as this project's scope is to mainly be a demo and a proof of concept, the model was not rigorously fine-tuned for robustness. With additional time and computational resources the model's performance could likely be significantly improved.