ETL Espacial (Spatial Data ETL)

Container: Python 3.10

Processes gridded climate data from global datasets and uploads the results for spatial visualization.

Calculate Monthly data

Aggregates daily gridded data into monthly spatial values for each climate variable.

Responsibilities

• Aggregate daily gridded data into monthly values
• Produce monthly spatial layers for each variable

Key functionalities detailed:

1. Monthly aggregation — Groups daily data into monthly values.
2. Spatial layers — Generates monthly rasters ready for use.

Calculate Indicators

Derives spatial indicators from the gridded data, such as dry spell durations and heat indices.

Responsibilities

• Apply formulas to calculate climate indices
• Produce indicator rasters for the geographic area

Key functionalities detailed:

1. Spatial indicators — Calculates drought and heat stress indices.
2. Raster outputs — Generates indicator layers.

Validation data

Verifies the integrity of spatial data before processing, identifying missing values and filling them using spatial climatology.

Responsibilities

• Check data completeness and consistency
• Fill missing values with climatology

Key functionalities detailed:

1. Integrity checks — Detects missing or inconsistent values.
2. Imputation — Fills missing values.

Connector AgERA5 (connector)

Downloads gridded meteorological data from the AgERA5 dataset at the required spatial resolution.

Responsibilities

• Download AgERA5 data
• Prepare data for processing

Cut data

Clips the global gridded data to the geographic region of interest.

Responsibilities

• Clip data to the target region
• Reduce the volume of data to process

Connector CHIRPS (connector)

Downloads precipitation data from the CHIRPS dataset for indicator calculations.

Responsibilities

• Download CHIRPS precipitation data
• Prepare rasters for indicators

Calculate Climatology

Computes spatial climatological averages per month from historical gridded data.

Responsibilities

• Calculate monthly climatological averages
• Produce reference values for analysis

Key functionalities detailed:

1. Spatial climatology — Calculates long-term monthly averages.

Main script

Coordinates the spatial ETL workflow: download, clipping, validation, indicators, climatology, and upload to GeoServer.

Responsibilities

• Orchestrate the full ETL sequence
• Manage uploads to GeoServer

Key functionalities detailed:

1. Pipeline orchestration — Runs all stages in sequence.
2. Upload coordination — Sends processed rasters to GeoServer.