The problem
GOES-16 sends images every 10 minutes in NetCDF (.nc), a heavy scientific format that browsers cannot open. The problem was distributing these satellite images on the internet quickly: turning them into a map that opens in seconds, for anyone.
Processing depends on NetCDF, GDAL and PROJ versions that must match. I packed the conda environment into an image with cron in the foreground, and products and logs live in host volumes.
Decisions and why
- Conda environment inside the image
- netcdf4, GDAL and PROJ pinned in environment.yml. It works the same on my machine and on Unicamp's server, and OS updates do not break the pipeline.
- cron in the foreground as the container process
- Docker restarts the container if it dies, and cron runs main.py at 03, 13, 23, 33, 43 and 53, three minutes after each scan is published.
- Host volumes
- Logs, products and dates live outside the container: updating the image or restarting deletes nothing, and Nginx reads the same folder.
- One container per responsibility
- Processing and the tile server are separate: either can be updated without taking down the other.
requestresponse
Pick a scenario above to watch the message travel through the architecture.
What it looks like
Real screenshot of the platform, click to open plataforma.labsat.cpa.unicamp.br

Camadas