Development takes place using the map5topo GitHub repo. For now this is a single repository that contains all that is needed to prepare the data, generate the map (tiles) and use various tools to inspect and apps to view (map viewers). Website and documentation is in the map5topo.nl GitHub repo
There are three main servers in the development chain:
- topo.map5.nl the DEV server
- test.map5.nl the TEST server
- map5.nl the PROD server
The latter two are identical, only the final tiling (in GeoPackages) is done on TEST and then copied to PROD. But you can run everything locally!
Below are guidelines how to develop using the map5topo GitHub repository.
0. Startup (once)
- the repo is currently private, you need to be added with proper rights
git clone https://github.com/map5nl/map5topo.git
1. Keeping current
Before changing any file be sure to do a
git pull to become current with the
latest version of the repo.
./refresh-git.sh if you want to wipe any local changes.
Warning: your changes are lost forever!.
Instead you may do
git stash to keep your changes locally or use branching.
2. Issues and Milestones
GitHub Issues and Milestones are used track progress and releases.
Working with Issues
Issues can be opened by anyone. They usually represent a new feature or a bug. Through Labels, Issues can be tagged with a Priority and other characteristics like Bug, Enhancement, Documentation. See map5topo issues.
When changing and commit/pushing code to the repo, you are strongly encouraged to add the Issue number in the Commit message. That way we know that an issue is worked on and that your changes are related to that issue number. The issue number is prefixed with a 'hash' symbol. For example:
git add tools/mapnik/styles-map5/mystyle.xml
git commit -m "#128 changing zoom levels"
For trivial changes or quickfixes there may not be an issue number, so not required.
Working with Milestones
A Milestone is basically a list of Issues with a name, state and description. In the GitHub UI you can add or remove an Issue to/from a Milestone. A Milestone corresponds to a Release. Milestones, thus Releases, are named YYYY-MM, for example 2023-05. See our Milestones here: https://github.com/map5nl/map5topo/milestones.
This way we know "what went into a Release" which greatly helps making release notes. If an Issue is not finished, it can be moved to a next Milestone, thus Release.
3. GitOps Automation
The "DEV" server is topo.map5.nl. This server always has the latest version of the GitHub repo
Whenever something is changed in the repo, that code is refreshed (via a GitHub Workflow). Further Workflow actions depend on
the directory in which the change was made.
Whenever something is committed/pushed, a GitHub Workflow is started (from within GitHub) that applies to the directory where a change was made. If multiple directories are changed, multiple Workflows are started.
In general for
services, the (
docker-compose) service will be restarted. If a new version is configured, the Docker
Image is downloaded first.
For changes under
tools the following applies:
tools/mapnik: the MapProxy service will automatically reload the Mapnik config (Layers/Styles)
tools/etl: no effect, any ETL needs to be run manually on the server, as these may be long-duration processes
tools/dem: no effect, as these may be long-duration processes
tools/qgis: no effect, basically the QGIS Inspector project connects to the
service/featureservvia OGC Feature REST API
Several tools and apps are available to inspect data and maps.
- https://topo.map5.nl/app/ several apps
- the main inspection app is https://topo.map5.nl/app/sidebyside to compare among versions
map5topo - DEV - DIRECTis a Layer that renders tiles from the DEV server without caching, so the Mapnik Styles are directly applied.
- this way you may compare your changes with TEST and PROD
map5topo - DEV - DIRECTrenders tiles 256x256 directly, without metatiling and gutter. Reason is performance. This impacts label placement!
pg_featureservis run to provide ALL PostGIS data schemas plus tables via OGC API Features endpoint
- in QGIS we connect to that endpoint
- QGIS: start the project from
- PostGIS: use PGAdminIV webapp to inpect PostGIS data: https://topo.map5.nl/pgadmin
You are encouraged to write comments and READMEs, but the main documentation is has its own repo: the map5topo.nl GitHub repo using mkdocs to automatically rebuild the website map5topo.nl on commit/push.
Docs are in Markdown.
Created 2024-02-09, Updated 2024-02-09
Authors: justb4 (1)