Custom map style (2) - advanced modifications

This page describes modifications to the OpenStreetmap style that are more involved than those from the previous page. The old standard OpenStreetmap style in XML format will remain out starting point, and the OpenStreetmap database will still be the main data source.

Changing the map projection

The projection in which the map is rendered is specified in the srs attribute of the Map tag in the XML mapnik style file. In the standard OpenStreetmap style, it is located at the start of the top-level XML style file, osm.xml. Mapnik supports changing the projection of the map to be rendered independently of the projections of data sources. So modifying the srs attribute of the Map tag gets you a style suitable for rendering a map in a different projection. There is one caveat: Mapnik expects the map region to render in the target coordinate system. So your rendering program has to use the projection from the style file to convert the map region from latitude/longitude or whatever coordinates it is specified in. My example rendering program supports that.

Why would you want to change the map projection? Both Wikipedia and the proj manual agree that the Mercator projection is not well suited for world maps, and the pseudo-Mercator default projection of OpenStreetmap is a close relative. The region the Mercator projection is best suited for is near the equator. If you render a map of a small to medium-size country somewhere else, you can find a projection with much less distortion. Many projections have parameters that determine where they are most accurate. The the proj manual contains a list of those supported by the proj library, which Mapnik uses.

I have generated a comparison of three renders of Scotland and northern England with different projections. Start here and cycle through them to see the differences. The first is the Mercator projection, the second the transverse Mercator centred on the map region, and the third the Lambert cylindrical equal area projection. The first two are conformal, i.e. angle-preserving. The Mercator projection shows significant distortion so far from the equator, while the transverse Mercator is much more accurate for being centred on the map. It would also have been possible to use the oblique Mercator projection with parameter +alpha=90 to get an equivalent of the ordinary (non-transverse) Mercator projection centred on the map:

<Map srs="+proj=omerc +lat_0=56.351 +lonc=-4 +alpha=90">

But the distortion near the centre point is so small that the result does not differ visibly from the transverse Mercator map.

Hill shading and contour lines

Obtaining elevation data

The data in the OpenStreetmap data base are purely two-dimensional and contain no topographic information. In order to render topographic maps, additional data sources have to be made available to Mapnik.

Elevation data, i.e. data giving the height above sea level of different locations, is available from NASA, who determined it in a space shuttle mission using synthetic aperture radar. The data are available in the form of georeferenced pixmaps, with a pixel value representing the elevation of that point. (This is somewhat ironic considering that after a lot of processing, we will end up with a pixel image of a map again.)

This page in the OpenStreetmap wiki lists the different available data sets. I have always used data from the Shuttle Radar Topography Mission (SRTM), which seems to be the only data set freely available for non-commercial use. It is available in two resolutions, 3 arc seconds (about 90 metres) and one arc second. The coverage differs between download locations: The US geosurvey server provides 3 arc second data up to 60 degrees latitude and 1 arc second data for the US. Viewfinderpanoramas.org provides 3 arc second data worldwide and 1 arc second data for some European regions. Some other data sets linked from the OpenStreetmap wiki seem to be for sale.

The SRTM data files are in a custom but simple "HGT" format. As documented here, the files contain big-endian signed 16-bit values in a square array in row-major order. The file names contain the centre coordinates of the lower left corner pixel, and there are no metadata in the file contents. The size of each tile is 1201 by 1201 values for SRTM3 data, and the edge rows and columns are duplicated between adjacent tiles. The elevation values are in metres relative to the WGS84 ellipsoid, and the grid constant is 3 arc-seconds in latitude and longitude (approximately 90 metres).

No worries — you will not have to deal with the format yourself. The GDAL toolkit for processing georeferenced raster data also supports SRTM .hgt data. gdalinfo lists properties of such files and can extract information such as the minimum or maximum value (here: elevation). All GDAL tools are well documented in manual pages, though their command-line format is not very consistent.

Preprocessing

Mapnik cannot use elevation pixmaps directly. They have to be converted to contour lines or shading images. Both can be done with the GDAL toolkit.

Because the GDAL programs generating contours and shading accept only one input file, the SRTM tiles have first to be combined to a single elevation file. This can be done in two ways: gdalwarp can be used to create a single elevation pixmap, and can also change the projection while doing so. The other option is gdalbuildvrt, which creates a virtual data set by listing all tiles in an XML file. The latter is much faster because only the metadata of the tiles have to be read. If you have all SRTM tiles you need collected in one directory, you can simply run:

gdalbuildvrt output.vrt *.hgt

Other GDAL programs will treat output.vrt as though it were a large georeferenced image (here, an elevation pixmap) containing all the tiles.

Contour lines

The GDAL program creating contours from an elevation pixmap is gdal_contour. Depending on if you want contour lines annotated with the elevation, you can run one of the following commands:

gdal_contour -i 50 output.vrt contours.shp
gdal_contour -i 50 -a elev output.vrt contours.shp

The parameter of the -i option is the elevation increment between successive contour lines in the same unit as the elevation data (metres in the case of the SRTM data). The -a option adds the elevation as an attribute with the given name (here elev) to the .dbf file, where Mapnik can access it.

If you do not want equidistant contour lines, you can use the option -fl instead of -i. It is followed by the elevation of one of the contours to generate and may be used multiple times. For example, the following command generates contours at 100, 200 and 500 metres:

gdal_contour -fl 100 -fl 200 -fl 500  output.vrt contours.shp

Either -i or -fl has to be present, there is no default. Strictly speaking one should add the option -snodata -32768 when processing SRTM data, which makes gdal_contour ignore the designated "void" value of those data indicating that no elevation value exists for that point. Separately, I have seen it recommended to generate a shape file index with the shapeindex program that comes with Mapnik. This is supposed to speed up rendering regions smaller than the shape file such as tiles; as I have only used contour lines with large map images, I cannot say how effective it is.

In order to make Mapnik render the contour lines, you have to reference contours.shp as an additional data source and define the rendering style. For example:

<Style name="contours">
    <Rule>
        <LineSymbolizer stroke="#d0a030" stroke-width="0.7" />
    </Rule>
</Style>
<Layer name="contours" status="on" srs="+proj=longlat +datum=WGS84 +no_defs">
    <StyleName>contours</StyleName>
    <Datasource>
        <Parameter name="type">shape</Parameter> 
        <Parameter name="file">contours.shp</Parameter>
    </Datasource>
</Layer>

The projection in the srs attribute of the Layer tag is given as latitude/longitude &emdash; gdal_contour has kept the coordinate system from the SRTM raster files. The same projection could also have been specified as srs="+init=epsg:4326", which uses the appropriate EPSG index (listed in /usr/share/proj/epsg, which is installed with proj).

If you have used the -a elev option with gdal_contour and want Mapnik to print the elevation along the contours, add the following after the LineSymbolizer in the style file fragment above (in a slightly darker colour for better readability):

<TextSymbolizer size="9" fill="#8a6a20" placement="line" spacing="300" fontset-name="book-fonts" halo-radius="1">[elev]</TextSymbolizer>

If you want elevation annotation only for some contours, you need to generate two sets of contours and insert two sets of Style and Layer tags into the style file, with the one for the annotated contours last. Then the elevation annotation and possibly greater thickness of the annotated contours will hide the unannotated contours if their increment is a multiple.

Hill shading

Medium-scale topographic maps sometimes represent elevation by shading instead of contour lines. There are different kinds of shading that can be generated from elevation information. gdaldem from the GDAL toolkit supports a wide range of them.

Probably the most basic kind of shading is the simulated appearance of the topographic relief under oblique lighting. This is the hillshade mode of gdaldem.

gdaldem hillshade output.vrt hillshade.tiff
gdaldem hillshade -of png output.vrt hillshade.png

gdaldem can generate a GeoTIFF (default) or other image formats. Unlike GeoTIFF, PNG cannot contain metadata such as the projection and extents of the georeferenced image. So for such formats, gdaldem writes an additional XML file with the extension .aux.xml that contains this information. That does not matter for further processing, since Mapnik also uses the GDAL library which will read the XML file automatically if the PNG is specified, but you have to remember to include the XML if you copy or move the files. On the other hand, the GeoTIFF is uncompressed and therefore significantly larger.

The options -az and -alt allow to adjust the azimuthal angle and the angle above the horizon of the lighting direction, both in degrees. The -s option gives the ratio between the elevation unit and the horizontal raster pitch or the other way round depending on whether you believe the documentation text or the example; the -z option sets a vertical exaggeration factor. Experimenting with both, I have not obtained significantly different results.

[Aside: Apparently gdaldem assumes the raster pitch to be the same in both coordinate directions, which is only true at the equator for latitude/longitude coordinates used by the SRTM data. So the hill shading will be increasingly warped the closer you get to the poles. If you really care about this, you could use gdalwarp to generate a big raster of your region in a projection with a low local distortion, such as transverse or oblique Mercator, see above. On the other hand, elevation shading is more for appearance than information anyway, so it is probably not worth the bother.]

A nice-looking kind of shading is to change the background colour with the elevation, so that valleys appear green and mountain peaks white. This is generated by gdaldem color-relief. Because you have to assign colours to height ranges, this is more complex than the previous shading. A required argument is a text file with four numerical columns containing the elevation value and red, green and blue values between 0 and 255 (plus an optional opacity column). By default colours are interpolated betwee the given elevation values.

gdaldem color-relief -of png output.vrt colours.txt relief.png

Other shading modes of gdaldem are steepness of slopes (slope), direction of slope (aspect) and three different measures of local roughness. Bjørn Sandvik's blog (referenced below) shows rendered examples of different kinds of shading.

Even without Mapnik, a colour relief can be manually annotated to create a simple map. For example, this relief of the Pyrenees cites SRTM data as its only source, so it may have been created that way. (The rivers and borders point to some additional data sources, though.)

In order to use shading images with Mapnik, you have to use the RasterSymbolizer tag. The following layer is the simplest possible addition to the old standard OpenStreetmap style:

<Style name="shading">
  <Rule>
    <RasterSymbolizer opacity="0.15" mode="multiply" />
  </Rule>
</Style>
<Layer name="hillshading" srs="+init=epsg:4326">
  <StyleName>shading</StyleName>
  <Datasource>
    <Parameter name="type">gdal</Parameter>
    <Parameter name="file">path/to/shading-image</Parameter>
  </Datasource>
</Layer>

I have placed this layer after the "land use" layer of the standard style; then the shading will affect the land use colours and pattern fills, but not more specific information drawn above it. The style above makes the shading translucent, which is achieved by multiplication. Contrary to some examples elsewhere on the web, in my experience the srs attribute of the Layer tag is obligatory (its value means latitude/longitude coordinates, which could also be spelled out as above).

The simple style above is only satisfactory if there are no large bodies of water in the region you are rendering. gdalwarp correctly excludes ocean regions where no SRTM tiles exist at all, which is carried through to transparency in the result image of gdaldem. But bodies of water within the SRTM tiles are not treated the same way. For the color-relief mode, a colour map could be constructed that assigns complete transparency to sea level. A smaller imperfection is that if the resolution of the elevation data is much higher than that of the rendered map, the shading can appear quite noisy. This could probably be fixed by reducing the resolution of the elevation data (or possibly the shading image) with gdalwarp. In any case, creating a beautifully shaded map is going to require some manual work.

See also

Including elevation information in a map is common enough that there are/were other web tutorials about it:

• A series of posts on digital elevation models, hill shading, colour reliefs, rendering them and contour lines on Bjørn Sandvik's blog
• The OpenStreetmap wiki page on relief maps in general and contour lines in particular
• A report from Czech Technical University
• Part 1 and part 2 of a hill-shading tutorial by Michael Corey

Adding supplementary data

There are two ways of adding non-OpenStreetmap data to your map, creating a separate data source to be used by Mapnik or inserting the data into the database created by osm2pgsql. We will discuss several options in order of increasing complexity.

Supplementary data in CSV files

It is a longstanding but little-known feature of Mapnik to support CSV (comma-separated value) data sources, and the feature was created just with our use case in mind. CSV is easy to write manually or output from a simple program. What is more, the data can either come from a file (using <Parameter name="file">/path/to/file</Parameter> in the Datasource tag, or be included in the style file using <Parameter name="inline">...</Parameter>. That makes it extremely convenient for quick or temporary changes.

Mapnik requires the first row of the CSV data to contain the column keys. As I found out accidentally from an error message, the keys interpreted by Mapnik itself are latitude, longitude, x, y, wkt or geojson. Putting a marker at a specific location works like this:

<Style name="marker">
  <Rule>
    <PointSymbolizer file="path/to/icon.png" allow-overlap="true" ignore-placement="true"/>
  </Rule>
</Style>
<Layer name="marker" status="on" srs="+proj=longlat +datum=WGS84 +no_defs">
    <StyleName>marker</StyleName> 
    <Datasource> 
        <Parameter name="type">csv</Parameter> 
        <Parameter name="inline">
        longitude,      latitude
        1.2345,        67.89
        </Parameter> 
    </Datasource> 
</Layer>

As far as I can tell, a longitude column is treated the same as an x column, and a latitude column the same as a y column &emdash; the projection longlat has to be given explicitly, there is no default based on the column names.

If you add columns beyond those needed for geography, you can use them in rendering as additional columns in a database, for example:

<Style name="marker">
  <Rule>
    <TextSymbolizer size="12" fill="#ff0000" fontset-name="bold-fonts" halo-radius="1">[name]</TextSymbolizer>
  </Rule>
</Style>
<Layer name="marker" status="on" srs="+proj=longlat +datum=WGS84 +no_defs">
    <StyleName>marker</StyleName> 
    <Datasource> 
        <Parameter name="type">csv</Parameter> 
        <Parameter name="inline">
        longitude,      latitude,   name
        1.2345,        67.89,       foo
        </Parameter> 
    </Datasource> 
</Layer>

That said, text annotation only worked for me for the last row in the CSV table, and only if the name string did not contain spaces (even with quotes).

So we have spent two examples marking points. Can CSV data be used to include other objects? Yes, if you use a wkt column. The acronym stands for well-known text, a rather ambiguous term for a standard plain-text geography object description language. Because the object notation can contain commas, it will usually have to be quoted. For example, drawing a polyline would use the linestring data type:

<Style name="specialline">
  <Rule>
    <LineSymbolizer stroke="#ff0000" stroke-width="2" />
  </Rule>
</Style>
<Layer name="specialline" status="on" srs="+proj=longlat +datum=WGS84 +no_defs">
    <StyleName>specialline</StyleName>
    <Datasource>
        <Parameter name="type">csv</Parameter>
        <Parameter name="inline">
        wkt
        "linestring(1.234 5.678, 1.235 4.678, 1.325 4.768)"
        </Parameter>
    </Datasource>
</Layer>

Mapnik's CSV data source is quite flexible, according to its documentation. Column separators, quote and escape characters can be configured. What is called an "option" on that page seems to mean a <Parameter name="option">value</Parameter> tag. The column keys can also be given in a header option instead of the first table row.

Supplementary data in shape files

Before I learnt about Mapnik's CSV data source, I used shape files, which we already know from rendering above. Even though the shape file format is a pain, there are standard tools supporting them, and they can contain the three kinds of data also used by OpenStreetmap &emdash; points, polylines and polygons. On the down side, shape files require a set of several files for a data set, and annotation data has to be entered in the binary .dbf database file. If you have no particular reason for using shape files, you are better off with CSV.

Here is a Perl script generating shape files from the simplest human-writeable format, a plain-text file with longitude-latitude pairs of floating-point numbers in each line. The script can generate multipoint, polyline and polygon shape types. (Multipoint is treated like points by Mapnik but was easier to implement for being similar to the other two types.) Polylines and polygons can be drawn with a LineSymbolizer as the contour lines, or possibly be filled with a PolygonSymbolizer; points would typically be marked with a PointSymbolizer.

The values you convert to shape files using my script need not be longitude/latitude coordinates if you pass a different coordinate system to Mapnik in the srs attribute of the Layer tag. The shape file specification states that the x coordinate comes first.

Supplementary data in an SQLite database

This is something I would like to mention even though I have not tried it. Mapnik supports SQLite databases as data sources. SQLite is a file-based database that is somewhere between file-based data and a server-based database. It is easy to copy and move around like any other file, but can be queried like a database. Therefore it may be an option for use cases that fall between the two "pure-bred" alternatives. The page on the Mapnik wiki describes how to generate a SQLite database with geodata using the GDAL tools.

Maps with transparent background

Even though this seems to be rarely used, Mapnik can easily create maps with a transparent background. Starting from the old XML-based standard OpenStreetmap style, one first has to set the background colour in the Map tag to #00000000, which is a black colour that is completely transparent. (The last two digits are the alpha channel, so partial transparency would also be possible.) The Map background colour is normally set to the colour of the oceans, onto which the continents are superimposed by the file inc/layer-shapefiles.xml.inc. If we want a transparent background, we should remove its include line in inc/layers.xml.inc, and while we are at it, also remove the inclusion of inc/layer-landcover.xml.inc. If you render a map with those modifications to the standard style, the only remaining large covered areas will be bodies of water (remove inc/layer-water.xml.inc to be rid of them too, but rivers will also be gone).

Why would one want to do such a thing? It would allow rendering map layers (or sets of layers) independently of the background and of each other. A special viewer could allow enabling and disabling layers for viewing depending on what information was of interest in a given situation. I have not yet written such a viewer and am not sure when or if I will get around to it. But the possibilities of different layers rendered to separate image files with transparent backgrounds should be kept in mind.

Licensed under the Creative Commons Attribution-Share Alike 3.0 Germany License

TOS / Impressum