|↑ OpenStreetmap Hacker's guide||↑↑ Net & Web|
This page gives an overview of how geospatial information from OpenStreetmap is turned into map tiles for online viewing. This is the most common usage of OpenStreetmap data, well-maintained tools exist for it and can serve as a basis for customisation.
This diagram shows the successive data repositories and processing steps:
|Source||OpenStreetmap, Geofabrik or other sources|
|Tools||Non-interactive downloaders like wget or curl helpful|
|Result||After download, file on your local disk|
These can be huge files that are continually being updated. You can (possibly should, depending on your use case) download a partial database of only one country or region.
They contain most things you can see on a map, such as rivers, roads, houses, towns, etc. They do not contain sea shore lines or elevation data. Shore line data is generally needed in addition to the OSM database (unless perhaps you are rendering only a landlocked region). The three archives from this directory are needed, as well as borders and inhabited regions from naturalearthdata.com. This shell script from this git repository can be used to download most of them.
|Documentation||Wiki page; more doc links|
|Result||Live geospatial database|
This step copies the information from the OpenStreetmap database packages into a live database to allow querying and selecting. If you downloaded only regional data, the rest of the world will contain no features, but this will not prevent it from being rendered as blank tiles later.
This step is very demanding in terms of memory usage and disk access.
|Tools||PostgreSQL and PostGIS|
|Documentation||PostgreSQL manual; plenty of manual pages installed with PostgreSQL; PostGIS manual|
|Result||Default storage in /var/lib/postgres/data/, default port 5432|
PostgreSQL is a freely-licenced database. PostGIS adds data types and queries for geometrical objects and relations between them, such as intersections, distance, area etc.
|Documentation||Rendering style configuration documentation, wiki main page, outdated Python API documentation|
|Style examples||Old OpenStreetmap style, Style share wiki page|
Mapnik is a very versatile and powerful map rendering toolkit. It is written in C++, but is almost undocumented. The old versions 2.x came with Python bindings that were documented; with version 3.0.0 they have been moved to a separate repository and there is no longer any online documentation. Obviously a library on its own does not do anything, but Mapnik is so frequently mentioned that it warrants a section of its own. Multiple renderers using the Mapnik library exist.
The appearance of the map it produces is determined by a configuration file in XML format. For each map scaling, it contains layers to be rendered in order from bottom (occluded by higher layers) to top. For each layer it can be selected which features are shown, to the point of complicated SQL queries. Such style files can be very complex. The old OpenStreetmap style which is written in XML is 6469 text lines. To handle this complexity better, several style generators with simpler input languages have been written.
The Mapnik library can read information from various sources, not just from a live database. (There is even an input plugin that reads packed OSM data directly, but it seems to be considered obsolete.) In the canonical toolchain, the archives of continent outlines, borders and city limits we downloaded above are read by Mapnik in addition to data from the live database.
|Tool||renderd (part of distribution of Apach module mod_tile)|
|Documentation||Manual page and supplementary documentation|
|Configuration||/etc/renderd.conf (or config file specified with -c option)|
|Control||Default socket /var/run/renderd/renderd.sock|
|Result||Metatiles; default output directory /var/lib/mod_tile|
renderd is a background daemon designed to render tiles on demand from the Apache module mod_tile or other programs. By default it generates "metatiles", mini-archives of 8 by 8 tiles. Rendering larger metatiles helps to render efficiently because there has to be some overlap between neighbouring tiles, and storing metatiles helps keep the total number of files down and reduces the load on the file system driver.
The individual tiles are in PNG format. The world is made up of 4z tiles in zoom level z because each successive level zooms in by a factor 2. To avoid having directories with too many subdirectories or files, the metatiles are stored in a deep directory structure corresponding to bit fields in the tile coordinates:
<base dir>/<style>/<zoom>/<16*A+a>/<16*B+b>/<16*C+c>/<16*D+d>/<8*(16*E+e)>.meta x = AAAABBBBCCCCDDDDEMMM y = aaaabbbbccccddddemmm
The index of the tile within the metatile file is 8*M+m. The <style> directory allows generating different map appearances on the same server.
As a curiosity, the tiles served by openstreetmap.org are actually in JPEG format instead of PNG. Everything else is the same, even the file extension of the tile URLs is .png, but the images are JPEGs. Apparently this is a non-standard conversion performed after tile generation to save network bandwidth.
|Tool||Apache with module mod_tile|
|Documentation||Apache V2.4 documentation home page; wiki page on mod_tile|
|Result||Tiles served at path http://server.foo/osm_tiles/<zoom>/<x>/<y>.png|
|Tools||Emerillon; various browser plugins|
This step is strictly out of the scope of this page, but of course the tiles are pretty useless without a viewer. I don't have much to say on this, as I have mostly used OSM data offline.
You can also search the web or the OpenStreetmap Wiki for other viewers, but be wary, there are many obsolete ones which do not compile any more or crash.
This workflow is fairly standard, so there actually exist tutorials about recreating it. They take the form of "wetware programs", lists of steps readers are supposed to follow blindly, with no explanation. Both tutorials assume Ubuntu Linux, apparently what the OSM project itself uses.
Licensed under the Creative Commons Attribution-Share Alike 4.0 Germany License
TOS / Impressum