| ↑ OpenStreetmap Hacker's guide | ↑↑ Net & Web |
This page gives an overview of how geospatial information from OpenStreetmap is turned into map tiles for online viewing. This is the most common usage of OpenStreetmap data, well-maintained tools exist for it and can serve as a basis for customisation.
This diagram shows the successive data repositories and processing steps:
| Source | OpenStreetmap , Geofabrik or other sources |
| Tools | Non-interactive downloaders like wget or curl helpful |
| Result | After download, file on your local disk |
These can be huge files that are continually being updated. You can (possibly should, depending on your use case) download a partial database of only one country or region.
They contain most things you can see on a map, such as rivers, roads, houses,
towns, etc. They do not contain sea shore lines or elevation data. Shore
line data is generally needed in addition to the OSM database (unless perhaps
you are rendering only a landlocked region). The three archives from
this directory are
needed, as well as
borders
and
inhabited
regions from naturalearthdata.com. This shell
script from this git
repository can be used to download most of them.
| Tool | osm2pgsql |
| Documentation | Wiki page; more doc links |
| Result | Live geospatial database |
This step copies the information from the OpenStreetmap database packages into a live database to allow querying and selecting. If you downloaded only regional data, the rest of the world will contain no features, but this will not prevent it from being rendered as blank tiles later.
This step is very demanding in terms of memory usage and disk access.
| Tools | PostgreSQL and PostGIS |
| Documentation | PostgreSQL manual; plenty of manual pages installed with PostgreSQL; PostGIS manual |
| Result | Default storage in /var/lib/postgres/data/, default port 5432 |
PostgreSQL is a freely-licenced database.
PostGIS adds
data types and
queries for geometrical objects and relations between them, such as
intersections, distance, area etc.
| Tool | Mapnik library |
| Documentation | Rendering style configuration documentation, wiki main page, outdated Python API documentation |
| Style examples | Old OpenStreetmap style, Style share wiki page |
Mapnik is a very versatile and powerful map rendering toolkit. It is written
in C++, but is almost undocumented. The old versions 2.x came with Python
bindings that were documented; with version 3.0.0 they have been moved to a
separate repository and there is no
longer any online documentation. Obviously a library on its own does not do
anything, but Mapnik is so frequently mentioned that it warrants a section of
its own. Multiple renderers using the Mapnik library exist.
The appearance of the map it produces is determined by a configuration file in XML format. For each map scaling, it contains layers to be rendered in order from bottom (occluded by higher layers) to top. For each layer it can be selected which features are shown, to the point of complicated SQL queries. Such style files can be very complex. The old OpenStreetmap style which is written in XML is 6469 text lines. To handle this complexity better, several style generators with simpler input languages have been written.
The Mapnik library can read information from various sources, not just from a live database. (There is even an input plugin that reads packed OSM data directly, but it seems to be considered obsolete.) In the canonical toolchain, the archives of continent outlines, borders and city limits we downloaded above are read by Mapnik in addition to data from the live database.
| Tool | renderd (part of distribution of Apach module mod_tile) |
| Documentation | Manual page and supplementary documentation |
| Configuration | /etc/renderd.conf (or config file specified with -c option) |
| Control | Default socket /var/run/renderd/renderd.sock |
| Result | Metatiles; default output directory /var/lib/mod_tile |
renderd is a background daemon designed to render tiles on demand from the Apache module mod_tile or other programs. By default it generates "metatiles", mini-archives of 8 by 8 tiles. Rendering larger metatiles helps to render efficiently because there has to be some overlap between neighbouring tiles, and storing metatiles helps keep the total number of files down and reduces the load on the file system driver.
The individual tiles are in
PNG format. The
world is made up of 4z tiles in zoom level z because each successive level
zooms in by a factor 2. To avoid having directories with too many
subdirectories or files, the metatiles are stored in a deep directory
structure corresponding to bit fields in the tile coordinates:
<base dir>/<style>/<zoom>/<16*A+a>/<16*B+b>/<16*C+c>/<16*D+d>/<8*(16*E+e)>.meta x = AAAABBBBCCCCDDDDEMMM y = aaaabbbbccccddddemmm
The index of the tile within the metatile file is 8*M+m. The <style> directory allows generating different map appearances on the same server.
As a curiosity, the tiles served by openstreetmap.org are actually in JPEG format instead of PNG. Everything else is the same, even the file extension of the tile URLs is .png, but the images are JPEGs. Apparently this is a non-standard conversion performed after tile generation to save network bandwidth.
| Tool | Apache with module mod_tile |
| Documentation | Apache V2.4 documentation home page; wiki page on mod_tile |
| Configuration | /etc/httpd/conf/, /etc/httpd/conf/extra/mod_tile.conf |
| Result | Tiles served at path http://server.foo/osm_tiles/<zoom>/<x>/<y>.png |
The Apache web server together with the mod_tile module deliver the tiles to the client (which is usually running a JavaScript "slippy map" that chooses which tiles to download and puts them together to form the map). mod_tile connects to a running renderd to render metatiles on demand when they do not yet exist or are out of date. The decision which tiles are out of date seem to be taken purely based on their last rendering time, no check for an updated database seems to be done.
| Tools | Emerillon; various browser plugins |
This step is strictly out of the scope of this page, but of course the tiles are pretty useless without a viewer. I don't have much to say on this, as I have mostly used OSM data offline.
You can also search the web or the OpenStreetmap Wiki for other viewers, but be wary, there are many obsolete ones which do not compile any more or crash.
This workflow is fairly standard, so there actually exist tutorials about recreating it. They take the form of "wetware programs", lists of steps readers are supposed to follow blindly, with no explanation. Both tutorials assume Ubuntu Linux, apparently what the OSM project itself uses.
by the OpenStreetMap Foundation with some additional information and recommendations
Licensed under the Creative Commons Attribution-Share Alike 3.0 Germany License 