Abstract
This document describes the procedures for using OGR vector data sources within MapServer applications.
Last Updated: 2003-10-17
Table of Contents
Starting with version 3.5, MapServer included the ability to access vector data sets in formats other than Shapefile in their native format using the OGR library. The following document describes the process for implementing OGR support within MapServer applications.
Note that experimental OGR support was included in MapServer version 3.4 but this initial implementation had some limitations and is not covered in this document.
This document assumes that you are already familiar with certain aspects of MapServer:
MapServer application development and especially setting up .map files.
Some compilation skills if you don't have ready access to a precompiled installation and need to compile your own copy of MapServer with OGR support.
The OGR Simple Features Library is a C++ open source library (and commandline tools) providing read (and sometimes write) access to a variety of vector file formats including ESRI Shapefiles, and MapInfo mid/mif and TAB formats.
The OGR Simple Features Library allows MapServer users to display several types of vector data files in their native formats. For example, MapInfo Mid/Mif and TAB data do not need to be converted to ESRI shapefiles when using OGR support with MapServer.
See http://ogr.maptools.org/ogr_formats.html for the latest list of supported formats. At the date this document was written, the following formats were supported:
Note1: Some of the above formats (e.g. OGDI) have external dependencies and are not always included in the precompiled binary distributions of MapServer with OGR support.
Note2: Some of the above formats are not well suited for random access by nature, that's the case of MapInfo MIF/MID files which is a TEXT format and will give very poor performance for a web application. On the other hand, some binary formats such as MapInfo TAB are better suited for random access and will give performance comparable to native shapefile access in Mapserver.
More information on the OGR Simple Features Project can be found at http://ogr.maptools.org/.
The main developer of the OGR library is Frank Warmerdam.
The integration of OGR within MapServer was done by Daniel Morissette.
TODO: this is still incomplete
Follow the instructions on the OGR page to compile/install OGR (actually GDAL since OGR is part of the GDAL library): http://ogr.maptools.org/.
Obtain the MapServer source from: http://mapserver.gis.umn.edu/dload.html.
For UNIX users, see the README.CONFIGURE file in the MapServer source, or see the UNIX/MapServer HOWTO at http://mapserver.gis.umn.edu/doc/unix-install-howto.html. If GDAL/OGR is normally installed it should be sufficient to add --with-ogr to the configure line before (re)building MapServer.
For Windows users, it is recommended to look for a precompiled binary on the MapServer site at http://mapserver.gis.umn.edu/win32binaries.html. If you want to compile your own then see the README.WIN32 file in the MapServer source.
The only change that is needed to integrate OGR support with a MapServer application is with the .map file. The LAYER's DATA parameter is expanded to three parameters (CONNECTIONTYPE OGR, CONNECTION and DATA).
The syntax for this differs depending on the type of data being used. In OGR, a data source can be either a set of files that share a common basename (e.g. .shp/.shx/.dbf for ArcView Shapefiles, or .tab/.map/.dat/.ind/.id for MapInfo TAB files) or a whole directory of files (e.g. TIGER).
Let's call the former "File-based data sources" and the later "Directory-based data sources". When accessing a file-based data source you specify the filename of one of the files in the set (e.g. roads.shp or roads.tab) and when accessing a directory-based data source you specify the directory name and OGR reads all the files in the directory as a single data source with potentially several layers (e.g. TIGER files).
Some OGR drivers (e.g. SHP, TAB) can have dual behaviors, that is if they're pointed to a single file then they behave as a file-based data source and if they're pointed to a directory then they will behave as a directory-based data source and then every file in the directory becomes a new layer in the data source.
See the OGR documentation for more info on the specific file format you're using: http://ogr.maptools.org/ogr_formats.html.
The .map file LAYER definition for file-based sources is as follows:
LAYER
...
CONNECTIONTYPE OGR
CONNECTION "<datasource_name>"
DATA "<layer_definition>"
...
END
<datasource_name> is the name of the datasource to read from and is prefixed by the CONNECTION keyword. The exact organization depends on the format driver in use. The format driver to use is automatically selected by OGR based on the nature of the string passed as the datasource, and/or the format of the file referenced by it.
For file based datasources this is the name of the file, including the extension, using an absolute path, or a relative path. Relative paths are interpreted relative to the SHAPEPATH first, if not found then we try again relative to the .map file location. (Note: before version 4.1 the SHAPEPATH used ignored for OGR datasources).
For directory based datasources, such as TIGER/Line, or Arc/Info Binary Coverages this is the name of the directory containing the files. If the path is relative it is interpreted relative to the .map file.
For virtual datasources such as database systems, and OGDI this is the service connection string and is generally not related to the filesystem. For instance, for Oracle Spatial this might be "OCI:warmerda/Password@gdal800.velocet.ca".
<layer_definition> is the name, number or SQL definition of the layer to use from the datasource. It is indicated via the DATA keyword in the map file.
Layer Name: The (case insenstive) layer name may be used to select a layer.
Layer Number: The layer number (starting from 0 for the first layer) may be used to select a layer. Generally the layer name is preferred to this since it is more self describing.
Omitted: If no DATA keyword is provided, this is equivelent to selecting layer 0.
SQL SELECT: If an SQL SELECT statement is used, it is interpreted in a driver specific manner to try and generate a temporary pseudo-layer. For some formats this a restricted subset of SQL is interpreted within OGR. For RDBMS based drivers (such as PostGIS and Oracle) this is passed through to the underlying database.
The OGRINFO program can be used to find out the list of layers and their names in a data source.
Example 1. MapInfo TAB file
LAYER
NAME "Builtup_Areas_tab"
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "data/tab/092b06_builtup_a.tab"
STATUS ON
CLASS
...
END
...
END
Example 2. Microstation DGN file using <layer_index>
The entire DGN file is represented in OGR as one layer (see http://ogr.maptools.org/drv_dgn.html for more info):
LAYER
NAME "Water_dgn"
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "data/dgn/wat.dgn"
DATA "0"
STATUS ON
CLASS
...
END
...
END
Example 3. TIGER/Line file using <layer_name>
LAYER
NAME "Roads_tig"
TYPE line
CONNECTIONTYPE OGR
CONNECTION "../data/tiger_michigan"
DATA "CompleteChain"
STATUS ON
CLASS
...
END
END
Example 4. Directory of Shapefiles using SQL JOIN
LAYER
NAME "Parks_cov"
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "data/shppoly"
DATA "SELECT eas_id, idlink.Name FROM poly LEFT JOIN idlink ON poly.eas_id = idlink.eas_id"
STATUS ON
CLASSITEM "idlink.Name"
CLASS
NAME "At Risk"
COLOR 0 0 255
OUTLINECOLOR 255 0 0
EXPRESSION "_158_"
END
CLASS
NAME "Normal"
COLOR 0 255 0
OUTLINECOLOR 255 0 0
END
...
END
OGRINFO is part of the GDAL/OGR distribution. It is an executable that can be used to obtain layer information about OGR supported files. The parameters are:
ogrinfo [-ro] [-q] datasource_name [layer [layer...]]
-ro opens the file as read only (optional)
-q executes in quiet mode, only the layer idex line will be returned (optional)
datasource_name is the filename including extension (eg. roads.tab); for TIGER/Line files, datasource_name is the directory containing the TIGER files (eg. ogrinfo TGR25001)
Example 5. To get the list of layers in a file:
$ ogrinfo popplace.tab
Had to open data source read-only.
INFO: Open of `popplace.tab'
using driver `MapInfo File' successful.
1: popplace (Point)
which shows that there is one point layer in the popplace.tab file.
Example 6. To get a dump of a specific layer, including field names, projection, etc:
$ ogrinfo popplace.tab popplace
Had to open data source read-only.
INFO: Open of `popplace.tab'
using driver `MapInfo File' successful.
Layer name: popplace
Geometry: Point
Feature Count: 497
Layer SRS WKT: PROJCS["unnamed",GEOGCS["unnamed",DATUM["North ...snipped...
AREA: Real (15.3)
PERIMETER: Real (15.3)
POPPLACE_: Real (11.0)
POPPLACE_I: Real (15.0)
NAME: String (50.0)
OGRFeature(popplace):1
AREA (Real) = 0.000
PERIMETER (Real) = 0.000
POPPLACE_ (Real) = 1
POPPLACE_I (Real) = 1
NAME (String) = Port Hope Simpson
POINT (2437287.249 1153656.751)
OGRFeature(popplace):2
AREA (Real) = 0.000
PERIMETER (Real) = 0.000
POPPLACE_ (Real) = 2
POPPLACE_I (Real) = 1
NAME (String) = Hopedale
...
...
Example 7. To get a list of layers in a TIGER/Line Directory:
$ ogrinfo TGR25001
Had to open data source read-only.
INFO: Open of `TGR25001'
using driver `TIGER' successful.
1: CompleteChain (Line String)
2: AltName (None)
3: FeatureIds (None)
4: ZipCodes (None)
5: Landmarks (Point)
6: AreaLandmarks (None)
7: KeyFeatures (None)
8: Polygon (None)
9: EntityNames (Point)
10: IDHistory (None)
11: PolyChainLink (None)
12: PIP (Point)
13: TLIDRange (None)
14: ZipPlus4 (None)
The above example shows that there are 14 layers in the TGR25001 directory.
Example 8. To get a dump of a specific TIGER layer, including field names, projection, extent, etc:
$ ogrinfo TGR25001 Landmarks
Had to open data source read-only.
INFO: Open of `TGR25001'
using driver `TIGER' successful.
Layer name: Landmarks
Geometry: Point
Feature Count: 777
Extent: (-70.674324, 41.519817) - (-69.969211, 42.046868)
Layer SRS WKT: GEOGCS["NAD83",DATUM["North_American_Datum_1983",
SPHEROID["GRS 1980",6378137,298.257222101]],PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433]]
MODULE: String (8.0)
FILE: String (5.0)
STATE: Integer (2.0)
COUNTY: Integer (3.0)
LAND: Integer (10.0)
SOURCE: String (1.0)
CFCC: String (3.0)
LANAME: String (30.0)
OGRFeature(Landmarks):1
MODULE (String) = TGR25001
FILE (String) = (null)
STATE (Integer) = 25
COUNTY (Integer) = 1
LAND (Integer) = 1
SOURCE (String) = J
CFCC (String) = H31
LANAME (String) = Northeast Pond
...
...
OGR layers can be queried the same way as regular shapefiles in MapServer.
OGR layers can utilize tile indexes in a similar fashion to Shapefile based layers. The TILEINDEX keyword should contain the connection string for the tile index file. The tile index file may be any supported OGR format, including shapefiles.
The TILEITEM keyword in the LAYER definition indicates what attribute from the tile index file should be used as the datasource location. If omitted, the default TILEITEM value is "location". The value in the location field should be a connection string the same as would have been used in the CONNECTION field for OGR layers. The CONNECTION keyword is not needed (and will be ignored) for layers using the OGR connection type and having the TILEINDEX keyword.
Tileindex files can be prepared in an external GIS, or using the OGR utility ogrtindex. Details can be found on the OGR Utilities Page at http://ogr.maptools.org/ogr_utilities.html.
The following is a simple example a point layer using a tile index.
LAYER
NAME "ogr_points"
TYPE POINT
CONNECTIONTYPE OGR
TILEINDEX "PIP_ogr_tiles.shp,0"
STATUS ON
CLASS
SYMBOL "default-circle"
COLOR 255 0 0
SIZE 6
END
END
OGR tileindex layers should support all normal query and attribute fetching mechanisms, including from MapScript; however, this has not been heavily tested as of April/2002. Please report problems via the MapServer Bugzilla. If auto projection support is used for tileindexed OGR layers, the tileindex is read for the projection (not the component tiles). Problems may (or may not) be encountered if the component tiles have differing schemas (different sets of attributes).
NOTE: This feature is only supported with MapInfo TAB and Microstation DGN files at the moment, but eventually other formats that carry colors and styles at the shape-level may also be supported through OGR.
In MapServer, ArcView, and other shapefile-based applications, colors and styles are usually defined at the layer level. This means that all the shapes in a given layer are usually rendered using the same color and styles.
On the other hand, some formats supported by OGR such as MapInfo TAB do have color and style information attached to each shape. OGR adds support for the 'STYLEITEM "AUTO"' layer parameter which allows you to request that the shapes in a layer be rendered using colors and styles coming from the data source instead of being driven by CLASSes as was traditionally done with MapServer.
In order to have a layer rendered using colours and styles coming from the OGR data source, your must do the following:
Your layer definition must contain the STYLEITEM "AUTO" parameter.
Your layer definition needs to contain at least one CLASS (which may be empty) and optionally a CLASSITEM to match the expressions if your CLASS contains an expression. The empty CLASS in the layer will be updated dynamically at runtime to contain colours and styles coming from the data source for each shape.
Example 9. Layer Definition Using STYLEITEM "AUTO" without a CLASSITEM
LAYER
NAME "test_dgn"
STATUS ON
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "../data/dgn/test.dgn"
# This enables use of colors and styles from the source file.
STYLEITEM "AUTO"
# Define an empty class that will be filled at runtime from the color and
# styles read on each shape in the source file.
CLASS
END
END # layer
Example 10. Layer Definition Using STYLEITEM "AUTO" with a CLASSITEM
LAYER
NAME "Builtup_Areas_tab"
TYPE POLYGON
CONNECTIONTYPE OGR
CONNECTION "data/tab/092b06_builtup_a.tab"
STATUS ON
# This enables use of colors and styles from the source file.
STYLEITEM "AUTO"
# Define an empty class that will be filled at runtime from the color and
# styles read on each shape in the source file.
CLASSITEM "CATEGORY"
CLASS
EXPRESSION "1"
END
END
CLASS EXPRESSIONs are still working, so it is still possible to query and classify layers that are using STYLEITEM "AUTO". The only difference is that instead of using static class definitions, the colors and style will be read from the data file.
NOTE 1: Even though MapInfo and other OGR data sources may support layers with mixed geometry types (e.g. points, lines and polygons in the same file) this is not yet supported in MapServer. So you still have to define a layer 'TYPE' and make sure that all the shapes in the OGR data source are compatible with that layer type, otherwise MapServer may produce an error about incompatible geometry types at runtime.
NOTE 2: Due to the dynamic nature of this feature, it is not compatible with the labelcache, so the labelcache is automatically disabled for layers that make use of 'STYLEITEM "AUTO"'.
NOTE 3: When you use STYLEITEM AUTO, MapServer tries to match symbol names returned by OGR to names in your symbol file. For a quick solution, try using the following symbol file:
http://www2.dmsolutions.ca/msapps/yk_demo/etc/symbols_mapinfo.sym
The name of the symbols returned by OGR to MapServer depends on the file format. In the case of MapInfo files, it will be:
For "old-style" symbols (default MapInfo 3.0 symbols numbered 32 to 67) the symbol name will be 'mapinfo-sym-##' where '##' is the symbol number, e.g. 'mapinfo-sym-32'.
For "Font Symbols", the symbol name is also 'mapinfo-sym-##' where '##' is the symbol number in the font. In this case, the name of the font itself is ignored by MapServer.
MapInfo also supports "custom symbols" (bitmap symbols)... I'm not sure what you would get from OGR for this, but I'm pretty sure that MapServer doesn't do anything useful with them.
The OGRINFO utility can be used to find out exactly which symbol names OGR will return to MapServer. Look at the "Style" string in the ogrinfo output for each shape that is read.
Here is the list of style parameters that are currently supported from OGR data sources and how they're mapped in MapServer:
The line colour is mapped to CLASS.COLOR
The default will be 1 pixel line (as it always is with MapServer). In MapServer, in order to get lines thicker than 1 pixel, one has to define a circle symbol that will be used to render the line (by applying a size to the circle symbol).
So if your data source contains lines thicker than 1 pixel then you need to provide a circle symbol in your symbolset and you must name it "default-circle". If this symbol is present then the lines will be drawn using their real thickness, otherwise all lines will be 1 pixel wide.
Polygon fill color is mapped directly to CLASS.COLOR
If a polygon has an outline color and thickness defined in the data source then the same rule as for line color and thickness above will apply, except that the outline color is mapped to CLASS.OUTLINECOLOR
Point symbol color is directly mapped to CLASS.COLOR
Point symbol size is directly mapped to CLASS.SIZE
It is also possible (with a bit of work) to control which symbol gets used in rendering point symbols. OGR provides MapServer with symbol names, and if the symbol name returned by OGR to MapServer matches the name of one of the symbols in your symbolset then this symbol will be used.
For MapInfo point symbols (numbered 32 to 67 in the MapInfo MIF spec), the name returned by OGR is "mapinfo-sym-X" where X should be replaced with the MapInfo symbol number (e.g. "mapinfo-sym-35" is the star symbol).
The text string is mapped to CLASS.TEXT
Text color is mapped to CLASS.LABEL.COLOR
Text background color is mapped to CLASS.LABEL.BACKGROUNDCOLOR
Text height is mapped to CLASS.LABEL.SIZE
Text angle is mapped to CLASS.LABEL.ANGLE
Text font mapping follows the following rules:
If TTF fonts are supported:
If the native font name (e.g. "Arial") is found in your fontset then this font will be used.
If 1a. failed and a font called "default" is present in your fontset then this "default" font will be used.
If TTF fonts are not supported or if all above cases failed, then BITMAP MEDIUM font will be used.
The following sites use OGR's STYLEITEM "AUTO" feature:
The following site uses OGR, as well as MapInfo's 'Seamless Map Layers' feature:
The following site uses OGR to display TIGER 2000 files:
| Q: | What Does "OGR" Stand For? |
| A: | Basically, OGR does not stand for anything. For a detailed explanation of how OGR was named, see Frank Warmerdam's OGR site at http://ogr.maptools.org/ in the section 'OGR and OpenGIS'. |
| Q: | When using STYLEITEM AUTO, what should I have in my .sym symbols file? |
| A: | When you use STYLEITEM AUTO, MapServer tries to match symbol names returned by OGR to names in your symbol file. For a quick solution, try using the following symbol file: http://www2.dmsolutions.ca/msapps/yk_demo/etc/symbols_mapinfo.sym The name of the symbols returned by OGR to MapServer depends on the file format. In the case of MapInfo files, it will be:
The OGRINFO utility can be used to find out exactly which symbol names OGR will return to MapServer. Look at the "Style" string in the ogrinfo output for each shape that is read. |
Copyright (c) 2003, Jeff McKenna, DM Solutions Group Inc.
This documentation is covered by the same Open Source license as the MapServer software itself. See MapServer's License and Credits page for the complete text.
No liability for the contents of this document can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies that may be damaging to your system. Although this is highly unlikely, the author(s) do not take any responsibility for that: proceed with caution.
| Add a Comment |