Exporting, importing and projecting data layers with DIVA-GIS

Using Farmer Field Schools Approaches to Overcome Land Degradation in Agro-Pastoral Areas of Kenya

Land degradation assessment – Baseline survey on spatial analysis of land cover / degradation trends and Toolkit Development.

Lesson overview

In this lesson you will learn how to import and export data from DIVA-GIS to other GIS software and applications; a short introduction to ArcView and using the dataset in ArcView; you will also learn how DIVA-GIS can project data from Geographic coordinates to other projections.

Managing files in DIVA-GIS

DIVI-GIS file manager

DIVA-GIS can be used for managing the dataset, and for exporting and importing data from and to DIVA-GIS.

Note that if you delete files using the File Manger, they will not be moved to the Recycle Bin in windows, but will be permanently deleted. If you rename or remove a file that was included in a project, the project will no longer have access to the file, and it will be omitted when you open the project again.

For grid files there is a special tool for retrieving statistical data about grid layers and for managing grid data headers, under the menu: Grid – Describe,

. Apart from getting statistical information, this tool can be used to update erroneous grid file header (*.grd – see lesson 5), but should only be used for such purposes if you are familiar with data types and there is an error in your grid file.

Importing vector files to DIVA-GIS

DIVA-GIS can import both vector and grid data, reached under the menu: Data – Import Points to Shapefle / Import Text to Line/Polygon. For vector data it can be a useful tool for importing point data with X and Y coordinates, including data obtained with a GPS receiver. DIVA-GIS can import data with coordinates from either text files (*.txt), dBase files (*.dbf) or from Microsoft Access (*.mdb). Text files (*.txt) can be imported either as points or as lines/polygons.

On screen digitizing

You can also digitize your own vector files directly on the screen using DIVA-GIS, from the menu: Data – Draw Shape. You can digitize points, lines or polygons, and save the data to a shape file (*.shp). In this way you can create a vector layer identifying e.g. land degradation hotspots (the example used here).

You can digitize several points, lines or polygons in one digitizing session, closing a line of polygon by clicking the Stop Drawing button,

, or double clicking when you digitize the last point of a line or polygon feature. You can also add values and labels in the Draw Shape window, as shown below.

Save the digitized features by clicking the Save button,

, in the Draw Shape window, and the new vector will be added to the DIVA-GIS project. You can then symbolize the new shape, as illustrated in the example below.

Converting between vector and grid data

DIVA-GIS can convert data from grid to vector, and vice versa. The polygon to grid conversion was introduced in lesson 5, and is a good tool. The vector to grid conversion is a bit cruder, but is useful when e.g. projecting a grid (see below). The exported shape file will consist of as many features as there are cells in the grid file. To export a grid file to a raster use the menu entry: Data – Export Gridfile, then choose either to generate Shapefile (polygons) or Shapefile (points), as shown below.

From the illustration above you can see that each cell from the grid is exported as a feature in the new shapefile (points in the example). The generated shapefile will hence have as many features as there are cells in the grid file, and will be very slow to work with (if you try to symbolize this vector file it will take a very long time for DIVA-GIS to process it).

Importing and exporting grid files

DIVA-GIS can import and export grid files in several formats, including files used by Idrisi (Kilimanjaro), ArcView/ArcGIS, and general grid files (BIP, BIL and BSQ – explained below). The grid file import/export functions are under the menu: Data – Export Gridfile / Import Gridfile, and allows export/import of multiple files in one go.

There are more options for exporting compared to importing, but the conversions that DIVA-GIS can do are sufficient for sharing data with almost any other GIS software. If you want to use the data in the dataset with e.g. Idrisi or ArcView, you can simply choose Data – Export to GridFile – Multiple Files, select all files in a folder and export them (to the same folder or another folder). First the interface for exporting a single grid file for use in Idrisi.

The data exported to Idrisi (or GRASS, which is a very powerful GIS software available for free, but more difficult to use) can be directly used in Idrisi (GRASS) once exported.

Then an example of exporting multiple files for use in ArcView/ArcGIS (produced by ESRI – Earth System Resources Incorporated). There are two options for exporting data to ESRI (ArcView/ArcGIS); as ESRI binary (FLT); or as ESRI ASCII. The former is better for real data (see lesson 5 to learn about data formats in grid files), and the latter for byte or integer files. In the example below the three land degradation index files created in lesson 6 are exported to ESRI ASCII format, as they contain integer data (whole numbers).

Data exported to ESRI formats are generated in special exchange formats used by ArcView/ArcGIS, and must hence be imported to these programs before they can be used.

Importing grid to ArcView

This section will deal with importing grid (e.g. exported from DIVA-GIS) to ArcView. To follow this section you must have an ArcView licence, including the extension Spatial Analyst.

Start ArcView, and open an empty View. First you must install the extension for Spatial Analyst (the module that handles grid data in ArcView) from the ArcView menu: File – Extensions.

In the Extensions window, scroll down and click the checkbox for Spatial Analyst (if you do not have the extension Spatial Analyst you will not be able to use grid data with ArcView, but you can still all the datasets in the dataset by using the jpg images instead). It is not enough to click the text, you must click in the checkbox.

Click OK for ArcView to load the extension. Make sure the View is the active document in ArcView (the frame of the active document - View in this case, appears dark blue by standard). Then go to the import function via the menu: File – Import Data Source, as shown below.

Dependent on whether you exported an ASCII file (*.asc) or a binary file (*.flt) from DIVA-GIS you have to choose the corresponding file type in the Import Data Source window of ArcView, as shown below.

In the Import window that opens navigate to the folder where you put the exported grids from DIVA-GIS, and select the files you want to import, you can import multiple files in one go (hold down ctrl+alt while selecting), as shown below.

Click OK to start the import to ArcView. In the Output Grid window that opens you can navigate to the folder where you want to put the data, and give a name you want the ArcView grid to have. If you look carefully, ArcView will show existing grids in fuzzy type fonts. And if you navigated to the original directory of the dataset that came with the project CD you will find that all datasets under \data_spatial\grid are already available in ArcView grid format, but that the data under \data_spatial\rs are not. Actually the whole project dataset was produced by scripts (program pieces) written in ArcView, and the files you have used with DIVA-GIS where all produced from ArcView. You can choose to save your grid file with another name, but note that ArcView accepts a maximum of 14 letters for grid names (grid files are saved under individual folders as compressed data in ArcView).

If you Imported ASCII files to ArcView you will be prompted if you want Cell values as integers, which is preferred if the exported data was either byte or integer (see lesson 4).

You can then choose to add your data to the View which was the active document when you started the import.

To add other data to ArcView, including the shape files (which is a format invented by ESRI and the default format for ArcView) and jpg image files, use the Add Theme tool,

, in ArcView. To add vectors you must set the Data Source Types: Feature Data Source in the Add Theme window, as shown below.

To add the jpg files you must change the Data Source Types: to Image Data Source (and to add grid files it must be set to Grid Data Source).

In the example below the image data source (*.jpg) for z-score trend in Rain Use Efficieny, and the Feature data source (*.jpg) for sub-districts (Mbeere district) have been added to ArcView.

Export /import BIL images

Both DIVA-GIS and ArcView can handle the general BIL (Byte Interleaved by Line) data format. This is a file format where the data is stored row by row (row = line) for images with multiple bands (more than one data layer in one file) as one long string of binary values. In comparison BIP (Byte Interleaved by Pixel) files store data values pixel by pixel. And BSQ stores that data Band SeQuential (first band one, then band two etc). If there is only one band in the file (as is the case for all data in the dataset of this project) BIP, BIL and BSQ files are exactly the same, it is only if more than one data layer is stored in the same file that the format becomes different. To convert all the satellite (remote sensing - rs) data in the dataset (i.e. all grid layers under the folder \data_spatial\rs), it is faster to export them as BIL files from DIVA-GIS, as shown below.

And they can then be opened in ArcView without being imported first. In ArcView go via the Add Theme tool,

, navigate to the folder with the exported BIL files, and add them as images. As there are both jpg files (included by default in the dataset – you looked at them in several of the lessons before) and BIL files in the folder after the export from DIVA-GIS you must be careful in selecting the correct files. ArcView will accept both the jpg image and the BIL file, but will only understand the information content in the BIL file, the jpg file will only have color values (no values related to e.g. vegetation or rainfall). Both files must be added by setting the Data Source Types: Image Data Source as shown below.

In the example below one of the Rain Use Efficiency BIL files exported from DIVA-GIS have been added to ArcView (you can see the file extension “bil” if you look at the full file name).

The BIL file can now be saved as grid file and used for calculations (ArcView and DIVA-GIS have very similar interface as you can see, and also the calculation functions fro grid layers are similar). To save the BIL file as grid go via the ArcView menu: Theme – Convert to Grid.

To make things even more complicated ArcView can not import real data in BIL format. The trend (and trend correlation – r2) grid files that we used (and generated) from DIVA-GIS must hence be exported and imported as binary (*.flt) files when converted from DIVA-GIS to ArcView (see above).

ArcView project files

ArcView and DIVA-GIS also share the concept of a project. Both programs store map compositions, legends and design (called layout in ArcVIew) in project files (*.div for DIVA-GIS, *.apr for ArcView). As mentioned above the datasets for this project was generated using ArcView. When the dataset was created, ArcView was programmed to store all the information in a project (.apr) file. If you have an ArcView licence you can hence open the complete dataset (except the satellite data under the folder \data_spatial\rs) by opening the project file for each site (Kenya, Mbeere, Mwingi and Narok). The ArcView project files are under the site folder under \data_spatial\ of each site. The ArcView project file comes in two versions, one containing all framework datasets, and one containing only framework datasets. The former project file demands that the extension Spatial Analyst is installed, otherwise the grid files can not be loaded. The latter version (with the suffix _nogrid) are for those who have a basic ArcView license, and does not demand Spatial Analyst.

Use Windows explorer (or start ArcView and at startup choose Open an existing project) and navigate to the locations indicated in the table above, and double click the respective *.apr files. If you have the extension Spatial Analyst installed open the project without suffix, if you lack the extension Spatial Analyst instead open the project file with the extensions _nogrid. There are some free extensions added on the Project CD, under the folder \software\arcview. These extensions have different functions that can be added to ArcView by putting them in the default extension folder of ArcView GIS (with a normal ArcView installation the folder to copy these files to is C:\ESRI\AV_GIS30\ARCVIEW\EXT32). With the extension JFIF.avx added you can view all the data layers (including all the satellite derived vegetation and rainfall datasets) in ArcVew by adding them as images (see above). You will however not be able to use these images for any map calculations.

Projecting layers

All the data on the project CD are in geographic (longitude – latitude) coordinates. The geographic system is not a true projection, but with DIVA-GIS you can project your vector layers to almost any other projection. Unfortuantely DIVA-GIS can not project grid files, but if you need to project a grid file you can first export it to a Shapefile (polygon), then project, and then convert it back to a grid. In DIVA-GIS you can not project an already projected layer.

Projecting a vector file

Use the Add Theme tool to add the vector line that shows the border of Kenya (\data_spatial\ke\mapdata\politic\ke_keadmin1_line_adds.shp). With the layer added, from the DIVA-GIS menu select: Tools – Projection.

In the Projection window select the Kenya border file as Input (default if it is the active theme in the Legend). Select an Output file name, there is a small sample dataset projected to Universal Transverse Mercator (zone 36 – the standard zone for Kenya) in the dataset – under the folder \data_spatial\ke_utm36\mapdata\politic. Add the ending utm36 to the output file, and put it in the sample folder. Make sure the tab for Standard is selected in the Projection window (see picture below), click the drop down menu for Category and select UTM – 1983. Then click the drop down menu for Type and set it to Zone 36. The standard projection parameters for UTM 36 will be shown in the Projection window as illustrated below.

You only need to click the Apply button,

, and DIVA-GIS will project the layer for you. The new layer will be automatically added to the Data View, but if you check the checkbox for the projected layer, it will not be seen. To see it make it the active theme and click the Zoom to Theme tool,

. The Data View zooms to the projected layer, but you might not see much difference between the layer with geographic coordinates and the projected layer.

But look at the lower left of the Project window, and you will see that the coordinates have changed.

You can check the coordinates and scale out by toggling between the projected and the geographic layer, most easily done by using the Zoom to Previous Extent,

, and Zoom to Next Extent,

, buttons. Note how the Scale changes despite the Data View looking very similar. The projected data is better at depicting true distances, areas and angles (depend a bit on what projection you are using).

The standard UTM zones use the spheroid GRS80 to describe the roundness of the Earth (the Earth is a bit flattened due to its rotation, and can not be described as a perfectly round sphere, but one that has a larger horizontal radius compared to the vertical radius). More modern data, however, use the WGS84 spheroid (the difference is small as you will see, but for other spheroids that might not be so). To project the Kenya border to UTM 36, but using the WGS84 spheroid, make sure the original layer (in geographic coordinates) is the active theme, and again open the Projection window (form the menu: Tools – Projections). This time click the Custom tab, set projection to Transverse Mercator in the Projection drop down menu, and the spheroid to WGS84 in the Spheroid drop down menu, as shown below.

You then have to fill in the projection parameters yourself shown below (as you want to use the same parameters as for the Standard UTM36 zone you can toggle to the Standard settings to find the parameters by using the Standard and Custom tabs).

Click the Output button and give a name, but this time add the suffix utm36_wgs84, as shown above. Click Apply and DIVA_GIS projects the Kenya border.

Add the projected layer showing Kenya as a polygon prepared in the dataset - \data_spatial\ke_utm36\mapdata\political\ke_keadmin1_utm36.shp. Put this layer at the bottom of the Legend and turn all the projected layers on and use the Zoom to Layer tool to see them in the Data View.

If you turn the layers you projected on and off, you will see that they fit the prepared (projected) layer perfectly. Even if you zoom in you will not see any difference between the layers. This is because Kenya lays on the equator, further away from the equator the different spheroids will generate slightly different results.

You can use the projection tool in DIVA-GIS to project data from geographic coordinates to almost any projection, but you can not project data from one projection to another, or back to geographic coordinates. As all the dataset supplied on the project CD are in geographic coordinates you can however change them into your preferred projection, and then combine the supplied datasets with other data that you have available, even if they are projected.

Projecting a grid file

DIVA-GIS does not contain any function for directly projecting a grid layer from geographic coordinates to other projections. It is a rather complicated process to project a grid as you must first convert it to shape polygon, then project the polygon, then create a template to set the new grid parameters manually, and finally convert the projected polygon back to a grid. But it is possible to do. In general it would be better to do any grid calculation using the original (geographic) data, and then just extract the key information and convert the grid to a shape file. You can then apply the Select Feature tool to the exported shape file to extract only the features you are interested in and Save the selection as a shape file (see lesson 3). Then project the shape file with your selected feature as shown above. You can also use the digitizing options, and just project the digitized features to your preferred projection.

To project a grid file you must first Export it to a polygon shape file, as done above in this lesson. In the example below, the gridded land degradation hotspot layer derived from the Multi Criteria Evaluation (MCE) (\data_spatial\xx\grid\land_degrad\hotspots.grd; where xx is the name of the district) was exported to a polygon.

Exporting grids can be slow, but all the information in the grid will be converted to a shape file containing as many features as there are cells in the grid. To symbolize this vector file takes a long time as it consist of as many records as there are cells in the grid, as you can see in the zoomed Data View showed below.

To project a file is a heavy calculation, hence you should first select the records (rows) in the attribute table that you want to export. In this case you only want to export the hotspots, and other polygons can be omitted. To select the polygons identified as hotspots, you should use the Select (Query) tool,

. In lesson 3 you did a simple selection of records in the attribute table, now you should instead used the more advanced query tool. In the Select window, click the Select by Query tab. Under the Select by Query tab click the drop down menu for Field and select the VALUE field, then enter the criteria, less than “<” in this case, and enter the Value to be 0. Then click the Add button,

, and the query will be shown in the main area. You can add more conditions (using other fields) and build very advanced queries in DIVA-GIS, but here this simple query is enough. Click Apply to perform the query.

If you turn on both the themes, with the polygons on top you can see that the selected features exactly matches the grid cells identified as hotspots, as shown below.

To project the land degradation hotspot, make the shape file with valid records for hotspots the active layer (hotspots_only.shp), then open the Projection window (via the menu: Tools – Projections). Set the projection parameters in the Projection window and click the Apply button to project the polygons.

If your shape file contains many records the projection will take a (very) long time (depends on the power of your computer). The projected shape will be added to the Data View, and if you look carefully (zoom in) you will see that the polygon boundaries are now longer exactly horizontal and vertical (as they are in the geographic coordinates), but they are a bit skewed.

Before you can convert the projected polygon to a grid, you must first create a grid template for DIVA-GIS to use for the conversion. The template defines the projection, and number of columns and rows. First you have to make a copy of the original grid file you wanted to project. It is better to do it using windows explorer. Open Windows explorer, navigate to the folder with your original grid, and copy both the file with the extension .grd and .gri. Then rename both files to e.g. lmce_index_template_utm36.

You can copy the files using DIVA-GIS by first exporting the grid to e.g. Idrisi (see above) and then import the Idrisi grid again to DIVA-GIS, but give it a name like utm36_template.

You now have a copy of the original grid file, and you must turn it into a template for your new projection. You have to do a bit of editing using any text editor. The best option is to use the Notepad text editor that comes with Microsoft windows. To open it go via the Windows desktop: Start – Programs – Accessories – Notepad. In Notepad go via the menu: File – Open, and navigate to the folder were you saved your template for UTM36 (i.e. the copy of the original grid file that you want to project). In the bottom of the Open window from Notepad change File of Type: All Files, as shown below. Then open the *.grd (the GRD file contains the header for the grid data file, which has the same name, but with the extension GRI, you can see both files in the Open window below).

Open the *.grd file for the template that you created (the file with suffix utm36). You will then see the content of the standard DIVA-GIS header file for grid data., also shown below.

The content of the header file is described in detail in lesson 5. You now have to edit the header manually in order to turn it into a template for the new grid you want to make. You must change the projection, extent and resolution of the header to reflect the projected data.

With Notepad still open go back to DIVA-GIS and open the Properties window for the projected shape file, e.g. by double clicking it in the legend.

The third row of the properties window contains the information you want: the extent of the projected grid,

. The four numbers refers to MinX, MaxY, MaxX and MaxY in the projected file. Copy those numbers to the grid header you want to use as a template. You can use the windows copy – paste functions to do it (e.g. Ctrl-c to copy and Ctrl-v to paste). Then also make the following changes in Notepad to reflect the new projection.:

Finally you have to calculate the resolution of the template grid in the projected units (meters in this case), as

Below is an example of how to do it using Microsoft Excel, also showing the formula you have to enter.

Enter the calculated X-resolution and Y-resolution into the GRD file using Notepad. Your template is now finished, hence save the edits you did in Notepad.

When you have created a template you can convert the projected polygon vector back to a grid file. Make the projected hotspot file (hotspots_utm36.shp) the active layer, and go via the menu: Data – Polygon to Grid. In the Polygon to Grid window, Input file is defaulted to the active shape file (otherwise click the Input File button and select the correct file – the projected hotspot polygon layer). The field to use for grid values is the VALUE field. Then set the Existing Grid for Parameters to the template that you just created, and give a logical Output Grid file name (hotspots_utm36.grd in the example below). Click apply to convert the hotspots back to a (projected) grid.

Projecting a shape file can be done very exact, but projecting a grid always leads to a loss in geometric accuracy (true for all GIS software). In the example below you can see the difference between the projected shape file and the projected grid file (the symbolization of the grid was done with the option Read from File, that you used in lesson 4). There is a small mismatch between the shape and grid layers that was created first by our definition of template, and secondly by the fact that the polygons are not perfectly straight (horizontal and vertical) when projected. Nevertheless you now also know how to project grid data to almost any projection using DIVA-GIS.

Dataset	Folder and file
Kenya	\data_spatial\ke\ke.apr (ke_nogrid.apr)
Mbeere	\data_spatial\mbeere\mbeere.apr (mbeere_nogrid.apr)
Mwingi	\data_spatial\mwingi\mwingi.apr (mwingi_nogrid.apr)
Narok	\data_spatial\narok\narok.apr (narok_nogrid.apr)

Extension	Function
Afrcver.avx	customized functions for the Africover land cover layer data (in the folder \data_spatial\ke\rs\landcover\ke_ke-spatial-agg.shp).
JFIF.avx	for handling jpg images in ArcView
mrsid.avx	for handling mrsid images in ArcView
tiff.avx	for handling tiff images in ArcView

Original entry	New entry
Projection=GEOGRAPHIC	Projection=UTM36
Datum=	Datum=WGS84
Mapunits=DEGREES	Mapunits=METERS