# Presentation: 895624

## Use of GIS for Drawing Samples

**Use of GIS for Drawing Samples**

**May 3, 2007**
**Andrew Gilbert**
**Allan O’Connell, Jr.**
**Patuxent Wildlife Research Center**
**U. S. Geological Survey**

**Notes:**
This presentation will feature a program designed to generate a randomized systematic sample.
The program was developed for ArcGIS and can be loaded into ArcMAP as a special tool.
This program was developed in support of inventory and monitoring work for the NPS  (mammals and long-term stream monitoring) as well as support for site occupancy models developed at Patuxent. 

## Geospatial data

**All GIS related projects rely entirely on geo-referenced data**
**The first step is finding the spatial data or creating it if it is not available**
**Paper maps**
**Topographic maps, tax maps, road maps, surveys, etc.**
**Digital maps**
**Digitized versions of above**
**New integrated products such as the National Map**
**Increasingly data is collected digitally when produced resulting in better quality, more available digital maps**
**Aerial photographs/satellite images**
**Ortho-rectified (made true to the ground) aerial photos, satellite images**
**Must be rectified to be useful with other geospatial data**

**Notes:**
One of the MOST critical steps in the whole process is coming up with reasonable data with which to designate sampling strata

Many state and federal agencies now provide data online free of charge for use. Make use of as much available data as you can.

Some example sites are listed here.

## Spatial data resources

**Most states now have GIS data servers for maps **
**Maine office of GIS: apollo.ogis.state.me.us/catalog/**
**Maryland: dnrweb.dnr.state.md.us/gis/data/**
**Hawaii: www.hawaii.gov/dbedt/gis/**
**State GIS data clearinghouse: libraries.mit.edu/gis/data/statecenter.html**
**Federal GIS data servers**
**USGS – National Map: nationalmap.gov/**
**FWS – National Wetlands Inventory: wetlandsfws.er.usgs.gov/**
**NOAA – Observing Systems Architecture: map.ngdc.noaa.gov/website/nosa/viewer.htm **
**NPS – Data clearinghouse: www.nps.gov/gis/data_info/**
**Private entities **
**Survey companies**
**NGO’s such as The Nature Conservancy, The Audubon Society, etc.**

## What if the data is not available electronically?

**Search and then search again!**
**Much data are available electronically or is becoming available electronically **
**However, data are not always served online if not approved for public distribution or if it only resides on a local computer **
**Ask around, contact the likely sources of the data**
**As a last resort and only if you must – digitize**
**This is very time-consuming and costly but is sometimes the only way**
**This often requires more sophisticated equipment**

**Notes:**
Keep looking if data is not found initially – in some case the data is not served online but may be available elsewhere.

Don’t resort to digitization or other means of getting data unless you absolutely have to

## Bringing the data together

**Theoretically a point on this earth is in the same location no matter what system you use to describe where that point is**
**Loading data layers into a map and checking for consistency will often make you think otherwise**
**Coordinate system errors most common**
**Check for UTM (measured in feet or meters) vs. lat-long  (measured in degrees) coordinate systems**
**Check the datum (geographic reference point)**
**e.g., NAD 1927, WGS 1984**

- A shrunken habitat layer? – a datum error!

**Notes:**
Bringing data together can sometimes be a challenge.

Differences in coordinate systems and lack of information about coordinate systems used for data layers can lead to problems with displaying data properly.

## Coordinate systems

**Coordinate systems defines how the numerical coordinates of a map relate to true coordinates on the earth**
**Project from one coordinate system to another – ARC does this automatically in the map if the coordinate system for the data is defined correctly in the first place**
**Metadata should specify exactly what coordinate system was used for the data,  but don’t trust it explicitly **
**Always check the data against the metadata file and known data sets if possible to correct errors before processing geographic data**

- A much better fit...

**Notes:**
Know what coordinate systems you are working in – and what your data sources are based on.

Metadata files should tell you this information, but it is not always accurate.

The most common problem seems to be Lat/long coordinates being displayed as UTM coordinates – check the values displayed – do they make sense for the projection you are using. 

Once you have the coordinate systems properly defined – ArcMap should make the necessary changes even if there  are differences  among layers.

## Change projections – spatial reference

**Change projections – spatial reference**

**Use ArcGIS 9.X toolbox to correct coordinate system errors**
**Sometimes layers come without projection information included and must be defined**
**Use Data Management – Projections and Transformations – Define Projection**
**This can also be used to modify projection parameters for a data layer if they are incorrect**

**Notes:**
One last thing.  Remember that a GPS has a predefined coordinate system as well and even if you have your sampling points, you need to make sure that that the coordinate system and datum match what you generated from the GIS maps in order to find them on the ground.

## NPS GIS Data Standards

| Datum | North American Datum 1983 |
| --- | --- |
| Spheroid | GRS 1980 |
| Units | Decimal degrees |

**Park unit standard**

| Projection | Universal Transverse Mercator |
| --- | --- |
| Datum | North American Datum 1983 |
| Spheroid | GRS 1980 |
| False Easting | 500,000 |
| False Northing | 0 |
| Units | Meters |

**NPS GIS Data Standards**

## Drawing samples using a randomized stratified sampling design program

**This program will generate random, systematic samples for an area or a set of linear features **
**Uses ArcMap 9.X **
**The program was written for Personal Geodatabase (GDB) features and not shapefiles or ARCInfo coverages**
**Preferred type of GIS data type because...**
**There is more you can do with it **
**Is portable and doesn’t require exporting/importing  **
**Is fully relational**
**Shapefiles and coverages must be converted to GDB features**

## Required data layers

**Boundary  - or some data layer with the geographic extent that you wish to sample**
**The program will use this layers extent to define the area for sampling**
**The extent is a rectangle that would enclose the entire geographic data layer**
**If you wish to develop samples for only one region of an area within a GIS layer, export this area to a new boundary file for use, or assign this area a different strata (more later)**
**Strata data layer **
**Elevation map**
**An amalgamation of different data layers**
**E.g., elevation, slope, soil type**
**Strata are fixed once set, be careful how you choose them**
**Must have a text field with the type of strata or habitat assigned**
**Linear features layers with defined strata if required**

**Notes:**
The grid generation program will use the extent (lower left coordinates and upper right coordinates defining the rectangle that includes the geographic data) to develop the grid. 

The grid has a random start +/- the size of the specified grid cell X units on a side.

The habitat/strata data layer will be used to assign each grid cell a habitat/strata association for later drawing samples
	- This can be a veg map, a combination of veg and geology layers, or any layer defined by the user as long as it has a text field called habitat with uniquely assigned strata

## How does this program select points?

**Place a dense grid over the area of interest**
**User assigns strata to grid cells**
**Select a strata to sample**
**Systematically select grid cells for each strata to place points based on a random starting cell**

## Selection within a strata

**All cells (or linear features) within a stratum are arranged “linearly” from one corner of the park to the other**
**The sampling frame is determined from the total number of cells within a stratum divided by the number of sampling points desired**
**Points are assigned to a cell based on the frame length from a random starting cell (up to the length of the first frame)**
**Sampling linear features works similarly except that the lengths of the features are used in place of grid cells**

## The sampling program files

**SystematicGrid.dll and Add_sample_points.dll**
**These programs work within ArcMAP 9.X**
**SystematicGrid.dll generates a randomized grid having random and center points identified in each cell (this can be used alone for other applications)**
** ****AddSamplePoints.dll uses the generated grid to draw samples, creating a separate sample points dataset in the geodatabase**
**You must have administrative rights to install these in ArcMap (or have your IT administrator install them)**

**Notes:**
These programs are executable files that can be loaded into ArcMAP 9.X. The method for loading these will be shown in the following slides.

Grid.dll creates a randomized grid

add_sample_points.dll draws sample points using a systematic scheme with random start and adds points from each strata to a new sample point file

## Create a custom toolbar

**Create a custom toolbar**

**If you wish to have the toolbar present in any map you open, save it in the Normal.mxt file (the normal template) otherwise it will available only in the current map layout**
**Click OK, the new toolbar will open in the window and can be dragged onto the toolbar area with the other toolbars or dragged anywhere else on the screen**

**Pull down the tools menu and open the customize dialog box**
**In the toolbars tab, click the new button**

**Assign a name to the new toolbar (e.g., Sampling Design) **

**Notes:**
Create a custom toolbar.
Pull down the tools menu and open the customize dialog box.
In the toolbars tab, click the new button.
Assign a name to the new toolbar. 
If you wish to have the toolbar present in any map you open, save it in the Normal.mxt file (the normal template) otherwise you may specify the toolbar to be only available in the current map layout. 
Click OK, the new toolbar will open in the window and can be dragged onto the toolbar area with the other toolbars or dragged anywhere else on the screen. 

## Add the programs to the toolbar

**Note: you must have admin rights to add programs**
**Open the custom tools.**
**Click on the commands tab to open the new tools to add**
**Click the Add from file... button**

**Browse to the location of the two program files: CreateSystematicGrid.dll and AddSamplePoints.dll **
**Select one file than the other to open. Each time you select a file, the “Added Objects” dialog box (see left) will appear when successful **
**Click OK **

## New commands added

**A new category will be created “Sampling tools” and the tools will appear in the “Commands:” area of that category**

**Drag the installed tools: “Add Sampling Points” and “Create sampling grid” from the “Commands:” area to the newly created toolbar to be able to use the tools **

## Newly added tools

**Newly added tools**

**Note: the Add Sampling Points icon      will be grayed out until a data layer is added to the map layout in data view. This will also be grayed out in layout view.**

**Notes:**
The newly created tools are now visible and ready for use. Except the Add Sampling Points icon will be grayed out until a data layer is added to the map layout in data view. This will also be grayed out in layout view. 

## Intro to create systematic grid program

**Intro to create systematic grid program**

**Create Systematic Grid        will create a sampling grid X units on a side covering the area of interest. **
**The grid measurement units will depend on the measurement units of the map**
**A grid of cells is initially placed over the entire extent of the date layer used as the area of interest with a randomized start +/- the length of one grid cell in both X and Y directions  **
**The number of grid cells are reduced to cells having their centers within the polygon representing the area of interest (e.g., a park boundary)**
**Each cell has both the center coordinates and a random location within each cell as an attribute of that cell for use for sampling**

## Add a boundary or other extent layer 

**Add a boundary or other extent layer**

**First add a data layer     to the ArcMAP for which you want to generate a sampling grid **

**In the example the red outline is the boundary for Saint-Gaudens National Historic Site.**

## Open Create Systematic Grid dialog box

**Clicking on the create grid icon      will open the grid creation dialog box**
**The dialog box asks you to select the layer that grid will be created for (e.g., the park boundary) as well as the dimensions of the grid cell (in map measurement units) **
**The pull down list box will show all data from the data layers to pick from**
**The boundary data layer must be in an editable personal geodatabase as a polygon data type in order for the grid to be generated **
**This ONLY works with personal geodatabase data types and will NOT work with shapefiles or ArcInfo coverages **

## How big (or small) a grid cell?

**This depends on what you are trying to measure and how many sample points you need to establish**
**Rare features may be lost at very large grid cell sizes depending on how you determine strata**
**Spatial autocorrelation may be important at small grid sizes and dense sampling schemes or many strata (species specific)**
**Larger grid cell sizes will insure greater separation between sampling locations**
**Species with very small homeranges and/or largely sedentary species may call for smaller sampling grids **

**1 km                             or            25 m  ****on a side**

## Generated grid cells

**Generated grid cells**

**The grid (orange cells) is generated by the program and saved as a data layer within the personal geodatabase that the boundary layer is saved within  **
**The grid is automatically added to the map**
**Some grid cells have areas outside of the boundary and some areas of the park do not have grid cells  (i.e., not all of the park is sampled)**
**You can add multiple grids at a time or redo a grid  **

**You will be asked if it is ok to delete the old one if you choose a grid size that has already been created. **

**Notes:**
Only the grid cells with the centers falling within the park boundary will be selected. This assures that most of the grid cell will have their areas within the boundary of the park.

	- Cells at the edges of the park will have some portion outside the park and may result later on in points being found outside the park if random points are selected (vs. center points)
	- Some areas within the boundary may not have any grid cell coverage
	- This is more of an issue for smaller parks as with this park of 165 ac.

## Grid attribute table

**Grid attribute table**

**A view of the attributes table shows the following fields: **
**ObjectID: ID created by ARCGIS**
**Shape: the data type (polygon in this case)**
**Centroid_X, Centroid_Y: point locations for the center (centroid) of each grid cell **
**Random_X, Random_Y: random point locations within each grid cell**
**GridID: a unique value for each cell**
**Strata: a text field for specifying strata  **
**Shape length: perimeter length of the grid cell**
**Shape area: area of the cell**

**Notes:**
The attribute table for the newly created “sampling_grid_X” layer shows you what fields are created.

	- GridID is a unique number assigned to each grid cell allowing future linkage between the habitat and grid layer
	- This layer can be used alone for sampling as point coordinates are already given in this file

## Strata data layer

**Create or use a pre-existing strata data layer**
**Strata can be:**
**Habitat/community types**
**Soil type**
**Wetland drainage size**
**Accessibility (e.g., distance to road or trail)**
**Cost to sample an area**
**Stream order**
**Proximity to road**
**Some combination of any or all of these, etc.**

## Example strata layer

**Example strata layer**

**This is a habitat/community type map for Saint Gaudens NHS**
**Each polygon represents a contiguous community type which will become our strata in this example**
**How do we use this to develop our samples?**
**Assign each strata to a grid cell and use “Random Systematic Sampling” tool to generate sample points**
**Assignment of strata is up to the user.  The strata field can be edited and assigned manually, or assigned using ArcMAP or another program**
**Example follows using MS Excel to assign strata to grid cells**

## Clip the habitat/strata layer to the grid

**Clip the habitat/strata layer to the grid**

**In ArcGIS 9.1 use the toolbox to access the “clip” tool within Analysis tools - Extract**

## Clip to the grid

**Clip to the grid**

**Specify **
**input features to clip (habitat layer)**
**clip features (the grid)**
**output feature class**

## The clipped habitat layer

**The clipped habitat layer**

**Notice that because the habitat layer did not extend beyond the boundary, some cells will not be well classified**
**Extend strata classification, exclude cells, or ignore?**

## Intersect the clipped file with the grid

**Intersect the clipped file with the grid**

**In ArcGIS 9.1 use the toolbox to access the “intersect” tool within Analysis tools - Overlay**

## Intersect the habitat and grid layers

**Specify **
**input features to intersect (clipped habitat and habitat)**
**the output feature class**

**Intersect the habitat and grid layers**

## Intersected habitat features

**Intersected habitat features**

**Habitat features are now split in each grid cell in the new intersected layer**
**The habitat-grid intersect layer references the old grid cell ID and habitat (strata)**
**Use these to assign strata to each grid cell**

## Export the data as a text file

**Export the data as a text file**

**Export the data from the attribute table**
**Options button, choose Export**
**Save as type “Text File” (DBF files can truncate data)**
**Import to Microsoft Excel**

**Notes:**
Options – export the attribute table for the intersected layer as a .txt file and import into MS Excel.

Note that .DBF files (Dbase) can truncate some of the information (I don’t know why this happens.)

## Use MS Excel Pivot tables to summarize by grid cell

**Use MS Excel Pivot tables to summarize by grid cell**

**Save as an Excel file and create a pivot table using the wizard as shown**
**This will allow you to sum the area of each strata type in each grid cell to determine the dominant type**
**The dominant type (by percent of area) can be used to assign strata to each grid cell**

## Select the entire dataset for import to the pivot tables.

**Select the entire dataset for import to the pivot tables.**

**Put the table in a new sheet.**

## Drag the gridID field to the rows

**Drag the gridID field to the rows**
**Drag the habitat type to the columns**
**Drag the shape_area to the data section**

**Notes:**
Drag the gridID field to the rows
Drag the habitat type to the columns
Drag the shape_area to the data section

This will summarize over the area by each habitat type and for each grid cell.

## Slide 36

**Notes:**
The generated table with areas in square meters for each strata by gridID

## Use the max function to find the habitat having the largest area in each grid cell.
Use the equation as shown to match this value to habitat.
Save this worksheet as a text (.csv) file 

**Use the max function to find the habitat having the largest area in each grid cell.**
**Use the equation as shown to match this value to habitat.**
**Save this worksheet as a text (.csv) file **

**Notes:**
Use the max function to find the habitat having the largest area in each grid cell.
Use the equation as shown (fx = ...) to match this value to habitat.

Using this function you are effectively assigning the habitat in greatest area within each grid cel
	- This function could be anything of your choosing.

Save this worksheet as a dbase (.dbf) file.
	- You must save as a dbase file now in order to import back into ArcMAP.

## Join the newly created table to Sampling_grid_100

**Join the newly created table to Sampling_grid_100**

**Right click on the grid layer**
**Select Joins and Relates – Join**

**Notes:**
Right click on the sampling_Grid layer to join the newly created table. 

## Join dialog box

**Join the summary .csv file to the grid layer by the gridID**
**This will allow you to copy the assigned habitat (strata) to the strata field of the grid layer**
**Use the “Field Calculator”, accessed through the attribute table**

**Notes:**
Choose the GridID field to join by. Join the new .csv table with habitat assignments choosing again the GridID column to match to. This will add another set of fields in the attribute table of the sampling_grid_X layer which belong to the .csv file.

## Use the Field Calculator to add strata attributes

**Use the Field Calculator to add strata attributes**

**The strata field (blank normally) in the Sampling_grid_X layer can be assigned a strata based on the joined pivot table**
**Use the field calculator to do this easily**

**Notes:**
Copy these habitat assignments using the “Field Calculator”
	- Start an editing session if you want changes to be reversible – otherwise changes are permanent.
	- Select the Strata Column in the Sampling_grid_x and set equal to the Habitat column derived from the newly joined table. 

## Random Systematic Sampling Tool

**This program will draw systematic samples for the previously generated grid cells but has also been adapted for systematically drawing samples for linear features as well**
**The tool uses previously assigned strata for either the grid cells or linear features **
**All cells may be within a single stratum (unstratified design) or there may be as many strata as there are cells (complete census?)**
**Inputs:**
**Feature type (grid or linear)**
**Sampling layer**
**Field containing strata information**

## Adding sample points

**The program generates a list of all strata and the number of features (cells or lines) within that strata **
**To draw samples for a stratum:**
**Click on the strata type to highlight it.**
**Enter the number of points desired (up to the number of features)**
**Select either random or center points to generate**
**Click done**
**Repeat steps 1-4 for each stratum desired.**
**The program will create a point feature class layer called “Sample_points”, if it does not already exist, and append sample points to it.  **
**The layer will automatically be added to the current map. **

## Sample_points file warning

**Sometimes when adding additional sample points, you will get this warning**
**The domain of the automatically created Sample_points layer is not large enough and will not allow additional points to be added**
**A new layer is created with a larger domain, but any additional fields added by the user will be lost**
**Save this layer to a new data layer if you would like to keep these before overwriting the file**

## Samples picked!!

**Samples picked!!**

**Notes:**
The new sample points layer is created and displayed.

You can add as many points as you like to this file. It will prompt to add or delete sample points.

## Delete or append samples?

**Delete or append samples?**

**The program will ask if you want to delete the samples and start over or append to the samples layer**
**Additional samples appended in this example**
**Notice that some points fall outside of the boundary**
**If grid cells outside the boundary, then some random points may be too**
**Center points always in the boundary**

## Sample points attribute table

**Sample ****points**** attribute table**

**The sample points attribute table contains point information and their coordinates that can be exported as a text file for uploading to a GPS or printed, etc. **
**The sample points layer can be exported as different data types (shapefiles, coverages, etc.) **

**Notes:**
The sample points attribute table contains the points and their coordinates that can be exported as a text file for uploading to a GPS or printed, etc. 
Sample: the unique sample number.
Strata: the strata from which the samples were drawn. 
Sample_type: automatically generated as an identifier
FeatureID: the Object ID from the cell that the point was chosen (or line the point was chosen). 
X_Coord, Y_Coord: the X and Ycoordinates of the generated sample point

## Draw sample points for linear features

**The program can also be used to generate sampling points for linear features**
**Linear features can be stratified**
**Features are selected based on total number of points desired and total length of features within a stratum**
**Can be used with any linear feature type: roads, streams, etc.**
**Inputs:**
**Feature type (grid or linear)**
**Sampling layer**
**Field containing strata information**

## Select strata and samples

**Select the stratum (in this case the stream)**
**# features is number of segments within a stratum**
**Select the number of points to draw**
**Click done**
**Again, the Sample_points point feature class will be created and added to the map if not already present (even if from creating grid samples)**
**Repeat if samples for additional strata are desired**

## Linear sample points added

**Linear ****sample**** points added**

**Sample points were added to two streams**
**Points were appended to the end of the other sample points previously drawn**

## Other programs for sampling

**ArcView 3.X programs :**
**Alaska Pack  for ArcView 3.1– NPS program**
**http://www.nps.gov/akso/gis/**
**Program Sample – ArcView 3.2**
**http://www.quantdec.com/sample/**
**ArcGIS 9.X programs:**
**AlaskaPak for ArcGIS 9.X**
**http://www.nps.gov/akso/gis/**
**GRTS sampling program developed by Dave Theobald**
**Reversed Randomized Quadrant-Recursive Raster algorithm**
**Uses ArcGIS 9.0 toolbox**
**http://www.nrel.colostate.edu/projects/starmap/rrqrr_index.htm**
**Standalone programs**
**S-Draw by West, Inc.**
**GRTS for the average Joe**
**http://www.west-inc.com/computer.php**
**Program Distance**
**http://www.ruwpa.st-and.ac.uk/distance/**

**Notes:**
Other programs exist for ArcView that are not compatible with ArcGIS 9.X, but can be used in ArcView 3.X

## Final thoughts...

**GIS is not that scary or difficult**
**Improvements in interface design has made it much easier for the lay-person to use**
**Much more thought and work must be done beforehand to conceptualize your design and determine the appropriate strata**
**Generating sampling points can be done quickly and easily**
**Contact me with program issues/suggestions**
**agilbert@usgs.gov**