Today we'll have a look at how to turn paper map resources into electronic maps. For this, we'll be using mouse digitization (digitizing tablets are also commonly used, especially for industrial-sized jobs). Michael Reiskind used this technique in his project, to create an electronic database of houses in a study of Dengue fever in Peru (as shown in this figure).
Before getting into this new technique, however, warm up with a review of prior labs, and ArcView: create a project in ArcView for the French Flu data we'll be studying in the surveillance unit. Once we've loaded in that project, we'll throw in a wrinkle and have you try out an "ArcView extension" which allows one to calculate simple spatial statistics on the data.
Lab Objectives: at the end of this lab you should have
- loaded in the French Flu data and geography;
- loaded and used ArcView extensions (MapStat.avx, JPEG Image
Support);
NB: it appears that this extension does not work in ArcView version 3.0a. It works in version 3.1. - calculated neighbor adjacency relationships and Moran's I for the French flu data; and
- digitized point and polygon themes using the mouse and ArcView.
Lab
- French Flu Data: Review of previous lab work
UMich students: work in C:\temp. If you want to save anything to your personal space you can do it in the end. In the meantime this may make ArcView more stable, and reduce problems of storage.
In preparation for the surveillance module you need to begin to use the flu data. We will thus create a new project (e.g. flu.apr). You should have already downloaded the historical (through 2000) flu data (from the first module). Now we need to combine that with the geography of France, which is available from the French Flu data page of the GeoMed project.
Pick up the zipped shapefiles of France (w/o Corsica); uncompress this file, and add the france_wo_corsica.shp shapefile as a theme. Now you will want to add in the flu data, which you downloaded in the first lab (called flu.txt - here provided as a zipped file).
UMich students: go ahead and replace the flu.txt in your directory with the zipped file above: it is more current than the one you downloaded the in the first lab.
You must first add flu.txt as a table, then join it to the attribute table for the france_wo_corsica theme, as was done in the GIS and Spatial data handling lab.
Convert the resulting joined theme to a new shapefile: with the france_wo_corsica.shp theme active, click
Theme -> Convert to Shapefile
and save this theme as flu.shp. [This shapefile takes up about a meg of space.] ArcView will ask if you want to add it to your view: go ahead.The value in creating the new shapefile is that everything is included in one set of files with the names flu (and extensions like .dbf, .shp, etc.). It's easy to load everything rapidly into ArcView this way, or to share this information with a friend. Taking someone through the join process, etc. can be rather unpleasant.
- First Spatial Statistics
We're going to use an extension to ArcView to compute a spatial statistic (Moran's I) from within ArcView. Extensions are written, often by interested users, to augment the capabilities of ArcView. ArcView doesn't do any spatial statistics by default. The MapStat extension was created by Dr. Daniel Griffith, of Syracuse University, one of the consultants on the GeoMed project. We will download the zip file and extract MapStat.avx to the C:\temp directory (this directory is important! ArcView looks in this directory by default for any new extensions). Download and extract this file now.
Now let's practice using some spatial statistics with the flu.shp theme which you just created. By double clicking on its legend you will enter the legend editor: choose "Graduated Color" as your Legend Type. For Classification Field choose variable 199951 (the 51st week of 1999), and click the Apply button.
We now need to load the MapStat extension:
File menu -> Extensions -> MapStat
A new menu item ("Statistics") should appear on the menu bar of the View of the French data.Try two of the options:
UMich students: DO NOT run the Local Moran's I - it's been crashing computers on this data set....
- Neighbor: clicking on this option will lead to a file
(neighbor.txt by default) with the neighbor relationships in it. You need to
"Select a field for identification": for that you should choose
Station. Take a look at the neighbor.txt (open it with Notepad, say): can you check that the regions are correct?
- Global Moran's I: this will compute Moran's I for the region (select the variable 199951), and provide a Z-Score for the significance of the result. Does this appear reasonable?
- Neighbor: clicking on this option will lead to a file
(neighbor.txt by default) with the neighbor relationships in it. You need to
"Select a field for identification": for that you should choose
Station. Take a look at the neighbor.txt (open it with Notepad, say): can you check that the regions are correct?
- Digitizing
The maps we are using in this lab were scanned in from books. Scanning is a skill worth acquiring, as you may wish to scan and digitize something of your own someday (perhaps even today!). Scanning text is also sometimes very useful (e.g. some of the glossaries on the GeoMed site were scanned in, and the text retrieved using OCR (Optical Character Recognition) software; and the next module includes a table of data read in this way).
UMich students: PHISA offers short courses in scanning. Check one out! The PHISA library multimedia center has scanners that you can use anytime their facilities are open.
- Two images are needed for the rest of this exercise: snow.jpg
contains polygon and line information, while smallpox.jpg
contains point and polygon information. If you have some other image
that you want to use instead of these, that's fine! Go ahead, and just
replace the names of these file with your own.
If you're going to use the default images, download them (from the browser) by right-clicking on the links to the images above - you'll be offered the option of saving the links as....
- Now that you have your images, here's what to do: launch ArcView, first of all, and start a New Project (you will decline their charitable offer to add data for you - we're going to do that the hard way!). You should end up with a fresh View1 open.
- You need to load an "extension" to allow ArcView to import jpeg files (by default, it only handles tiffs). Make the project window active, then click on the File menu -> Extensions -> JPEG Image Support. Now we'll be able to import your file.
- Add image smallpox.jpg to an empty view. You can do this from the View menu item: click on the Add Theme item, and you will be offered the choice of themes you want to add. In the dialog window that appears you will see Data Source Types. Rather than Feature Data Source, choose Image Data Source. Now browse to the directory containing smallpox.jpg, and it should appear as a choice. Select it, and you will see it added to your view (although it doesn't appear until you've clicked its legend's check box).
- Next, add a new theme to the view: the theme of the new digitized data. You can do this from the View menu item. When you click on the New Theme item, you will be offered the choice of themes you want to create. This depends on the type of feature (point, line, or polygon) that you want to add. If sites, point; if linear features (e.g. rivers), line; if you want to add counties, choose polygon. We'll start with a point theme (villages which show smallpox scars versus those which don't), so select Point.
- At this point, you'll want to make your ArcView window as large as
possible, to help you see better. You'll also want to Zoom in on areas so
that you can get right in the center of the points on the plot. Do that now:
zoom in on an area of points that you'll want to digitize. Notice that there
are both black filled and unfilled dots: we'll start with the filled dots.
- When you elected to create a new theme, a new icon appeared next to
the T icon - a little point, or dot icon.
This is the Draw Point icon; click on it, and you can now start digitizing. As you click on a black dot, a small dot will be drawn in; choose a good color for the new theme, so that it is clearly visible against the black dot. This way you will be able to easily see that you have digitized all the dots (haven't missed any).
If you've zoomed so far that you can't see the whole image, then as you digitize, you may want to switch back and forth between the Draw Point button and the Pan button (the little hand icon), which allows you to move the image to new areas to digitize.
- When you think that you're done digitizing all the dots, Zoom to Theme (from the View menu). Again, if you chose a good color, you should be able to clearly see that all the black filled dots are covered. Go back and digitize any that you missed. We are now ready to edit the attribute table, to indicate that these points are villages having pock marks.
- Have a look at the attribute file by choosing Table from the Theme
menu. It doesn't look very interesting, does it?! We want to add a field to
the table, which we do from the Edit menu. You might call your new field
"pox", and leave it a Number, with perhaps width 1 (since it will be binary,
0 or 1). Now we want to edit the pox column, which we do by choosing the
I-arrow button on the lower right of the button bar. We want to put a 1 in
every box, and it doesn't appear that there's any nice way of doing this so
we'll do it the hard way. Click on the first pox box. Type a 1. Hit the
enter key, and do it again. And again, and again, and ....
When you're done, run statistics on the field, to make sure that you have a max, a mean, and a min of 1:
- click on the Attribute table for the theme, and click on the variable name of the new field in that table.
- Now click on the Field menu item, and click statistics. It will report the mean (and several other) statistics for that variable.
- Now we'll go back and add in the unfilled dots, corresponding to villages without pox marks. Back to view, zoom to an area, choose the Draw Point button, and away you go! By default the table values will be zeros, so we don't need to worry about our table any more. When you have digitized all the points, then you should choose Stop Editing from the theme menu.
- At this point, you can change the symbol used to display the villages by double clicking on the legend. Use Graduated Color instead of Single Symbol, and choose pox as the Classification Field. Choose nice colors for the symbol (by double clicking on each symbol), apply the result, and you should be looking at a nice digitized data set, created from the smallpox.jpg file.
- You can have a look at the data set in a different GIS (GRASS) via
WebGIS. Select
- the smallpox raster map,
- the smallpox vector map,
- the smallpox site map, and
- the smallpox region.
I created this GRASS-based data because I was frustrated when I digitized the smallpox data as you just did and couldn't figure out how to look at the coordinates of the points I created. They're stored in ESRI's proprietary shapefile format, and I couldn't figure out how to get ArcView to show the coordinates in the attribute table (I've since discovered that there is an ArcView Script to add coordinates to the attribute table). So I converted the shapefile to an Arc export file, then imported that into GRASS (which is a free, public domain GIS).
- Now let's move on to polygon themes. We'll create a polygon theme for
the snow.jpg file. You may want to start a new project, saving the old one
(as well as all the shapefiles) to an appropriate directory.
Follow the instructions for adding an image theme, only this time add snow.jpg. Once you have it loaded you will again create a new theme (let's call it snow.shp), only this time it will be a polygon theme.
- Notice that, where before you saw a point icon, you now see a
rectangle icon:
Click (and hold!) the rectangle button. You have your choice of what sort of polygons you want to add, and we'll only be adding two types. The first one to choose is the odd polygonal shape following the circle:
- We'll use that to make our first region. Start in the upper right
corner with the large area labelled ND (again, you'll want to zoom in so
that you can see what you're doing). Click on the corner and click again on
the northwest tip. Now drag the mouse down a little, and you'll see a
region getting traced out. Click as the mouse winds along the edge of the
area, and when you get to the southeastern tip, click twice rapidly. A
region should be displayed.
Beware spurious rapid double clicks! That signals "all done". I try to consciously pause a bit between clicks as I move.
- From now on, we want to create areas that adjoin this first region
(or other regions), so we need to switch tools: go to the rectangle button
again, choosing the last button (the one on the bottom):
This cryptic symbol means that you're going to add an adjoining region. To do so, let's work on the 37/103 region next to the ND region. The strategy is to click inside the ND region, then click on a point where our new region will join the ND region (the northeast corner is good). Then click on the northwest corner, and descend the winding region edge, clicking as you go. If all goes well, you'll click all the way back to the first region. Go a little bit into the first region, double click, and the second region should magically appear.
- Again, we probably should add the data as we go (i.e. the 37 103 part). Add two fields to your table, Population and Deaths, and edit them to incorporate the data (you'll need to choose some missing value indicator).
- You can start in one region and end in another. So if you wanted to
next digitize the 31/298 polygon, you could start in the 37/103 region near
the northwest corner of 31/298, and end in the ND region.
Keep this up until your hand hurts, until time runs out, or until you run screaming from the lab shouting "I never want to see another mouse again as long as I live!": then quit.
- Just for fun!
- You might try some geocoding
via the internet. This is one way (a very painful way!) of collecting
information on the spatial location of an address (Census information
and Latitude/Longitude of the Tract/BNA). Try it with some addresses
you know, to see if it gets them right!
UMich students: as of this lab - approximately Wed Jan 26 14:58:53 - the Federal site which provides the data is down, so the geocoder will not work.
When you get down to the tract level, you could consult the Census TIGER files to find the lat/longs of the centroids of the tracts (these too are available from the CIESIN site) - but this would be a lot of work. I'm sure that soon (if not already) some good soul will put a really good full address -> lat/long geocoder on the web. Right now this one is the best we've got!
- Making a Black/White (BW) plot
snow.jpg has the same geographical structure as the map and data set that was used for the BW join statistics example that you saw in your reading and lecture. If you finish this map, then you could create the BW map by adding a rate variable (derived from Population and Deaths), and coding by above and below the median. Here's one way to do it:
- click on the Attribute table for the areas theme, and click on the rate variable you created in that table.
- Now click on the Field menu item, and click statistics. It will report the mean (and several other) statistics for that variable. (Unfortunately, ArcView doesn't report the median - sheesh!)
- You now want to edit your legend, reducing the number of classes to 2 (use the Classify button), and then setting those classes to be from 0 to the mean, and from the mean to some value exceeding all values in the table. To do so, double click on the legend. You can then double click on the symbol (to change one symbol to black, and the other to white), and on the Value of that symbol to set the ranges the way you want.
- You might try some geocoding
via the internet. This is one way (a very painful way!) of collecting
information on the spatial location of an address (Census information
and Latitude/Longitude of the Tract/BNA). Try it with some addresses
you know, to see if it gets them right!
- Don't forget to evaluate this lab.
Principal Readings:
- Jacquez, G. M. 1998. GIS as an enabling technology. In GIS and Health (Eds. A. C. Gatrell and M. Loytonen). Taylor and Francis.
- Pages 33-35 in Cliff, A. D. and P. Haggett. 1988. Atlas of Disease Distributions: Analytic Approaches to Epidemiological Data. Blackwell, Ltd., Oxford, UK. (for BW-join statistics)
- Jacquez, G. M. 1994. Chapter 4: The Statistical Frame of Mind. In: Stat! Statistical Software for the Clustering of Health Events. BioMedware Press, Ann Arbor.
- Pages 241-242 in Sokal, R. R. and F. J. Rohlf. 1983. Biometry. W. H. Freeman and Company, New York. (multiplicative inequality)
- Pages 5-51, 5-52 in Jacquez, G. M. 1994. Chapter 5: Which test do I use?. In: Stat! Statistical Software for the Clustering of Health Events. BioMedware Press, Ann Arbor.
- Aickein, M. and H. Gensler. 1996. Adjusting for multiple testing when reporting test results: The Bonferroni vs. Holm methods. American Journal of Public Health 86:726-728.
- Thomas, D. C. 1985. The problem of multiple inference in identifying point-source environmental hazards. Environmental Health Perspectives 62:407-414.
- We're digitizing our own!
- The next step: using spatial statistics with these lab project files:
- Two chapters ( Chapter Seven - Evaluating the Visualisation of Disease Data - and Chapter Eight - Toward a Visualisation Framework for Epidemiology) from a thesis on the visualization of disease data (a good intro to some techniques that are available for displaying info in GIS: The Visualisation of Disease Data: An Evaluation).
- Making maps easy to read.
- The mathematics of Cartography
- An on-line course in Graphical Data Analysis, including source code for the examples presented using S-PLUS, xlispstat, and R.
- S-PLUS home page.
- R home page. R is a knock-off of S-PLUS. Here's a great R site for spatial spatial methods.
- Spatial Modeling and Visualization from the University of Illinois, including a link to spatial interpolation with lots of interesting examples of a variety of techniques.
- The XGobi home page; a Windows port of XGobi; and XGobi linked to (the UNIX version of) ArcView. Note: the Windows port still requires an X Server.
- How about a description of projection pursuit? This is one option in XGobi (the Tour button), although we won't pursue it (ooh, sorry!). Dianne Cook has a very interesting paper illustrating the method, entitled Calibrate your eyes to recognize high-dimensional shapes from their low-dimensional projections in the electronic journal Journal of Statistical Software.
- Color Space demonstration. Very interesting way to understand many aspects of color (hue, saturation, etc.) and color mixing.