Parcel Data – Who Aggregates and Distributes It?

Generally state agencies, some regional agencies, and for a fee, a plethora of data vendors. 

The first step is aggregation, which is the process of taking many locally developed parcel data sets and standardizing each to a common format and in some rare cases providing quality control and/or spatially reconciling at the boundaries.  A national standard for parcel attributes exists, (http://nationalcad.org/CadStandards/CadStand.html) but as with locally produced data, each state has its own standards to meet its state business needs.  A recent review of state parcel standards found over 20 states had developed state parcel publication standards with many common attributes but no commonality of field names, types, or lengths.  In all states reviewed the state aggregated data did have data definitions and was easier to understand and interpret than individual producer data sets.

The tools used to build aggregated data sets range from brute force to Safe Software’s FME, Esri’s Community Parcel tools, and customized state specific tools.  Most states are either using or moving to web based processing for aggregation. 

Update frequency is typically annually, some are twice a year, and a few are daily or continuous updates. 

Some of the nuances and challenges for data aggregation are described in this article (http://www.esri.com/esri-news/arcnews/winter16articles/making-local-parcel-data-open-at-state-national-levels). 

Distribution has typically been zip file download and a web based viewer.  Files may be a single statewide file but often is individual files for each data provider, such as each town or each county.  A noticeable trend for aggregated and distributed parcel data is the use of feature services. Many national parcel data in federal agencies require a data download to incorporate information into agency systems, but even federal agency applications are increasingly using feature services.

Paul Ramsey presented an intriguing twist to data aggregation and distribution in 2015.  (http://s3.cleverelephant.ca.s3.amazonaws.com/2015-ccog.pdf).  Mr. Ramsey discusses relevancy in terms of frequency of use.  If data are not used it is less relevant than data that are used.  Let’s go with the parcel data are important and has many uses, and the parcel data must be relevant.  It must be available to be used and recognized as a useable source to be relevant.

The Moment of Opportunity, as described by Mr. Ramsey, is that small window when data (parcel data) can be provided in a way that developers can easily harvest and embed it in applications that can be seen and used by many on mobile devices.  An interesting implication is that data needs to be distributed in a way that the data can be used and accessed by developers, rather than focusing on end user consumption. 

This is an interesting perspective and important to consider.  As Mr. Ramsey states “it just means that governments need to accept the way that the technology ecosystem is going to want to consume their data, and change their behavior to fit. The first step is to recommit to the idea of data as a public good. If this data (parcel data) is critical infrastructure, as we believe it to be, making it available to all members of civil society, without restriction, is a basic requirement.  … Commit to simplicity in distribution. Follow the lead of NASA and publish raw data, with computer readable manifests, with stable URLs, close to the point of consumption on public cloud infrastructure”

Who distributes parcel data? Generally data aggregators, but we should all keep an eye on distributing our data in ways that will keep it relevant.

Bounding Rectangles - A Case for Something Better

It’s 2015. Almost 2016. And the best we can do to show the extent of a data set is a bounding rectangle?  By their very nature bounding rectangles are not particularly helpful, especially for geographic footprints that are not at all a rectangle. 

Anyone who has searched for, generated metadata for, published, or tried to download geographic data has run into the bounding rectangle.  It is usually displayed in a rather small, set to the side, unusable map graphic. It certainly seems that in this day and age we could do more than that. 

The Alexandria Digital Library Project is almost 20 years old (http://legacy.alexandria.ucsb.edu).  This project described a gazetteer for geospatial data that would use a real footprint as a location index that could be embedded in any number of applications. 

What if we could select a basic set of geopolitical boundaries to register our data and the inset map would actually show the boundary of the state, county, city, or park that the data was describing or contained within.  Granted there might be issues in finding an agreeable boundary to use for a city especially with changing annexations and de-annexations and in many parts of the world the boundaries of countries are changing and evolving.  But clearly we should be able to do better than a bounding rectangle.

Maybe we could even develop a method for using a real map to search for available geospatial data. 

Imagine if the Geographic Names Information System (GNIS) (http://geonames.usgs.gov/) had more than a point for each name? What if there was also a polygon of the area covered by that name?

While it is relatively simple to link lists of names or point locations to Census geography, shouldn’t it be routine to identify the footprint of published data by the actual data extent?  This could be extended to symbolize the footprint by the date last updated and the data steward or data publisher. 

Maybe it’s time to start thinking outside the bounding rectangle box.

Aereo, ABC and Parcel Data

-->

In late April the Supreme Court heard arguments on if Aereo, a company with thousands of small (tiny actually) digital antennas, could collect programs broadcasted over the air, package them and distribute the freely collected information over the internet, for a price.  The data is compiled onto cloud storage and can be consumed for a price on any screen. 

Several news articles called this a disruptive business model  (http://freakonomics.com/2014/04/28/whats-at-stake-in-the-aereo-case-maybe-the-future-of-the-cloud/). The issue at hand was could Aereo take the freely broadcasted signals and sell them through cloud services.  The problem being that ABC and other broadcasters sell those same signals to cable and dish networks for a fairly significant amount.

In my opinion there are many elements of this case that are similar to parcel data, of course.  Many states and counties and cities produce parcel data for their internal use and then “broadcast” or provide the data freely.  As we have seen time and time again, commercial vendors assemble that free data, repackage it, and sell it.

Most recently for example, a firm sent out a notice that users could purchase high-resolution aerial photography for southeastern Ohio (www.emap-int.com).  This is the part of Ohio with the Utica Oil Shale fracking activities.  But this very same product is freely available through the State of Ohio OGROP program http://ogrip.oit.ohio.gov/ProjectsInitiatives/OSIPDataDownloads.aspx.

So why is capturing freely broadcast programming and reselling it disruptive and taking freely available GIS data and reselling it not disruptive?  Maybe neither are disruptive, maybe it’s just the American free market. 

AGOL Maps and Applications

The difference between web maps and applications (maps and apps) puzzled me at first but a little more exploring on the AGOL site has demystified this a bit.

The web map is just as it sounds, it is a base map with a set of layers that can be seen through a ArcGIS.com viewer or ArcGIS Explorer, provided you have shared the map service and the web map.

An application on the server side, on AGOL in this case, packages the map and adds some functionality and look and feel to the web map.  It also creates a single URL to the map and the added functionality so it is much easier to share and view.  The AGOL application can be viewed in any browser. 

It’s fairly easy to wrap an AGOL web map into an application.

Open the web map in AGOL and open the share option.  The share gives you a URL that links directly to the AGOL map viewer that you can share with people to view the map and the code for adding the map to an existing web site.  But you can also select Make a Web Application and chose from about 24 basic web application templates.  I have tried several of these web applications including the basic viewer, the social media, editing, and a few of the story telling web applications.  Most of the applications are configurable to some extent.

Here is a link to the state parcel stewardship that will open in the AGOL viewer 

and here is that same map in a basic viewer web application

Both links connect to maps with the same content; they are just wrapped differently.

The real deal with applications is mobile applications.  For these applications you don’t have to do anything on the server side other than have the web maps configured to be shared or edited, depending on your application.  The mobile application connects to the URL for the web map and the work is done on the mobile side.  There are some catches.  For the data collection application, I did set up a separate service so the edits would be made on a copy of the feature service and not on my original service. Also the data on the AGOL service has to be structured and configured so the mobile application recognizes the data fields.

There are a few remaining glitches.  If you are publishing a viewer to a data set that is updating frequently and you want to track status or updates on your desktop and push the updates to AGOL, make sure your services are all “awake”, sometimes there is a lag between when the service is over ridden and when the changes are reflected in updates. Another way to handle this is to do the edits on the AGOL site and then users will see changes more quickly.  You have to be wise about how you manage the sharing on the services and viewer if you want to have a few people edit the data but have everyone see the results. 

AGOL is not likely to replace your existing ArcGIS Server, but it can lighten the load.  It is a good place to set up some quick data publishing, to set up some relatively static views of the data, experiment with new or proposed data or map services, set up a story map about your organization, use web services for data collection within a defined organization, and to learn how web services work. 

There are emerging capabilities.  Such as using drop box to link to content or possibility to link to data downloads, additional templates and application offerings, expanded analysis tools like buffer and data summaries and analysis, and the ability to track credit consumption for each tool. 

The AGOL experience takes a little getting used to but it is a relatively painless way to gain familiarity with web publishing, web and feature services, and providing access to map content.  This is also a good way to gain familiarity with the use of web applications and available templates

AGOL, Web Services, and Mashups

My next foray with Arc GIS Online is to explore some the available web services that can be added to AGOL and how they might be used.  Many of the maps and web services I work with are national status maps or inventories.  The search for available services started when I wanted to add labels on county polygons for one of my web applications.

Web Services Terms

Skip this section if web services are second nature to you.  Some of the nuances among these terms are important to understand for AGOL.  WMS is a Web Map Service.  This is a completed map provided with symbology and scale visibilities, essentially providing an image of the map.  Within AGOL a single web service may have multiple layers that can be turned on and off, and have customized popups.  In AGOL these are added as A WMS OGC Web Service.

WFS is a Web Feature Service.  This service provides the actual features, not a map image of them.  Features can be analyzed, downloaded, added to a desktop canvas, and are typically provided in an extensible markup language (XML) for geography (GML).  This means the definitions of the features are provided and the features can be added to a variety of software platforms.  AGOL does not have the capability to add WFS, but links to the WFS for a feature could be added to the customized popup, if you wanted to provide information on the WFS address.

REST or RESTful Services is a Representational State Transfer (did that clarify the term?) and this is another standard or architecture for providing access to web served data and services.  In AGOL these services are added as ArcGIS Server Web Service.  The transparency and visibility ranges (minimum and maximum display scale) can be configured for Restful Services in AGOL.

Services

There are many WMS, WFS, and REST Service available to supplement your AGOL map.  I did a web search from my browser, but there is also a search engine within the AGOL map to find available services. 

These are fairly easy to add to any map, just add data from the web and select the type of service.  The service adds fairly quickly, if you pick the right type.  After the service appears on your map, click on the service in the details information on the map viewer and the layers within that service can be added and configured individually. 

Since county labels were my first requirement I started with Tiger Web. http://tigerweb.geo.census.gov/tigerwebmain/TIGERweb_wms.html

This WMS had exactly what I needed for state and county labels as well as American Indian Areas with labels.  School districts and federal congressional districts are also available along with many of the Census geography features. This service provides a lot of rich Census content as well and it is an easy way to display place names and geography with some population information.  The geography and the labels are offered separately and the labels scale nicely.  The WFS services on Tiger Web are really helpful to view your current Census geography with your local data sets on an ArcGIS desktop map canvas as well.  No need to download and re-project the Census geography files.  This is for desktop, not AGOL.

The National Map Web Services is another fairly easily navigated web site to find services. http://viewer.nationalmap.gov/example/services.html

This site has a good variety of imagery, elevation, and federally managed data sets available as WMS and WFS.  All of the links worked and the data added as advertised.

There are a lot of map and feature service available at the federal Geospatial Platform.  Finding the available WMS and WFS services is not particularly obvious at this site but a search for WMS found 26 results on the Geoplatform and 1,043 harvested items.  However, many of these did not work as advertised, some had data in them some did not, some were web services and some were viewers only, but it’s a start.  Many states also have some excellent web services.  Montana and Arkansas are two states I have used.

It is interesting to add the same data from different web service sources and see the differences in the representations.  State, county, and city boundaries have a variety of representations and all seem to be sourced differently.  I suspect as the use of the web services because more accepted there will more reliance in single authoritative representations of commonly used layers.