Friday, August 12, 2016

Parcel Data – Who Aggregates and Distributes It?

Generally state agencies, some regional agencies, and for a fee, a plethora of data vendors. 

The first step is aggregation, which is the process of taking many locally developed parcel data sets and standardizing each to a common format and in some rare cases providing quality control and/or spatially reconciling at the boundaries.  A national standard for parcel attributes exists, (http://nationalcad.org/CadStandards/CadStand.html) but as with locally produced data, each state has its own standards to meet its state business needs.  A recent review of state parcel standards found over 20 states had developed state parcel publication standards with many common attributes but no commonality of field names, types, or lengths.  In all states reviewed the state aggregated data did have data definitions and was easier to understand and interpret than individual producer data sets.

The tools used to build aggregated data sets range from brute force to Safe Software’s FME, Esri’s Community Parcel tools, and customized state specific tools.  Most states are either using or moving to web based processing for aggregation. 

Update frequency is typically annually, some are twice a year, and a few are daily or continuous updates. 

Some of the nuances and challenges for data aggregation are described in this article (http://www.esri.com/esri-news/arcnews/winter16articles/making-local-parcel-data-open-at-state-national-levels). 

Distribution has typically been zip file download and a web based viewer.  Files may be a single statewide file but often is individual files for each data provider, such as each town or each county.  A noticeable trend for aggregated and distributed parcel data is the use of feature services. Many national parcel data in federal agencies require a data download to incorporate information into agency systems, but even federal agency applications are increasingly using feature services.

Paul Ramsey presented an intriguing twist to data aggregation and distribution in 2015.  (http://s3.cleverelephant.ca.s3.amazonaws.com/2015-ccog.pdf).  Mr. Ramsey discusses relevancy in terms of frequency of use.  If data are not used it is less relevant than data that are used.  Let’s go with the parcel data are important and has many uses, and the parcel data must be relevant.  It must be available to be used and recognized as a useable source to be relevant.

The Moment of Opportunity, as described by Mr. Ramsey, is that small window when data (parcel data) can be provided in a way that developers can easily harvest and embed it in applications that can be seen and used by many on mobile devices.  An interesting implication is that data needs to be distributed in a way that the data can be used and accessed by developers, rather than focusing on end user consumption. 

This is an interesting perspective and important to consider.  As Mr. Ramsey states “it just means that governments need to accept the way that the technology ecosystem is going to want to consume their data, and change their behavior to fit. The first step is to recommit to the idea of data as a public good. If this data (parcel data) is critical infrastructure, as we believe it to be, making it available to all members of civil society, without restriction, is a basic requirement.  … Commit to simplicity in distribution. Follow the lead of NASA and publish raw data, with computer readable manifests, with stable URLs, close to the point of consumption on public cloud infrastructure”

Who distributes parcel data? Generally data aggregators, but we should all keep an eye on distributing our data in ways that will keep it relevant.


Wednesday, July 27, 2016

Parcel Data – Who Shares It?

Some local, some state, some not at all.  Remember when Google first started and the one liner about them was “if you have it, we want it”.  That’s the story for parcel data too.

There are lots and lots of cities and counties that share parcel data with attribution on a regular or periodic basis.  Some update continuously others provide quarterly or annual updates.  A recent survey of parcel producers found there are approximately 6,500 parcel producers (cities, towns and counties) in the US (see Parcel Data - Who Builds it) and of these nearly 70% provided at least viewer or REST Service access to their data and a smaller percentage (around 50%) provided free download and/or open data access. Those that do provide open and/or downloadable access cover almost 80% of the US population and parcel count.

As awesome as digital parcel data are there are certainly understandable reasons why all parcel data is not shared for everyone to have.

The vendor made me do it or proprietary formats - It’s not the mapping formats, it’s the attributes.  Many local governments are tied into real estate tax system (or CAMA) software that has the attributes locked up in either a difficult to extract or expensive to extract data structure.  In a few rare cases the mapping and attribution is managed by outsourcing and the local government doesn’t even own it’s own maps or data.

Once bitten, twice shy - Many local governments have experienced sharing their data with an application or a customer only to be criticized because the supplied parcel data did not meet expectations.   As examples “Why doesn’t this data indicate which homes are owner occupied and which are rented?” “I wanted to see a recent inspection on condition of the structures, where is that information?” I checked this data against my iPhone and the coordinates you have are wrong, I am calling my councilman.” “Where is the mortgage amount in this data set?”  Many of the concerns downstream users have can be explained if the data producer has a chance and if the recipient wants to understand.  Unfortunately, there are too many instances where the criticism gets widely distributed and the explanation (metadata) not so much. 

Show me the money - This has several variants including - We paid good money for this; we are not going to give it away and I need this revenue to fund my office.  These are all, well mostly all, good reasons.  There are many jurisdictions that run short of allocations and automation is not enough to make up the shortfall.  In the Parcel Dial Tone, I mentioned the need for more rural counties and under budgeted offices in particular to charge for data.  Sadly data sales are not likely to be a sustainable revenue stream.  It’s just the way it is.  Consider the path imagery and addresses and elevation and building footprints and so many other data sets have taken.  As technology advances the data becomes more widely and more freely available.  This is not ridicule or undervalue the resource needs of local assessors and parcel mappers.  The needs are real; we just need to find more sustainable and effective funding sources.

It’s mine - Much like parenting sometimes it just comes down to you can’t go to the movies because I say so. Period.  Sometimes this rational is clothed in privacy concerns or “what are you going to do with it?” concerns (see once bitten, twice shy).  Sometimes it’s just that so much personal dedication and commitment has been put into the data, it’s hard to let it go. It’s hard to not take ownership of it.  I have always thought that parcel data is personal.  It’s the most personal of all locally maintained data sets.  The data producer’s fingerprints are all over this data set and having a sense of ownership is certainly understandable.


Who shares parcel data? The most recent detectable trend is more and more parcel producers are sharing, especially cities, lots of cities.  

Friday, July 15, 2016

Parcel Data – Who Builds It?

Local governments mostly.  And often the most local government meaning the town, city, township, or county.  In some cases, the state prepares parcel data on behalf of the local government. Parcels and addresses are the most local government data.  Constructed closest to the citizen. 
In addition to the obvious tax assessment, real estate billing, and real estate tax management, parcels, generally tax parcels, are used many local functions like permitting, land use planning, zoning management, land banks, public lands management, and many many more. 

Parcel data aggregation combines data from multiple data maintainers into single standardized data sets.  Most parcel data aggregation occurs at state or regional government levels.  There are also many cases where counties (and their equivalents) are aggregating digital parcel data from cities and towns).  County level aggregation often requires maintaining and reconciling common boundaries for a seamless representation.  Regional and state level aggregation is less likely to reconcile common boundaries. There is an increasing trend for states to build standardized aggregated parcel data sets to support statewide value equalization, state disaster response and recovery functions, broadband access management, and many other cross jurisdictional business needs.

An inventory map of the local governments that have digital parcel maps can be found at this link (http://fairview-industries.com/USParcelData/USParcelData.html).  This is a voluntarily maintained inventory and is not positively complete and accurate but it is a good indication that about 90 % of the 150 million parcels in the US have been mapped into a GIS or automated mapping software of some type.

That is just the mapping.  Real estate valuation and tax attributes are 100% automated at some level.  Yes, there are still jurisdictions that have hand written, hardcopy individual property assessment cards, but a recent national inventory did not find any jurisdiction that hand generated real estate tax bills.  Every jurisdiction was covered by or represented in a Computer Aided Mass Appraisal (CAMA) or similarly functioning system even if the most local officials did not have the software.
The attribute data are by no means standard.  Similar attributes are named and structured differently, data are collected or entered at different times, the basis of the attributes values, and extent of attribution vary greatly.  Adding to the variation, there are well over 75 software vendors each with local installation customizations.

Local governments build parcel data and each one builds it and maintains it uniquely. 


Tuesday, July 5, 2016

Parcel Data, Who Needs it

Just about everyone.  At a recent parcel gathering there was some discussion about re-discovering which federal agencies need parcel data. 

As our computing and analysis capability expands so does our need for more granular and more detailed information. An overview of the granularity of geographic units in the US shows the increasing density and decreasing size of the unit of land about which we can compile information.

50 states
3,200 counties or equivalents
40,000 functional governmental units
57,000 7 ½ minute quadrangles to cover continental US and Hawaii
66,000 Census tracts
3.8 million square miles in the US
11 million Census blocks
133 million residential housing units in the US
(5.6 million commercial buildings)
150 million parcels
324 million people in the US

Parcels are the logical division of land for compiling information about use, value, and ownership. And who and what agencies need to know about use, value, and ownership?  Almost every agency at some level needs this data.

In 2008 the Cadastral Subcommittee interviewed 30 federal agencies to ascertain their use of or need for locally produced parcel data and for the responding agencies 16 had a clear identified need.  (http://nationalcad.org/download/Federal_Agencies_-_final.pdf)
Since 2008 as agencies have become more familiar with the availability of more detailed information and the ability to use and analyze bigger data sets (think GIS big data) has arrived, the identified needs and uses for parcel data have grown.

And what information about parcels is needed?  Other than guarded concerns about personally identifiable information (PII) the basic information needs are as originally defined nearly a decade ago (http://nationalcad.org/download/CadNSDI-Version2-Documentation.zip) and include use, value, and ownership.  Several agencies have expanded data needs, typically in more defined areas, such as urban areas, commercial parks, wildland fire interface areas, or farm program participation parcels.  Some of the extended attributes in more focused areas include, rental or owner occupied, mortgage status, tax bill payment status, farm field activity, new building activity, structure condition, and commercial lease values. 


Defining which attributes to aggregate and what standard field names and attribute types to use are not barriers to building a national parcel dial tone.  These things have all been studied and are known well enough to start.  There will be updates of course.  How many times a year does your mobile phone provider update your operating system or connection parameters? 

Who needs parcels?  There are enough needs and uses that they should just be there.

Friday, June 24, 2016

The Parcel Dial Tone


As a bit of background, I come by the telephone/communication analogies historically or maybe genetically.  The man on the left in the picture is my dad.




He didn’t invent the dial tone. He was a pioneer in implementing it for long distance dialing.  Before direct distance dial (DDD) all long distance calls were placed through an operator.  The innovation was not the dial tone; it was the toll center.  This was the ability to identify the call line that placed the call, the target, the time of day, and the length of the call so that billing could be correctly assigned.  It was all about the toll center, even if the dial tone made it all possible.

The parcel dial tone technology and data sets are already here.  It should be possible to pick up any map product and add the parcels.  We shouldn't have to go through an operator.  Granted there are still availability, access, content, source, and liability topics to be resolved, but technologically all is possible.  

The toll center is something we may not have addressed adequately.  I was struck by a presentation by Louisville Kentucky (Jefferson County KY) that one third of the revenue necessary to support the core services of their assessment department is reliant on data sales. Even further we see rural and less endowed jurisdictions that do not have sufficient resources to collect, maintain, and provide web access to their parcel data.  It is not just economics. Some of the more endowed counties are reluctant to add their parcels to the national dial tone.  

After 30 plus years the technology has advanced nicely, but the institutional data sharing and acquisition funding issues that were identified many years ago are still with us.

A little follow up.  Note from Jack Dangermond's keynote at the Esru UC 2016  http://video.esri.com/watch/5149/enabling-a-smarter-world

He had a parcel map behind it. I said, "Well, that's interesting. Where is that parcel map coming from? Is that on your desktop?" He said, "No, no, no, it's coming as a service." It turns out that the parcel map is coming as a service from L.A. County, who does about 150 updates and transactions on parcels every day. He was no longer copying the data, having to update it. He was reading in like dial tone behind his map a basemap of parcel data from the county. The city and the county are working together. They're sharing their data. As services, they're leveraging each other's transactional work in real time.