Stop and Smell the Data – Part(s) Unknown

Data marketplaces fill the education gap for new users of location data

Thu Sep 06 13:16:00 EDT 2018

In part one, I discussed how many businesses are feasting on location data, both the experts in geospatial technology and the neogeo’s, such as those in business intelligence who are new to the smorgasbord of location intelligence. Now let’s discuss why it’s so hard to find the data that many are most interested in acquiring and what solutions are available.

Location-based data comes in many flavors: street networks (sometimes referred to as “spaghetti” line work); points of interest; administrative boundaries and attributed data, such as demographics. In addition, there is a banquet of Earth observation satellite data available in varied spatial and spectral resolutions. Godspeed to those that attempt to digest the variants. Comprising both vector and raster data, this feast of geospatial data looks less appetizing because of the complexity of finding, then buying, the right data for the right task.

In the golden era of desktop mapping, several companies, including my own, provided data catalogs, first paper, and then digital. For those loyal MapInfo users, the several inch thick data catalog that came in the box of software was an essential part of the project workflow. While some sample data was provided with the software, users often needed more detailed, regionally specific data to compliment location analytics and map production. Today, users have higher expectations for both more variety and specificity plus immediate purchasing options.

Enter the data marketplaces. In the course of the last few years, several data marketplaces have opened with the intent to both educate the market and provide a means to “try before you buy” products before making the investment. The challenge of understanding what to buy, however, can be daunting.

In Pitney Bowes’ Software and Data Marketplace, data are arranged into five primary categories: Addresses, Streets, Boundaries, Points and Demographics. In addition, there are augmented data types added that include a selection of “geoenriched” data that are intended to provide a rich foundation of data for property locations and risk. All categories allow the user to visualize and download sample data sets. To purchase data, the “Shop” section of the marketplace provides an eCommerce experience to select, purchase and download each data product.

While vector data types (points, lines and boundaries) as well as demographics, are somewhat intuitive, determining the regional varieties is more challenging. Most data are available to purchase by administrative boundary, such as purchasing demographics at the postal or ZIP code level for an entire state, county, province or municipal zone (e.g. city or municipal boundary). Data, on a global scale, are often required by multinational companies doing business across many regions and thus are sometimes localized by language. As a result, the combination of data category and geographic level makes the choice of selecting the optimum product akin to choosing the most tantalizing flavor of gelato.

Earth observation data are even more challenging. However, it’s important to explore the nuances of these data types. Images of the Earth’s surface are not bounded. Data are continuously acquired by orbiting satellites without regard for political or administrative boundaries, which introduces the complexity of where and how much to purchase. In addition, satellite data are available in various spatial resolutions. Data acquired with 1-meter accuracy reveal more detailed surface information than images with 30-meter pixel resolution. Buying imagery for 100 square kilometers at 1-meter resolution is more expensive as well. Now consider spectral resolution, which recognizes how much reflected solar radiance (sunlight!) is captured by satellite sensors. Sensors acquire different “bands” of the electromagnetic spectrum that are required for certain analyses. Some projects need imagery with natural color, such as you would see in the visible part of the spectrum; other projects need near-infrared, or the non-thermal part of the spectrum that would reveal variations in plant or soil type. Each needs to be well understood to make a sound purchase. DigitalGlobe and Harris provide marketplaces for Earth observation data.

One aspect of data marketplaces that are part of some existing web interfaces is the ability to select regions from an interactive map, thus allowing the user to “cookie cut” only the areas they want to purchase. It is a fantastic feature that requires some element of dynamic pricing as well but is and will be a feature highly desired by users.

As data marketplaces continue to improve the ability to find new sources of data and to explore new data types to satiate the immediate demand for location-based data, expect each to expand in functionality, add new sources of authoritative data, and provide better means to fulfill the needs of a new crop of users.

More Ideas and Insights