4.8 Quality Assurance and Control of GIS Data

GIS data editors should follow basic GIS editing best practices when editing MDOT MAA’s GIS data and before running ESRI Data Reviewer. Pre-checks should be performed to help spot errors. These pre-checks can be accomplished by querying the attribute tables and symbolizing data. Before running the Data Reviewer checks, verify that required attributes are populated, and topology rules have been followed. The Data Reviewer checks will then help the Data Editors catch errors that may be missed during editing.

4.8.1 ESRI Data Reviewer

MDOT MAA requires that all consultants preparing GIS data and related files for MDOT MAA must use Esri Data Reviewer to ensure data consistency and adherence to the MDOT MAA GIS database design as outlined in the GIS Standard and its appendices.

4.8.2 Availability & ESRI Support

Data Reviewer can be purchased directly from ESRI by visiting ESRI’s website at http://www.esri.com/software/arcgis/extensions/arcgis-data-reviewer/pricing. MDOT MAA will not reimburse the consultants for the purchase of this software.

Data Reviewer Support is available for registered users directly from ESRI’s website at http://resources.arcigis.com/en/communities/data-reviewer/

4.8.3 System Requirements

Because Data Reviewer runs only inside ArcMap and cannot be run as a stand-alone application, ArcMap must be installed on any computer before Data Reviewer can be installed. Computers running Data Reviewer must therefore meet ArcMap’s minimum system requirements, which can be found at http://desktop.arcgis/en/system-requirements/latest/arcgis-desktop-system-requirements.htm.

4.8.4 Automated Quality Control Software: Esri Data Reviewer and Custom ArcToolbox

MDOT MAA has an established set of automated quality control tests for file geodatabases that utilize ESRI’s Desktop Data Reviewer extension. These batch files, known as “.rbj” files, will be shared with all Consultants to set expectations for data quality. The “.rbj” files will be provided at the same time as the geodatabase checkout or replica.

A custom ArcToolbox will also be provided at the same time as the geodatabase checkout or replica. This toolbox automatically creates two feature classes necessary for the Data Reviewer batch files to run. Consultant’s Data Editors are expected to run the ArcToolbox and batch files on edited data prior to delivery to MDOT MAA. It is suggested that the tests be used early and frequently to ensure data integrity while establishing and executing editing processes, however the only requirement will be prior to delivery to ensure that the data passes the tests.

4.8.5 Usage

MDOT MAA will create all of the baseline tests to be performed on data during editing and make the tests available to the Consultant’s Data Editors. The Data Editors will first run the custom ArcToolbox to generate two feature classes and then run the batch files until the data returns a clean report, meaning there are no errors. Both the ArcToolbox and batch files can be run multiple times as the data errors are cleaned up. The data will then be submitted to MDOT MAA, at which point the same set of tests will be re-run by MDOT MAA to ensure compliance. Any failure noted once data has been delivered to MDOT MAA will result in the entire dataset being rejected and returned to the Consultant for corrective action and resubmittal.

4.8.5.1 Required Tests

Automated quality control tests are developed for each feature class within a feature dataset and are self-documenting. The tests will share basic similarities across similar features, with customizations occurring on checks for logical consistency between attributes.

The following automated checks have been set up for each feature class:

A.      Database Validation Checks – Validates coded value domains to ensure that all values meet domain constraints.

B.      Default Checks – Invalid Geometry Check under Default Checks finds features whose geometry is empty, nothing, or not simple, as well as features with empty envelopes.

C.       Advanced Checks – Custom checks for logical consistency between attributes that return values that do not conform to custom SQL statements.

D.      Feature on Feature Checks – Evaluates the spatial relationship from the same or two different feature classes.

E.       Duplicate Geometry Checks – Find features of the same geometry type that are collocated.

4.8.5.2 Data Acceptance

Data acceptance will be based on data performance using automated tests and visual inspections. Any single failure of edited data will constitute total failure of the data delivery and it will be returned to the Consultant for corrective action and resubmittal.

4.8.5.3 Exceptions

In cases where features fail a data reviewer check but have a valid exception, the Consultant shall provide MDOT MAA with the file geodatabase used for the Data Reviewer session. This geodatabase must have all corrected features removed from the reviewer table and should only contain the exceptions. The Consultant shall include a brief reason that the feature is exempt from the check.

4.8.6 Visual Quality Control Tests Using Sampling

While automated data review will catch systematic errors, the only way to evaluate data content is through a visual quality control process. Visual quality control processes will ensure that features are stored in correct feature classes, drawings are interpreted correctly, and attributes are correctly populated. Data Reviewer can be used to facilitate this process by selecting a random sampling of features for visual review.

Visual review will be based on the best practices and editing guidelines for a specific feature class.