There are several different uses of metadata for geospatial data. The most common uses are:
For each of these uses, the emphasis on what is read in metadata may vary slightly. For instance, identification information (what, why) is of primary importance to someone searching for data; data quality becomes important in evaluating its fitness-for-use; and lineage/process steps are important for reviewing that data's history.
To be able to use metadata efficiently, it is important to have a grasp of the scope and elements of its content. With an understanding of content, metadata becomes easier to read and use, despite the confusing variety of formats it may appear in. The following sections will provide you a brief overview of the content and basic format of metadata.
Both for learning how to use metadata, and learning how to create it, the first step is to become familiar with its content. For a data-producer, the content will appear familiar though some of the terms may be new. For a data-user that has never actually created data, or an occasional user with little or no GIS background, learning metadata content is equally important. The content of metadata ties in directly to the conceptual foundations of spatial information, and the implications of its use: proper vs. improper use, garbage in = garbage out, and potential legal liability.
There are seven main sections that correspond to complete metadata documentation. Each of the sections is linked to a brief description of what is required for that section:
For a more detailed description of metadata content, you'll need to get a copy of the Metadata Workbook (sometimes refered to as the "Green Book"), with detailed field descriptions and answers to frequently asked questions. See the resources section for more information.
Familiarizing yourself with the content of metadata is the first step to being able to use metadata. It is also important to be able to distinguish if the metadata information is complete and appropriate for the dataset that it describes. For instance, if you are familiar with metadata content, you'll know that you can find out information about the accuracy of the dataset in the Data Quality section. However, this information may have been filled out incompletely or incorrectly in the metadata document: there may only be a description of the dataset's precision, instead of its accuracy. Review the topic on data types and some good examples of metadata documents to see what complete and appropriate content includes for different data types.
Different metadata formats
Even metadata that is produced according to a standard, such as the FGDC's content standard (CSDGM), can still appear in different "display" or "use" formats. See the topic on content, format, and use standards for more information. Here are some examples of different formats:
- unstructured notes or "log" files
- plain text (ASCII) file
- HTML file: "outline" format (example)
- HTML file: "frequently-anticipated questions" format (example)
- several short or summary formats
- several formats used for the purpose of indexing/searching, not for viewing (SGML)
- new formats are being introduced specific to metadata software or GIS software that includes capability for viewing, creating, and managing metadata
- new customizable formats of metadata may become more common with extensible markup language (XML): metadata producers and user can customize how metadata will appear using style sheets (the same metadata document may be presented in different ways)
The following section briefly introduces two of the most common formats for standardized metadata.
- Outline format
Elements of metadata are organized hierarchically underneath seven basic headings (as described above in the Focus on Content Approach). As of 1999, this is the most common format of metadata, the result you will most likely find when performing a search on the NSDI geospatial clearinghouse. This format may be confusing to read because of the existence of many headers, which do not actually contain information but exist as a means of structuring the metadata. For instance, the dataset's title follows after 3 headers (Identification information, citation, and citation information). Please see the USGS page CSDGM - An Image Map for more information about this format.
- Frequently-asked question format
This new format was introduced in early 1999 and presents an easier-to-read, "plain English" format, in which the content is organized based on an "interview format", starting with six basic questions:
- What does the dataset describe and what area does it cover?
- Who produced the dataset?
- Why was the dataset created - for what purpose, based on what requirements/specifications?
- How was the dataset created - what sources where used and what procedures?
- How reliable are the data, what problems remain in the dataset?
- How can someone get a copy of the dataset?
This format is especially helpful for non-technical users with little or no GIS background. Regular GIS-users may prefer the more technically oriented "outline" version, especially as they become familiar with the terms. Please see the USGS web page on Metadata in plain language for more information about this format.
Issues regarding metadata implementation
The high level of detail of metadata content, technical terminology, and confusing variety of metadata formats are all reasons that may impede the use of metadata. These same reasons also contribute to the avoidance of creating metadata. Data creation is time-consuming and complicated by itself; metadata creation is often considered an added burden. In one of the first formal studies of metadata implementation (Gelbman 1999), it was discovered that even though a majority of data-producers recognize the benefits of metadata, there continues to be reluctance to commit time to creating metadata. This is because a commitment to metadata would take too much time away from more important or necessary endeavors/responsibilities. The time required for training in order to implement metadata is especially controversial.
"How do we deal with people who complain that metadata implementation is too hard? The solution in most cases is to redesign the workflow rather than to develop new tools or training. People often assume that data producers must generate their own metadata. Certainly they should provide informal, unstructured documentation, but they should not necessarily have to go through the rigors of fully structured formal metadata. For scientists or GIS specialists who produce one or two data sets per year it simply isn't worth their time to learn the FGDC standard. Instead, they should be asked to fill out a less- complicated form or template that will be rendered in the proper format by a data manager or cataloger who is familiar (not necessarily expert) with the subject and well-versed in the metadata standard. If twenty or thirty scientists are passing data to the data manager in a year, it is worth the data manager's time to learn the FGDC standard. With good communication this strategy will beat any combination of software tools and training."
(Excerpt from Frequently Asked Questions maintained by Peter Schweitzer).
As suggested above, probably the best way of dealing with the complexity of the standard is not to try to reduce its complexity (and therefore its usefulness) but rather, to create a "metadata specialist" position. Not all users and data producers should be required to know the standard, though they should be aware of it and understand the elements of it content (see the content vs. format discussion in "Why metadata is important"). The problem with this is that, of course, not all organizations or educational/research institutions have the resources to create a metadata specialist position, or even the resources to re-direct 20% of a GIS producer's time to this endeavor. In answer to this dilemma, most states have GIS coordinating councils, state data centers or GIS centers, or state libraries with a metadata specialist who would be willing to help out in exchange for access to the final metadata to add to a data catalog or clearinghouse. Contact the National States Geographic Information Council to find out your state's GIS coordinating center or contact a nearby clearinghouse from the list of clearinghouse websites maintained by the FGDC.
Metadata specialists play a crucial role, but their existence raises another issue: if one person is familiar with the standard and can create formal metadata, but many people are actually involved in geospatial data production, how is metadata collected? How is the information transferred and validated between the producers and the metadata specialists? From past implementation experiences, there has been a common recommendation in providing simplified templates to GIS staff to help them collect metadata. Information from these templates is then re-formatted into standard format by the metadata specialist. Here is a link to some examples of templates that have been successfully used.
Both for filling out a simplified template, or for the standard itself, experience has shown that having examples of completed metadata on hand is extremely helpful, even more so than having workbooks, glossaries/definitions and metadata creation software.
For those determined to learn the standard, metadata software can help alleviate some of the difficulties by providing helpful tools for stepping through the standard and providing contextual help for the fields. For the average data producer, who may only need to create one or two metadata documents a year, or update them occasionally, metadata software may still create information-overload. Also, NO software is easy to use if you only use it once or twice a year.
The standard does allow for some creative flexibility. Perhaps it is not appropriately designed for every single dataset out there: but for unique/unusual datasets, there are enough descriptive fields where explanations can be provided for the unusual aspects of the data. Again, a metadata specialist is the best person for this job because he/she is most familiar with the standard and aware of how to coax it to handle difficult/unique datasets.
Some sections of the standard can be "recycled": information will be the same for multiple data sets and only has to be entered once, then copied/pasted into subsequent metadata documents.
Things you should consider before starting implementation
Organizations that have been successful in their implementation of a metadata initiatives have addressed the following needs (WAGIC):
- Adopt an operational plan
: two to three pages that from a planning perspective address a typical approach to training staff & management, data set inventory and prioritization, adopting procedures & best practices, etc. Here is WAGIC's working draft of a template operational plan
- Establish Procedures
: what level of granularity to collect metadata at? what level of detail to collect metadata at?
- Train people
: identify needed skills for all associated personnel and set aside time for training (e.g. 2 days for a metadata specialist; 2 hour orientation for other GIS staff)
- Assess technology infrastructure
: identify collection tools compatible with existing IT structure and that supports procedures for creation, maintenance and distribution of metadata. Also important to consider if tool is adaptable/upgradable.
- Identify suitable metadata repository
: where your metadata resides should be based on consideration of internal access, external access and maintenance. Options: resides with data, stand-alone metadata database, clearinghouse, paper-based
The sophistication and complexity of the successful approaches varies with the needs and resources of the organization. It is possible for a one-person-GIS-shop to be successful with metadata implementation.
Copyright material on implementation of metadata and metadata management issues by Bruce Westcott of RTSe (USA), Inc.
Specifics to consider
- Database vs. discrete metadata documents
How many GIS datasets do you have? How many of them need to have their own individual metadata documents, or how many can be considered part of a "series" that shares the same metadata? How often is your data updated, requiring updates to the metadata? Answering these questions will help you determine whether you can create discrete metadata documents or if it is wiser, in the long-run, to create metadata as part of "database", which is easier to query and update. Discrete metadata documents can be as simple as a file folder with notes,
requiring no special software or training. However, discrete documents are harder to keep track of and to update. Before making your decision, carefully consider the issue of data granularity (listed below) as well.
- Data granularity
Do you create metadata for each dataset, for each feature in a dataset, or for a data series (many closely-related datasets, such as the
U.S.G.S. topographic map series)? Take a careful inventory of all the datasets you have. Do any of the datasets have certain basic things in common, such as the method they were digitized, their projection, certain attributes, etc? If there are only a few differences between each dataset, such as the name and date, you might consider creating series-level metadata. Series-level metadata is just one document for multiple datasets. Individual differences between the datasets can be stored in a simple table within the metadata, or as a seperate file (text, spreadsheet, database) that is referenced within the metadata. Check out this
example of series-level metadata.
What about very "fine-grained" data - suppose your dataset is so detailed that it is important to record metadata for each feature, such as its source, date, last update, etc. Feature-level
metadata is usually stored within tables or data dictionaries, which are then referenced within the metadata document.
- Generic standard, or specific profile?
Specific profiles of the metadata standard exist for special types of data such as cadastral, biological, or orthoimagery. These profiles may contain additional elements that are useful for describing specific elements of a dataset. Profiles also may remove other elements that are too generic and do not apply to the dataset being profiled. Profiles are designed to make the documentation for a dataset more efficient (getting rid of unnecessaries) and more complete (adding specific characteristics). See the Federal Geographic Data Committe web-pages for a listing of existing profiles as well as working groups in the process of developing new profiles and standards for different types of data.
- What type of software tool should be used?
Tools for creating metadata range from simple text templates
to step-by-step fill in the blank programs with context-sensitive help. Tools also exist for management of large repositories of metadata. Some tools are
free, some are not; some are geared for specific GIS software, some are
not related to any GIS software at all. Some tools are best for individuals
or small one-to-three GIS shops; some tools are better for large organizations with many data users and producers. The Metadata Tools web-site provides a helpful summary of different available tools, and the
The Metadata Primer provides good background about making choices.
- Redesigning workflow and training
Some thought must be given to how metadata implementation will fit into the existing workflow, whether the environment is a single-person GIS shop or a large, multi-department organization. The frequency of use/creation of metadata and the volume of metadata understanding will vary depending on a person's role: occasional data-user,
frequent data-user and data producer. Both managers and staff should have an
awareness of the importance and benefits of metadata; all data-user and data-producers should have a grasp of metadata content. A metadata specialist should
know the ins and outs of metadata content, standards, formats, and some tools. See the section above on Issues regarding metadata implementation for more information.
- Top ten most common metadata errors
Compiled by the experts (coming soon)
Resources to help implementation of metadata
Experiences from several different implementation efforts have resulted in these suggestions. Note that "implementation" in the following examples refers to creating, formatting, and distributing metadata according to the FGDC's Content Standard for Digital Geospatial Metadata. Read the topic Overview of the metadata content standard for a general overview and if you didn't cover the Issues of metadata implementation and focus on content approach first, do so now!
The original source
The Content Standards for Digital Geospatial Metadata Workbook, or "Green Book", was published by the FGDC in 1995. Though highly criticized, this remains the definitive source for learning the standard; no other material has been developed at a level of detail sufficient to actually learn the standard.
In June, 1999 FGDC sponsored a meeting of metadata and education experts, the purpose of which was to discuss existing obstacles in metadata implementation and education. At this meeting, the necessity of revising the Green Book into a more user-friendly, "plain English", better-annotated guide was considered a top priority (second only to better marketing strategies for metadata).
Until such an upgrade occurs, the Green Book is still a necessary but frustrating tool in metadata implementation. The following sources provide some suggestions to help ease the process of implementation, though they all still recommend use of the Green Book. Good metadata software and image maps assist in understanding the organization, compound elements, and mandatory/optional elements of the standard. Some software contains context-sensitive help files, but no software has, to date (1999), incorporated all the elements of the Green Book or replaced it with comparable resources.
Other resources
A great deal of work has already been done to provide tutorials, exercises, and recommendations for the implementation process. Here are several useful sources:
The Metadata Primer:
a "how to" guide on metadata implementation. Next to the Green Book, this is the most comprehensive source for actually creating metadata. It has a succinct introduction to the CSDGM, including an amusing "minimum metadata document", lots of links to other resources, and a good discussion on choosing an appropriate tool for entering metadata.
Metadata tool exercise: developed for use with the Metadata Primer. In this exercise, you actually install software, work with metadata files in MetaMaker, parse them for errors, and fill in a text template if you actually want to create some metadata. Includes Cliffy's notes or Important / Difficult / Misinterpreted Elements of the CSDGM, practical notes from an experienced metadata creator that can get you through the entire CSDGM in "11.3 minutes instead of the typical 2 hour workshop".
Metadata Tools:
a frequently-updated site that contains information on the latest developments in metadata tools and also provides a functional description for each of the most common tools.
Spatial Metadata Samples Library: a collection of metadata samples for the seven "framework" datasets (geodetic control, elevation, orthoimagery, transportation,
hydrography, governmental units, and cadastral). Also includes a few samples
from datasets that don't fall in the above categories. A good site for "learning by example."
North Carolina's Center for Geographic Information and Analysis recommended process, based on experiences (both positive and negative) from four separate implementation projects. This tutorial has some elements specific to North Carolina policies, but is generic enough to be used by anybody. It goes through all steps from defining management resources, to creating an implementation plan, to collecting information, choosing a tool, formatting metadata, creating "compliant" metadata by parsing (the tricky "mp" process), and getting metadata into a clearinghouse. What it does not do is guide you through the standard itself or provide examples.
BLM metadata tutorial that walks you through all entering all the mandatory elements of the CSDGM, step by step, though some of the information filled out could be considered too "technical" (only another BLM person would understand it) or incomplete. (Note: there is a whole topic on good and bad examples of metadata).
Louisiana Geographic Information Center's Metadata Workbook On-Line. Recommends specific metadata creation software. Refers to the Metadata Creation Exercise included as part of the Metadata Primer (listed above) for specifics. Includes steps for parsing and distributing metadata.
NOAA Coastal Services Center Metadata Reference Guide
References
Washington Geographic Information Council instructor's presentation
Gelbman, R. and T. Mathys. 1999. Making Metadata Part of Your Daily Diet. Proceedings of the 1999 Urban and Regional Information Systems Association (URISA) annual conference.
North Carolina Center for Geographic Information and Analysis, "Accessing Data Tutorial" http://cgia.cgia.state.nc.us:80/tutorials/index.html
Berendsen, Margo. "Metadata for the Rest of Us". University of Wyoming Spatial Data and Visualization Center. http://www.wygisc.uwyo.edu/metadata
Back to Course Topics
