Metadata · Aug 13, 2026 · 5 min read

Metadata quality metrics that matter

A small set of record and retrieval metrics for metadata teams: what to measure, how to compute it, and how to keep a dashboard from becoming busywork.

A reel-to-reel tape recorder and record sleeves in an archive

Every metadata team says it cares about quality. Fewer can say what they measure. Without a measure, quality becomes a matter of taste, and cleanup work gets scheduled by whoever complains loudest. This post sets out a small set of metrics that tell you something useful, how to compute them, and how to keep them from turning into busywork.

The aim is not a perfect score. It is a short dashboard that answers one question: can people find, understand and reuse what we describe?

Two kinds of quality

It helps to separate two ideas that often get mixed. The first is record quality: whether an individual record is complete, accurate and consistent with your rules. The second is retrieval quality: whether users, searching a real system, get results they judge to be good. The two are related but not identical. A perfectly formed record can still be invisible if the search system ignores the fields it fills.

The MetaScholar Initiative explored the second idea directly. Its Study of User Quality Metrics, a collaboration between Emory and Virginia Tech supported by the Institute of Museum and Library Services, used production digital library services to examine what criteria shape users' preferences in metasearch systems. The team added custom ranking metrics to open source search engines and tested how users reacted to different retrieval algorithms and weightings. The planned outcomes were empirical data on user preferences, a theoretical model of user quality metrics, and an open source search engine that could accept explicit ranking algorithms.

That framing is still the right one. Record metrics are what you control. Retrieval metrics are what users feel. A good program tracks both and connects them.

Record-level metrics worth tracking

The literature offers several quality frameworks. The one most often cited in libraries, from Bruce and Hillmann, names dimensions such as completeness, accuracy, provenance, conformance to expectations, logical consistency, timeliness and accessibility. In practice, a handful of measurable checks covers most of what matters.

MetricWhat it measuresHow to compute
Required-field completenessShare of records with every field your profile requiresCount records passing a schema or profile check
Controlled-value conformanceShare of subject, place and name values that match an authorityMatch values against vocabularies or authority files
Date parseabilityShare of date values a machine can interpretParse against an agreed date format, such as EDTF
Rights coverageShare of records with a clear rights statementCheck for a value from an agreed list
Link healthShare of identifiers and URLs that resolveScheduled HTTP checks
FreshnessTime since a record was last reviewed or harvestedCompare datestamps with a review cycle

Each of these produces a percentage you can track over time and break down by collection or contributor. That breakdown is where the value is. A single overall number hides the one collection that drags everything else down.

Figure 1 · interactive illustration

Aggregation changes the picture

Metrics behave differently once records leave home. A local catalog can tolerate idiosyncratic subject terms because local staff know what they mean. An aggregation cannot. When the AmericanSouth and MetaArchive portals harvested records from many partners in the early 2000s, the lack of shared vocabulary and authority control in the underlying Dublin Core made subject browsing across institutions nearly impossible. The MetaCombine proposal later listed this as a core problem and proposed semantic clustering as a remedy.

The 2006 Metadata Enhancement and OAI Workshop took up the same thread. It considered automated and semi-automated ways to improve harvested metadata, including normalization of date and format fields and the assignment of subjects through clustering and classification. The workshop's goal was an agenda of shared experiments, not a single fix.

For anyone running an aggregation today, the implication is straightforward. Measure quality per source, report it back to the source, and decide in advance which problems you will remediate centrally and which you will ask contributors to fix. Central fixes are faster. Source fixes last.

A simple feedback loop makes this work. After each harvest, send every contributor a one-page report: how many records arrived, which checks failed, and three example records for each failure. Keep the format identical every time so people learn to read it quickly. Contributors rarely ignore a concrete list of their own records. They often ignore a general request to improve quality. Over a few cycles, the report also becomes evidence for which training or documentation would help most, because the same failures tend to recur across institutions that share a cataloging system.

Retrieval metrics you can actually run

Retrieval quality is harder to measure, but not impossible for a small team. Three approaches work without a research budget.

  • Known-item tests. Keep a list of fifty or so items that users regularly look for, with the query a typical person would type. Run the list after every significant change and record where each item ranks.
  • Zero-result review. Pull queries that returned nothing. Many point to missing synonyms, variant spellings or fields that are not indexed.
  • Task-based sessions. Ask a few real users to complete specific research tasks and observe. Even five sessions reveal patterns that logs hide.

These methods echo the quality metrics study in miniature. They test what users experience, not what records contain, and they make it possible to see whether a change in metadata practice changed anything that matters.

A magnifying glass over a handwritten manuscript next to its digital scan
Photo illustration

Keeping metrics useful

Metrics fail in familiar ways. Teams track too many, report them without context, or optimize the number instead of the outcome. A few habits help.

  1. Start with five. Pick the metrics tied to your most common user complaints and add others only when you have a reason.
  2. Set thresholds, not targets. A threshold says what is unacceptable. A target invites gaming.
  3. Automate the counting. Run checks on every ingest or harvest so numbers stay current without manual effort.
  4. Pair each metric with an owner. A number with no one responsible for it is decoration.
  5. Review quarterly. Drop metrics that have not prompted a decision in a year.

A small dashboard reviewed regularly beats a large one nobody reads. The point is to direct scarce cataloging time to the problems that most affect discovery.

Where to start this month

If you have no metrics today, begin with required-field completeness and date parseability. Both are easy to compute, both show up in every aggregation, and both are cheap to improve. Add a known-item test list next, because it connects record work to the user experience. Once those three are running, you will have enough evidence to decide where the next round of effort belongs.

The early harvesting projects learned that quality is not a property of a single record. It is a property of a system: records, vocabularies, software and the people who use them. The metrics you choose should reflect that whole system, and they should be few enough that someone actually looks at them. For background on the source documents behind this history, see the MetaScholar documents archive.

Questions

What is the single most useful metadata metric?
Required-field completeness, broken down by collection. It is easy to compute, shows where gaps cluster, and directly affects whether records can be found and displayed.
How is retrieval quality different from record quality?
Record quality describes the records themselves. Retrieval quality describes whether users get good results from a real search system. Strong records can still perform poorly if search ignores key fields.
Should aggregators fix metadata centrally?
Sometimes. Central normalization is fast and helps users now, but fixes made at the source last longer. Measure per source and agree in advance which problems each side owns.