FAQ · About this site

Questions about the archive

Short answers about what MetaScholar covers, how we document the original projects and how to reach the editors. The sections below also explain the core ideas behind the programme.

Covers2001–2008 projects
TopicsMetadata, preservation
FormatArchive and journal
UpdatesOngoing
MetaScholar — FAQ · About this site
record.xmlnotes.mdsources.txt
All collectionsSymposia06Projects07Photographs02Documents06Journal20
FAQ · About this site
oai_dcindexedneeds review
1<record status="active">
2 <dc:relation>/faqs.html</dc:relation>
3</record>
Record
Covers2001–2008 projects
TopicsMetadata, preservation
FormatArchive and journal
UpdatesOngoing
Publish

Metadata harvesting, briefly

Many MetaScholar projects were built on one idea: institutions can share descriptions of their holdings without sharing the holdings themselves. That practice is called metadata harvesting.

A library, archive or museum exposes its catalogue records through a standard protocol. A separate service collects those records on a schedule, stores them together and offers one search across all of them. The protocol most of these projects used was the Open Archives Initiative Protocol for Metadata Harvesting, usually shortened to OAI-PMH.

In the early 2000s the Andrew W. Mellon Foundation funded a group of harvesting projects to test the approach on real collections. AmericanSouth and the first MetaArchive project were among them. Their findings, collected in the 2003 workshop proceedings, describe both what worked and what did not.

The short version: harvesting made records findable, but records from different places rarely matched. Much of the later research tried to repair that.

Focused crawling and clustering

Harvesting only reaches institutions that publish their metadata. A great deal of useful material sits on ordinary websites. The programme looked at two techniques to bring that material in and organise it.

Focused crawling is a way of collecting web pages on one subject rather than the whole web. A focused crawler follows links that look relevant and ignores the rest. It builds a collection faster, uses less storage and returns more precise results inside its topic than a general crawler would.

Semantic clustering groups items by similarity of meaning. In this programme the term covered several methods: forming and visualising clusters, automatic classification and adding subject tags. The MetaCombine project tested two of them in depth, support vector machines for classification and multidimensional scaling for visual maps.

Both techniques aimed at the same goal. A researcher should be able to browse a subject across many sources, even when those sources never agreed on headings.

MetaScholar — Archive
All collectionsSymposia06Projects07Photographs02Documents06Journal20
FAQ · About this site
oai_dcindexedneeds review
1<record status="active">
2 <dc:relation>/faqs.html</dc:relation>
3</record>
Record
Covers2001–2008 projects
TopicsMetadata, preservation
FormatArchive and journal
UpdatesOngoing
Publish

Frameworks and open source tools

The programme paid attention to theory as well as software. A digital library framework is a formal way to describe how a library is organised, and it can then be realised in code. Frameworks save time because each new project does not have to start from nothing.

Two strands of work were especially relevant. The Open Digital Libraries approach from Virginia Tech described libraries as networks of extended open archives, each component speaking a lightweight protocol. The 5S model, also developed at Virginia Tech, described digital libraries in terms of five elements: streams, structures, spaces, scenarios and societies.

On the practical side, the projects studied a range of open source crawlers, indexers and classifiers. Examples included the Internet Archive's Heritrix crawler, the Lucene search library, the SWISH-E indexer and the SVM light classifier.

The commitment to open source was deliberate. Software paid for by grants was meant to be reused by other institutions after the projects ended. The Metadata Migrator is one example.

How to use this archive

The site is organised by project and by event. Each project page explains what the work set out to do, who took part, what it produced and what it means now. Event pages cover the symposia and workshops the programme organised between 2004 and 2008.

The documents library lists our briefings on the programme's reports and proceedings. Start there if you want the findings in one place. The journal carries newer essays that apply lessons from the archive to current practice.

Some pages describe collections built by the projects, such as the Highlander Folk School photographs. Where we show an image that is a reconstruction rather than an archival photograph, the page says so plainly.

Every page ends with related links. Follow them to move between a project, the event where it was presented and the report that closed it.

Wooden library card catalog drawers
Photo illustration

Questions

What is MetaScholar today?
An independent editorial archive and journal. We document the MetaScholar Initiative, a digital library programme that ran at Emory's Woodruff Library from 2001 to about 2008, and publish new writing on metadata and preservation.
Is this the original MetaScholar website?
No. The domain once hosted the initiative's own site. The current site is a rewrite by independent editors and is not affiliated with Emory University.
Which projects does the archive cover?
AmericanSouth, both MetaArchive projects, Music of Social Change, MetaCombine, OCKHAM, the Study of User Quality Metrics, the Trans-Atlantic Slave Trade Database project, the Highlander photograph collection and the programme's symposia.
Where do the documents in the library come from?
Each entry is an editorial briefing we wrote about an original report or proceedings volume. The briefings summarise the source; they are not reproductions of it.
Can I use material from this site?
You may quote short passages with a link back to the page. For anything more, write to us first. Original reports remain the work of their authors and institutions.
I found a mistake. What should I do?
Send the page address, the passage and your source through the contact page. We check every request against the record and correct pages when the evidence supports it.
Do you hold original archival materials?
No. We are not a repository. If you have materials related to these projects, we can point you to institutions that collect them.
How can I reach the editors?
Use the contact page. It explains what to include for corrections, archival leads and guest essay proposals.