Questions about the archive
Short answers about what MetaScholar covers, how we document the original projects and how to reach the editors. The sections below also explain the core ideas behind the programme.

Metadata harvesting, briefly
Many MetaScholar projects were built on one idea: institutions can share descriptions of their holdings without sharing the holdings themselves. That practice is called metadata harvesting.
A library, archive or museum exposes its catalogue records through a standard protocol. A separate service collects those records on a schedule, stores them together and offers one search across all of them. The protocol most of these projects used was the Open Archives Initiative Protocol for Metadata Harvesting, usually shortened to OAI-PMH.
In the early 2000s the Andrew W. Mellon Foundation funded a group of harvesting projects to test the approach on real collections. AmericanSouth and the first MetaArchive project were among them. Their findings, collected in the 2003 workshop proceedings, describe both what worked and what did not.
The short version: harvesting made records findable, but records from different places rarely matched. Much of the later research tried to repair that.
Focused crawling and clustering
Harvesting only reaches institutions that publish their metadata. A great deal of useful material sits on ordinary websites. The programme looked at two techniques to bring that material in and organise it.
Focused crawling is a way of collecting web pages on one subject rather than the whole web. A focused crawler follows links that look relevant and ignores the rest. It builds a collection faster, uses less storage and returns more precise results inside its topic than a general crawler would.
Semantic clustering groups items by similarity of meaning. In this programme the term covered several methods: forming and visualising clusters, automatic classification and adding subject tags. The MetaCombine project tested two of them in depth, support vector machines for classification and multidimensional scaling for visual maps.
Both techniques aimed at the same goal. A researcher should be able to browse a subject across many sources, even when those sources never agreed on headings.
Frameworks and open source tools
The programme paid attention to theory as well as software. A digital library framework is a formal way to describe how a library is organised, and it can then be realised in code. Frameworks save time because each new project does not have to start from nothing.
Two strands of work were especially relevant. The Open Digital Libraries approach from Virginia Tech described libraries as networks of extended open archives, each component speaking a lightweight protocol. The 5S model, also developed at Virginia Tech, described digital libraries in terms of five elements: streams, structures, spaces, scenarios and societies.
On the practical side, the projects studied a range of open source crawlers, indexers and classifiers. Examples included the Internet Archive's Heritrix crawler, the Lucene search library, the SWISH-E indexer and the SVM light classifier.
The commitment to open source was deliberate. Software paid for by grants was meant to be reused by other institutions after the projects ended. The Metadata Migrator is one example.

How to use this archive
The site is organised by project and by event. Each project page explains what the work set out to do, who took part, what it produced and what it means now. Event pages cover the symposia and workshops the programme organised between 2004 and 2008.
The documents library lists our briefings on the programme's reports and proceedings. Start there if you want the findings in one place. The journal carries newer essays that apply lessons from the archive to current practice.
Some pages describe collections built by the projects, such as the Highlander Folk School photographs. Where we show an image that is a reconstruction rather than an archival photograph, the page says so plainly.
Every page ends with related links. Follow them to move between a project, the event where it was presented and the report that closed it.
