Blog

The National Library of Scotland has recently launched a Digital Scholarship Service, to enable new, computational uses of the collections. With three elements to the service – making data available; external collaboration and engagement; and internal culture change and skills training – the Library is seeking to equip both its collections and staff for advances in technology and research methods.

A core part of this activity is around making collections available in machine-readable format – as ‘data’ – for computational use.

Digital scholarship spans across such a broad range of research methods and areas, and our users might wish to use collections to carry out analysis at scale, such as Text and Data Mining, to uncover trends and patterns that the human eye wouldn’t be able to see, or which we simply couldn’t have time to discover; they may wish to use machine learning and AI to create new artworks based on the collections, or to enhance our metadata; they could geolocate places in texts onto maps; or at a smaller scale, they may wish to mark up texts using TEI to create new digital editions.

Ensuring that our collections are made available in suitable formats is the first step in enabling this kind of activity, and gives our special collections material a whole new lease of life.

Inspired by many others working in this area, such as Library of Congress Labs, British Library Labs and British Library Digital Scholarship, and KB Labs, as well as work by Thomas Padilla around the ‘Collections as Data’ movement, we launched the Data Foundry in September 2019.

This is our platform for making collections available as data, from digitised material, to metadata collections, geospatial data, organisational data and, we hope in the future, audiovisual data and web archive data.

The core principles of the way we present collections on the Data Foundry are:

• Open: ‘The National Library of Scotland publishes data openly and in re-useable formats’
• Transparent: ‘We take the provenance of our data seriously, and are open about how and why it has been produced’
• Practical: ‘We present datasets in a variety of file formats to ensure that they are as accessible as possible’

The file formats and the rights statements/open licenses under which we make collections available are all detailed on our Rights and Standards pages.

The main focus of the Data Foundry so far has been on digitised material: the National Library of Scotland has an in-house mass-digitisation programme underway, as one strand of activity leading into its strategic aim to make one third of its collections available in digital format by 2025. This has involved reassessing our digitisation workflow and making some changes to processes: digital scholarship brings a new use case to digitisation – rather than digitising solely with the idea of online image galleries in mind, we now need to provide additional file formats and contextual information about the collections.

As a result, we have changed our approach to ensure that standard file formats (METS and ALTOXML) are produced; suitable quality JPEGs are provided for bulk download; files are stored in the cloud for faster access; and DOIs are generated for the final ‘datasets’. Additional ‘digitisation provenance’ information (such as why an item was digitised; what funded it) is stored in the metadata files, to keep the thread running back from the dataset to the original item.

Working with curators, we also provide an introduction to each dataset, to set it in a clear collections, and historical, context.

The most popular of the Data Foundry’s collections to date is the first 100 years (8 editions) of Encyclopaedia Britannica, but we have also had a lot of interest around collections including Chapbooks printed in Scotland; Scottish School Exam Papers; and a collection from the India Papers (official publications form the UK government).

Aspects of these collections which make them particularly relevant for computational methods include the broad timescales they span, the extent of the collections, and amount of OCRed text, which enables researchers to explore changes over time in the collections. However, there are always challenges for researchers around quality of OCR: only one of our collections currently has cleaned-up OCR; other collections need some manual work on the part of the user. Given the scale of these datasets, however, it is still often possible to extract clear patterns in the material.

We’ve had some great examples of uses of the collections, even very early on in our new service, from a text and data mining project exploring Encyclopaedia Britannica, to a student project visualising aspects of Britannica in Minecraft, complete with video tour.

We also have an Artist in Residence, using machine learning techniques with the collections; academics using the Gazetteers of Scotland to improve a geoparser tool; and a ‘Song of Scottish Publishing’ created from 3,000 Scottish chapbooks. Find out more about all of these on our Projects page, and look out for a future CERL blog on digital scholarship uses of our Chapbooks Printed in Scotland collection with curator, Dr Anette Hagan.

These uses of Data Foundry datasets are making us think about the collections in a whole new way – seeing the creative, as well as the academic, and the very practical, uses that computational methods can facilitate and enable with library special collections material.

Sarah Ames, National Library of Scotland

Pingback: Exploring Digitised Official Publications: Large-scale Text Analysis of “Britain and UK Handbooks” – OfficialPapersUK

Pingback: Exploring Digitised Official Publications: Large-scale Text Analysis of “Britain and UK Handbooks” – SWOP Forum

Pingback: Old meets new: Chapbooks as digital scholarship resources – Europe’s printed and hand-written books in the spotlight