Blog

When CERL was founded in the early 90s, we lived in simpler, happier times. Books were books, libraries collected them and catalogued them, researchers visited those libraries to see the books they had found in our catalogues, and all was well. Today, a global pandemic is raging across the globe, no one knows exactly what a book is supposed to be, libraries are constantly having to justify their existence, and our catalogues have turned into ‘data’, and – even more troubling – data that is supposed to be linked and open.

Illustration 1: Roughly what the 90s looked like? (Image source)

One of the reasons why I enjoy working with Early Modern prints and the people who research them is that it serves as a constant reminder to take history seriously and to not fall for oversimplified narratives of an imagined past – such as the one opening this blog post. History matters, and although I most urgently want to share our new and exciting ideas for Linked Open Data at CERL with you, I will ask you to bear with me first while I briefly sketch the development of our databases – and what we can learn from it going forward.

There are two databases relevant to this discussion, the old and venerable HPB – now “Heritage of the Printed Book”, but created in 1994 as the “Hand Press Book” database – and its companion database, the CERL Thesaurus (CT) – now a massive data hub for our entire database ecosystem, but originally envisioned as a tool for expanding search terms to their many variants (latinizations, misspellings, translations etc.) in the HPB when it was created in 1999.

You may be surprised to hear that the CT has been offered as LOD – Linked Open Data – for quite a while now. It was open (that is, both openly accessible and openly licensed) from the beginning, and starting from 2012, it has also quietly offered RDF representations (RDF being the widely accepted standard for publishing LOD on the web) of all the entities collected in the database: persons, places, cooperations, and most importantly, printers. Since then, we have constantly increased the amount of links in the data set, both within the CT – for example between people and the places they were active in – and to external data providers. We have always linked out to “library-ish” locations, be it source records from our data suppliers or authority files like the Gemeinsame Normdatei or VIAF, but the CT now also includes links to Wikidata – the emerging central data hub of the LOD universe –, and almost more importantly, Wikidata also includes CT identifiers, making them visible to anyone looking up the relevant entities in their system.

Illustration 2: Wikidata property P1871 – the CT identifier

As LOD in libraries is a relatively young topic, and definitely still in its exploratory phase, not too many projects have had long runtimes – something certainly not helped by the often short-term funding for such projects –, so here are a few things we learned from having the LOD version of the CT up and running for eight years now:

[1] Choose your ontology wisely. You are going to be wrong anyhow. Choosing the right vocabulary to describe your data is one of the most important steps in preparing an LOD data publication, and it is important to follow best practices and standards. But standards only really emerge over time – and so it is important to think about the choice of ontology not as a one-off decision, but as something that will need to be revisited in the future as part of your data maintenance. The CT mostly uses RDA vocabulary, which sadly has not seen a large uptake in the community (and is listed as deprecated in the open metadata registry), so we will probably have to think about what other standards may be better suited to represent the data at some point in the near future.

Illustration 3: The RDF representation in the CT, a child of its time (Source record)

[2] Continuity matters. It also costs money. The LOD data publication process crucially does not end when your data is published. That is just when the fun is getting started! You are in for a lifetime commitment of maintaining and migrating that data (and data can and should live a very long and healthy life), and you will need the expertise and funding to follow through on that commitment – so make sure your institution knows what they are getting into, or help them talk to other players that are in a better position to make such long-term commitments (Hint: You know at least one such player if you are reading this blog). The CT is a living database that is constantly being updated, cleaned (and yes, in many cases that still means manual editing by human beings) and adapted to new technologies and standards – LOD has never been a standalone project for us, but simply a part of those ongoing efforts.

[3] Accessibility is key. Know your audience. When adopting new standards and technologies, communication with the community you serve is highly important. There is no point in running after the latest standards and innovative visualizations if they don’t integrate well with the workflows of your researchers. For this reason, we always try to keep a stable, long-term supported technology stack with easy-to-use, familiar interfaces while at the same time adding new access points to our data. These new access points are both for those parts of our existing audience that are running up against the limitations of traditional modes of representation, and for those communities that may only discover they are part of our audience by providing them with such access points, like the DH community.

Illustration 4: alking about our plans with the community (Slides)

We tried to heed these lessons when thinking about bringing the HPB – our oldest database, which still lives outside of our main technology stack based on CouchDB / ElasticSearch – into the LOD world. So in 2019 we ran a first little study into the potentials and possibilities of converting that database from its original format – which is PICA+ – to RDF and making it accessible through a so-called SPARQL endpoint, an interface which lets you write incredibly powerful (but maybe equally complicated?) search queries. We also looked into building a more familiar interface powered by that SPARQL endpoint and exposing some of its power to users unfamiliar with SPARQL itself. You can see the description of a first prototype and the results of discussing it with the community in a thesis I wrote, so let me mostly skip over them here (probably worth another blog post though!) Suffice it to say that we learned a lot, but also decided to not continue hosting that particular interface after the evaluation phase. The main reason is what I outlined above: We want any technology we are running to be well integrated with our main technology stack, and to be able to support it in the long-term. That was not the case for this particular prototype, but it will be the case for what we are planning to do with the new knowledge we gained from developing it.

Illustration 5: A peek at the abandoned prototype

So, here is where we shift from history to futurology, a brief outline of what LOD activities the Data Conversion Group and CERL have planned together:

[1] We want to set up a SPARQL endpoint. There is a lot to be said about the usability of SPARQL endpoints, but they are, without doubt, incredibly useful for anyone who has invested the time in learning how to write SPARQL queries (I said above that it is complicated, but that doesn’t mean it’s hard). They are also useful for people – like us at DCG – who want to write tools and interfaces that expose some, but not all of the complexity of SPARQL queries to end users. So we want to run a triple store and start hosting our data in there – probably starting with the CT and then thinking more about our other databases, not least the HPB. But we intend this to be an additional layer on top of our record-based infrastructure, not a full replacement.

[2] We want to collect and create tools for working with our data. Rather than aiming towards a full interface that answers all possible research questions, we want to highlight the possibility of creating new views on our data sets – and those of other data providers – from the existing infrastructure. Some forms this might take: scripts that demonstrate how to analyze records from our databases, modifiable SPARQL queries that show how to search for things that cannot be found with the existing search tools, software written by researchers for working with our data. We want to look into ways of presenting these kinds of tools, for example as annotated notebooks with executable code – such as Jupyter notebooks –, and to start talking more with the DH community, presenting and explaining our data and learning from them. CERL has just created its own Digital Humanities working group to facilitate that dialogue.

[3] We want to continue talking and linking to other data providers. LOD becomes more and more useful with the amount of links it contains (although one of the surprising results of our prototype was that you might not always want to display all those links to your users.) Our databases have always been aggregated from many data providers, and we aim to find the right balance between ingestion and linking (or both), so we want to continue talking to people hosting databases in the domain of Early Modern prints – or one of the infinitely many adjacent areas –, to see how we can productively link our data sets with one another, increasing their usefulness to researchers all over the world. This will hopefully also serve to close gaps in our records, as new research emerges and new (or long overdue) topics arise. Are you a researcher that has created an interesting data set? Or work in an institution that has an LOD resource not yet linked with the CT? Come talk to us!

Andreas Walker, Data Conversion Group, Göttingen