Archive, Museen & Information Retrieval

Jun.-Prof. Dr. Mark Hall

Sommersemester 2019

Photo of the library stacks
https://www.flickr.com/photos/carmichaellibrary/5670360555/
Photo of archival data in tubes
https://www.flickr.com/photos/27892629@N04/15958244721/
Photo of museum archive shelves
https://www.flickr.com/photos/khowaga/8579074312/
Photo of boxes in the stacks
https://www.flickr.com/photos/usnavalwarcollegeri/6941929111/
Photo of movable shelves in the stacks
https://www.flickr.com/photos/-jvl-/13091907034/

Wer hat Zugriff?

Keine Kugelschreiber!

Photo of a box with unknown content
https://www.flickr.com/photos/withassociates/3169531275/

Wie oft kann man etwas im Museum sehen?

Information Need

Textsuche

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Curabitur lacus sem, sodales sed consectetur in, viverra vitae velit. Sed sagittis ex et nunc elementum, at ultrices lorem ornare. Aliquam ornare leo sed risus aliquam, vel dignissim dui fermentum. Nullam nisi magna, luctus ac lacus et, posuere egestas augue. Integer auctor leo eu tempor mollis cocacola. Pellentesque tincidunt semper hendrerit. Donec sollicitudin quis erat vitae volutpat. Sed dapibus nisl velit, vel porttitor elit ultricies eget. Suspendisse fermentum aliquet nisi, non faucibus leo egestas nec. Sed finibus id nunc nec varius.

Pellentesque lorem mauris, fringilla eu molestie non, ullamcorper vitae lorem. Sed nec nunc ac dolor molestie ultrices vel non lorem. Curabitur cursus eget ante sed consequat. Maecenas eget viverra mauris cocacola. Nulla aliquam dictum dolor a pretium. Aliquam et augue vitae dolor egestas accumsan. Curabitur bibendum libero quis felis laoreet tempus. Morbi pharetra dui consectetur tempus ultricies. Nam auctor lorem eget faucibus vehicula. Fusce fringilla nec sapien id tempor. Ut tincidunt ante turpis, sed congue arcu commodo nec cocacola. Aliquam molestie finibus eros sed tristique. Curabitur eget ullamcorper leo, in tincidunt nisi. Vestibulum dapibus ac mauris eu lacinia.

Textsuche

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Curabitur lacus sem, sodales sed consectetur in, viverra vitae velit. Sed sagittis ex et nunc elementum, at ultrices lorem ornare. Aliquam ornare leo sed risus aliquam, vel dignissim dui fermentum. Nullam nisi magna, luctus ac lacus et, posuere egestas augue. Integer auctor leo eu tempor mollis cocacola. Pellentesque tincidunt semper hendrerit. Donec sollicitudin quis erat vitae volutpat. Sed dapibus nisl velit, vel porttitor elit ultricies eget. Suspendisse fermentum aliquet nisi, non faucibus leo egestas nec. Sed finibus id nunc nec varius.

Pellentesque lorem mauris, fringilla eu molestie non, ullamcorper vitae lorem. Sed nec nunc ac dolor molestie ultrices vel non lorem. Curabitur cursus eget ante sed consequat. Maecenas eget viverra mauris cocacola. Nulla aliquam dictum dolor a pretium. Aliquam et augue vitae dolor egestas accumsan. Curabitur bibendum libero quis felis laoreet tempus. Morbi pharetra dui consectetur tempus ultricies. Nam auctor lorem eget faucibus vehicula. Fusce fringilla nec sapien id tempor. Ut tincidunt ante turpis, sed congue arcu commodo nec cocacola. Aliquam molestie finibus eros sed tristique. Curabitur eget ullamcorper leo, in tincidunt nisi. Vestibulum dapibus ac mauris eu lacinia.

Für große Datenmengen unmöglich

Lösung: Pre-processing

Invertierte Liste

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause of the Indian Continent

Invertierte Liste

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of British the Empire
  4. Common Cause the Indian of Continent

Invertierte Liste

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause the Indian of Continent
  • Functions = #1
  • of = #1 #2 #3 #4
  • the = #1 #2 #3 #4
  • Empire = #1 #3
  • throne = #2
  • that = #2
  • empire = #2
  • integrity = #3
  • British = #3
  • Common = #4
  • Cause = #4
  • Indian = #4
  • Continent = #4

Indexing Pipeline

  1. Tokenisieren
  2. Stop-wörter filtern
  3. Pre-processing

Stop-wörter

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause of the Indian Continent
  • Functions = #1
  • Empire = #1 #3
  • throne = #2
  • empire = #2
  • integrity = #3
  • British = #3
  • Common = #4
  • Cause = #4
  • Indian = #4
  • Continent = #4

Pre-processing (groß/klein)

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause of the Indian Continent
  • functions = #1
  • empire = #1 #2 #3
  • throne = #2
  • integrity = #3
  • british = #3
  • common = #4
  • cause = #4
  • indian = #4
  • continent = #4

Pre-processing (stemming)

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause of the Indian Continent
  • function = #1
  • empire = #1 #2 #3
  • throne = #2
  • integrity = #3
  • britain = #3
  • common = #4
  • cause = #4
  • india = #4
  • continent = #4

Suche: Empire

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause of the Indian Continent
  • function = #1
  • empire = #1 #2 #3
  • throne = #2
  • integrity = #3
  • britain = #3
  • common = #4
  • cause = #4
  • india = #4
  • continent = #4

Suche: Empire = empire

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause of the Indian Continent
  • function = #1
  • empire = #1 #2 #3
  • throne = #2
  • integrity = #3
  • britain = #3
  • common = #4
  • cause = #4
  • india = #4
  • continent = #4

Suche: Empire = empire

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause of the Indian Continent
  • function = #1
  • empire = #1 #2 #3
  • throne = #2
  • integrity = #3
  • britain = #3
  • common = #4
  • cause = #4
  • india = #4
  • continent = #4

Suche: Britain or India

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause of the Indian Continent
  • function = #1
  • empire = #1 #2 #3
  • throne = #2
  • integrity = #3
  • britain = #3
  • common = #4
  • cause = #4
  • india = #4
  • continent = #4

Suche: Britain or India = britain india

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause of the Indian Continent
  • function = #1
  • empire = #1 #2 #3
  • throne = #2
  • integrity = #3
  • britain = #3
  • common = #4
  • cause = #4
  • india = #4
  • continent = #4

Suche: Britain or India = britain india

  1. Functions of the Empire
  2. the throne of that empire
  3. integrity of the British Empire
  4. Common Cause of the Indian Continent
  • function = #1
  • empire = #1 #2 #3
  • throne = #2
  • integrity = #3
  • britain = #3
  • common = #4
  • cause = #4
  • india = #4
  • continent = #4

In 1815 few could have predicted what was to happen to the British Empire . The country may have been celebrating the triumphs of Wellington and Nelson but the nation was at a crossroads and experiencing economic and social change that for many meant the very real possibility of imminent revolution. The existence of the British Empire had been threatened by the American revolution but since the loss of the American colonies in 1783 and the destruction of France's military and naval might, Britain stood at the threshold of a new era which would be dominated by Britain's monopoly of the techniques of steam power and the existence of a group of entrepreneurs who existed in a political and economic climate that encouraged innovation. In 1815 British industry was based on the textile industry but within a few decades the United Kingdom was regarded by the British as the 'Workshop of the World' and the growing empire, both formal and informal, provided the market and raw materials for this growing economy which was not challenged until the turn of the twentieth century.

Term Frequency

$$tf(t, d) = \frac{f_{t,d}}{|d|}$$

In 1815 few could have predicted what was to happen to the British Empire. The country may have been celebrating the triumphs of Wellington and Nelson but the nation was at a crossroads and experiencing economic and social change that for many meant the very real possibility of imminent revolution. The existence of the British Empire had been threatened by the American revolution but since the loss of the American colonies in 1783 and the destruction of France's military and naval might, Britain stood at the threshold of a new era which would be dominated by Britain's monopoly of the techniques of steam power and the existence of a group of entrepreneurs who existed in a political and economic climate that encouraged innovation. In 1815 British industry was based on the textile industry but within a few decades the United Kingdom was regarded by the British as the 'Workshop of the World' and the growing empire, both formal and informal, provided the market and raw materials for this growing economy which was not challenged until the turn of the twentieth century.

britain america france
Term Frequency 6 2 1

In 1815 few could have predicted what was to happen to the British Empire. The country may have been celebrating the triumphs of Wellington and Nelson but the nation was at a crossroads and experiencing economic and social change that for many meant the very real possibility of imminent revolution. The existence of the British Empire had been threatened by the American revolution but since the loss of the American colonies in 1783 and the destruction of France's military and naval might, Britain stood at the threshold of a new era which would be dominated by Britain's monopoly of the techniques of steam power and the existence of a group of entrepreneurs who existed in a political and economic climate that encouraged innovation. In 1815 British industry was based on the textile industry but within a few decades the United Kingdom was regarded by the British as the 'Workshop of the World' and the growing empire, both formal and informal, provided the market and raw materials for this growing economy which was not challenged until the turn of the twentieth century.

britain america france
Term Frequency 6 / 182 = 0.033 2 / 182 = 0.011 1 / 182 = 0.005

Inverse Document Frequency

$$idf(t, D) = log \frac{|D|}{|\{d \in D : t \in d\}|}$$

In 1815 few could have predicted what was to happen to the British Empire. The country may have been celebrating the triumphs of Wellington and Nelson but the nation was at a crossroads and experiencing economic and social change that for many meant the very real possibility of imminent revolution. The existence of the British Empire had been threatened by the American revolution but since the loss of the American colonies in 1783 and the destruction of France's military and naval might, Britain stood at the threshold of a new era which would be dominated by Britain's monopoly of the techniques of steam power and the existence of a group of entrepreneurs who existed in a political and economic climate that encouraged innovation. In 1815 British industry was based on the textile industry but within a few decades the United Kingdom was regarded by the British as the 'Workshop of the World' and the growing empire, both formal and informal, provided the market and raw materials for this growing economy which was not challenged until the turn of the twentieth century.

britain america france
Term Frequency 6 / 182 = 0.033 2 / 182 = 0.011 1 / 182 = 0.005
Document Frequency 20 1 5

In 1815 few could have predicted what was to happen to the British Empire. The country may have been celebrating the triumphs of Wellington and Nelson but the nation was at a crossroads and experiencing economic and social change that for many meant the very real possibility of imminent revolution. The existence of the British Empire had been threatened by the American revolution but since the loss of the American colonies in 1783 and the destruction of France's military and naval might, Britain stood at the threshold of a new era which would be dominated by Britain's monopoly of the techniques of steam power and the existence of a group of entrepreneurs who existed in a political and economic climate that encouraged innovation. In 1815 British industry was based on the textile industry but within a few decades the United Kingdom was regarded by the British as the 'Workshop of the World' and the growing empire, both formal and informal, provided the market and raw materials for this growing economy which was not challenged until the turn of the twentieth century.

britain america france
Term Frequency 6 / 182 = 0.033 2 / 182 = 0.011 1 / 182 = 0.005
Inverse Document Frequency 20 / 20 20 / 1 20 / 5

In 1815 few could have predicted what was to happen to the British Empire. The country may have been celebrating the triumphs of Wellington and Nelson but the nation was at a crossroads and experiencing economic and social change that for many meant the very real possibility of imminent revolution. The existence of the British Empire had been threatened by the American revolution but since the loss of the American colonies in 1783 and the destruction of France's military and naval might, Britain stood at the threshold of a new era which would be dominated by Britain's monopoly of the techniques of steam power and the existence of a group of entrepreneurs who existed in a political and economic climate that encouraged innovation. In 1815 British industry was based on the textile industry but within a few decades the United Kingdom was regarded by the British as the 'Workshop of the World' and the growing empire, both formal and informal, provided the market and raw materials for this growing economy which was not challenged until the turn of the twentieth century.

britain america france
Term Frequency 6 / 182 = 0.033 2 / 182 = 0.011 1 / 182 = 0.005
Inverse Document Frequency log(20 / 20) = 0 log(20 / 1) = 2.99 log(20 / 5) = 1.38

TFIDF

$$tfidf(t, d, D) = tf(t, d) \cdot idf(t, D)$$

In 1815 few could have predicted what was to happen to the British Empire. The country may have been celebrating the triumphs of Wellington and Nelson but the nation was at a crossroads and experiencing economic and social change that for many meant the very real possibility of imminent revolution. The existence of the British Empire had been threatened by the American revolution but since the loss of the American colonies in 1783 and the destruction of France's military and naval might, Britain stood at the threshold of a new era which would be dominated by Britain's monopoly of the techniques of steam power and the existence of a group of entrepreneurs who existed in a political and economic climate that encouraged innovation. In 1815 British industry was based on the textile industry but within a few decades the United Kingdom was regarded by the British as the 'Workshop of the World' and the growing empire, both formal and informal, provided the market and raw materials for this growing economy which was not challenged until the turn of the twentieth century.

britain america france
Term Frequency 6 / 182 = 0.033 2 / 182 = 0.011 1 / 182 = 0.005
Inverse Document Frequency log(20 / 20) = 0.000000001 log(20 / 1) = 2.99 log(20 / 5) = 1.38
TFIDF 0.0000000000033 0.98 0.069

Vector Suche

britain america france
Dokument 0.0000000000033 0.98 0.069

Vector Suche

britain america france
Dokument 0.0000000000033 0.98 0.069
britain 1 0 0

Vector Suche

$$sim(Q, D) = \frac{Q \cdot D}{||Q|| ||D||}$$

Vector Suche

britain america france
Dokument 0.0000000000033 0.98 0.069
britain 1 0 0 0.000000000000336

Vector Suche

britain america france
Dokument 0.0000000000033 0.98 0.069
britain 1 0 0 0.000000000000336
america 0 1 0 0.998

Vector Suche

britain america france
Dokument 0.0000000000033 0.98 0.069
britain 1 0 0 0.000000000000336
america 0 1 0 0.998
britain america 1 1 0 0.705
Screenshot of the Nachtragswörterbuch des Sanskrit
http://nws.uzi.uni-halle.de
Screenshot of the National Museums Liverpool landing page
http://www.liverpoolmuseums.org.uk
Screenshot of the Tate Modern gallery landing page
https://www.tate.org.uk/visit/tate-modern
Screenshot of the Deutsches Museum landing page
https://www.deutsches-museum.de/
Screenshot of the Europeana landing page
http://www.europeana.eu

Recall vs Precision

Screenshot of the V & A Collection Inteface for the Medieval Content
https://www.vam.ac.uk/collections/medieval
Screenshot of the Tyne & Wear Collection Dive
http://www.collectionsdivetwmuseums.org.uk

Literatur

  • Baeza-Yates, R. and Ribeiro-Neto, B. (2011). Modern Information Retrieval (2nd ed.). Addison-Wesley.
  • Ruthven, I. and Chowdhury, G.G. (eds.)(2015). Cultural Heritage Information. Access and management. Facet publishing.

Übungsaufgabe