This module is intended to introduce you to the basics of data cleaning and analysis. The readings and discussion will introduce you to the (in)famous data/capta debate and some of the basics of data processing. The technical activities will let you try data cleaning with Open Refine and some basic (qualitative) data analysis.
Readings/Discussion
1. Johanna Drucker, “Humanities Approaches to Graphical Display,” DHQ 5, no 1 (2011).
2. Matthew Lavin, “Why Digital Humanists Should Emphasize Situated Data over Capta,” DHQ 15, no 2 (2021).
3. Clodomir Santana, et al, “Unveiling the Critical Nexus of Data Preprocessing and Transparent Documentation for Result Quality and Reproducibility in Digital History,” DHQ 19, no 2 (2025).
4. Tableau’s “Guide to Data Cleaning: Definition, Benefits, Components, And How To Clean Your Data”
5. OPTIONAL: Ashley R. Sanders, Visualizing History’s Fragments: A Computational Approach to Humanistic Research (Springer 2024) – you will need to request this via the libraries, it will arrive in 3-4 days
6. write a blog post or use the Slack channel to make note of your thoughts as you read and complete these exercises; depending on how many people are completing this optional module, this may or may not turn into an active discussion (that’s okay).
Technical Activities (Required)
1. download/install Open Refine and complete this Open Refine tutorial
NOTE: if you’re having trouble, you can also reference this Programming Historian tutorial to see if its explanations make more sense to you
2. create your own (small!) historical dataset and re-complete the tutorial with your own documents. What kinds of historical questions can you ask and answer with your dataset and this tool?
Technical Activities (Optional)
3. try this Programming Historian tutorial on fetching data from the internet with OpenRefine
4. annoyed by the messy OCR text we’ve generated elsewhere? try this pair of Programming Historian tutorials on regular expressions and cleaning messy OCR text using regular expressions