The BeeProject: Advanced Digitisation and Creation of a Dataset for the Monitoring of Beehives

The digitisation of historical documents, particularly those containing tabular data, is becoming increasingly critical for the preservation of information and analysis of long-term trends. However, this task presents significant challenges, particularly with semi-formal documents like handwritten records, which often need more consistent structure. This paper addresses the challenge of developing an automated approach for transcribing historical handwritten tables. Our presented method works on a mixture of computer vision tools and optical character recognition (OCR) to detect the grid and content of the table. The dataset we collected contains records from beekeepers, consisting of hive weight gain and loss and meteorological conditions. The Institute of Bee Protection at JKI gathered this information from the German beekeeper associations of Lower Saxony, Hesse, Mecklenburg-Vorpommern, Thuringia, and Brandenburg in Germany within the collaborative research project MonViA. This data is crucial for understanding the impact of climate change on bee vitality and contains daily information from each beekeeper over decades, holding valuable insights into past environmental conditions. The success rate of automatically transcribed hive scale data from Lower Saxony was compared with the accuracy of transcription done by human power. Our dataset of 14,738 handwritten scans, out of which 3819 were manually digitised, provides a large ground truth for future research, paving the way for further exploration and uncovering other historical knowledge.