Historical censuses are one of the most challenging datasets to compare over time. While many (successful) efforts have been made by researchers to harmonize these types of data, a lack of a generic workflow thwarts other researchers in their endeavors to do the same. In order to use historical census data for longitudinal analysis, a common process currently often loosely referred to as harmonization is inevitable. This process becomes even more challenging when dealing with aggregate data. Current approaches, whether focusing on micro or aggregate data, mainly provide specific, goal-oriented solutions to solve this problem. The nature of our data calls for an approach which allows different interpretations and preserves the link to the underlying sources at all times. To realize this we need a flexible, bottom-up harmonization process which allows us to iteratively discover the peculiarities of these types of data and provide different interpretations on the same data in an accountable way. In this article, we propose an approach which we refer to as source-oriented harmonization. We use the Resource Description Framework from (RDF) as the technological backbone of our efforts and aim to make the process of harmonization more graspable for others to stimulate similar efforts.