Systematic document analysis.

By Prof. Dr. Alfred-Joachim Hermanni

1 Introduction
The purpose of conducting a systematic document analysis in a business context is to support an organization's goals for its entire business activities systematically and productively on the basis of concrete results. The goals can relate to different organizational areas of a company or industry, for example, management, corporate relations, marketing, corporate communication, human resources, logistics, legal, finance or controlling.

The underlying problem: in a medium-sized company belonging to an online shoe retailer, two academic employees were tasked with analyzing the customer complaints that had reached the retailer via social media platforms over the course of a fiscal year, with the aim of suggesting improvements for excellent customer service. The two employees first collected all written statements that had been submitted to the company's complaints office and then carried out a document analysis consisting of four steps: (1) formulating the research question; (2) defining what should be considered a document; (3) source criticism; and (4) interpretation of the document. When the two employees reached step 4 of the document analysis, they were already able to gain insights into the research question, understand the context of frequent customer concerns and explain the causes with reference to them (in the sense of Max Weber's interpretative sociology[1]). They then discussed with each other why a content analysis with a further six or seven steps (according to Mayring or Kuckartz) was actually still necessary and whether the overall ten- or eleven-part process could not be abbreviated.

The solution: systematic document analysis offers a five-step solution for such or other cases in a business context that are based on documents. In general, the following reasons can be decisive for an analysis of documents in a business context (such as the evaluation of publications, analysis of publications on the Internet, examination of events, evaluation of photographic images or films to extract certain information contained therein, for business process optimization, for job descriptions or for determining personnel requirements):

• Problems that arise at short notice and require prompt solutions,
• work and action requirements that have not yet been institutionally organized,
• on-the-job tasks in the sense of a learning, continuously developing organization,
• competitor analyses to identify best practices,
• knowledge generation for the organization's future planning.

The evaluation process is documented systematically and comprehensibly from the outset and contributes to problem solving and knowledge development.

1.1 Cooperation between employees/researchers and the organization
Organizations usually provide researchers with time, space, and access to information so that they can conduct a document analysis. In exceptional cases, funding is also provided, for example, to purchase and evaluate external data or to use technical equipment.

In the context of scientific work, the cooperation between employees/researchers and the organization should take place in a motivating work environment that is characterized by opportunities for participation, transparency and self-determination. [2] Only if acceptance within the company is ensured can the intended success of the collaboration and the desired result be achieved.

1.2 Definition of document
The word/term “document” comes from the Latin documentum and means something like piece of evidence, example, proof or testimony. Documents are therefore available and can be viewed as information and data on the area under investigation, for example:
• documents of all kinds (e.g. statistics, annual reports, organizational instructions),
• computer files that are available on a data carrier or storage medium or can be accessed via a database,
• court rulings, expert opinions, protocols or files (e.g. from authorities),
• non-fiction literature and research reports (also in electronic form),
• articles in newspapers, magazines (also in electronic form),
• visual, audiovisual or multimedia documents such as photos, illustrations, podcasts, film recordings (e.g. from radio and television programs),
• artistic works such as fiction or in the context of exhibitions in museums or art projects,
• websites, online articles, social media profiles,
• transcripts of interviews or diaries.

2 Differentiation from conventional document analysis
Conventional document analysis is an interpretative method and, on superficial examination, is difficult to distinguish from content analysis. The main difference between the two evaluation methods is that in a conventional document analysis, only a source evaluation or critique is carried out, and the evaluation process is carried out in the course of the content analysis. The focus of the explanations is on the business management approach.

Mayring regards conventional document analysis as a central method of action research[3], with the aim being to understand the meaning of the documents.[4] However, document analysis can also be used to systematically collect and evaluate texts.[5] In this case, the selected documents are analyzed using a defined category system in order to filter out central findings/results.

The evaluation of the selected documents is based on the interpretative paradigm (perceptions, interpretations and judgments of the researcher). It is therefore not surprising that conventional document analysis is combined with other evaluation techniques in order to provide a comprehensive answer to a research question. In many theoretical case studies or theses, it primarily serves to orientate oneself in the research field (in the sense of a preliminary investigation).

In this context, the systematic document analysis method appears above all as a planned and consistent procedure for gaining insights. This is particularly the case when other methods of obtaining scientific knowledge require a very difficult or cumbersome approach (e.g. by combining several methods or models; cf. introduction). As a prerequisite for using this method, documents must be available that can be examined epistemologically. The effort and duration of the investigation depend on the scope of the documents to be evaluated and the required depth of analysis.

3 Methodical approach “Systematic Document Analysis” (according to Hermanni)
Conventional document analysis is a technique for collecting data that is available as documents and requires further processing of the data through a content analysis. In contrast to this, systematic document analysis offers more than an introduction to a subject area and concludes with a research result. What is important is that the processes of examining communication material (texts, statistics, computer data, images, films, etc.) can be presented in a streamlined and simplified way.

The following figure 1 illustrates the process of a systematic document analysis, which enables a self-sufficient approach. Without having to resort to further methods or models for data evaluation, systematic document analysis is suitable for the scientific examination of communication material (texts, statistics, computer data, images, films, etc.). The analysis of the material's content was integrated into the method (see steps 3-5).

Source: Hermanni, A.-J. (2020)

Step 1: Selecting the documentation area
In advance, researchers identify or name the relevant documentation area, based on the research question/objective. If necessary, the documentation area is coordinated with the client commissioning the research. For example, researchers derive the documentation area from the company's request to analyze the content of the websites of the five largest competitors in order to determine differences or similarities with their own website.

The research question should be formulated as specifically as possible (what should be investigated?). Based on a research plan, the course of the investigation is recorded as specifically as possible from the perspective of a later implementation, for example:

(1) Population of documents
• Which material is used for the investigation and can be identified as a document?
• How many documents are evaluated?
• Is a sample (subset of a population) drawn because of the large number of documents? Why is a sample necessary and how large should it be?

(2) Type of documents
• What type/system unit of documents are involved (e.g. computer data, films, texts, images, files, disks, newspaper reports)? The documents should be identifiable and classifiable.
• External characteristics of the documents: condition of the material, etc.
• Internal characteristics of the documents: significance or content
• Intended effect of the documents: was the result aimed at a specific outcome? E.g. to influence target groups?
• Temporal, spatial, social or other proximity to the object of investigation
• Tradition and origin of the documents: authenticity (from a reliable source), source criticism (validity of the documents), etc.

(3) Time frame of the document analysis
• In which time frame does the investigation and the evaluation of the documents (in the form of results) take place?

(4) Number of analyses
• Does the investigation of the documents take place only once?
• Are the documents examined by one or more organizations/companies?

(5) Operationalization
• Are further methods or measuring instruments used in the course of the investigation in addition to systematic document analysis?

Step 2: Categorization of a document corpus
In order to be able to classify, interpret and evaluate the documents at a later stage, the necessary categories are derived from scientific theories or developed with the help of a structure tree[6] or a creativity technique (e.g. brainstorming, mind mapping or design thinking). In this context, subcategories can also be formed. In short: a category system arranges a comparability of the results.

Categories are defined according to Kant as a priori forms of thought in the sense of tools of knowledge and judgment that are to be regarded as given.[7] In this respect, the formation of categories is a fundamental cognitive process that is related to the perception, learning and knowledge processes of the human mind. Kuckartz points out that in a social science context, categories are to be regarded as classifications of units, for example of ideas, arguments or processes[8], which can have a greater or lesser degree of complexity.[9]

The document can be classified and evaluated according to different criteria, depending on the research question/objective/hypotheses of the work, for example:
• type of document (research report, certificate, article, radio report, photos, newspaper article, etc.),
• origin (specific historical, social, national, cultural area),
• distribution (offline or online, media reach, print run, etc.),
• authorship (one or more authors; male or female authors who have created a work),
• external characteristics: (preservation) state and material properties (e.g. easily legible texts, clear signs of wear in photos, complete pages),
• design characteristics (e.g. page format, typography, color scheme, image design, corporate identity, web layout),
• content-related characteristics (e.g. first publication, news value, attractiveness of the topic for the target group, image motifs, keywords, expert source references).

The following categories have proven useful for the analysis of websites, posts (in the field of social media), videos, print products or photo series:
• Communication strategy (target group, media use, advertising medium),
• Content (theoretical treatment, foundation of the information and comprehensibility of the argumentation),
• Concept and design (corporate design, form, color, images, etc.),
• Visual and verbal implementation (overall impression).

Step 3: Evaluation
The relevant documents are selected and the central findings/results are recorded. In systematic document analysis, a descriptive analysis is generally carried out. For example, technical literature is evaluated, whereby the information contained in the individual data is condensed and presented transparently. The essential information is presented in a clear and transparent way, for example in an Excel file, using key figures, in graphical representations, tables or formulas. The results are described objectively, but not commented on or evaluated. Interpretations and paraphrases are to be avoided.

The categories can also be evaluated using a survey method such as the multi-level Likert scale [10] (e.g. the researcher either “strongly disagrees” or “fully agrees” with the communication strategy). Nuanced answer options are possible and can be evaluated quickly.

Step 4: Review of relevant documents
The relevant documents are checked for contradictions, ambiguities, meaningfulness, etc., and irrelevant material is sorted out. Documents that have been sorted out are no longer considered for the subject of the study. The final selection of relevant documents is made, if necessary, in consultation with the client of the study (e.g. the company).

Step 5: Discussion
The results are interpreted and critically examined in relation to the research question/objective: What insights can be gained for the research question? In this context, the researcher's own approach is also reflected upon according to the scientific quality criteria of objectivity, reliability and validity.

Subsequently, the analysis is made available to the relevant scientific field and to the client who commissioned the study, and may be published. The steps described correspond to an ideal-typical process. In practice, it may be necessary to carry out individual steps of the process chain several times. Therefore, the method is to be understood in such a way that there can be interdependencies in the form of repetitive cycles.

4 Advantages and disadvantages of a systematic document analysis
In general, the advantages and disadvantages of different research techniques should be critically examined in order to choose a methodical procedure for the research project. The following table compares the characteristic advantages and disadvantages of systematic document analysis with conventional document analysis + content analysis.

Source: authors’ own work.

5 Conclusion
Systematic document analysis combines traditional document analysis with content analysis and thus shortens the research process. It is useful for researchers, facilitates the understanding of the scientific approach by providing a transparent insight into the evaluation process, and saves a significant amount of time in the research process.

In any research area where there is no direct access through a questionnaire or interview technique or through “on-site” experiments, existing data material can be reliably collected, analyzed and evaluated for the research project on the basis of the research question. The documents can be a wide range of materials: websites, texts, videos, books, print products or information on business process optimization, for job descriptions or for determining personnel requirements.

To sum up, it can be said that systematic document analysis has a lot of potential and can be integrated into almost any research plan. It is open to all questions and all types of documents. This is one of the method's great strengths, enabling new perspectives on existing problems, progress of developments, analysis of change, weaknesses of a process, systematic errors and a gain in knowledge.

References
Diekmann, Andreas (2011), Empirical Social Research, Fundamentals, Methods, Applications, 5th edition, Hamburg.
Hupfer, Barbara (2006): Die Gestaltung von Wissenskontexten, Wissensmanagement – von der lernenden zur wissenden Organisation, Institut für Wirtschaftsgestaltung, Munich. First published by the Federal Institute for Vocational Education and Training, Bonn. http://www.ifw01.de/text_pdfs/wirtschaftsphilosophie_wissen_2.pdf, accessed on 04.05.2020.
Kant, Immanuel (1930). In: Eisler, Rudolf (ed.), Kant-Lexikon, Nachschlagewerk zu Kants sämtlichen Schriften, Briefen und handschriftlichen Nachlaß, chapter category.
Kromrey, Helmut et al. (2016), Empirische Sozialforschung, 13th edition, Konstanz and Munich.
Kuckartz, Udo (2014), Qualitative Inhaltsanalyse: Methoden, Praxis, Computerunterstützung, 2nd edition, Weinheim/Basel.
Kuckartz, Udo (2016), Qualitative Inhaltsanalyse, Methoden, Praxis, Computerunterstützung, 3rd edition, Weinheim and Basel.
Likert, Rensis (1932), A technique for the measurement of attitudes. In: Archives of Psychology, 22, 140, p. 55.
Mayring, Philipp (2016), Einführung in die qualitative Sozialforschung, eine Anleitung zu qualitativem Denken, 6th edition, Weinheim and Basel.
Weber, Max (1922), Grundriss der Sozialökonomik, III. Abteilung Wirtschaft und Gesellschaft, edited by Max Weber, Tübingen.
Weber, Max (1980), Economy and Society, an outline of interpretative sociology, 5th edition, edited by Johannes Winckelmann, Tübingen.
[1] Cf. Weber (1922/1980)
[2] Hupfer (2006), p. 3
[3] Cf. Mayring (2016)
[4] Cf. Kromrey (2006)
[5] Cf. Dieckmann (2011), p. 576
[6] Based on a core concept, facets of a term are explored and weighted.
[7] Cf. Kant, Immanuel (1930)
[8] Cf. Kuckartz, Udo (2014), p. 41
[9] Cf. Kuckartz, Udo (2014), p. 46
[10] Cf. Likert (1932), p. 55