This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
To provide health research teams with a practical, methodologically rigorous guide on how to conduct direct observation.
Synthesis of authors’ observation-based teaching and research experiences in social sciences and health services research.
This article serves as a guide for making key decisions in studies involving direct observation. Study development begins with determining if observation methods are warranted or feasible. Deciding what and how to observe entails reviewing literature and defining what abstract, theoretically informed concepts look like in practice. Data collection tools help systematically record phenomena of interest. Interdisciplinary teams--that include relevant community members-- increase relevance, rigor and reliability, distribute work, and facilitate scheduling. Piloting systematizes data collection across the team and proactively addresses issues.
Observation can elucidate phenomena germane to healthcare research questions by adding unique insights. Careful selection and sampling are critical to rigor. Phenomena like taboo behaviors or rare events are difficult to capture. A thoughtful protocol can preempt Institutional Review Board concerns.
This novel guide provides a practical adaptation of traditional approaches to observation to meet contemporary healthcare research teams’ needs.
Keywords: Direct Observation, Methods, Qualitative Methods, Ethnography, Health Services Research
Health research studies increasingly include direct observation methods [[1], [2], [3], [4], [5]]. Observation provides unique information about human behavior related to healthcare processes, events, norms and social context. Behavior is difficult to study; it is often unconscious or susceptible to self-report biases. Interviews or surveys are limited to what participants share. Observation is particularly useful for understanding patients’, providers’ or other key communities’ experiences because it provides an “emic,” insider perspective and lends itself to topics like patient-centered care research [1,5,6]. This insider perspective allows researchers to understand end users’ experiences of a problem. For example, patients may be viewed as “non-compliant,” while observations can reveal daily lived experiences that impede adherence to recommended care [[7], [8], [9], [10]]. Observation can examine the organization and structure of healthcare delivery in ways that are different from, and complementary to, methods like surveys, interviews, or database reviews. However, there is limited guidance for health researchers on how to use observation.
Observation has a long history in the social sciences, with participant observation as a defining feature of ethnography [[11], [12], [13]]. Observation in healthcare research differs from the social sciences. Traditional social science research may be conducted by a single individual, while healthcare research is often conducted by multidisciplinary teams. In social science studies, extended time in the field is expected [11]. In contrast, healthcare research timelines are often compressed and conducted contemporaneous with other work. Compared to social science research questions, healthcare studies are typically targeted with narrowly defined parameters.
These disciplinary differences may pose challenges for healthcare researchers interested in using observation. Given observation’s history in the social sciences there is a need to tailor observation to the healthcare context, with attention to the dynamics and needs of the research team. This paper provides contemporary healthcare research teams a practical, methodologically rigorous guide on when and how to conduct observation.
This article synthesizes the authors’ experiences conducting observation in social science and health services research studies, key literature and experiences teaching observation. The authors have diverse training in anthropology (GF, MM), systems engineering (BK) and psychology (MR). To develop this guide, we reflected on our own experiences, identified literature in our respective fields, found common considerations across these, and had consensus-reaching discussions. We compiled this information into a format initially delivered through courses, workshops, and conferences. In keeping with this pedagogical approach, the format below follows the linear process of study development.
Following the trajectory of a typical health research project, from study development through data collection, analysis and dissemination ( Fig. 1 ), we describe how to design and conduct observation in healthcare related settings. We conclude with data analysis, dissemination of findings, and other key guidance. Importantly, while illustrated as a linear process, many steps inform each other. For example, analysis and dissemination, can inform data collection.
Direct observation across a health research study.
In developing research using observation, the first step is determining if observation is appropriate. Observation is ideal for studies about naturally occurring behaviors, actions, or events. These include explorations of patient or provider behaviors, interactions, teamwork, clinical processes, or spatial arrangements. The phenomena must be feasible to collect. Sensitive or taboo topics like substance use or sexual practices are better suited to other approaches, like one-on-one interviews or anonymous surveys. Additionally, the phenomena must occur frequently enough to be captured. Trying to observe rare events requires considerable time while yielding little data. Early in the study design process, the scope and resources should be considered. The project budget and the timeline need to account for staffing, designing data collection tools, and pilot testing.
Research questions establish the study goals and inform the methods to accomplish them. In a study examining patients’ experiences of recovery from open heart surgery, the ethnographic study design included medical record data, in-depth interviews, surveys, and observations of patients in their homes, collected over three months following surgery [7]. By observing patients in their homes GF saw how the household shaped post-surgical diet and exercise. Table 1 provides additional examples of healthcare studies using observation, often as part of a larger, mixed-method design [14,15].
Example studies that use observation.
| Research Topic | Study Design | Use of Observation |
|---|---|---|
| Organization, structure and process of HIV care. | Mixed Methods (survey, interviews and observation) | Site visits with observations of clinical encounters and staff work routines [1,16] |
| Identification of contextual factors influential in the uptake and spread of an anticoagulation improvement initiative. | Mixed Methods (survey, interviews, observation, and Interrupted time series) | Observations of clinical processes and clinical encounters with patients and of site champion quality improvement team meetings [4] |
| Examination of how physicians respond verbally and nonverbally to patient pain cues. | Observation of clinical interactions | Observations of clinical encounters [17,18] |
| Determination of proportion of tasks that are commonly carried out by clinical pharmacists can be appropriately managed by clinical pharmacy technicians. | Mixed Methods (modified Delphi process and observation) | Observation of pharmacists carrying out work tasks in a time-motion study [19] |
The phenomena to observe should be clearly defined. Research team discussions create a unified understanding of the phenomena, clarify what to observe and record, and ensure data collection consistency. This explication specifies what to look for during observation. For example, a team might operationalize the concept of patient-centered care into specific actions, like how the provider greets the patient. Further, additional nuances within broader domains (e.g., patient-centered care) could be identified while observations are ongoing. The team may identify unanticipated ways that providers enact patient-centered care (e.g., raising non-clinical, but relevant psychosocial topics- like vacations or hobbies- prior to gathering biomedical information). It is also important to look for negative instances, or behaviors that did not happen that should have, or surprising, unexpected findings. A surprise finding during observation was the impetus for further analysis examining how HIV providers think about their patients. While observing HIV care, a provider made an unexpected, judgmental comment about patients who seek pre-exposure prophylaxis (PrEP) to prevent HIV. This statement was documented in the fieldnotes (see 3.1.3 for a further description of fieldnotes) and later discussed with the team, leading to review of other study data and an eventual paper (see Fix et al 2018) [1]. Leaving room, both literally on the template and conceptually, can provide space for new, unexpected insights.
The sampling strategy outlines the frequency and duration of what is observed and recorded. It requires determining the unit of observation and the observation period. Units of observation are sometimes called “slices” of data. Ambady and Rosenthal [20] coined the term thin slices, using brief exposures of behavior (6s, 15s, and 30s) to predict teacher effectiveness. While thin slices are predominantly used in psychology, healthcare researchers can apply this concept by recording data for set blocks of time in a larger process, such as recording emergency department activity for the first 15 minutes of each hour.
The unit of observation can be a person (e.g., patient, provider), their behavior (e.g., smiling, eye rolling), an event (e.g., shift change) or interaction (e.g., clinical encounter). Using interactions as the unit of observation requires consideration for repeat observations of some individuals. For example, a fixed number of providers may be repeatedly observed with different patients.
Observation frequency will depend on the frequency of the phenomena. Enough data is needed for variation while also achieving “saturation,” a concept from qualitative methods, which means the point in data collection when no new information is obtained [21]. For quantitative studies, when examining the relationship between a direct observation measure (e.g., patient smiling) and an outcome (e.g., patient satisfaction), effect sizes from past research should dictate the number of interactions needed to achieve power to detect an effect. The duration of observation (the data slice) can be constrained using parameters as broad as a clinic workday, to distinct events like a clinical encounter.
Observation data can be collected on a continuous, rolling basis, or at predefined intervals. Continuous sampling is analogous to a motion picture—the recorded data mirrors the flow of information captured in a video [22]. Continuous observation is ideal for understanding what happens throughout an event. It is labor intensive and time-consuming and may result in a small number of observations, although each observation can yield considerable data. For example, a team may want to know about the patient-centeredness of patient-provider interactions. Continuous sampling of a clinical encounter could start when the patient arrives through when they leave, with detailed data collected about both the verbal and nonverbal communication. This could be considered an N of one observation but would yield substantial data. This information could be collected over a continuous day of encounters across several providers and patients, resulting in a considerable amount of data for a small group of people.
In contrast, instantaneous sampling can be conceptualized as snapshots, and is analogous to the thin slice methodology. Psychology research sometimes uses random intervals, while in healthcare research it may be preferable to use predetermined criteria or intervals [23]. Instantaneous sampling is economical and data collection can happen flexibly across a variety of individuals or times of day or weeks. Disadvantages include losing some of the context that is gained through continuous sampling.
Data collection tools enable systematic observations, codifying what to observe and record. These tools vary from open-ended to highly structured, depending on the research question(s) and what is known a priori. We describe below three general tool categories—descriptive fieldnotes, semi-structured templates, and structured templates.
Descriptive fieldnotes, common in anthropology, are open-ended notes recorded with minimal a priori fields. Descriptive fieldnotes are ideal for research questions where less is known. An almost blank page is used to record the phenomena of interest. Key information such as date, time, location, people present and who recorded the information are useful for later analysis. These notes are jotted sequentially in real-time to maximize data collection, and are filled out and edited later for clarity and details. The flexible and open format facilitates the capture of unanticipated events or interactions.
Descriptive fieldnotes describe in detail what is observed (e.g., who is present, paraphrased statements), while leaving out interpretation. Analytic notes, that interpret what is being observed, can accompany the descriptive notes (e.g., the doctor is frowning and seems skeptical of what the patient is saying), but these analytic notes should be clearly marked as interpretation. One author (GF) demarcates interpretive portions of her fieldnotes using [closed brackets] to identify this portion of the fieldnote as distinct from the descriptive data. Interpretive notes should explain why the observer thinks this might be the case, using supporting data from the observation. Building on the example above, an accompanying interpretive note might say, “[the doctor raised their eyebrows, and does not seem to believe what the patient is saying, similar to what was observed in another encounter- see site 5 fieldnote). This information can be valuable during analysis to contextualize what was recorded and used in a later report or paper. Observation experience builds comfort and expertise with the open-ended, unstructured format.
A semi-structured template comprises both open-ended and structured fields ( Fig. 2 ). It includes the same key information described above (i.e., date, time, etc.), then provides prompts for a priori concepts underlying the research questions, often derived from a theoretical model. These literature-based, theoretical concepts should be clearly defined and operationalized. For example, drawing from Street et al’s [24] framework for patient-centered communication, we can use their six functions (fostering the patient-clinician relationship, exchanging information, responding to emotions, managing uncertainty, making decisions, and enabling self-management) to develop categories for semi-structured coding a template. Like descriptive fieldnotes, the template also provides open-ended space for capturing contextual details about the a priori data recorded in the structured section.
Semi-Structured Observation Template.
