METHODS

By Thomas S Weisner

1 May 2012

Methods are systematic, socially agreed upon ways to represent the world. Mixed methods integrate qualitative and quantitative evidence through intentional efforts to focus “on research questions that call for real-life contextual understandings, multi-level perspectives, and cultural influences” (Cresswell, et al, 2011, Best Practices for Mixed Methods Research in the Health Sciences, p 4).

Good anthropology will always benefit from the widest variety of data. High quality examples of combining qualitative and quantitative methods abound in anthropology today and have done so throughout our history. Although ethnography and qualitative methods remain central, it has always been true that other methods are commonly used as well in every field of anthropology.

SOME EXAMPLES
Elinor Ochs and colleagues at UCLA assembled what is arguably the richest family database in the world today (combining video, sociolinguistic, ethnographic, questionnaire, daily diary, material possession, stress hormone and other evidence) in their study of the everyday lives of two-parent, middle class working Los Angeles families and their children (www.celf.ucla.edu). Robert LeVine and collaborators combined sociolinguistic, ethnographic, systematic observational, demographic, historical and child assessment methods in their study of the connections between women’s gains in literacy, lower completed family size, improved health and changes in maternal care in communities around the world (Literacy and Mothering: How Women’s Schooling Changes the Lives of the World’s Children, 2012). The New Hope community based work and family support study (Duncan, Huston and Weisner, Higher Ground: New Hope for the Working Poor and their Children, 2007) used a random-assignment social experiment, survey, questionnaire, child assessment and qualitative ethnographic fieldwork to discover why the program was successful in improving the well-being of parents and children, and yet why sometimes only selectively so.

Andrew Fuligni, Nancy Gonzalez and I currently collaborate on a study of the daily activities, family responsibilities and obligations, and academic and behavioral outcomes of 428 Mexican American immigrant teens and parents in Los Angeles (first, second and later generations, documented and not). Methods include 14-day consecutive daily diaries, survey and questionnaire data, and school and behavior assessments. In addition, a 10% nested random sample of parents and teens from this larger sample participate in a qualitative study in the homes of parents and children in addition. We gave cameras to adolescents in ninth and tenth grades with instructions to take 25 pictures of people, places, events and activities important to them. We plugged the cameras into our laptops and talked with the teens about their photos. We asked questions such as: Who are these friends; oh you have a boyfriend? Tell me more about your soccer team. That’s your Mom cooking; what do you do for chores? That’s one of your teachers? What class is it; how is school going? Teens take photos of other family members’ photos such as grandparents they cannot visit in Mexico; one took a photo of the moon, mentioning the film Under the Same Moon (La Misma Luna).

The narratives then can be recorded, transcribed and uploaded to a mixed methods software program such as Dedoose (www.Dedoose.com), a web-based mixed method software tool. Indexing and coding are a matter of dragging and dropping codes on the relevant portions of the text. Quantitative data from the larger study also are uploaded and linked to adolescent and parent narratives and photos. Narratives can be coded; patterns in quantitative data can be enriched qualitatively. The same fieldworkers who went to the homes and did interviews, also often worked on analyses of quantitative data.

STRENGTH OF INTEGRATED METHODS
Methods and research designs are languages understood across the social sciences. To the extent that we can speak those languages in our work, we more likely will draw in those in other disciplines into conversations with us. A study that creatively integrates quantitative and qualitative methods sends a positive message to those fluent in only qualitative or quantitative methods that we take their methods (and so their identities and ideas) seriously. The increased believability in our and others’ work which often results is itself a criterion for successful mixed methods research. The use of integrated methods is growing across the social sciences; psychology (eg, Yoshikawa, et al, Developmental Psychology 44[344–54]), sociology (eg, Mario Small in Annual Review of Sociology 37[57–86]), psychiatry (Palinkas, et al, Psychiatric Services 62 [3]), public health (Plano-Clark, Qualitative Inquiry16 [6]), political science, education, economics and other fields are benefitting and sometimes looking to anthropology for collaboration. Policy and practice research benefits hugely from integrating qualitative and quantitative methods. Funders increasingly see integrated methods as a strength in grant proposals.

The stark binary contrast of the “two Q’s”—qualitative vs quantitative—is not very useful; it restricts our thinking and limits our conversations. The two Q’s oversimplifies the debates and obscures important shared goals common to all methods. A better narrative and discourse about methods should use a richer conceptual framework. The actual contrast with quantitative levels of measurement (ordinal, interval, ratio scales) should be nominal or categorical levels (words, categories, narratives, themes, patterns); both are useful. The contrast with naturalistic research should not be experimental but research that is contrived or controlled in some systematic way to aid understanding. A useful framework for anthropology should distinguish person and experience-centered, or context-centered and variable-centered methods, not a qualitative/quantitative binary. Such a methods conversation could then focus on the most important Q—our common questions.

Many of us use ethnographic settings, events or activities as our units of analysis to be sure we do not bracket out context that provides essential meaning. However, inquiry across levels of analysis beyond settings and beyond projects often requires mixed methods. We often deal with suspicions about the “bias” of ethnographic and qualitative methods. Mixed methods do not necessarily lead to common findings; there is method variance just as there is expectable heterogeneity, conflict and inconsistency in cultural beliefs and practices themselves. A more useful question is whether our methods have been systematically context-examined or remain context-unexamined—since all methods (whether qualitative or quantitative) entail a context or a set of presumptions and methods effects of some kinds.

Quantitative methods and statistical analyses have guidelines and procedures (not uncontested of course) for deciding if they are done well—if they met accepted standards and should be published and disseminated for example. These include judgments of reliability, validity, sample size and representativeness or generalizability, power, and so forth. Qualitative and ethnographic work can and should have recognized criteria as well, such as breadth, depth, holism, veridicality, specificity of context, meaning centered, narrative and behavioral coherence, shared cognitions, interpretive richness, and others. These are of course more variable, and not so easy to define, yet they are valuable and defensible if carefully described. These should be in addition to explicit descriptions of sampling, setting, and so forth. Reasonable, flexible mixed methods criteria are being developed in these respects (Weisner and Fiese in Journal of Family Psychology 25[6]). Recent NIH guidelines have been developed for the use of mixed methods in health research and in applications for funding (Cresswell, et al, 2011).

METHODS PLURALISM IN ANTHROPOLOGY
I would guess—or at least hope—that most anthropologists are fairly tolerant pluralists regarding methods. Most of us appreciate the vast range of qualitative and ethnographic methods and their integration, as in Russ Bernard’s Research Methods in Anthropology. Qualitative and Quantitative Approaches (2011). I suspect many if not most of us generally agree with this view or use mixed methods in our own research and teaching, and regularly cite such work even if we don’t do this ourselves. If we don’t do quantitative research, we may have partnered with others who do and are interested in similar questions, or we may have taught courses using books and papers with quantitative evidence. And yet it is fair to say that those who critique quantitative methods, or dismiss systematic methods altogether, including mixed methods, sometimes, without justification in my view, seek to claim the dominant view. To the contrary: the future of our field and the social sciences is far more likely to be characterized by interdisciplinary methodological pluralism, often including integrated mixed methods. Anthropology should be at the forefront of such research and practice, not critiquing from the margins or simply ignoring important methodological and research design innovations.

Donald Campbell long ago described this more modest, pluralist, pragmatic, skeptical, empirically based approach to methods: he argued that all methods are valuable and important, but that all methods are also weak in the sense that they are incomplete representations of the incredibly complex world that we hope to understand. Hence we should use the widest range of methods, so that the weaknesses of one method can be complemented by the strengths of another, and so that phenomena in the world that are holistic qualities best or only to be represented by narrative, text, photos or sound are represented that way, and phenomena best or only to be represented with numbers, variables and models are represented quantitatively. As a result, we will get closer to understanding the world, and then persuading others of the truth of what we discover and believe.

Thomas S Weisner (www.tweisner.com) is anthropology professor in the departments of psychiatry and anthropology at UCLA, and director of Center for Culture & Health. His research and teaching interests are in culture and human development; medical, psychological and cultural studies of families and children at risk; mixed methods; and evidence-informed policy.