Exploring Mixed Methods

Explanatory Sequential (QUAN -> qual)

According to Creswell (2013), this mix method style uses qualitative methods to do a deep dive into the quantitative results that have been previously gathered (often to understand the data with respect to the culture).  The key defining feature here is that quantitative data is collected before the qualitative data and that the quantitative data drives the results from the qualitative.  Thus, the emphasis is given to the quantitative results in order to explore and make sense of qualitative results.  It is used to probe quantitative results by explaining them via qualitative results.  Essentially, using qualitative results to enhance your quantitative results.

Exploratory Sequential (QUAL -> quan)

According to Creswell (2013), this mix method style uses quantitative methods to confirm the qualitative results that have been previously gathered (often to understand the culture behind the data).  The key defining feature here is that qualitative data is collected before the quantitative data and that the qualitative data drives the results from the quantitative.  Thus, the emphasis is given to the qualitative results in order to explore and make sense of quantitative results.  It is used to probe qualitative results by explaining them via quantitative results.  Essentially, using quantitative results to enhance your qualitative results.

Which method would you most likely use?  If your methodological fit suggests you to use a mixed-methods research project, does your world view colors your choice?

Resources

Quasi-experimental

In the Quantitative Methodology, there are experimental (deals with the impact of an outcome, while having a controlling variable to see if the tested variable does have an impact), quasi-experimental (deals with a non-random sample but still measures the impact of an outcome) and non-experimental (deals with generalizing/inferring about a population) project designs.

For a non-experimental project design, surveys are used as an instrument to gather data and help produce quantitative/numeric data to help identify trends and sentiment from a sample of a total population (Creswell, 2013).  The Pew Research Center (2015), wanted to analyze the changing attitudes on Gay Marriage a few days after the Supreme Court struck down the bans as unconstitutional, have asked:

Do you oppose/favor allowing gays and lesbians to marry legally? What is your current age? What is your Religious Affiliation? What is your Political Party?  What is your Political Ideology? What is your Race? What is your gender?

Pew found that overall, since they were conducting this survey since 2001, they have seen that in every descriptive variable classifying people has shown an increase in acceptance for marriage, with an overall 55% approval rating to 39%.  This example is not trying to explain a relationship but rather a trend.

For an experimental project design, it usually follows the following steps: Identification of participants, gathering of materials, draft and finalize procedures and setting up measures so that you can conduct the experiment and derive some results from it (Creswell, 2013).

When a participant in a study is randomly assigned to a control group or in other groups in an experiment it is considered a true experiment, if the participants in a study are not randomly assigned then it is considered a quasi-experiment (Creswell, 2013).  In the famous Milgram Obedience Experiment (1974), an ad was posted to collect participants for a study on memory, but in fact, they were there to see if the presence of authority would compromise their internal morals to cause pain and sometimes delivering fatal shocks to another participant (an actor).  About 2/3 of people were willing to administer the deadly shock because they had the presence of authority (a man in a white coat) telling them to continue to the study.  Though this study will be hard to replicate today (due to IRB considerations), it wasn’t fully random, thus it’s a quasi-experiment, but it challenged and shocked the world.  This is a pivotal paper/experiment that defined behavioral science.

Resources

Methodological fit

Do you know what methodology you should use for your research project?

If there is a lot of extensive literature for a topic, then, according to Edmonson and McManus (2007) one could make a contribution to a mature theory then quantitative methodology would be the best methodological fit. If one strays and does a qualitative methodology in this case, they could run into reinventing the wheel error and may fail to fill a gap in the body of knowledge.

If there is just a little literature for a topic, then one could make a contribution to a nascent theory via qualitative methodologies, which in turn would be the best methodological fit (Edmonson & McManus, 2007).  If you do a quantitative research project here, you may be jumping the gun and running into possible false conclusions caused by confounding variables and may still fail to fill the gap in the body of knowledge.

Finally, one can stray from both pure qualitative and quantitative methodologies, and go into a mixed-methods study, and this can occur when there is enough research that the body of knowledge isn’t considered nascent, but not enough to be considered mature (Edmonson & McManus, 2007). Going one route here would do an injustice in filling in the gap in the body of knowledge, because you may be missing key insights that the each part of the mixed methodology (both qualitative and quantitative) can bring to the field.

So, prior to deciding which methodology you should choose, you should do an in-depth literature review.  You cannot pick an appropriate methodology without knowing the body of knowledge.

Hint: The more quantitative research articles you find in a body of knowledge, the more likely your project will be dealing with either a mixed-methods (low number of articles) or a quantitative method (high number of articles) project. If you see none, you may be working on a qualitative methodology.

Reference

  • Edmondson, A., & McManus, S. (2007). Methodological fit in management field research. Academy of Management Review, 32(4), 1155–1179. CYBRARY.

Worldviews and Approaches to Inquiry

The four worldviews according to Cresswell (2013) are postpositivism (akin to quantitative methods), constructivism (akin to qualitative methods), advocacy (akin to advocating action), and pragmatism (akin to mixed methods).   There are positives and negatives for each world view. For pragmatists, they use what truth and what methods from anywhere that works at the time they need it, to get the results they need.  Though the pragmatist research style takes time to conduct.  The advocacy places importance on creating an action item for social change to diminish inequity gaps between asymmetric power relationships like those that exist with class structure and minorities.  Though this research is noble, the moral arc of history bends towards justice, but very slowly, it took centuries for race equality to be where it is at today, it took over 60 years for gender equality, and 40 years for LGBT equality.  Yet, there are still inequalities amongst these groups and the majority that have yet to be resolved.  For instance: Equal Pay for Equal Work for All, Employment/Housing Non-Discrimination for LGBT, Racial Profiling, etc.  The constructivist viewpoint researchers seek to understand the world around them through subjective means.  They use their own understanding and interpretation of historical and cultural settings of participants to shape their interpretation of the open-ended data they collect.  This can lead to an interpretation that is shaped by the researcher’s background and not representative of the whole situation at hand.  Finally, postpositivism looks at the world in numbers, knowing their limitation that not everything can be described in numbers, they choose to propose an alternative hypothesis where they can either accept or reject the hypothesis. Numbers are imperfect and fallible.

My personal world view is akin to a pragmatist world view.  My background in math, science, technology, and management help me synthesize ideas from multiple fields to drive innovation.  It has allowed me to learn rapidly because I can see how one field ties to the other and makes me more adaptable.   However, I also lean a bit more strongly to the math and science side of myself, which is a postpostivism view.

Resource:

The Role of Theory

The theory is intertwined with the research process, thus a thorough understanding of theory must involve the understanding of the relationship between theory and research (Bryman & Bell, 2007).  When looking at research from a deductive role (developing and testing a problem and hypothesis) the theory is presented at the beginning.  The theory here is being tested, as it helps define the problem, its parameters (boundaries) and a hypothesis to test.  Whereas an inductive role uses data and research to build a theory.  Theories can be grand (too hard to pinpoint and test) or they can be mid-range (easier to test, but it is still too big to test it under all assumptions) (Bryman & Bell, 2007).

Where you write your theory depends on the type of world view you have (positivism at the beginning of the paper, or constructivism at the beginning or end of the paper) (Creswell, 2013).   My particular focus will be on the postpositivism view (quantitative methods), so I will dissect the placement of the theory primarily on a quantitative research study (which are mostly deductive in nature).  Placement of the theory in the introduction lit review, or after the hypothesis runs into the issue that it will make it harder for the reader to isolate and separate the theory from their respective sections (Cresswell, 2013).  There is another disadvantage from what Creswell (2013) states for the after the hypothesis approach: you may forget to discuss the origins and rationale for the theory.  Cresswell (2013), suggests as a research tip to separate the theory and create a brand new section for it so that it is easily identified and its origin and rationale can be elaborated on.

However, separating the theory section from the rest of the paper can still get the paper tossed out of being published in a journal if it is still fuzzy to decipher amongst your peers and the editor.  Feldman’s 2004 editorial states that if the question & theory is succinct, grammatically correct, non-trivial, and makes a difference, it would help you get your results published.  However, he also states (like many of our professors do) we need to find what are the key articles and references in the past 5 years, that we should be exhaustive yet exclusive with our dataset, and establish clear boundary conditions such that we can adequately define independent and dependent variable would help you get your results published (Feldman, 2004).  The latter set of conditions helps build your theory, whereas the first set of conditions speaks to the readability of the theory.  If it is hard to read your theory because it’s so convoluted, then why should anyone care to read it?

Resources:

  • Bryman, A. & Bell, E. (2007) Business Research Methods. (2nd ed.). Location: Oxford University Press.
  • Creswell, J. W. (2013). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 4th Edition. [VitalSource Bookshelf version]. Retrieved from http://online.vitalsource.com/books/9781483321479/epubcfi/6/24
  • Feldman, D. C. (2004). What are we talking about when we talk about theory? Journal of Management, 30(5), 565–567.

Literature reviews

Side Note: This particular post was on my to-do list for a long time.

A literature review as a process containing a deep consideration of the current literature, to aid in identifying the current gaps in the existing knowledge, as well as building up the context for your research project (Gall, Gall, & Borg, 2006).  The literature review helps the researcher to build upon the works of other researchers, for the purpose of contributing to the collective knowledge. Our goal in the literature review will be undermined if we conduct any of the following common flaws (Gall et al., 2006):

  1. A literature review that becomes a standalone piece in the final document
  2. Analyzing results from studies that are not sound in their methodology
  3. Include the search procedures used to create this literature review
  4. Having only one study on particular ideas in the review, which may suggest the idea is not mature enough

For a literature review, one should be learning their field by reviewing the collective knowledge in the field by studying:

  • The beginning of {your topic}
  • The essence of {your topic}
  • Historical overview {your topic}
  • Politics of {your topic}
  • The Technology of {your topic}
  • Leaders in {your topic}
  • Current literature findings of {your topic}
  • Overview of research techniques {your topic}
  • The 21st century {your topic} Strategy

Creswell’s (2014), proposed that a literature map (similar to a mind map) of the research is a useful way to organize the literature, identify ideas with a small number of sources, determine the current issues in the existing knowledge, and determine the reviewers current gap in their understanding of the existing knowledge.  Finally, Creswell in 2014, listed what a good outline for a quantitative literature review should have:

  1. Introduction paragraph
  2. Review of topic one, which contains the independent variable(s).
  3. Review of topic two, which contains the dependent variable(s).
  4. Review of topic three, which provides how the independent variable(s) relate to the dependent variable(s).
  5. Summarize with highlights of key studies/major themes, to state why more research is needed.

Cresswell’s is generally a good method, but not the only one.  You can use a chronological literature review, where you build your story from the beginning to the present. In my dissertation, my literature review had to tie multiple topics into one: Big Data, Financial forecasting, and Hurricane forecasts.  I had to use the diffusion of innovation theory to transition between Financial and Hurricane forecast, to make the leap and justify the methodologies I will use later on.  In the end, you are the one that will be writing your literature review and the more of them you read, the easier it will be to define how you should write yours.

Here is a little gem I found during my second year in my dissertation: Dr. Guy White (2014) in the following youtube video has described a way to effectively and practically build your literature review. I use this technique all the time.  All of my friends that have seen this video have loved this method of putting together their literature reviews.

References

Some Qualitative Methodologies

This blog post will differentiate among the following qualitative designs:

    • Phenomenology (e.g. Georgi, Moustakas, etc.)
    • Grounded theory (e.g. Glaser, Strauss, etc.)
    • Ethnography (e.g. White, Benedict, Mead, etc.)
    • Case Studies (e.g. Yin, etc.)

The Implicit goal of qualitative data analysis is truth, objectivity, trustworthiness, and accuracy of data (Glaser, 2004). All methods have the observer usually exercising little bias in their thoughts to help further their analysis or development of their core theory.  Researchers here are observers taking notes to help them in their study.

Phenomenology (Giorgi, 2006): It is the study of experiential phenomena through encountering an instance of it, describing it, and using free imagination variation to determine its essence. Thus, making the phenomena more generalizable.  Though it should be noted that the experience should exist without preconceived biases (a neutral party), and one way of doing so is listing out your entire biases related to the phenomena.  This removal of biases will help limit the claims to the way we experienced the phenomena.

Grounded Theory (Glaser, 2004): It is the study of a set of grounded concepts, which create a core theory/category that forms a hypothesis.  Data is collected, but as it is analyzed “line by line”, the researcher asks: “What is this data a study of?”, “What category does this incident indicate?”, “What is actually happening in the data?”, “What is the main concern being faced by the participants?”, and “What Accounts for the continual resolving of this concern?”  These questions are asked within the most minimum of preconception.  The use of literature is treated as another source of data to be integrated into the analysis and core theory/category.  However, literature is not used before the emergence of a core theory/category arises from the data.

Ethnography (Atkinson & Hammersley, 1994, Mead, 1933): It is studying the customs of people and cultures, usually on a few numbers of cases (maybe one case), through analyzing unstructured data (not previously coded) with no aim of testing a hypothesis.  Analysis of the data may revolve quantification and statistics on the explicit interpretation of the data.

Thus, grounded theory seeks to find meaning in data and find a core concept/category/theory/variable.  Ethnography tends to seek meaning in the customs of people, which can exist in a single case study.  Phenomenology seeks to study the phenomena that have occurred while keeping in mind all the possible variables that can influence it.  So, a certain topic can be explored using each of these methods, and they are looking at the same problem just with different preconceptions (or lack thereof), thus adding to the further understanding of that topic.  These are all collection of data methods, whereas case studies are a research strategy.

A problem needs to arise in order for research to occur.  A gap in knowledge can be seen as a problem.  Thus, case studies are a strategy that can be used to help shine some light at that gap and using any of the techniques aforementioned the research can try to fill in that gap of knowledge.  If you are aiming for grounded theory, you may have a ton of case studies to look through to seek common themes, whereas ethnography may be concerned about one or two cases and what happened in those cases.  Phenomenology can use as many case studies necessary to explore any particular phenomena in question.

Case Studies Research (Yin, 1981): Can contain both qualitative and quantitative data (e.g. fieldwork, records, reports, verbal reports, observations, memos, etc.), and it is independent of any particular data collection method.  Case studies concern themselves in a real-life phenomenon, and when the boundaries between phenomenon and context are not known, yet aim to be either exploratory, descriptive and/or explanatory.  It is a strategy similar to experiments, simulations, and histories.

Since, case studies can be “an accurate rendition of the facts of the case” (Yin, 1981), most of that data cannot be described quantitatively in a quick manner. Sometimes, descriptions and qualitative data paint the picture of what is being studied much more clearly than if we were to do this with just numbers.  Can you picture that over a million people saw the ball drop on Time Square in 2015, or 14 blocks of thousands of people adorned in foam Planet Fitness hats and waving purple noodle balloons, eagerly cheered as the ball dropped on Time Square in 2015. This is why most case study research involves the collection of qualitative data.

References:

  • Atkinson, P., & Hammersley, M. (1994). Ethnography and participant observation. Handbook of qualitative research, 1(23), 248-261.
  • Glaser, B. G., & Holton, J. (2004, May). Remodeling grounded theory. In Forum Qualitative Sozialforschung/Forum: Qualitative Social Research (Vol. 5, No. 2).
  • Giorgi, A. (2008). Difficulties encountered in the application of the phenomenological method in the social sciences. Indo-Pacific Journal of Phenomenology, 8(1).
  • Mead, M. (1933). More comprehensive field methods. American Anthropologist, 35(1), 1-15.
  • Yin, R. K. (1981). The case study crisis: Some answers. Administrative science quarterly, 58-65.