Methodological fit

Do you know what methodology you should use for your research project?

If there is a lot of extensive literature for a topic, then, according to Edmonson and McManus (2007) one could make a contribution to a mature theory then quantitative methodology would be the best methodological fit. If one strays and does a qualitative methodology in this case, they could run into reinventing the wheel error and may fail to fill a gap in the body of knowledge.

If there is just a little literature for a topic, then one could make a contribution to a nascent theory via qualitative methodologies, which in turn would be the best methodological fit (Edmonson & McManus, 2007).  If you do a quantitative research project here, you may be jumping the gun and running into possible false conclusions caused by confounding variables and may still fail to fill the gap in the body of knowledge.

Finally, one can stray from both pure qualitative and quantitative methodologies, and go into a mixed-methods study, and this can occur when there is enough research that the body of knowledge isn’t considered nascent, but not enough to be considered mature (Edmonson & McManus, 2007). Going one route here would do an injustice in filling in the gap in the body of knowledge, because you may be missing key insights that the each part of the mixed methodology (both qualitative and quantitative) can bring to the field.

So, prior to deciding which methodology you should choose, you should do an in-depth literature review.  You cannot pick an appropriate methodology without knowing the body of knowledge.

Hint: The more quantitative research articles you find in a body of knowledge, the more likely your project will be dealing with either a mixed-methods (low number of articles) or a quantitative method (high number of articles) project. If you see none, you may be working on a qualitative methodology.

Reference

  • Edmondson, A., & McManus, S. (2007). Methodological fit in management field research. Academy of Management Review, 32(4), 1155–1179. CYBRARY.

Worldviews and Approaches to Inquiry

The four worldviews according to Cresswell (2013) are postpositivism (akin to quantitative methods), constructivism (akin to qualitative methods), advocacy (akin to advocating action), and pragmatism (akin to mixed methods).   There are positives and negatives for each world view. For pragmatists, they use what truth and what methods from anywhere that works at the time they need it, to get the results they need.  Though the pragmatist research style takes time to conduct.  The advocacy places importance on creating an action item for social change to diminish inequity gaps between asymmetric power relationships like those that exist with class structure and minorities.  Though this research is noble, the moral arc of history bends towards justice, but very slowly, it took centuries for race equality to be where it is at today, it took over 60 years for gender equality, and 40 years for LGBT equality.  Yet, there are still inequalities amongst these groups and the majority that have yet to be resolved.  For instance: Equal Pay for Equal Work for All, Employment/Housing Non-Discrimination for LGBT, Racial Profiling, etc.  The constructivist viewpoint researchers seek to understand the world around them through subjective means.  They use their own understanding and interpretation of historical and cultural settings of participants to shape their interpretation of the open-ended data they collect.  This can lead to an interpretation that is shaped by the researcher’s background and not representative of the whole situation at hand.  Finally, postpositivism looks at the world in numbers, knowing their limitation that not everything can be described in numbers, they choose to propose an alternative hypothesis where they can either accept or reject the hypothesis. Numbers are imperfect and fallible.

My personal world view is akin to a pragmatist world view.  My background in math, science, technology, and management help me synthesize ideas from multiple fields to drive innovation.  It has allowed me to learn rapidly because I can see how one field ties to the other and makes me more adaptable.   However, I also lean a bit more strongly to the math and science side of myself, which is a postpostivism view.

Resource:

The Role of Theory

The theory is intertwined with the research process, thus a thorough understanding of theory must involve the understanding of the relationship between theory and research (Bryman & Bell, 2007).  When looking at research from a deductive role (developing and testing a problem and hypothesis) the theory is presented at the beginning.  The theory here is being tested, as it helps define the problem, its parameters (boundaries) and a hypothesis to test.  Whereas an inductive role uses data and research to build a theory.  Theories can be grand (too hard to pinpoint and test) or they can be mid-range (easier to test, but it is still too big to test it under all assumptions) (Bryman & Bell, 2007).

Where you write your theory depends on the type of world view you have (positivism at the beginning of the paper, or constructivism at the beginning or end of the paper) (Creswell, 2013).   My particular focus will be on the postpositivism view (quantitative methods), so I will dissect the placement of the theory primarily on a quantitative research study (which are mostly deductive in nature).  Placement of the theory in the introduction lit review, or after the hypothesis runs into the issue that it will make it harder for the reader to isolate and separate the theory from their respective sections (Cresswell, 2013).  There is another disadvantage from what Creswell (2013) states for the after the hypothesis approach: you may forget to discuss the origins and rationale for the theory.  Cresswell (2013), suggests as a research tip to separate the theory and create a brand new section for it so that it is easily identified and its origin and rationale can be elaborated on.

However, separating the theory section from the rest of the paper can still get the paper tossed out of being published in a journal if it is still fuzzy to decipher amongst your peers and the editor.  Feldman’s 2004 editorial states that if the question & theory is succinct, grammatically correct, non-trivial, and makes a difference, it would help you get your results published.  However, he also states (like many of our professors do) we need to find what are the key articles and references in the past 5 years, that we should be exhaustive yet exclusive with our dataset, and establish clear boundary conditions such that we can adequately define independent and dependent variable would help you get your results published (Feldman, 2004).  The latter set of conditions helps build your theory, whereas the first set of conditions speaks to the readability of the theory.  If it is hard to read your theory because it’s so convoluted, then why should anyone care to read it?

Resources:

  • Bryman, A. & Bell, E. (2007) Business Research Methods. (2nd ed.). Location: Oxford University Press.
  • Creswell, J. W. (2013). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 4th Edition. [VitalSource Bookshelf version]. Retrieved from http://online.vitalsource.com/books/9781483321479/epubcfi/6/24
  • Feldman, D. C. (2004). What are we talking about when we talk about theory? Journal of Management, 30(5), 565–567.

Literature reviews

Side Note: This particular post was on my to-do list for a long time.

A literature review as a process containing a deep consideration of the current literature, to aid in identifying the current gaps in the existing knowledge, as well as building up the context for your research project (Gall, Gall, & Borg, 2006).  The literature review helps the researcher to build upon the works of other researchers, for the purpose of contributing to the collective knowledge. Our goal in the literature review will be undermined if we conduct any of the following common flaws (Gall et al., 2006):

  1. A literature review that becomes a standalone piece in the final document
  2. Analyzing results from studies that are not sound in their methodology
  3. Include the search procedures used to create this literature review
  4. Having only one study on particular ideas in the review, which may suggest the idea is not mature enough

For a literature review, one should be learning their field by reviewing the collective knowledge in the field by studying:

  • The beginning of {your topic}
  • The essence of {your topic}
  • Historical overview {your topic}
  • Politics of {your topic}
  • The Technology of {your topic}
  • Leaders in {your topic}
  • Current literature findings of {your topic}
  • Overview of research techniques {your topic}
  • The 21st century {your topic} Strategy

Creswell’s (2014), proposed that a literature map (similar to a mind map) of the research is a useful way to organize the literature, identify ideas with a small number of sources, determine the current issues in the existing knowledge, and determine the reviewers current gap in their understanding of the existing knowledge.  Finally, Creswell in 2014, listed what a good outline for a quantitative literature review should have:

  1. Introduction paragraph
  2. Review of topic one, which contains the independent variable(s).
  3. Review of topic two, which contains the dependent variable(s).
  4. Review of topic three, which provides how the independent variable(s) relate to the dependent variable(s).
  5. Summarize with highlights of key studies/major themes, to state why more research is needed.

Cresswell’s is generally a good method, but not the only one.  You can use a chronological literature review, where you build your story from the beginning to the present. In my dissertation, my literature review had to tie multiple topics into one: Big Data, Financial forecasting, and Hurricane forecasts.  I had to use the diffusion of innovation theory to transition between Financial and Hurricane forecast, to make the leap and justify the methodologies I will use later on.  In the end, you are the one that will be writing your literature review and the more of them you read, the easier it will be to define how you should write yours.

Here is a little gem I found during my second year in my dissertation: Dr. Guy White (2014) in the following youtube video has described a way to effectively and practically build your literature review. I use this technique all the time.  All of my friends that have seen this video have loved this method of putting together their literature reviews.

References

Internal validity in qualitative studies

Internal validity is determining the accuracy of the findings in qualitative research from the viewpoints of the researcher, participants or reader (Creswell, 2013). There are many validity strategies like: Triangulation of different data sources, member checking, rich thick description of the findings, clarifying any bias, presenting negative or discrepant information, prolong the time in the field, peer debriefing, external auditor to review the project, etc.

Triangulation of different data sources for observational work is an idea where I would examine evidence from multiple sources of data to justify the themes that I create through coding.  Converging themes from multiple sources of data and/or perspectives from participants would add to the validity of the study.  Thus, in order to increase the validity of the thematic codes would be to present the thematic codes from analysis of multiple sources like:

  • Interviews from N number of participants (until data saturation is reached)
  • Observations of the participants
    • Repeated observations will be taken, during multiple different types of shifts, with or without the same participants and during different random days of the week over a one-month period.
    • Observational Goals: Tracking what information is used (type and time stamps, instrumentations, etc.)
    • Observational Goals 2: Through videotaping, I hope to track conversations between participants sharing the same shift. Field notes would contain: “Why the conversation was initiated?”, “What was discussed?”, “Were there decisions made regarding the area of study”, “What is the bodily-based behavior portrayed by the specialists in the discussion?”, and “What was the outcome of that discussion?”
  • Document Analysis

The aforementioned, in particular, will help ensure internal validity in quiet a few studies.

 References:

Ethical issues involving human subjects

In Creswell (2013), it is stated that ethical issues can occur at all phases of the study (prior to the study, in the beginning, during data collection, analysis, and reporting).  Since we deal with data from people about people, we as researchers need to protect our participants and promote the integrity of research by guarding against misconduct and improperly reflecting the data.  Because we deal with people, it is our obligation to assure that interviewees do not get harmed as a result of our research (Rubin, 2012). The following anticipated risks are from Crewell (2013) and Rubin (2012):

  • Prior to conducting the study
    • We must seek an Institutional Review Board (IRB) approval before we conduct a study.
    • I must gain local permission from the agency, organization, corporation for which the study will take place and from the participants to conduct this study.
  • Beginning the study
    • We will not pressure participants to sign consent forms. To make sure that you have high participation rates, you need to make sure that the purpose of this study is compelling enough that the participants will see that it would be a value-added experience to them as well as to the field of study that they don’t want to say no.
      • We should also conduct an informal needs assessment to ensure that the participant’s needs are addressed in the study, to ensure a high participation rate.
      • But, we will tell the participants that they have the right not to sign the consent form.
    • Collecting data
      • Respecting the site and keep disruption to a minimum, especially if I am conducting observations. The goal of the observation in this study is not to be an active participant, but taking field notes of key interactions that occur while the participants are doing what they need to do.
      • Make sure that all the participants in the study receive the same treatment to avoid data quality issues while collecting it.
      • We should be respectful and straightforward to the participants.
      • Discuss the purpose of this study and how the data will be used with the participants is key to establishing trust and this would allow them to start thinking about the topic of the study. This can be accomplished by sending them an email prior to the interview as to the purpose of the study and the time we are requesting of them.
      • As we are asking our interviewing questions, we should avoid leading questions. That is why questions may be asked in a particular order.  In some cases, questions can build on one another.
      • We should avoid sharing personal impressions. Given that we know what the final questions in the interview are, as we should ask them questions while not giving any indication of what we are looking for so that they don’t end up contaminating our data.
      • Avoid disclosing sensitive or proprietary information.
    • Analyzing data
      • Avoid only disclosing one set of results, thus we must report on multiple perspectives and report contrary findings.
      • Keeping the privacy of the participants, assuring that the names have been removed from the results as well as any other identifying indicators.
      • Honor promises, if I offer to the participant a chance to read and correct their interviews, I should do so as soon as possible after the interview.
    • Reporting, sharing and storing data
      • Avoid situations where there is a temptation to falsify evidence, data, findings or conclusions. This can be accomplished through using unbiased language appropriate for audiences.
      • Avoid disclosing harmful information of the specialist.
      • Be able to have data in a shareable format, however with keeping the privacy of the specialist as the main priority, while keeping the raw data and other materials for 5 years in a secure location. Part of this data should consist of the complete proof of compliance, IRB, lack of conflict of interest, for if and when that is requested.

References:

Observational protocol and qualitative documentations

As a researcher, you could be a non-participant to a full-on participant when observing your subjects in a study.  Thus, the observed/empathized behavioral and activities of individuals in the study are jotted down in field notes (Creswell, 2013).  Most researchers use an observational protocol to jotting down these notes as they observe their subjects.  According to Creswell (2013), this protocol could consist of: “separate descriptive notes (portraits of the participants, a reconstruction of dialogue, a description of the physical setting, accounts of particular events, or activities) [to] reflective notes (the researcher’s personal thoughts, such as “speculation, feelings, problems, ideas, hunches, impressions, and prejudices), … this form might [have] demographic information about the time, place, and date of the field setting where the observation takes place.”

Whereas, observational work can be combined with in-depth interviewing, and sometimes the observational work (which can be an everyday activity) can help prepare the researcher for the interviews (Rubin, 2012).  Doing so can increase the quality of the interviews because the interviewers know what the researcher has seen or read and can provide more information on those materials.  This can also allow the researcher to master the terminology before entering the interview. Finally, Rubin (2012) also states that cultural norms become more visible through observation rather than just a pure in-depth interview.

In Creswell (2013), Qualitative Documents are information contained within documents that could help a researcher out in their study that could be either public (newspapers, meeting minutes, official reports) and/or private (personal journals/diaries, letters, emails, internal manuals, written procedures, etc.) documents.  This can also include pictures, videos, educational materials, books, files. Whereas, Artifact Analysis is the analysis of the written text, usually are charts, flow sheets, intake forms, reports, etc.

The main analysis approach of this document would be to read the document to gain a subject matter understanding.  Document analysis would aid in quickly grouping, sorting and resort the data obtained for a study.  This manual will not be included in the coded dataset, but will help provide appropriate codes/categories for the interview analysis, in other words give me suggestions about what might be related to what.   Finally, one way to interpret this document would be for triangulation of data (data from multiple sources that are highly correlated) between the observation, interviews and this document.   

References

Organizational research & Participant Observer

For organizational research, some of their major goals for research are to examine their formation, recruitment of talent, adaption to constraints, types and causes, factors for growth, change and demise, which all fall under ethnographic studies (Lofland, 2005).  Ethnographic studies lend themselves much more nicely to participant-observers.

Participant observer is where the researcher/observer is not just only watching their subjects, but also actively participates (joins in) with their subject. The level of participation of the observer might impact what is observed (the more participation the harder it is to observe and take notes), thus low-key role participation is preferred.  Participating before the interviews will allow the observer to be sensitive to important issues otherwise missed. It is a more in-depth version of interviewing building on a regular conversation.  Participation may occur after watching for a while, focusing on a specific topic/question. (Rubin, 2012)

References:

Data Analysis of Qualitative data

Each of the methods has at its core a thematic analysis of data, which is methodically and categorically linking data, phrases, sentences, paragraphs, etc. into a particular theme.  Coring up these themes by their thematic properties helps in understanding the data and developing meaningful themes aiding in building a conclusion to the central question.

Ethnographic Content Analysis (Herron, 2015):  Thick descriptions (collection of field notes that describe and recorded learning and a collection of perceptions of the researcher) help in the creation of cultural themes (themes related to behaviors on an underlying action) from which information was interpreted.

Phenomenological data analysis (Kerns, 2014): Connections among different classes of data through a thematic analysis were used for which results could be derived from.

Case study analysis (Hartsock, 2014): Through the organization of data within a specific case design and treating each distinct data set as a case study, one could derive some general themes within each individual case.  Once, all these general themes are identified, we should look for some cross-case themes.

Grounded Theory Data Analysis (Falciani-White, 2013): Code data through comparing incidents/data to a category (by breaking down, analyzing, comparing, labeling and categorizing data into meaningful units of data), and integrating categories by their properties, in order to help you identify a few themes in order to drive a theory in a systematic manner.

References:

Interviewing strategy and qualitative sampling

As an interviewing strategy, open-ended questions leave the responses open to participant experience and categories and don’t close down the discussion or allow the participant to answer the question in one word (Snow et al, 2005).  Though in the past it was rejected because it did not involve a precise measurement, sometimes data that may not be easily measurable or counted, have value because of its intrinsic complexity and showcase of the “conditional nature of reality” (Rubin, 2012).  A whole field of text-analytics is aiming to prove that this data, considered as unstructured data, is an important part of knowledge discovery and knowledge sharing. Thus, Rubin (2012) says that open-ended questions grant the participant the chance to respond to the question in any way they choose, as elaborated on a response, allow participants to raise issues that are important to them, or even raise new issues not thought of by the interviewer.  Creswell (2013), further states that the more open the questions the better because it will allow the interviewer to listen to what people say and how they say, which can allow the participants to share their own views.  Usually, there are a few open-ended questions.  Finally, open-ended questions are used primarily in qualitative studies, but a mixture of both close-ended and open-ended questions could be asked in mixed methods studies.

One thing is to have the right questions as part of your interviewing strategies, it is another thing to have the right qualitative sampling plan.

Sampling Plans {purposeful/judgmental sampling, maximum variation sampling, sampling extreme or deviant cases, theoretical sampling, snowball/chain-referral sampling, cluster sampling, single-stage sampling, random sampling} (Creswell, 2013, Rubin, 2012, & Lofland et al, 2005). Here are just three of the many sampling plans listed in the sampling plan space.

  • Purposeful/judgmental sampling: In order to learn about a selective character, group, or category or their variations, you group the population into different characters, groups, or categories to collect data from with the participants now representing those divisions. (Creswell, 2013 & Lofland et al, 2005)
  • Maximum variation sampling: Allows for an analysis of error and bias in a phenomenon, through sampling and discovering the widest range of diversity in the phenomena of interest. (Lofland et al, 2005)
  • Snowball/chain-referral sampling: Asking your initial set of contacts with characteristics X, if they can refer to you their network that has the same characteristics X that you are studying. This is a means to enlarge your sample size and break down barriers to the entrance of your future participant. (Lofland et al, 2005). Depending on the characteristic X, like domestic violence, sexual assault, etc., this technique may run into IRB issues (Rubin, 2012).  Rubin (2012), stated that the way to avoid IRB issues if you have the current participants contact the future participants on your behalf to participate in the interview process, but this can drastically reduce the number maximum number of participants you could have gotten.

References: