In quantitative research, a study is valid if one could draw meaning and inferences from the results based on methodology employed. The three ways to look at validity is in (1) Content (do we measure what we wanted), (2) Predictive (do we match similar results, can we predict something), and (3) construct (are these hypothetical or real concepts). This is not to be confused with reliability & consistency. Thus, Creswell (2013) warns that if we modify an instrument or combine it with others, the validity and reliability of it could change, and in order to use it we must reestablish its validity and reliability. There are several threats to validity that exist, either internal (history, maturation, regression, selection, mortality, diffusion of treatment, compensatory/resentful demoralization, compensatory rivalry, testing, and instrumentation) or external (interaction of selection and treatment, interaction of setting and treatment, and interaction of history and treatment).
Sample Validity Considerations: The validity issues are and their mitigation plans
Internal Validity Issues:
Hurricane intensities and tracks can vary annually or even decadally. As time passes during this study for the 2016 and 2017 Atlantic Ocean Basin this study may run into regression issues. These regression issues threaten the validity of the study in a way that certain types of weather components may not be the only factors that can increase/decrease hurricane forecasting skill from the average. To mitigate regression issues, the study could mitigate the effect that these storms with an extreme departure from the average forecast skill have on the final results by eliminating them. Naturally, the extreme departures from the average forecast skill will, with time, slightly impact the mean, but their results are still too valuable to dismiss. Finding out which weather components impact these extreme departures from the average forecast skill is what drives this project. Thus, their removal doesn’t seem to fit in this study and defeats the purpose of knowledge discovery.
External Validity Issues:
The Eastern Pacific, Central Pacific, and Atlantic Ocean Basin have the same underlying dynamics that can create, intensify and influence the path of tropical cyclones. However, these three basins still behave differently, thus there is an interaction of setting and treatment threats to the validity of these studies results. Results garnered in this study will not allow me to generalize beyond the Atlantic Ocean Basin. The only way to mitigate this threat to validity is to suggest future research to be conducted on each basin separately.
- Creswell, J. W. (2013). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 4th Edition. [VitalSource Bookshelf version]. Retrieved from http://online.vitalsource.com/books/9781483321479/epubcfi/6/40