Nohel Corral

Overview

Miron and Kelaher-Young (2012) researched the Kalamazoo Promise, a scholarship program providing free college education for any local area high school student who graduates and is accepted to a public state school in Michigan. Specifically, this research considered the changes in student attitudes and behaviors as a result of the Kalamazoo Promise being implemented. The authors considered how student aspirations were informed by context and the Promise altering the setting where students in middle school and high school form their educational, career, and life goals.

Methodology

This explanatory retrospective study used a non-experimental research method. This study employed a mixed methods research design using quantitative surveys to identify trends, as well as open-ended questions and interviews to contextualize the quantitative ratings. Data was collected in three ways including student surveys, student interviews, and interviews with district principals, counselors, and teachers.

Miron and Kelaher-Young (2012) describe the reliability of their Likert-type survey instrument. The Likert-type scale utilized was the Students Aspiration Scale and was found to be reliable based on the use of Chronbach’s Alpha coefficient analysis, a form of internal reliability (Phelan & Wren, n.d.). The survey was administered to students at all three high schools in the district, resulting in a large, and according to the article, representative sample of 1,676 students. A large sample size of this magnitude is more representative of the greater population of students than a smaller sample size. The article does not mention any reliability testing for the multiple-choice questions or for coding the responses to the open-ended questions.

In terms of validity, the authors did not specifically mention conducting any validity testing. However, it can be assumed that formative validity was established as a result of using outcomes assessment to assess how well their survey helped provide information to validate students’ change in aspirations as a result of the Promise (Phelan & Wren, n.d.). Additionally, construct validity can also be inferred based on citations listed in the references.

The identified variables included dependent variables of short-term (e.g., increased attitudes, goals, and aspirations), and intermediate outcomes (e.g., improved school climate, improved enrollment in college prep courses/activities). The independent variables varied depending on the analyses and method utilized in the study (e.g., student gender, ethnicity, grade level).

Research Findings

With guiding questions based on the outcomes logic model, the researchers theme coded the interview data. Specifically, codes were created to track preexisting constructs such as student goals and positive student outcomes. Sub-codes were identified during the analysis to identify and track emergent themes. These codes were applied to all interviews and the open-ended survey responses. This qualitative analysis was important for researchers to triangulate data between interviews, survey responses, and relevant theoretical research.

Faith in the Results

My personal evaluation of this article is that the authors did a good job in explaining the importance of this work, developing the research questions, administering a robust methodology, analyzing the data, and explaining the findings and conclusions. However, a missing item in their research was to indicate their establishment of validity for this research within the article.  The establishment of some types of validity can be inferred, but without specifically stating them in the article, the researchers left a gap in their study. Furthermore, since this study was conducted at schools only in Kalamazoo, with findings on the impact of the Kalamzoo Promise on students, it would be difficult to generalize these findings to all community colleges with Promise programs, as student populations and promise programs vary across the nation.

References

Miron, G., Jones, J. & Kelaher-Young, A., (2012). The impact of the Kalamazoo Promise. American Secondary Education. Bowling Green Vol. 40 (2), (Spring 2012): 5-25

Phelan, C. and Wren, J. (n.d.). Exploring Reliability in Academic Assessment.  Retrieved from https://www.uni.edu/chfasoa/reliabilityandvalidity.htm