Chapter Three: Procedures and Methodology

Introduction

The goal of education is to increase student achievement and knowledge of the material being taught. It is imperative that, if this is the real goal of education, search for best practices that assist in increasing student achievement. While many different aspects impact student achievement, expanding the practice efforts of educators to help in the classroom is beneficial (Tucker & Strange, 2020). The idea for the study focused around the theory, while it may be considered old by many in education today, from Benjamin Bloom and mastery learning and the utilization of formative assessments and individualized learning to drive instruction (Guskey, 2010). The purpose of this quantitative study was to determine the strength and nature of the relationship between the level of implementation of the diagnostic assessment software PowerSchool and student achievement on the eighth-grade mathematics TCAP test in a semi-rural system in northeast Tennessee.

The target system for this study served a total enrollment of 5,290 students, grades pre-k through grade 12, and consists of 15 schools and one alternative placement setting. While many of the schools from the target district are considered to perform at proficient levels for student achievement, others in the system are or are in danger of becoming target schools by the Tennessee Department of Education based on student achievement. As with all public schools in the state of Tennessee, all third through eighth-grade students in the target district partake in yearly TCAP testing in ELA, mathematics, science, and social studies. As discussed in Chapter 2, TCAP is a criterion-referenced assessment that, when coupled with TVAAS and value-added, is a reliable and valuable source of data for educators statewide. Chapter 3 discusses the methodology of the research as well as the utilization of the TCAP and TVAAS as sources of data. The chapter begins with an introduction and research paradigm and design before moving through the sampling procedures, data collection sources, statistical tests being utilized, and a summary of the chapter.

Research Paradigm

The review of the literature discussed in Chapter two explained how the use of formative assessments and mastery learning could be used to increase student achievement. Furthermore, as mentioned in Chapter One, the state of Tennessee, as well as the nation, is facing a crisis with a large percentage of today’s students performing below grade-level expectations. For this reason, systems nationwide have implemented programs specifically for assisting in increasing student achievement in mathematics such as Response to Intervention (RTI), and continual search for programs that can further help in this goal of improving student achievement and understanding in mathematics.

The goal of this study was to investigate the relationship between the level of implementation of the diagnostic assessment software PowerSchool and student achievement in eighth-grade mathematics in a semi-rural northeast Tennessee school system. A quantitative study was chosen for the study since quantitative research “entails the collection of numerical data and exhibiting the view of the relationship between theory and research as deductive, a predilection for natural science approach, and as having an objectivist conception of social reality” (Bryman & Bell, 2015, p. 160). The dependent variables for this quantitative study focused on the student different data on achievement results based on the eighth grade Tennessee Comprehensive Assessment Program (TCAP) performed in the spring semester of the 2020-2021 school year. The independent variable was based on the different levels of implementation of the diagnostic assessment software PowerSchool during the 2020-2021 school year in a school district in semi-rural northeast Tennessee.

Research Design

The research design for this study includes a correlation design utilizing the independent samples t-test and Chi-square to measure the strength and nature of the relationship between student achievement and the level of implementation of PowerSchool. A correlation design was chosen due to the desire to realize if and how strong of a relationship exists between the level of implementation and student achievement. One group of classes uses PowerSchool as merely a benchmark testing, making up less than five percent of the time spent utilizing PowerSchool for instructional purposes. In contrast, the second group not only uses PowerSchool for the system-wide benchmark testing but also weekly as formative assessments to drive the daily instruction, making up 50% or more of instructional time spent utilizing PowerSchool for instructional purposes. The research used the PowerSchool program, including criterion-referenced benchmark exams based on Tennessee state standards provided through the program, as well as a second criterion-based test in the Tennessee Comprehensive Assessment Program (TCAP) during the 2020-2021 school year. PowerSchool was implemented through an online platform inside the individual classrooms as well as home access provided at home. The TCAP test was presented in the paper-pencil format during Spring 2021 by exiting eighth-grade students in Northeastern Tennessee.

The researcher chose three different sources to serve as dependent variables, one for each research question, and one for the independent variable for each. The dependent variables were based on the different degrees of measure of student achievement on the 8th grade TCAP mathematics test: individual composite scores, TVAAS value-added for each teacher participating in the study based on TCAP scores, and the level of achievement of each student. The basis for the independent variable was the two levels of implementation of the diagnostic assessment software PowerSchool: a full implementation that was used to drive the curriculum and implementation for benchmark testing purposes only.

Sampling Procedures

Prior to conducting this study, approval was asked for and obtained from the University of the Cumberlands Institutional Review Board (IRB). The target school system chosen for this study has acknowledged that a problem exists with student achievement in TCAP testing, especially in middle school mathematics. For this reason, the system implemented the mandatory use of benchmark testing (three total tests throughout the school year) utilizing the PowerSchool software system-wide during the 2019-2020 school year. Permission was granted to conduct research through the district in question by the curriculum supervisor (see Appendix A).

The targeted semi-rural district located in northeast Tennessee was relatively large for a single district. According to data obtained from the personnel department of the target school district, during the 2020-2021 school year, the district employed 473 professional employees: 10 supervisors, 16 principals, eight assistant principals, 18 system-wide support supervisors (curriculum coaches, testing coordinators, etc.), and 421 classroom teacher. Furthermore, the targeted district consists of 15 schools serving students in grades pre-kindergarten – twelfth grade and one alternative placement school. The 15 schools served 5,290 total students containing 756 students that qualify for special education services. The system is considered “direct serve,” which indicates all students kindergarten – eighth grade receive free breakfast and lunch. Each school in the system qualifies as Title 1 schools. The percentage of the ethnic diversity of the 5,290 total students served during the 2020-2021 school year consisted of 95.7% Caucasian, 2.28% Hispanic, and 2.02% identifying as other.

Due to the nature of the study, a non-random, convenience sampling method was chosen for participants. Convenience samples are defined as the “non-probability sampling method that relies on data collection from population members who are conveniently available to participate in the study” (Convenience, 2019). Because convenience sampling was utilized for this study, the study lacks the desired trait of randomness in sampling. However, the purpose of this study was to identify if a relationship between the level of implementation of the diagnostic assessment software PowerSchool in a local northeast Tennessee school system, thus the research and results may not produce data that can be generalized to an overall population. Furthermore, including all eighth-grade students in the targeted district helps to strengthen the validity of the study.

The targeted system consists of seven middle schools, three of which implementing full PowerSchool (50% of instruction) classified as Group X and five only utilizing the program for benchmark tests only ( less than 5% of instruction time) classified as Group Y. The convenience of using all seven middle schools was appropriate. Of the 5,290 total students served by the district, 404 students were served in eighth grade, represented 8.37 % of the population. For this study, the eighth grades were separated into two groups: Group X consisted of 188 individual students ( n = 188) and four teachers, and Group Y consisted of 216 individual students (n = 216) and four teachers.

Data Collection Sources

This study based the collection of data primarily from the results of the eighth grade Tennessee Comprehensive Assessment Program (TCAP) as well as the value-added results formulated from TVAAS. As previously discussed, the TCAP assessment is assumed to be valid and reliable criterion-based. The TCAP assessments will be completed during April 2021, and results will be finalized and reported back to the system during the summer of 2021. TCAP testing is implemented for all students grades three through eight throughout the state of Tennessee. Once the results are reported back to the system, the system will contact the researcher and provide access to the student’s results in coded form for each individual that are part of the study.