subject: Testing Programs
0-8 of 8
Optimizing State NAEP: Issues and Possible Improvements. NAEP Validity Studies.
description- – The National Assessment of Educational Progress (NAEP) has conducted state assessments in 1990, 1992, 1994, and 1996. At present, after the fourth round of state assessments, it is clear that the program is generally successful. A survey of state testing directors indicates that the NAEP has considerable credibility as a valid and reliable source of information. Despite the high regard in which the NAEP is held, today's environment of limited federal and state resources has led to level funding for the NAEP and intense scrutiny about the best ways to optimize all its aspects, including state assessment. Because the state component can account for nearly half the budget devoted to NAEP cooperative agreements, considering how to reduce effort and maximize utility is a good idea. This paper addresses several topics in relation to making the state NAEP more effective. The first is the need to examine how to reduce the burden for many states. The major way to reduce the burden is probably to conduct state assessments on a relatively infrequent schedule while keeping the number of subjects and grades to a reasonable level. The main challenge is to maximize the information gained from these assessments. There is also a need for a stable assessment schedule, commensurate with the burden currently required by states. The greatest need is to promote the use of state NAEP data. This could involve devoting greater attention to how best to link state assessment and NAEP results, developing more timely and user-friendly reports, and working with the states themselves and other organizations to address the data needs of different NAEP audiences. Promoting use will promote the participation and support necessary for the continued success of the state NAEP. (Contains 10 references.) (SLD)
- – 1997-05-01
Problem Choice by Test Takers: Implications for Comparability and Construct Validity. CSE Technical Report 485.
description- – For assessments that present problems that require extended responses and substantial amounts of time, there is often a desire to allow students to choose which problem they will respond to among two or more options. Student choice of problem may allow students a better opportunity to demonstrate what they know and are able to do. On the other hand, choice raises questions about the comparability of scores obtained by students who respond to different problems. Questions of comparability and validity of scores obtained when students are given a choice among alternative problems were investigated using data for approximately 30,000 students from the Oregon State Assessment Program for Grade 10 Mathematics Assessment administered in spring 1997. The assessment consisted of a multiple-choice section and a pair of extended-response problems. On each of six alternate forms, two problems were presented, and students were instructed to choose one. Data from the six forms were analyzed to evaluate the comparability of scores obtained from responses to different tasks and the validity of the results. It was found that problems differed in popularity, and that the scores students obtained differed systematically as a function of problem choice. On the other hand, confirmatory factor analysis results across forms for students choosing different problems suggest that there was similar validity for measuring the underlying constructs across problem choice. It is concluded that while choice may be justified, some form of equating adjustments would be needed before making high-stakes decisions based on performance of students on problems where choice is allowed. (Contains 23 tables, 2 figures, and 9 references.) (Author/SLD)
- – 1998-09-01
Implications for Use of the ACT within the Colorado Student Assessment Program
description- – This study was commissioned to investigate issues related to the inclusion of the American College Testing Program (ACT) assessment within the Colorado State Assessment Program (CSAP). Analyses were carried out using assessment materials, assessment results, and various data from the ACT and the Colorado State Department of Education. One focus of the investigation was the match between the Colorado Model Content Standards (CMCS) and the ACT Assessment. Data indicate that the ACT does not match well with the CMCS, and the match is especially poor in science. This finding casts doubts on the advisability of requiring all Colorado 11th graders to take the ACT. The poor match could potentially inject incoherence into the CMCS-based accountability program as a whole, and it raises questions about how ACT results might be combined with CSAP performance in the calculation of Student Accountability Reports for high schools. (SLD)
- – 2001-12-15
The Building Blocks of State Testing Programs. Statement Series.
description- – To ensure that educational tests do the least harm and bring the greatest good to the education of elementary and secondary school students, it is important that parents, teachers, educational administrators, and policy makers be actively involved in test construction and use. The purpose of this Statement is to provide a checklist of some components of large-scale testing programs and the choices they represent. This first part of the Statement describes six basic building blocks of state testing programs: (1) stakes levels (high, medium, or low); (2) performance standards; (3) information dissemination; (4) involvement of teachers; (5) technical design and data; and (6) range of measures used to assess educational performance. The second part demonstrates with two simple examples how these building blocks can be put together to form two very different state testing programs. The first scenario is essentially a developmental approach to learning and assessment, while the second scenario is a joint political and educational approach to setting educational standards and assessing educational outcomes. The six building blocks that are highlighted reflect the main dimensions along which current state testing programs differ and around which most debate and controversy occur. (SLD)
- – 2001-08-01
The Marketplace for Educational Testing. NBETPP Statements, Volume 2, Number 3.
description- – This paper considers the size of the marketplace for education testing. The discussion is focused on commercial standardized achievement tests produced for the elementary and secondary market, the"Elhi"market. A few testing companies now account for the bulk of the test sales in the Elhi market, but it is difficult to obtain exact figures on the size of the testing marketplace. Five indicators of indirect growth are examined: (1) number of state-mandated testing programs; (2) aggregate sales of tests; (3) revenues of four testing companies; (5) prices of test booklets, answer sheets, and scoring services; and (6) references to testing in the education literature. When all of these things are considered, it is evident that the test market is large and growing. The growth potential of the industry and the fact that it is largely unregulated mean that more attention must be paid to testing and the educational outcomes that result.
- – 2001-04-01
A Brief History of Attempts To Monitor Testing. NBETPP Statements, Volume 2, Number 2.
description- – This paper provides a brief history of attempts to monitor testing in the United States. It describes proposals for monitoring from the first attempts in the 1920s to similar proposals in the 1990s. The discussion focuses on: (1) Giles Ruche's proposal for a consumer research bureau on tests; (2) Oscar K. Burros' reviews of tests and efforts to establish a more active test monitoring agency; (3) the call of the American Psychological Association for a Bureau of Test Standards and a Seal of Approval; (4) the Project on the Classification of Exceptional Children's recommendation for a National Bureau of Standards for Psychological Tests and Testing; and (5) the efforts of various professional organizations to establish standards for test development and use. The concept of monitoring tests and the impact of testing programs has a long history, but was not translated into practice until the formation in 1998 of the National Board on Educational Testing and Public Policy. The National Board has finally begun the process of independently monitoring tests and testing programs that has been called for since the 1920s. (Contains 28 endnotes.) (SLD)
- – 2001-02-01
Accountability Systems: Implications of Requirements of the No Child Left Behind Act of 2001. CSE Technical Report.
description- – The recently enacted No Child Left Behind Act of 2001 amends the Elementary and Secondary Education Act of 1965. The new law substantially increases the testing requirements for states and sets demanding accountability standards for schools, districts, and states, including the setting of measurable adequate yearly progress objectives for all students, as well as for subgroups of students defined by socioeconomic background, race/ethnicity, and English language proficiency. Issues raised by variations among states in their content standards, the rigor of their tests, and the stringency of their performance standards are illustrated. In addition, the differences in types of tests are considered, as well as provisions related to how local tests might be integrated into the system. Some suggestions are provided for leveling the playing field among states and for improving the ways in which adequate yearly progress is evaluated. (Contains 3 tables, 9 figures, and 13 references.) (Author/SLD)
- – 2002-06-01
Perceived Effects of State-Mandated Testing Programs on Teaching and Learning: Findings from a National Survey of Teachers.
description- – Results from a national survey of teachers are reported for five types of state testing programs, those with: (1) high stakes for districts, schools, or teachers, and students; (2) high stakes for districts, schools, and teachers, and moderate stakes for students; (3) high stakes for districts, schools, and teachers, and low stakes for students; (4) moderate stakes for districts, schools, and teachers, and high stakes for students; and (5) moderate stakes for districts, schools, and teachers, and low stakes for students. Of the 12,000 teachers who received surveys, 4,195 returned responses. At least two themes emerged from these survey data. In several areas, teachers' responses differ significantly when analyzed by the severity of the stakes attached to test results. Pressure on teachers, emphasis on test preparation, time devoted to test content, and views on accountability are such areas. The second theme is that views of elementary, middle, and high school teachers regarding the effects of their state's test differed from each other in areas such as school climate and classroom use of test results. There are also instances in which stakes and grade level combined show interesting patterns in teachers' responses, and areas in which there are no differences. The summary is organized by major areas surveyed, and within each area, findings are presented for stakes levels, grade levels, and stakes combined with grade levels. Five appendixes contain supplemental information and data tables. (Contains 16 figures, 88 tables, and 53 references.) (SLD)
- – 2003-03-01
- – Pedulla, Joseph J.
- – Abrams, Lisa M.
- – Madaus, George F.
- – Russell, Michael K.
- – Ramos, Miguel A.
- – Miao, Jing
0-8 of 8