The NASFAA Conference in San Antonio earlier this month provided the Department of Education (ED) with a forum to start a conversation with financial aid administrators about how it wants future experimental sites to be conducted. The purpose of experiments is to generate evidence-based recommendations concerning possible changes to the law, regulations, and administrative procedures that impact a large number of students and schools. ED believes that some past experiments failed to achieve intended results because they were not well designed. Therefore, it is placing greater emphasis on the design of future experiments. In addition, ED is modifying its approach to the selection of new experiments by implementing a new more collaborative two-phase selection process.
During Phase 1, ED will publish a Dear Colleague Letter and a Notice in the Federal Register encouraging schools to submit suggestions for new experiments. Schools may submit proposals on their own or in collaboration with other schools. Proposed experiments should identify problems with current rules and include an alternative for meeting the rule's objective or intent. Each proposal should make the case that the current requirement imposes unintended consequences on students and/or creates undue burden on schools administering Title IV aid. In addition, the proposal should suggest criteria for evaluating the experiment.
The presenters cautioned that proposals should reflect real experiments. That is, experiments should be designed to include both experimental and control groups as well as rigorous data collection and analysis. Annual reporting about the experiment should identify what is working and what is not, as well as include recommendations for improvement. The presenters acknowledged that, by definition, some experiments will fail, and schools participating in the experiments must be willing to talk about what did not work and why.
ED will carefully review all proposals and will develop rigorous experimental designs using the ideas and design suggestions submitted. This will help to ensure that the new experiments will have the potential to provide clear cases in support of policy change.
During Phase 2, ED will share the experimental designs with all schools. Interested schools may submit an application to participate in one or multiple experiments. Schools will be able to participate either as an experimental group or as a control group. Priority for participation in an experiment will be given to schools that submitted the idea on which the experiment was based. If selected for participation in an experiment, the school must enter into a written agreement with ED and its Program Participation Agreement will be modified to reflect the terms of the experiment. The school will be encouraged to work with other schools participating in the experiment. Each year, the school will be required to submit a report on the experiment. The school's annual report must include descriptions of the experiment and the method used to collect evaluative data, an analysis of the evaluative data, and recommendations for modifications to the experiment.
Until the Dear Colleague Letter and the Notice in the Federal Register are published, schools are encouraged think about potential experimental sites proposals. The presenters noted that ED is particularly interested in experiments that might produce stronger student academic outcomes, such as improved persistence, a shorter time to degree completion, and reduced outside work.
You may download this session's handout from the NASFAA Conference Web site.
By Eileen F. Welsh
Assistant Director for Professional Assessment, Training, and Regulatory Assistance
Posted 07/29/09 to www.NASFAA.org. Redistribution to non-NASFAA institutions is prohibited. Please submit Web site questions or comments to Web@NASFAA.org.
