The number and type of research review organizations, often going by acronyms and initials, are proliferating. K-12 research review organizations launch and produce smart guidance to help educators compare products’ research base and evidence base, but the number of organizations and approaches are many and potentially confusing.
How many emails do you have in your inbox right now promoting K-12 research review webinars, conferences, and free reports? Is the influx of information inspiring your curiosity or making you numb to the topic? It’s time for a big-picture view.
As an educator or administrator committed to an evidence-based curriculum, you will encounter many potentially useful research review organizations as you assess which products meet your school, district, and state research requirements. Which review website should you follow and reference during your product review? Here is the scoop on the main players in the product review space.
External Review of Evidence-Base
What is the What Works Clearinghouse (WWC)?
- Who are they? A government-run website associated with the Institute of Education Sciences that reviews published research studies on ELA, math, and science products. They also review programs focused on early childhood, behavior (social and emotional learning), children and youth with disabilities, English learners, and more.
- What are they looking for? WWC has dozens of protocols that cover each area of research they review. WWC typically only reviews studies with a final study report or peer-reviewed manuscript in ERIC. It typically takes 3-5 years to complete and publicize an article in a journal or non-journal in ERIC.
- What does the rating look like? The WWC review describes a product’s effectiveness as positive, neutral, or negative on different sub-domains or skills related to the discipline (ELA, literacy, alphabetic knowledge).
- Which products to look for here? Products that have been around for at least five years, have a unique or innovative approach and have partnered with university professors who would have published articles in academic journals.
- Researcher note: It is not feasible for most companies to win grants and/or partner with a university to complete the research requirements to be reviewed by WWC.
What is Evidence for ESSA?
- Who are they? Center for Research and Reform in Education (CRRE) at Johns Hopkins University (JHU) School of Education and funded by the Annie E. Casey Foundation.
- What are they looking for? Rigorous research designs that match the criteria described in the Every Student Succeeds Act (ESSA) evidence standards. Unlike WWC, Evidence for ESSA accepts self-published research and researchers/companies can submit their studies for review directly. The seven pages of protocols are clear and specific, so companies would need to have them in hand before the research starts to meet the requirements.
- What does the rating look like? The descriptions match the levels in the ESSA laws, Strong, Moderate, and Promising. Educators can filter by learner criteria (whole class, struggling readers, English learners), grade range, community (rural, suburban, urban), demographic subgroups, and other features.
- Which products to look for here? Any product or program that is meant to be used for at least three months (usually supplemental or intervention) that supports student learning for Reading, Math, Social-Emotional, Attendance, Science, and Writing. Short-term interventions and summer programs are ineligible.
- Researcher note: Planning for meeting the Evidence for ESSA criteria would need to start before recruitment of the research partner and studies need to be quite large in order to meet the Strong criteria, even with randomization of groups.
What is the National Center for Intensive Intervention (NCII)?
- Who are they? Run by the American Institutes for Research, this website is focused on assessments (screening and progress monitoring tools) and intervention materials for students who are below grade level.
- What are they looking for? Their rubrics include the effect size of the impact and the fidelity of the implementation.
- Which products to look for here? Academic and behavior products and programs that target students who need intensive intervention support.
- Researcher note: NCII does include a version of fidelity of implementation in their rubric, that favors in-person observations, even for Edtech products.
Do you need a refresher on the difference between research-based and evidence-based programs? Check out our article on unlocking the research terminology.
External review of a Research Base
What is EdReports?
- Who are they? EdReports is an independent non-profit organization that reviews the curriculum itself, not the research conducted on that product.
- What are they looking for? EdReports looks for comprehensive coverage of standards and research-based practices, as well as implementation support materials and assessment. Their rubric puts a metric to how close a product’s research base is evident in its actual materials. For example, if a product says its instruction is explicit, does the EdReports expert educator reviewer think the instruction is explicit?
- What does the rating look like? Each product and criteria set gets a score for each grade, which is provided a rating of Meets, Partially Meets, or Does Not Meet Expectations.
- Which products to look for here? K-12 core curriculum for ELA, foundational literacy skills, math, and science.
What is the Council of Administrators of Special Education (CASE) Endorsement?
- Who are they? An international professional educational organization affiliated with the Council for Exceptional Children.
- What are they looking for? The brochure explains that they review the research base, product research, and testimonials from users (including CASE members). Specific research studies with data that supports the product’s efficacy and effectiveness with special education students are required.
- Which products to look for here? Products designed for intervention (Tier 3) instruction and special education
Alignment to Standards
What are the International Society for Technology in Education (ISTE) Standards?
- Competencies for learning, teaching, and leading in the digital age
- District leaders can use the standards to align their approach to student learning, education professional learning, leadership, and technology coaches.
- Certification of products: Seal of Alignment
What are Digital Promise Product Certifications?
- Competency-based framework for applying learning sciences to the development of educational technology tools.
- Criteria that product developers can use to create research-driven product development and testing.
- Certification of products: Open Badges
What are WIDA Correlations?
- WIDA stands for World-Class Instructional Design and Assessment. The WIDA Consortium is a group of 39 states dedicated to the design and implementation of high standards and equitable educational opportunities for English learners. Wisconsin Center for Education Products and Services has created WIDA PRIME, a tool to assist publishers and educators in analyzing their materials for the presence of key components of the WIDA Standards.
- The WIDA PRIME correlation process identifies how the components of the Kindergarten-Grade 12 WIDA English Language Development Standards are represented in instructional materials. These materials may include core and supplemental texts, websites and software (e.g., apps, computer programs), and other ancillary materials. If all of the key components are present, the product can earn a score of 100%.
- WIDA PRIME does not judge the effectiveness of published materials.
If a product is on the list, it may not be the best product for your students’ needs. If a product you are evaluating is not on one of these lists, it does not mean that it is not effective. Both things are true.
Are you rethinking things?
As you use the findings of research organizations to inform your curriculum evaluation and selection, keep in mind that each of these clearinghouses, organizations, and certifications has its own goals and protocols that may benefit some products more than others.
And, the big reveal, this list of organizations is not exhaustive or fixed. More important than any single organization are the questions modeled in this post — “Who are they?” and “What are they looking for?” These questions are your cheat sheet for contextualizing products reviewed on any and all K-12 research review sites.
***
No worries if you forget what any of the organizational acronyms or initials mean! LXDResearch will keep this article up for your reference and continued learning — it’s the second in our series on research education that we’re rolling out to support district leadership navigate evidence-based curriculum review and selection.
