Skip to content
Home » Resources & Successes » Understanding Top Tier ESSA Research Studies

Understanding Top Tier ESSA Research Studies

top tier choices

In the dynamic world of educational technology, understanding the impact of your products through effective research methodologies is crucial. Two prominent approaches to evaluating edtech interventions are retrospective studies that “look back” on learning and applied or “real-time” studies. Here’s a breakdown of what they entail and when to use each.

What is a Real-Time ESSA Evidence Research Study?

A real-time study involves collecting and analyzing data continuously as it is happening. This method provides an up-to-date understanding of the educational process and the impact of edtech interventions. Real-time studies are valuable for capturing immediate teacher, district, or school feedback. They typically require a commitment of time and resources to recruit and establish partnerships with schools, and they can take an academic year or more to complete, excluding the time spent on recruitment.

Choose a Real-Time Study if:

  • Current feedback and insights from educators would be helpful.
  • You have the resources to establish and maintain school partnerships.
  • You can commit to a long-term study.
  • You aim to generate evidence for products that potentially qualify for John Hopkin’s Evidence for ESSA website’s Strong or Moderate categories.

What is a Lookback ESSA Evidence Research Study?

On the other hand, a lookback study involves analyzing historical data to examine patterns, trends, or relationships related to edtech implementation or outcomes. Unlike real-time studies, lookback studies use existing data to understand the long-term effects of interventions. This approach is faster as it doesn’t require new data collection but may lack the timeliness of recent developments.

Choose a Lookback Study if:

  • You have a success story with substantial existing data.
  • You can identify trends and relationships from past data.
  • You require a faster study completion.
  • You are not looking to establish new partnerships for data collection.

Key Differences Between Lookback and Real-Time Studies

  1. Data Collection:
    • Real-Time Study: Ongoing and continuous data collection.
    • Lookback Study: Utilizes historical or past data.
  2. Feedback:
    • Real-Time Study: Provides immediate, current feedback.
    • Lookback Study: Offers insights from past data, which may not reflect recent changes.
  3. Time Commitment:
    • Real-Time Study: Requires long-term commitment, often an academic year or more.
    • Lookback Study: Can be completed faster as it relies on existing data.
  4. Resources and Partnerships:
    • Real-Time Study: Needs significant resources to establish and maintain school partnerships.
    • Lookback Study: This design does not require new partnerships, leveraging already available data.
  5. Evidence Tier:
    • Real-Time Study: This is necessary for ESSA Strong (Tier 1), which requires randomization of students or schools to the treatment group by the researcher.
    • Lookback Study: A retrospective study may align with ESSA Moderate (Tier 2) or ESSA Promising (Tier 3), depending on the specifics of the design and outcomes. John Hopkins only allows these studies to be considered Promising.

Conclusion

For edtech product marketing managers, choosing the right study type is crucial for demonstrating the effectiveness of your product. If immediate, actionable feedback is essential, and you have the resources for a long-term commitment, a real-time study is the way to go. However, if you need quicker results and robust existing data, a lookback study can provide valuable insights without requiring new data collection efforts.

Understanding these methodologies will help you make informed decisions and effectively communicate the impact of your edtech products to stakeholders, ensuring that your products meet the educational needs and standards required for success.