How are you measuring outcomes for your project? Measuring outcomes for NSF Regional Innovation Engines (RIEs) is a complex yet crucial task, considering the program’s unique focus on regional, collaborative, and use-inspired research. Here are some potential approaches grantees can consider:
Alignment with RIE Goals:
- Technological Advancements: Track progress in key technology areas like AI, semiconductors, etc., through metrics like patent filings, publications, prototype development, and technology licensing deals.
- Economic Impact: Monitor job creation, new business formation, increased investment in R&D, and regional GDP growth attributable to RIE activities.
- Workforce Development: Assess the number of students and professionals trained, skills acquired, and career placements stemming from RIE programs.
- Innovation Ecosystem Enhancement: Evaluate the formation of new partnerships, increased research engagement with industry and communities, and improvements in regional infrastructure for innovation.
- Societal Impact: Measure the progress in addressing specific societal challenges targeted by the RIE, using metrics like healthcare improvements, environmental sustainability, or community development indicators.
Quantitative & Qualitative Methods:
- Data Collection: Utilize surveys, interviews, focus groups, economic modeling, and bibliometric analysis to gather data on various RIE outcomes.
- Control Groups & Baselines: Establish control groups or pre-RIE baselines to compare with post-RIE data and isolate the program’s specific impact.
- Cost-Benefit Analysis: Assess the return on investment (ROI) of RIE funding by comparing costs with quantifiable economic and societal benefits.
Focus on Partnerships & Collaboration:
- Partnerships as Metrics: Track the types and number of partnerships forged between academia, industry, government, and communities, including research collaborations, joint ventures, and technology transfer agreements.
- Collaborative Outcomes: Evaluate the success of collaborative research projects in terms of achieving shared goals, generating joint publications, and addressing regional needs.
Tools and Resources:
- NSF Evaluation Framework: NSF is currently developing a framework with core benchmarks and guidelines for RIEs to measure their progress.
- Regional Data Sources: Utilize state and local economic data, workforce statistics, and innovation ecosystem indicators to assess regional impact.
- External Evaluation Teams: Consider engaging independent evaluators for objective assessments and a wider range of expertise.
Remember, the ideal measurement approach will depend on the specific goals and activities of each RIE. Flexibility and adaptation are key in capturing the multifaceted impact of these complex programs. By employing a blend of quantitative and qualitative methods, focusing on partnerships, and utilizing diverse resources, RIE grantees can effectively measure their progress and demonstrate the program’s success in transforming regional innovation landscapes.
It’s important to note that the NSF is still developing its evaluation framework for RIEs, so you may want to stay updated on their latest guidelines and resources to ensure your measurement approach aligns with their expectations.
This means your evaluator should…
- Be an experienced evaluator or evaluation firm (FYI – we are);
- Be committed to creating regional-scale, technology-driven innovation ecosystems throughout every region of the United States, accelerating emerging technologies, driving economic growth, addressing key societal challenges, and maintaining national competitiveness (FYI – we do) ;
- Have a working knowledge of all (not just the Innovation Engine program) NSF guidelines, protocols, processes, and evaluation requirements (FYI – we totally do).
Got questions on how to find an External Evaluator? We have you covered. Check out this post from August of 2020. https://www.ebhoward.com/choosing-the-right-external-evaluator/
Want to talk with us about how we serve as your External Evaluator for the NSF Regional Innovations Engines or any other similar project? Schedule a consult call today. https://calendly.com/ebhoward/15min
We work with high-growth startups and organizations that support the startup and innovation ecosystem. We build highly specific non-dilutive funding menus, provide proposal preparation services, and measure outcomes of funding through evaluation. Schedule a consult call with us HERE.