Program Evaluation & Impact Analyst

Job description

Job Type Full-time Description JOB TITLE: Program Evaluation and Impact Analyst Department: Family Care Job Status: Active FLSA Status: Non-exempt Reports To: Chief Program Officer Grade/Level: Administrative/Evaluator Amount of Travel Required: up to 10% travel time required Job Type: Full Time, Hourly Positions Supervised: Designated Interns Work Schedule: 30-35 hours, In Person, Monday-Friday (with very occasional evening/weekend hours) Pay & Benefits: $17.00-$23.00 per hour JOB IS LOCATED IN: Lynchburg Virginia and required in person work POSITION SUMMARY: The Program Evaluation and Impact Analyst ensures FIVE18 has reliable, meaningful, and accessible program data that equips the Chief Program Officer and program leaders to evaluate effectiveness, make evidence informed decisions, continuously improve services and strengthen a culture in which data and consumer voice are routinely used to understand and increase impact. The Analyst joins FIVE18 at an important stage in the development of its evaluation infrastructure, stewarding strong existing practices while expanding organization-wide measurement systems, dashboards, standardized outcomes, and reporting processes. This role is expected to build, maintain, and improved the scalable measurement infrastructure needed to support FIVE18's Evidence-Based Impact value and long term goals. This individual will communicate effectively with colleagues and consumers to create and cultivate a positive impression of FIVE18, striving to present a culture of hospitality and competence. ESSENTIAL FUNCTIONS Reasonable Accommodations Statement: To accomplish this job successfully, an individual must be able to perform, with or without reasonable accommodation, each essential function satisfactorily. Reasonable accommodations may be made to help enable qualified individuals with disabilities to perform the essential functions. Program Evaluation Systems • Build, maintain, and improve meaningful outputs, outcomes, indicators, evaluation tools, and recurring evaluation processes. • Maintain continuity of established evaluation systems while identifying thoughtful opportunities for refinement. • Document measures, definitions, methodology, data sources, reporting processes, and historical trends so evaluation knowledge is organizationally accessible. 2. Data Quality, Analysis, and Interpretation • Monitor the accuracy, completeness, consistency, and reliability of program data; identify and communicate data-quality concerns, gaps, anomalies, and barriers to useful analysis. • Analyze quantitative and qualitative data to identify meaningful trends, patterns, gaps, risks, and opportunities rather than simply reporting numbers. • Apply healthy skepticism, test assumptions, investigate inconsistencies, and distinguish what the evidence supports from what it does not. • Develop sufficient understanding of FIVE18's programs, consumers, strategy, goals, and evaluation systems to exercise sound judgment about what deserves further investigation, program-level attention, or CPO awareness. • Determine what to elevate, when to elevate it, and the appropriate level of analysis before doing so, without relying on the CPO to routinely direct what should be analyzed. 3. Program Impact, Learning, and Continuous Quality Improvement • Facilitate recurring evaluation processes that connect evidence to program learning, improvement, and decision-making. • Assess the impact of program adaptations and improvement efforts over time. • Integrate participant feedback, satisfaction data, qualitative information, and consumer voice alongside quantitative measures. • Maintain a systems-level understanding of who FIVE18 serves, who may not be represented in the data, and what quantitative measures may not fully capture. 4. Organizational Measurement and Reporting • Build, maintain, and improve dashboards, Right Performance Metrics (RPMs), organizational impact measures, and routine reporting tools. • Support reliable reporting of reach, engagement, outcomes, satisfaction, consumer characteristics, and other meaningful indicators. • Increase the efficiency and reliability of data collection, aggregation, analysis, visualization, and reporting, including appropriate use of automation and emerging technology. • Help build toward standardized, scalable outcome measurement and the infrastructure required to assess progress toward annual and long-term organizational goals. 5. Collaboration and Data Capacity • Work collaboratively and proactively with Focus Area Leaders to strengthen evaluation practices across programs and increase consistency where appropriate. • Help staff and leaders understand, interpret, and routinely use data for learning and improvement. • Communicate complex information clearly and respectfully to people with varying levels of comfort with data, welcoming questions and adapting explanations as needed. • Partner with Focus Area Leaders around data-quality needs while maintaining clear accountability boundaries: program staff own accurate and timely data entry, and Focus Area Leaders own program-level follow-through and performance. 6. Evidence-Informed Program Development • Research relevant evidence and promising practices and help establish appropriate measures for new or evolving initiatives. • Identify gaps between what FIVE18 measures today and what it will need to measure as programs, reach, partnerships, and organizational complexity grow. 7. Regular Impact and Learning Rhythms • Ongoing - Monitor data and system health, maintain dashboards and evaluation systems, investigate meaningful questions, and proactively collaborate with program leaders. • Monthly - Lead the Program Impact & Learning Review with the Chief Program Officer, setting the primary analytical agenda and bringing forward the reports, trends, concerns, opportunities, and evaluation questions most deserving of CPO attention. • Quarterly - Conduct deeper program and organizational analysis