Evaluation Specialists

Evaluation Specialists (2 FTE)

Work Arrangement: Remote (7am-4:00pm M-F)

Worksite Address: 3500 Pennsy Dr, Landover, MD 20785 (Mandatory onsite training with occasional on-site meetings at Carmen Turner Facility in Landover, Maryland or other Metro facilities.)

Agency Interview Type: Either Web Cam or In Person

Engagement Type: 1 Year initial Contract with a possible extra 2 Years extension on yearly basis

NOTE: Applications with resumes in PDF Format will be automatically rejected. Only Word format resumes will be considered.

Major Duties and Responsibilities:

Design Evaluation Strategies: Develop comprehensive evaluation plans and tools to measure the effectiveness, efficiency, and impact of training programs.
Develop and Deploy Data Collection Tools: Create and utilize surveys, rubrics, focus groups, observational protocols, interviews, etc. that are methodologically sound to gather data from learners, facilitators, SMEs, and stakeholders.
Translate Complex Data Into Actionable Insights: Through compelling data storytelling, visualizations, and narrative summaries tailored to diverse audiences.
Build and Maintain Interactive Dashboards and Executive Level Reporting Tools: Use platforms such as Power BI or similar to support real-time decision making.
Analyze and Interpret Evaluation Data: Apply statistical and thematic analysis techniques to identify trends, strengths, and areas for improvement in training delivery and outcomes.
Support Continuous Improvement: Collaborate with instructional designers and training teams to refine content and delivery methods based on evaluation insights and learner feedback.
Minimum Qualifications:
Bachelors degree in education, psychology, or related field
3+ years of experience in training evaluation
Experience with technical training programs, with a preference for transit
Demonstrated expertise in data storytelling and the ability to craft clear, engaging narratives from complex data sets
Proficiency with advanced data tools and platforms, including but not limited to:
o Power BI or other analytics platforms for dashboard creation
o Data collection tools (Microsoft Forms, Surveymonkey, or similar)
Strong command of Excel, but with a mindset that goes beyond spreadsheets seeking automation, integration, and visualization opportunities
Excellent communication skills, with the ability to present findings to both technical and nontechnical stakeholders
Familiarity with Kirkpatrick Model, Philips ROI Model, Learning-Transfer Evaluation Mode and other evaluation models to measure the impact of training
Proficiency in survey platforms and data analysis tools and methods
Preferred Qualifications:
Experience with SPSS, R, or Python for statistical analysis
Familiarity with other learning evaluation models like Philips ROI Model, Learning-Transfer
Evaluation Model, and TPACK or SAMR