Modern Technology Solutions, Inc. (MTSI) is hiring an AI/ML Test Engineer in Springfield, VA. Qualified candidates must be U.S. citizens with a TS/SCI clearance and CI Polygraph.
Responsibilities:
- Execute system-level AI/Machine Learning evaluation activities within the Test & Evaluation (T&E) environment supporting mission-focused operational testing.
- Perform integration testing, data capture, drift detection, and operational validation activities across AI/ML evaluation workflows.
- Support automated and semi-automated evaluation pipelines for AI/ML model testing and performance assessment.
- Develop, maintain, and refine data annotation workflows supporting operational evaluation requirements.
- Validate operational datasets and support testing activities across multiple models, datasets, and mission scenarios.
- Conduct integration testing of AI/ML models within operationally representative environments and workflows.
- Adapt evaluation procedures and workflows to address evolving GEOINT data conditions and operational mission requirements.
- Support high-throughput and parallel model evaluation activities to maintain alignment with mission timelines and operational priorities.
- Identify and mitigate data drift, performance degradation, and workflow issues impacting evaluation reliability.
- Collaborate with Senior AI/ML Test Engineers, data scientists, software engineers, and mission stakeholders to support scalable T&E execution.
- Contribute to technical documentation, evaluation reporting, and customer briefings supporting operational decision-making.
Qualifications Required:
- 4+ years of experience supporting AI/ML testing, system integration, operational testing, or evaluation workflows.
- Familiarity with automated testing pipelines, operational data validation, and model performance assessment concepts.
- Ability to support multiple evaluation activities across dynamic operational environments.
- Experience troubleshooting technical workflow or data-related issues in support of operational objectives.
- Strong analytical, organizational, and communication skills.
Education Requirements:
- Bachelor’s degree in IT or STEM related field, or additional relevant experience in lieu of a degree.
Desired Qualifications:
- Experience supporting DoD, NGA, or Intelligence Community programs.
- Familiarity with AI/ML model evaluation, GEOINT workflows, and operational testing environments.
- Experience working in Agile, DevSecOps, or cloud-based operational environments.
- Familiarity with annotation workflows, data capture pipelines, drift detection, and automated evaluation tools.
- Experience supporting scalable AI/ML evaluation or mission-focused T&E activities.
Clearance Requirements:
- Active TS/SCI clearance and the ability to pass a CI polygraph within 30 days.
#LI-RC2