COMPUTER SCIENTIST (AI TEST AND EVALUATION)
What you'd do
This position may be filled using Direct Hire Authority: Z5CAV/Direct-Hire Authority (Certain DoD Personnel) PL 118-31, Sec 125OB (i)(2), 12/22/2023.This position is being filled under the memorandum from the Under Secretary of Defense for Personnel and Readiness (USD(P&R)) "Expansion of Direct Hire Authority for Certain Personnel of the Department of Defense," dated August 12,2024. This position is part of the Defense Threat Reduction Agency,
Major duties
As a COMPUTER SCIENTIST (AI TEST AND EVALUATION) at the GS-1550-13 some of your typical work assignments may include: Plans and executes comprehensive test and evaluation (T&E) of AI/ML systems by developing test plans, defining quantitative and qualitative metrics (e.g., sensitivity, specificity, confidence levels, generalizability), conducting hypothesis testing, and performing verification and validation activities to ensure AI solutions meet performance specifications and requirements. Tests AI/ML systems for security, robustness, trustworthiness, and reliability by conducting adversarial testing in operationally realistic environments, performing risk assessments, evaluating functionality and compatibility across diverse scenarios, and building assurance cases that demonstrate the level of confidence in AI system capabilities. Develops and executes code-level testing and validation procedures for machine learning models, including automating testing processes, evaluating user experience and human-computer interaction factors, and recording and managing all test data with accuracy and integrity throughout the testing life-cycle. Prepares detailed T&E reports and communicates findings, risks, and recommendations to technical and non-technical audiences, including senior leadership, by translating complex test data and evaluative conclusions into clear, actionable courses of action that inform decision-making across the organization. Assesses AI systems for bias, security vulnerabilities, and high-impact risks by reviewing AI architectures, analyzing training data sets and system outputs for unintended bias, identifying failure modes and low-probability risks, and developing risk management plans and corrective actions to ensure ethical and trustworthy AI deployment. Collaborates with cross-functional teams - including developers, data scientists, cybersecurity professionals, and operational users- to integrate AI T&E frameworks into project test strategies and ensure seamless coordination across DataOps, MLOps, and DevSecOps pipelines, including AI deployment within cloud and IT infrastructure environments. Represents the organization in working groups, interagency forums, and professional eventsrelated to AI T&E and responsible AI, while monitoring evolving test and safety standards (e.g., MIL-STD 882E, DO-178C, ISO 26262) to ensure AI evaluation activities remain current, rigorous, and aligned with mission requirements and AI ethical principles.
What you need to qualify
You may qualify at the GS 13 , if you fulfill the following qualifications: A. One year of specialized experience equivalent to the 12 grade level in the Federal service as listed below: Planning and executing test and evaluation (T&E) of artificial intelligence and machine learning (AI/ML) systems, including developing test plans, defining performance metrics, and conducting verification and validation activities to assess whether AI/ML models and applications meet defined requirements and performance standards. Assessing AI/ML systems for risks related to security, robustness, bias, and trustworthiness and recommending risk mitigation strategies to ensure reliable system performance. Writing and maintaining code in programming languages commonly used in AI/ML development. Preparing technical reports and communicating complex AI/ML test findings, risks, and recommendations to both technical and non-technical stakeholders, including translating test data and evaluative conclusions into actionable recommendations for leadership and cross-functional teams. Experience refers to paid and unpaid experience, including volunteer work done through National Service programs (e.g., Peace Corps, AmeriCorps) and other organizations (e.g., professional; philanthropic; religious; spiritual; community, student, social). Volunteer work helps build critical competencies, knowledge, and skills and can provide valuable training and experience that translates directly to paid employment. You will receive credit for all qualifying experience, including volunteer experience. In addition to meeting qualifications, your application package must reflect the applicable experience to meet the Individual Occupational Requirements for the 1550, series as listed below: Bachelor's degree in computer science or bachelor's degree with 30 semester hours in a combination of mathematics, statistics, and computer science. At least 15 of the 30 semester hours must have included any combination of statistics and mathematics that included differential and integral calculus. All academic degrees and course work must be from accredited or pre-accredited institutions. Applicants should have sufficient knowledge to understand the fundamental concepts and techniques of computer science. Courses designed to provide an introduction to computer science techniques and methodologies, to problems of system design, and to other specialized fields are acceptable. Courses or experience in teaching elementary, business, or shop mathematics are not acceptable. https://www.opm.gov/policy-data-oversight/classification-qualifications/general-schedule-qualification-standards/1500/computer-science-series-1550/
Before you apply
Federal applications are different: your resume should be 3–5 pages and mirror the language of this announcement. Read our federal resume guide first — it's the #1 reason qualified people get screened out.
Don't miss the next one.
Get an email the moment a similar federal job opens — postings can close in as little as 5 days.