Senior AI QA Engineer
تم النشر:
18 نوفمبر 2026 (نُشرت قبل 2 ساعة)
آخر موعد للتقديم:
15 فبراير 2027
عدد الوظائف الشاغرة:
1 عدد الوظائف الشاغرة
ملخص الوظيفة
**Title:**
Senior QA Automation Engineer AI & Agentic Systems Sovereign AI
**Location:**
Abu Dhabi
**About the role:**
We're hiring a Senior QA Automation Engineer for one of the leading players in Sovereign AI a UAE-based company with frontier-model access and agentic AI products live across critical sectors including large-scale construction government healthcare and defense.
This is not traditional QA. You'll own quality for AI and agentic systems defining what "good" looks like when a system reasons calls tools and can be wrong in subtle ways. If you've figured out how to test something whose output is different every time this is for you.
**What you'll do:**
* Test agentic systems end to end: tool-use and trajectory evaluation grounding and citation verification honest-failure and refusal calibration guardrail and adversarial testing (prompt injection jailbreaks)
* Build evaluation for non-deterministic AI: scoring for hallucination consistency and accuracy against gold datasets including LLM-as-judge calibrated against human judgement
* Own prompt and model regression suites that catch quality drift before it ships
* Run continuous evaluation on live production traffic with quality monitoring alerting and incident response
* Cover the fundamentals too: integration and API testing secure gateway validation (RBAC PII) and performance/load testing
**What we're looking for:**
* 5 years in QA or quality engineering with 2 years testing ML LLM or AI agent systems
* Hands-on with LLM/agent evaluation frameworks (DeepEval TruLens RAGAS or custom Python evaluators) and LLM-as-judge
* Agent observability and tracing (LangSmith Langfuse OpenTelemetry)
* Expert Python (Pytest custom harnesses) plus Selenium/Playwright and API testing
**The setup:**
* Abu Dhabi hybrid contract-based
* Onboarding ASAP
* World-class environment: elite talent from FAANG and frontier AI labs
Senior QA Automation Engineer AI & Agentic Systems Sovereign AI
**Location:**
Abu Dhabi
**About the role:**
We're hiring a Senior QA Automation Engineer for one of the leading players in Sovereign AI a UAE-based company with frontier-model access and agentic AI products live across critical sectors including large-scale construction government healthcare and defense.
This is not traditional QA. You'll own quality for AI and agentic systems defining what "good" looks like when a system reasons calls tools and can be wrong in subtle ways. If you've figured out how to test something whose output is different every time this is for you.
**What you'll do:**
* Test agentic systems end to end: tool-use and trajectory evaluation grounding and citation verification honest-failure and refusal calibration guardrail and adversarial testing (prompt injection jailbreaks)
* Build evaluation for non-deterministic AI: scoring for hallucination consistency and accuracy against gold datasets including LLM-as-judge calibrated against human judgement
* Own prompt and model regression suites that catch quality drift before it ships
* Run continuous evaluation on live production traffic with quality monitoring alerting and incident response
* Cover the fundamentals too: integration and API testing secure gateway validation (RBAC PII) and performance/load testing
**What we're looking for:**
* 5 years in QA or quality engineering with 2 years testing ML LLM or AI agent systems
* Hands-on with LLM/agent evaluation frameworks (DeepEval TruLens RAGAS or custom Python evaluators) and LLM-as-judge
* Agent observability and tracing (LangSmith Langfuse OpenTelemetry)
* Expert Python (Pytest custom harnesses) plus Selenium/Playwright and API testing
**The setup:**
* Abu Dhabi hybrid contract-based
* Onboarding ASAP
* World-class environment: elite talent from FAANG and frontier AI labs