Lead QA automation for AI/LLM systems: define validation criteria for non-deterministic models, build automated and performance tests, validate RAG systems, perform API and integration testing, ensure release readiness and trace defects to data/models/requirements.
N-iX is looking for a Senior QA Automation Engineer to lead the validation and verification strategies for EDGE Group’s AI transformation. You will be responsible for defining "what good looks like" for non-deterministic AI systems, ensuring that Large Language Models (LLMs) and predictive engines meet the strict reliability standards of the defense sector.
Qualifications
- 5+ years in QA Automation
- Experience testing data-driven systems, AI, or ML models
- Strong analytical skills for defining test criteria in non-deterministic systems
- Ability to collaborate with engineers, data scientists, and product teams
- Experience in regulated industries (defense, aerospace) will be a plus
- English level - at least Upper-Intermediate
Technical Requirements
- Strong experience in Python and test automation frameworks (Pytest, Selenium/Playwright, API testing)
- Experience with performance testing tools (JMeter, Locust, K6)
- Familiarity with LLM evaluation tools (e.g., DeepEval, TruLens)
- Knowledge of SQL and data validation tools
- Experience with CI/CD pipelines (GitLab) and version control (Git)
- Exposure to test reporting tools (Allure, TestRail)
Key Responsibilities1. AI & LLM Testing
- Build automated tests to evaluate AI outputs for accuracy, consistency, and reliability
- Validate Retrieval-Augmented Generation (RAG) systems using metrics like relevance and correctness
- Create regression tests to detect changes in AI behavior (prompt drift)
- Test integrations between AI systems and enterprise platforms (e.g., SAP)
- Perform API testing, including security and access control validation
- Design performance tests for latency, scalability, and system throughput
- Align test cases with system requirements and user needs
- Support release readiness through structured testing reviews
- Track and manage defects, ensuring traceability to data, models, or requirements
We offer*:
- Flexible working format - remote, office-based or flexible
- A competitive salary and good compensation package
- Personalized career growth
- Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
- Active tech communities with regular knowledge sharing
- Education reimbursement
- Memorable anniversary presents
- Corporate events and team buildings
- Other location-specific benefits
*not applicable for freelancers
Similar Jobs
Fintech • Legal Tech • Software • Financial Services • Cybersecurity • Data Privacy
Provide corporate governance and legal entity administration support: coordinate board and shareholder meetings, maintain statutory registers and corporate records, prepare filings and legal documents, liaise with directors, auditors, and advisers, manage share capital administration, and support operational tasks such as bank account administration.
Top Skills:
Document Management SystemsExcelMicrosoft OutlookMicrosoft Word
Enterprise Web • Hardware • Internet of Things • Software
Embed with manufacturing customers to design, build, and deploy Tulip solutions. Lead OT/IT integrations (PLCs, MES/ERP, quality systems), troubleshoot production issues, and deliver scalable, production-ready applications. Advise on architecture, document solutions, and represent customer needs to Product and Engineering.
Top Skills:
Cloud-NativeEmbedded AiErpMachine Data IntegrationsMesNo-Code PlatformPlcsQuality SystemsTulip Frontline Operations Platform
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
As a Finance FP&A Manager, you will lead financial planning processes, ensure compliance, analyze financial performance, and support decision-making within a Finance team, while driving continuous improvement initiatives.
Top Skills:
Data AnalysisFinancial ModelingFinancial PlanningReporting
What you need to know about the Dublin Tech Scene
From Bono and Oscar Wilde to today's tech leaders, Dublin has always attracted trailblazers, with more than 70,000 people working in the city's expanding digital sector. Continuing its legacy of drawing pioneers, the city is advancing rapidly. Ireland is now ranked as one of the top tech clusters in the region and the number one destination for digital companies, with the highest hiring intention of any region across all sectors.
