(792) QA Automation Engineer
AltorosPolska14 days ago
Job descriptionOwn the test strategy across FE, BE, and LLM / agent workflows; define the test pyramid and coverage goals Build and maintain automation for : API / contract testing (REST / GraphQL), schema / compatibility checks Data & integration tests across services and external partners LLM Quality & Evals : Set up regression / eval harnesses for prompts, tools, and agents; track drift and hallucination rates Implement guardrails and adversarial / red‑team tests for safety, privacy, bias, and toxicity Non‑functional testing : baseline and monitor performance, reliability, and resilience for key user journeys CI / CD ownership : integrate tests into pipelines, gate releases, reduce flakiness, and surface quality KPIs (e.g., failure rate, MTTR, coverage) Test data & environments : manage fixtures, synthetic data, and stable seeds for deterministic runs Quality operations : triage defects, drive root‑cause analysis, and keep crisp, proactive communication with PM, Design, and Engineering Documentation : maintain clear test plans, playbooks, and release criteria Experience with agent frameworks, tool calling, retrieval / RAG, or tracing Familiarity with evaluation / observability tools for LLM apps (e.g., prompt testing frameworks, tracing dashboards) Performance testing (e.g., k6 / JMeter) and basic security / safety testing practices
Altoros – an established provider of software development services, is looking for a QA Automation Engineer.
Requirements
- It’s a part‑time position.
Responsibilities
UI / E2E flows (critical paths, multi‑step agentic scenarios)
Design golden datasets and rubrics for LLM output evaluation (pass / fail thresholds, human‑in‑the‑loop review)
Nice to have
Join Altoros Team!
#J-18808-Ljbffr
Create a job alert for this search
Qa Automation Engineer • Polska