Remote Senior / Principal QA

We're growing our team and looking for a Senior/Principal QA Engineer to own quality across our AI voice assistant: from prompt and agent behavior testing to mobile, TV, and backend. Senior / Principal, 5+ years of manual QA - a high level of ownership and self-direction: the ability to take on the project as a whole, be fully accountable for quality, and get up to speed on new things independently. Experience testing AI agents / assistants: checking scenario and prompt following (prompt / instruction following), response quality and relevance.

Experience testing "under the hood" via AI observability / tracing tools (LangSmith and analogs): analyzing an agent's traces - tool calls, memory handling, the model that ran, inputs/outputs, token consumption, spotting bottlenecks. Experience with mobile and/or TV-app testing; experience specifically with TV devices is less critical. Experience testing backends and integrations: API testing (REST / gRPC), verifying the seams between services, reading logs and traces (Kibana / Grafana / Loki, etc.).

Solid QA fundamentals: test design, test documentation, defect handling. Hands-on experience using AI tools in the day-to-day QA process. Nice to have Experience testing speech technologies (ASR / TTS) and working with audio.

Evals - building and running AI-assistant quality evaluations: scenario-based self-play, LLM-as-judge, manual validation as human-in-the-loop. CI/CD / deployment / test environments: configuring stands, building and deploying applications (GitHub / GitLab, etc.). Device testing experience / ADB.

Willingness to move into test automation over time. AI assistant: Assessing assistant quality: prompt / instruction following, compliance with requirements; catching regressions between prompt and model versions. Testing the assistant "under the hood" via observability / tracing tools: correctness and parameters of tool calls (MCP / integrations with external APIs), memory handling, the bot's reasoning, traces, etc.

Devices and applications: End-to-end manual testing of the assistant in the mobile app and on the devices (TV, speaker). Testing voice interaction with the assistant, recognition (ASR) and synthesis (TTS) quality, voice UX. Testing the assistant's product scenarios: agentic e-commerce (voice order -> cart -> confirmation on TV via remote), media content and a movie showtimes guide, flight tickets, basic scenarios (weather, currency rates, time, timers, Q