Test Harness Engineer

Allata is a global consulting and technology services firm founded in 2016, with 350+ employees across the US, India, and Argentina. We partner with some of the world's largest enterprises to move AI from idea to production — building intelligent agents, data foundations, and governance frameworks that scale.

Our teams work at the intersection of strategy, engineering, data, and design, helping clients modernize their technology, unlock data value, and create meaningful digital experiences. At Allata, you'll join an agile, cross-functional team that works closely alongside clients — making a real impact and building lasting partnerships.

We're looking for a Test Harness Engineer to join a client product team and design and build the testing systems that protect the quality of a growing technology platform. This role goes beyond writing test scripts: you will build AI-powered test harnesses where agents help plan, execute, validate, and analyze testing activities.

You'll combine strong software engineering with automation, exploratory testing, and a product-oriented mindset. You'll collaborate closely with engineering and product teams in a fast-paced environment where quality is a shared responsibility and continuous improvement is highly valued. You will make architecture and tooling decisions and take ownership of quality from design to release.

Responsibilities::

• Design and extend AI-powered test harnesses where agents plan, execute, validate, and analyze test scenarios.
• Integrate LLMs (OpenAI, Claude, Gemini, or similar) with tools, APIs, and automation frameworks, with evaluation and guardrails to prevent false positives.
• Build and maintain automated UI, API, and integration test suites that run reliably in CI/CD pipelines, keeping them fast and maintainable.
• Turn Jira acceptance criteria and Figma designs into test plans, and run exploratory, regression, and functional testing across QA and UAT.
• Identify and communicate defects with clear reproduction steps and evidence, and drive triage with developers and product.
• Use AI tools to accelerate test design and failure analysis, critically validating their output.
• Mentor engineers on harness design and automation practices.

Requirements::

• Proven background in software engineering or test automation, including designing QA automation frameworks that teams rely on in production.
• Hands-on experience building AI agents or agent-based workflows, including tool/function calling, context management, and prompt engineering.
• Experience with at least one agent orchestration framework (LangGraph, LangChain, CrewAI, or similar).
• Ability to implement evaluation, validation, and guardrail mechanisms for AI-driven test execution.
• Strong JavaScript/TypeScript skills, with working knowledge of Python or Java.
• Hands-on experience with UI and API test automation tools (e.g., Cypress, Playwright, REST Assured, Postman/Newman, Karate).
• Experience testing REST APIs and asynchronous, event-driven systems (message queues, pub/sub, or similar).
• Experience working with SQL or NoSQL databases for test data setup and validation.
• Experience integrating automated tests into CI/CD pipelines.
• Fluent English.

Nice to have::

• Experience with MCP (Model Context Protocol) or other tool-integration standards for AI agents.
• Experience with performance testing tools such as k6 or Gatling.
• Experience with accessibility testing.
• Experience testing feature-flagged releases and gradual rollouts.
• Experience working with complex SaaS or consumer-facing products.

Soft Skills::

• Senior-level autonomy: you make sound decisions with little oversight and drive solutions end to end.
• Strong analytical skills with a risk-based approach, and clear communication with developers and product.
• Pragmatic about what to automate versus what to test exploratorily.
• Dedication to continuous learning and development, for yourself and for others.