UserTrace
UserTrace is an AI agent evaluation platform that sends simulated users into realistic multi-turn conversations with your voice or chat agent, so product and engineering teams in healthcare, banking and insurance can catch failures before release.
What is UserTrace?
UserTrace is an AI agent evaluation platform from UserTrace, Inc. Rather than scripted test cases, it puts simulated users in front of your agent, lets them hold realistic multi-turn conversations, and scores every exchange.
The documentation builds the product on four concepts. A Scenario is the situation to test — the example given in the docs is "A 30-year-old migrant delivery worker based in Bengaluru, under financial stress". A Persona sets communication style, emotional state, knowledge level and decision-making mode. Evals are the pass/fail criteria. A Simulation is the run itself. Personas and evals attach to the chosen scenario automatically, with "No manual persona creation", "No evaluation setup" and "No complex configuration". Coverage multiplies quickly: one scenario, four personas and two sessions already produce eight tested conversations.
Agent coverage is broad. On voice: telephony agents (PSTN, VoIP, inbound and outbound, multi-line) and WebRTC agents, through Vapi, Retell, LiveKit, Pipecat or custom SIP. On chat: support chatbots, virtual assistants, WhatsApp Business agents, Slack bots, web chat widgets and prompt-only agents. Architectures include RAG (knowledge base, document retrieval, source verification), tool-calling agents (function calls, database queries, multi-step workflows) and multimodal agents mixing voice, text, images, documents and video. OpenAI function calling, Anthropic tool use, LangChain and LlamaIndex are documented, alongside any custom API in Chat Completions format.
Integration runs through an OpenAI-compatible POST /v1/chat/completions endpoint with Bearer authentication and optional user_id, session_id, metadata and scenario_id parameters; Health Check, User Management and Webhooks endpoints complete the API. During a run you get real-time monitoring, live evaluation scores, conversation flow tracking and incident detection. Reports cover response quality, conversation breakdowns and safety issues, friction points, satisfaction and improvement recommendations.
Two verticals have dedicated pages: Healthcare, where the vendor claims 85% less evaluation time for a clinical decision support team, 3x more edge cases caught in triage and 100% regulatory coverage per run, and BFSI, covering fraud, credit, claims, support and financial advice. The founders trace the idea back to a hospital pilot that failed at Zealth AI after a reply breached medical protocol, and to the 80+ AI product managers and engineers they interviewed before building the tool.
What it does
- Connect an existing agent through a prompt, an API sandbox, voice, WhatsApp, Slack or a workflow
- Generate personas, journeys and evaluation criteria automatically from the use-case context you provide
- Run multi-turn simulations at scale against the agent
- Score safety, quality, policy adherence, tone, business rules or any criterion you define
- Catch regressions before every prompt, model or release change goes live
- Produce interactive reports with root-cause analysis and prompt suggestions
- Check regulatory coverage on every run (FDA and HIPAA on the healthcare side, financial rules on the BFSI side)
When to use UserTrace / When not to
A quick filter to help you decide if UserTrace is the right fit.
When to use UserTrace
- Teams shipping conversational or voice AI agents to production, who need to validate every prompt or model change before release
- Healthcare AI teams: a dedicated page covers clinical use cases and per-run checks against FDA and HIPAA expectations
- Banking, financial services and insurance teams testing agents on fraud, credit, claims, support and financial advice
- Product teams with no dedicated test engineering, since the site promises "No engineering dependency. No long setup." and pre-fills personas and evaluations
- AI product managers, engineers, conversational designers and clinicians who need broad multi-turn coverage: one scenario x four personas x two sessions already yields eight tested conversations
When not to use UserTrace
- Individuals and hobbyists: the contract is a signed enterprise Order and the minimum age is 18
- Buyers who need a published price list, since no pricing page exists and every path leads to a demo request
- Teams looking for production observability: UserTrace simulates users, it does not monitor real user traffic
- Teams that want to build agents rather than test them, since you must already have a working agent to connect
- Anyone expecting a mobile app, or an environment cleared by default for live production end-user data, patient data or identifiable health information, which the privacy policy excludes unless explicitly agreed by contract
How to use UserTrace
A typical end-to-end flow, from setup to results.
- Get in touch first: the site funnels through "Start Simulations" and "Request a Demo", and the documentation offers a Calendly slot to book a demo
- Connect your existing agent through a prompt, an API sandbox, voice, WhatsApp, Slack or a workflow
- For a custom chat agent, expose an OpenAI-compatible POST /v1/chat/completions endpoint and authenticate with your API key as a Bearer token
- For a voice agent, use the prebuilt Vapi, Retell, LiveKit or Pipecat integrations, or a custom SIP connection
- Describe your users and your goals; UserTrace generates the personas, journeys and evaluation criteria that fit the use case
- Browse the available scenarios in the dashboard and select the ones matching what you need to test
- Start the simulation: pick the configured agent, choose the scenarios (personas and evals fill in automatically, extra ones optional), then click Start Simulation
- Watch the run in real time, with live evaluation scores, conversation flow tracking and incident detection
- Read the results report: what failed, why and what to change, with root-cause analysis and prompt suggestions
- Re-run before each release to measure progress and catch regressions
Pros & Cons
Pros
- Broad agent coverage: telephony and WebRTC voice, chat, WhatsApp, Slack, prompt-only, RAG, tool-calling and multimodal
- Personas and evaluation criteria are generated automatically, so there is no manual test configuration to write
- An OpenAI-compatible Chat Completions API makes connecting an in-house agent a standard integration
- Explicit contractual commitment not to train or fine-tune models on Customer Data
- The customer keeps ownership of Customer Data and Outputs (Terms 5.2), export is available, and a DPA is required before any personal data is processed
- Regulated-industry focus is assumed rather than incidental, with per-run checks against FDA and HIPAA expectations and financial rules
- Structured public documentation on Mintlify, with llms.txt, an API reference and use-case guides, plus named customer testimonials (Wysa, Adaface, Virtual Internship, RedDoorz, Turn.io)
Cons
- No public pricing at all: no pricing page, no tiers, no entry price, and the marketing site is only five pages
- No documented permanent free plan; free evaluation goes through a Pilot granted at UserTrace's sole discretion (Terms 3.2), and no self-serve trial is advertised
- GDPR compliance is claimed by a footer badge alone: neither the privacy policy nor the terms mention the GDPR, and no Article 27 EU representative or DPO is named
- No subprocessor list is published, even though the platform connects to third-party services such as LangSmith, LangFuse and Datadog; vendors are described by category only
- No data hosting location is disclosed and no retention period is published, the privacy policy having no retention section at all
- No published security certification (no SOC 2, no ISO 27001), and the HIPAA badge is backed by no public document
- Very young company and product (domain registered 05/05/2025, first Wayback capture 11/07/2025), legal documents shipped as PDFs rather than versioned web pages, and homepage FAQ answers that are unreadable without JavaScript
Pricing & Plans
UserTrace publishes no pricing. There is no pricing page on the website, none is listed in the sitemap, and /pricing merely returns the homepage HTML. The terms are titled USERTRACE ENTERPRISE TERMS OF USE and are entered into between UserTrace, Inc. and the customer entity identified in a signed Order; they refer to Fees and to a commercial subscription without ever stating an amount. As a result, no free plan is documented and no starting price, currency or billing unit can be reported. Section 3 of the terms describes a Pilot Period during which Promotional Credits may be granted at UserTrace's sole discretion, with no purchase necessary and no dormancy fee; those credits are redeemable like paid credits, although their redemption value may differ, and the Pilot converts automatically into a commercial license at the end of the period. Prospective customers should note clause 3.6: Customer Data entered during the Pilot may be permanently lost if they do not purchase and have not exported it beforehand. Figures are obtained by contacting the vendor through Start Simulations, Request a Demo or the Book a Demo link in the documentation.
- No named or priced plan is published on the website.
- an evaluation license covering the Pilot Period
- with Promotional Credits granted at UserTrace's sole discretion (Terms sections 2.1 and 3).
- the commercial license the Pilot converts into
- once Promotional Credits are exhausted or when the customer decides to move to commercial terms.
- No information is published on the number of seats
- simulations or credits included in any tier.
Data, GDPR & hosting
A consolidated view of how UserTrace handles your data.
GDPR overview
A "GDPR Compliant" badge sits in the footer of all five pages, but it is only image alt text: neither the privacy policy nor the terms of use mention the GDPR anywhere. What is documented is contractual. Terms section 2.3 requires the customer to sign a Data Processing Addendum before any personal data is processed, failing which the customer warrants that no personal data covered by applicable data protection laws is submitted; where DPA and terms conflict, the more protective clause wins. The privacy policy casts UserTrace as a data processor acting on customer instructions, confirms US headquarters and possible transfers to the United States or other less protective jurisdictions, and describes data subject rights generically (account access, marketing opt-out, cookie refusal, account closure on request). No Article 27 EU representative and no DPO are named. Privacy contact: contact@getusertrace.com. Both documents are dated 15 June 2025.
Who owns the data?
Section 5.2 of the terms of use is explicit: "The Customer Data hosted by UserTrace as part of the Services, the Outputs and all worldwide Intellectual Property Rights in and to the foregoing, are the exclusive property of Customer". Customer Data covers Connected Account Data, AI Agent Data, Simulation Data and anything submitted by CSV, API or interface. In return, the customer grants UserTrace a non-exclusive, worldwide, royalty-free license for the term of the Order, to provide and improve the Services and to create de-identified Aggregated Data. The platform, its documentation and Integration Tools remain UserTrace property (5.1), and customers can export their Customer Data (4.4).
Reuse rights
Because Customer Data and Outputs belong to the customer (Terms 5.2), the customer can reuse, export and redistribute simulation results freely, without asking permission. On the publisher's side, the privacy policy states that "UserTrace does not use Customer Data, including AI Agent Data or Simulation Data, to train or fine-tune general-purpose artificial intelligence or machine learning models", and that Customer Data is processed solely to provide the Services to that customer. De-identified aggregated data may be used to understand service performance and improve reliability, and is not shared externally in regulated or healthcare use cases without a contractual agreement. By design the platform is meant to run on simulated, test or sandbox data. Website data is a separate matter: personal information collected through the site is used for service delivery, personalization, analytics, direct marketing and targeted advertising, and may be shared with vendors (hosting, IT, support, emailing, marketing, analytics), advertising partners, third parties the user designates, professional advisers, authorities and an acquirer in a corporate transaction. Session replay of clicks, mouse movements and keystrokes is also declared. No training opt-out is documented, the publisher's position being that no such training happens.
Data retention & training
Hosting summary
No hosting country, region or provider is disclosed. The privacy policy offers a transfer clause rather than a hosting statement: "We are headquartered in the United States and may use service providers that operate in other countries. Your personal information may be transferred to the United States or other locations where privacy laws may not be as protective as those in your state, province, or country." Service providers are described by category only, hosting, IT, customer support, emailing, marketing, consumer research and analytics, and are never named, so there is no subprocessor list to check. The website is served behind Cloudflare (IP 104.21.95.219, AS13335, anycast), which tells you about the CDN edge and nothing about where data is stored. The company itself has offices in San Francisco and in India (Sonata, Miramar, Panaji, Goa). On security, the policy commits only to "a number of technical, organizational and physical safeguards", with no detail and no certification. In practice: assume US jurisdiction, since UserTrace, Inc. is a US company, but ask for the hosting region, the provider and the subprocessor list in writing before signing.
Things to keep in mind
Risks and trade-offs to weigh before adopting UserTrace.
- The "GDPR Compliant" footer badge is not supported by a single mention of the GDPR in the privacy policy or the terms; treat it as a marketing claim until the vendor produces documentation
- The "HIPAA Compliant" badge is displayed with no public attestation, certification or audit report behind it, which matters if you are planning clinical use
- Customer Data entered during the Pilot Period may be permanently lost if you decide not to buy and have not exported it first (Terms 3.6)
- No retention period is published anywhere, and no subprocessor list exists even though the platform connects to third-party services such as LangSmith, LangFuse and Datadog
- The website's own privacy policy declares targeted advertising and session replay of clicks, mouse movements and keystrokes, and states that browser do-not-track requests are ignored: "We currently do not respond to 'Do Not Track' signals."
- With no public price, you cannot budget without going through a salesperson, and you have no reference point to judge the quote you receive
- Simulated users are not real users: a green run reflects the personas and criteria the platform generated, so reading it as proof of production safety is a genuine risk, all the more with a vendor online for less than two years whose San Francisco and Goa addresses come with no stated local legal entity
Setup & Integrations
Technical difficulty
Low to moderate, depending on the agent. The site promises "No engineering dependency. No long setup." and the docs say you can connect your agent and start testing in minutes with minimal setup. Prompt-only testing needs no integration at all, and WhatsApp Business or Slack agents connect with no customer-side code. Prebuilt voice integrations exist for Vapi, Retell, LiveKit and Pipecat; anything else goes through custom SIP. An in-house chat agent means exposing an OpenAI-compatible POST /v1/chat/completions endpoint with Bearer authentication, real engineering work the marketing claim does not mention. Personas and evals come pre-configured, removing the manual setup.
Deployment
Integrations
Behind UserTrace
Social
Resources
All the official URLs gathered for verification and reference.
Frequently asked questions
What does UserTrace actually test?
Which types of agent are supported?
Which platforms and frameworks does it integrate with?
How do I connect an in-house agent?
How much does it cost?
Is there a free trial?
Is my data used to train models?
Who owns the data and the results?
Is a DPA available?
Where is the data hosted?
Should you pick UserTrace?
UserTrace is a specialist, not a generalist: it evaluates conversational and voice agents before they reach production, and it neither builds them nor monitors live traffic. Its strongest cards are the breadth of agent types covered — telephony and WebRTC voice, WhatsApp, Slack, web chat, prompt-only, RAG, tool-calling and multimodal — and the fact that personas and evaluation criteria are generated for you rather than configured by hand. The stated focus on regulated sectors is real: healthcare and BFSI each get a dedicated page, and the product claims regulatory coverage on every run.
The data commitments read well for a buyer. The privacy policy states plainly that Customer Data is never used to train or fine-tune models; the terms leave ownership of Customer Data and Outputs with the customer and allow export; a DPA is required before any personal data is processed.
Transparency about everything else is where it falls short. There is no public pricing whatsoever — no page, no tier, no entry price — so budgeting means talking to a salesperson. The legal documentation is thin: the GDPR badge in the footer is not supported by a single word in the policy or the terms, the HIPAA badge rests on no public attestation, no subprocessor list exists, no retention period is published and no security certification is claimed. The company is also very young, with a domain registered on 05/05/2025 and a first Wayback capture on 11/07/2025.
Worth a serious look if you ship AI agents in a high-stakes domain and want systematic pre-release testing rather than ad-hoc prompting. Come to the demo with procurement questions ready: price, hosting location, retention, subprocessors and the exact basis of those compliance badges.
- Choosing a selection results in a full page refresh.
- Opens in a new window.