elluminate
elluminate is a German-built evaluation platform for LLM prompts and AI agents. It scores outputs against criteria you define, compares experiments, traces every agent action, and produces EU AI Act evidence before anything reaches production.
What is elluminate?
elluminate is the evaluation platform built by ellamind GmbH, a Bremen company entered in the German commercial register on 2 February 2024. ellamind presents itself as The Agent Lab and ships five products; elluminate is the entry point and the only one the site offers for free.
Its purpose is to turn AI evaluation into something structured and repeatable. Instead of spreadsheets and ad-hoc scripts, a team works with versioned prompt templates, reusable test collections, binary criteria expressed as yes/no questions, and experiments that can be replayed. Running an experiment across a collection returns pass rates, failure patterns, a breakdown per criterion, token usage and latency. Versions can be compared side by side, so a prompt change or a model swap stops being a coin flip.
What separates it from output-only testing is agentic evaluation. Agents such as Claude Code or Codex run on real tasks inside isolated containers, and the platform captures the whole trajectory: every tool call, file edit, shell command and reasoning step. A team can therefore see not only what an agent produced but how it got there, which is where silent failures hide.
A second strand is regulatory. Pre-built compliance packages cover EU AI Act prohibited practices and Fraunhofer IAIS assessment dimensions, and produce audit-ready PDF reports carrying criterion-level evidence. The vendor is careful to say these reports are not certifications or audit opinions.
The platform is deliberately built for two audiences at once. Product managers and subject-matter experts design evaluations in a visual builder without writing code; engineers use a Python SDK and a REST API to run the same evaluations inside CI/CD. Any OpenAI-compatible provider can be connected, judge models are included on every plan, and bring-your-own-key is available on the paid tiers. The product runs on its own domain rather than the marketing site: the application sits at app.elluminate.de, the documentation at docs.elluminate.de, and the SDK is published on PyPI.
Hosting is in the EU or EEA by contract, with on-premise and air-gapped deployment on Enterprise. Named customers include the health insurer hkk Krankenkasse, the insurance IT provider BITMARCK and the consultancy JUST ADD AI.
What it does
- Score prompt and agent outputs against binary, auditable pass/fail criteria you define yourself
- Run one experiment across a whole test collection and read pass rates, failure patterns, token usage and latency in a single view
- Evaluate a complete agent trajectory — tool calls, file edits, shell commands and reasoning steps — instead of the final answer alone
- Compare versions, models and configurations side by side to catch regressions before a release
- Produce EU AI Act evidence from pre-built compliance packages and export audit-ready PDF reports with a criterion-level breakdown
- Let the Eval Agent build or analyse an evaluation from a plain-language brief, driven through MCP from Claude Code, ChatGPT or Codex
- Trigger evaluations straight from a CI/CD pipeline using the Python SDK or the REST API
When to use elluminate / When not to
A quick filter to help you decide if elluminate is the right fit.
When to use elluminate
- AI and ML engineering teams that have to prove a model or an agent is reliable before it goes live
- Regulated organisations assembling EU AI Act evidence — health insurers, banks and public bodies are the customers the vendor names
- Mixed teams where subject-matter experts write the evaluation criteria and engineers wire the results into CI/CD
- Companies bound by EU data residency rules, including those that need on-premise or fully air-gapped deployment
- Groups already running coding agents such as Claude Code or Codex who need to measure the whole execution trace, not just the final answer
When not to use elluminate
- Teams looking for a runtime to deploy agents in production — that is ellarun, a separate product from the same vendor
- Buyers who need a published price before speaking to sales: only the free plan carries a figure
- Anyone expecting a compliance certificate, since the vendor states its reports are neither certifications nor audit opinions
- Mobile-first users: there is no iOS or Android application, and none is announced
- Solo builders who need more than one project, more than 300 rated responses a month or a long data history without entering a sales conversation
How to use elluminate
A typical end-to-end flow, from setup to results.
- Request access through the free sign-up form on the pricing page: first name, last name, email, company and phone number are all mandatory
- Wait for ellamind to review the request — access to the free plan is granted by invitation, not instantly
- Connect an LLM provider, either an OpenAI-compatible endpoint or your own key on the paid plans
- Create a prompt template with variables and keep it under version control
- Build a test collection covering the real user scenarios and the edge cases that matter to you
- Define binary criteria as yes/no questions that technical and non-technical colleagues can both read
- Launch an experiment across the collection and read the dashboard: pass rate, criterion performance, tokens, latency
- Drill into individual responses and, for agents, into the full trace to find failure patterns
- Compare runs across versions, models and configurations before deciding to release
- Create an API key from the project's Keys dashboard, install the SDK with pip install elluminate and set the key and service address as environment variables
- Wire the SDK or the REST API into CI/CD so that evaluations run on every change
Pros & Cons
Pros
- Evaluates whole agent trajectories rather than final outputs alone, capturing tool calls, file edits and reasoning steps
- The free plan is permanent and includes full SDK access and every judge model, not a countdown trial
- EU and EEA data centres are a contractual commitment, with on-premise and air-gapped options on Enterprise
- An Article 28 data processing agreement is written into the terms instead of being negotiated separately
- The vendor commits in writing not to train any model on customer data unless the customer agrees in text form
- Exit rights are unusually strong: structured machine-readable export, Data Act alignment, at least 30 days to retrieve data, and no switching fees from 12 January 2027
- A 99.5% monthly availability target applies by default, without a negotiated service level
Cons
- No public price beyond the free plan: Pro and Enterprise are quoted after a thirty-minute scoping call
- The free plan is not self-service — the form demands a phone number and access is granted manually after review
- Free is tightly capped at one project, one collection, 300 rated responses a month and 30-day retention, with judge models limited to cost-efficient ones
- Audit trails, SSO/SAML and custom EU AI Act packages are reserved for Enterprise
- The documentation, the application and the API live on a separate domain (docs.elluminate.de, app.elluminate.de) that no page of the marketing site links to and that the sitemap omits entirely
- The terms are written in German and only the German text is legally binding; the English version is a courtesy translation supplied on request
- The vendor holds no security certification — the terms say only that its security management is oriented towards ISO/IEC 27001 and BSI C5
Pricing & Plans
elluminate offers a permanent free plan priced at 0 EUR, covering one project, one collection, 300 rated responses and 40 generated responses per month with full SDK access. Beyond it, the Pro and Enterprise plans are quoted on request: the vendor describes its pricing as value-based, set after a thirty-minute scoping call according to team size, evaluation volume and deployment requirements. No paid price point is published for this product, which is why no starting amount is recorded here. Readers should note that the amounts visible elsewhere on the same pricing page — 490 EUR per month, and from 1,800 EUR per month — belong to elladoc, a different product of the same vendor, and do not apply to elluminate. Prices on that page are stated exclusive of VAT.
- Free — 0 EUR — 1 project
- 1 collection
- 300 rated responses per month
- 40 generated responses per month
- all judge models
- full SDK access
- Eval Agent with 5 prompts
- 30-day data retention
- Pro — on request — 3 projects
- 3 collections
- 300 ratings included then billed per 100
- 200 generated responses per month or bring-your-own-key
- Eval Agent with 20 prompts
- 90-day data retention
- cloud only
- email support and one 30-minute onboarding session
- Enterprise — on request — unlimited projects
- collections and ratings
- custom EU AI Act compliance packages
- SSO
- SAML and audit trails
- cloud
- on-premise or air-gapped deployment
- custom retention
Data, GDPR & hosting
A consolidated view of how elluminate handles your data.
GDPR overview
GDPR implementation is contractual rather than declarative. Clause 11 of the terms is the Article 28 processing agreement: it fixes subject matter, purpose, data categories and instructions, and commits ellamind to process only on documented instruction, to bind authorised staff to confidentiality and to apply Article 32 measures. The customer is named as controller. Subprocessors run under a general authorisation, with text-form notice and a fourteen-day objection right. Special-category data, social data, professional and tax secrets, patient data and minors' data are forbidden unless expressly agreed. An external data protection officer is designated: heyData GmbH, Berlin. The website privacy policy lists the rights of access, rectification, erasure, restriction, objection, portability and consent withdrawal, and states that no Article 22 automated decision-making takes place.
Who owns the data?
Under the terms and conditions the customer keeps every right to its own materials and to any pre-existing customer code. ellamind receives only a non-exclusive, worldwide, sublicensable licence strictly limited to performing the contract: hosting, storing, processing and transmitting that material to the agreed LLM providers and subprocessors. Once the contract ends, the licence survives only for the agreed export window, statutory retention and ordinary backup cycles. ellamind does not sell customer data and does not use it for its own advertising. Rights in AI outputs pass to the customer after full payment, with no warranty of protectability or exclusivity. The customer is the controller under the GDPR.
Reuse rights
The customer may reuse its own material without asking ellamind for permission — the terms place no condition on that reuse. Rights in AI outputs are granted for the customer's own business purposes once payment is complete: an unrestricted, non-exclusive right to use and adapt them, although the vendor gives no assurance that outputs are protectable, unique or free of third-party rights, and the conditions of the underlying model providers continue to apply. Individual work results may be sublicensed to affiliates and to appointed service providers where the customer's own purposes require it, provided those parties are properly bound to confidentiality. Exclusive rights, transfers of ownership, resale, use as a standalone commercial product, source-code release and escrow are all excluded unless separately agreed and paid for. Feedback and improvement suggestions handed to ellamind may be reused by the vendor without charge or limit, as long as no customer material or confidential information is disclosed.
Data retention & training
Hosting summary
The terms commit ellamind to running the standard service from data centres in the European Union or the European Economic Area; moving the primary hosting region outside that area is only possible under the order documents and applicable data protection law. The marketing pages go further, claiming German data centres and stating that data never leaves the EU — a claim the contract itself does not narrow to Germany. The handover point is the data centre router output, so connectivity, devices, browsers and firewalls remain the customer's responsibility. Enterprise customers can choose cloud, on-premise or fully air-gapped deployment, in which case backups become their own responsibility. Backups of the standard cloud are taken at least daily and kept for 30 days by default. Transfers to third countries are permitted only under applicable data protection law. The public website is a separate matter: it is served by Vercel, a US company, which processes the data on servers in the EU.
Things to keep in mind
Risks and trade-offs to weigh before adopting elluminate.
- Automated scores invite false confidence. The vendor states plainly that AI outputs are probabilistic and may be incomplete, outdated, wrong, discriminatory or hallucinated, and that every business, safety, legal or compliance-relevant output must be reviewed by a competent person before use
- Compliance reports produced by the tool are explicitly not certifications, legal opinions or audit attestations. Presenting them to a regulator as formal proof would be a misreading the vendor warns against in its own terms
- Handing quality judgement to an evaluator can quietly erode a team's own feel for how a model behaves. The criteria still have to be written by people who understand the domain, and a platform cannot supply that understanding
- Prompts, configurations and customer content are transmitted to the LLM providers you select. When you bring your own keys or endpoints, responsibility for their data use and training settings shifts to you, not to the vendor
- Binary pass/fail criteria compress nuance. A system can score well on the checks a team thought to write and still fail on the cases nobody imagined, which makes a green dashboard a weak substitute for judgement
- The single pricing page carries five products in sequence, and the only visible euro amounts belong to a different one. A reader in a hurry can budget for the wrong tool
- Nothing on the marketing site points to where the product actually lives: the application, the documentation and the API reference all sit on a separate domain that the sitemap omits, so an evaluator can conclude the tool is less open than it is
- The company was entered in the German commercial register on 2 February 2024, so its track record is short relative to the regulated workloads — health insurance, banking, public administration — that it targets
Setup & Integrations
Technical difficulty
Low for the business side, moderate for engineering. Getting in is administrative rather than technical: a five-field form, then a manual review before an invitation arrives. From there the vendor says most teams run their first evaluation within a day — connect a provider, create a prompt template with test cases and criteria, launch an experiment. Subject-matter experts need no code, since criteria are yes/no questions in a visual builder. Engineers face more work wiring the Python SDK into CI/CD, made harder by the absence of public API documentation. Enterprise deployments with SSO, on-premise or air-gapped hosting are infrastructure projects in their own right.
Deployment
Integrations
Supported languages
Behind elluminate
Social
Resources
All the official URLs gathered for verification and reference.
Frequently asked questions
What exactly does elluminate do?
Is there a free plan, and what does it include?
How much do the paid plans cost?
Where is the data hosted?
Is customer data used to train AI models?
Is a data processing agreement available?
Does elluminate have an API, and is it documented?
What happens to the data if the contract ends?
Should you pick elluminate?
elluminate is a credible, well-documented answer to a real problem: proving that a language model or an agent behaves acceptably before it reaches production, and being able to show the working afterwards. Its strongest card is agentic evaluation — capturing an entire execution trace rather than a final answer — and its second is a European posture that is contractual rather than cosmetic: EU or EEA hosting written into the terms, an Article 28 processing agreement built in, a written commitment not to train on customer data, and exit rights aligned with the Data Act.
The reservations are commercial more than technical. Only the free plan carries a price, and that plan is neither self-service nor generous: a phone number is required, access is granted manually, and the caps are low enough that any serious use leads to a sales conversation. Audit trails, SSO and custom compliance packages sit behind the Enterprise tier. No API reference is published, so the integration effort cannot be assessed in advance. And although the tool is built to produce compliance evidence, its own vendor holds no security certification and says explicitly that its reports are not certifications or audit opinions.
Prospective buyers should also read the pricing page carefully: it presents five products at once, and the only visible euro amounts belong to elladoc, not to elluminate. The company itself is young — entered in the German commercial register in February 2024 — though its founders have a substantial open-source model track record and named customers in German statutory and private health insurance give the references some weight.
For a team in a regulated European industry that needs measurable AI quality and an audit trail, it deserves a serious look. For anyone wanting a priced, self-service tool, it is not that.
- Choosing a selection results in a full page refresh.
- Opens in a new window.