Groq
Groq is an AI inference cloud that serves open-weight language, speech and vision models through an OpenAI-compatible API, running on its own LPU hardware across thirteen data centres, with a permanent free tier and usage-based pricing.
What is Groq?
Groq describes itself as the premier neocloud for fast inference. Behind that phrase sits Groq LLC, an American company that designs its own inference silicon and sells access to it as a cloud service. The platform is presented in three stacked layers, each including the ones below it: GroqMetal provides dedicated bare-metal infrastructure, GroqCore adds a tested inference stack that turns that capacity into production-ready performance, and GroqAssured layers on enterprise governance, auditability and control.
The hardware story is the differentiator. Groq pioneered the LPU, and its announced LPX generation places 256 LPU accelerators per rack alongside NVIDIA's Vera Rubin, quoting 40 PB/s of SRAM bandwidth, 128 GB of on-chip SRAM per rack, 315 PFLOPS of FP8 compute and 1,000 tokens per second per user. That capacity runs from thirteen data centres on four continents, with named sites in Liberty Lake, St Paul, Dallas and Houston, Kamloops, Vaudreuil-Dorion and Calgary, Vantaa, Sydney, London and Riyadh.
For developers, all of this is reached through GroqCloud, an inference API deliberately made compatible with OpenAI's: point the OpenAI client at https://api.groq.com/openai/v1 and existing code keeps working. Official SDKs exist for Python 3.7+ and JavaScript/TypeScript. The catalogue is made of third-party open-weight models rather than a house model, including GPT-OSS 120B and 20B, Qwen3.6-27B, Llama 3.1 8B and Llama 3.3 70B, MiniMax M2.7, Whisper large-v3 and its turbo variant, Orpheus text-to-speech in English and Saudi Arabic, and Llama Prompt Guard 2. Documented capabilities cover text generation, speech to text, text to speech, OCR and vision, reasoning, content moderation, structured outputs, prompt caching, tool use, remote MCP, batch processing and LoRA inference. Two agentic systems, Compound and Compound Mini, combine models with built-in tools at around 450 tokens per second.
Groq claims trillions of tokens served weekly to Fortune 500 companies and thousands of AI-native firms, and says it is an NVIDIA Cloud Partner. Developer counts quoted on the site range from two million to six million depending on the page.
What it does
- Run open-weight language models through an OpenAI-compatible API at up to 1,000 tokens per second
- Transcribe audio with Whisper and generate speech with Orpheus text-to-speech
- Read images and documents with the OCR and image recognition endpoints
- Give models built-in tools: web search, website visits, code execution and Wolfram Alpha
- Connect remote MCP servers and ready-made Gmail, Google Calendar and Google Drive connectors
- Process large workloads asynchronously with batch jobs, or run fine-tuned and LoRA weights
- Filter prompts and responses with content moderation and Prompt Guard safeguard models
When to use Groq / When not to
A quick filter to help you decide if Groq is the right fit.
When to use Groq
- Backend and full-stack developers building latency-sensitive LLM features who want an API they can call with the OpenAI client they already use
- Teams already running on OpenAI-compatible tooling who want to switch providers by changing a single base URL rather than rewriting their integration
- AI platform, MLOps and LLMOps engineers who need production controls such as service tiers, batch processing, spend limits, projects and Prometheus metrics
- Builders of real-time voice and agent products, who can combine Whisper speech-to-text, Orpheus text-to-speech and the documented LiveKit integration
- Enterprises with procurement and privacy requirements, served by the published Services Agreement, Data Processing Addendum, Business Associate Addendum and Zero Data Retention setting
When not to use Groq
- Non-technical users looking for a ready-made assistant: Groq sells an API and a console, not a finished consumer product
- Anyone who needs a proprietary frontier model, since Groq serves third-party open-weight models rather than a house model of its own
- Teams that need a fixed monthly subscription for budgeting, because every paid plan is metered per token with no flat fee
- Organisations wanting customer data kept inside the EU, as all retained customer data sits in Google Cloud buckets in the United States
- Products aimed at minors or building unsupervised automated decisions in employment, healthcare, finance, legal, housing, insurance or social welfare, which the acceptable use policy forbids
How to use Groq
A typical end-to-end flow, from setup to results.
- Open an account on console.groq.com from the Start building button on the website, accepting the Services Agreement
- Try a model straight away in the Playground, without writing any code
- Generate an API key in the console under API Keys and expose it as the GROQ_API_KEY environment variable
- Install an official SDK with pip install groq for Python, or the JavaScript/TypeScript package
- Alternatively, keep your OpenAI client and simply set the base URL to https://api.groq.com/openai/v1
- Pick a model from the Models page, checking its price, speed, context window and rate limits
- Make a first call, following the Quickstart or the cookbook examples published on GitHub
- Add a payment card, US bank account or SEPA debit to move from the Free tier to the Developer tier
- Set spend limits and watch consumption in Dashboard then Usage as traffic grows
- Review Settings then Data Controls to configure retention, including Zero Data Retention, before going live
Pros & Cons
Pros
- Published per-model throughput, from 260 to 1,000 tokens per second, so speed claims can be checked rather than assumed
- OpenAI compatibility makes migration a base-URL change instead of an integration project
- A permanent free tier, and no fixed fee on the paid Developer tier
- Progressive billing thresholds at $1, $10, $100, $500 and $1,000 that prevent a surprise first invoice
- No retention of inference inputs and outputs by default, with Zero Data Retention available to every customer
- A contractual ban on training or fine-tuning models on customer inputs and outputs
- Unusually complete public legal documentation: Services Agreement, DPA, BAA, acceptable use policy and a public subprocessor list
Cons
- No pricing page on groq.com at all: rates live only inside the console documentation, model by model
- Metered token pricing makes budgets hard to estimate before you have measured real traffic
- Several models, including Llama 3.1 8B, Llama 3.3 70B and MiniMax M2.7, are marked Enterprise and Contact Sales, with neither price nor rate limits published
- Preview models are explicitly unsuitable for production and may be discontinued at short notice
- All retained customer data sits in United States buckets, whatever region the inference itself ran in
- The Trust Center is a JavaScript application, so certifications cannot be read without a browser
- Chat support is reserved for the paid Developer tier
Pricing & Plans
Groq offers a permanent free plan. The Free tier requires no payment method and imposes reduced rate limits. The paid Developer tier carries no fixed subscription fee: it is billed purely on consumption, in arrears, on a monthly cycle. Published rates are unit rates rather than plan prices. The lowest documented entry point is USD 0.03 per million tokens, input and output, for Llama Prompt Guard 2 22M; among general purpose models, GPT-OSS 20B is billed at USD 0.075 per million input tokens and USD 0.30 per million output tokens, and GPT-OSS 120B at USD 0.15 and USD 0.60 respectively. Audio transcription starts at USD 0.04 per hour with Whisper large-v3 turbo, and speech synthesis at USD 22.00 per million characters with Orpheus V1 English. Enterprise-only models carry no public price.
- no cost
- no payment method required
- reduced rate limits
- pay-as-you-go with no fixed fee
- higher rate limits
- chat support
- flex service tier
- batch processing and spend limits
- unpriced
- covering models marked Contact Sales and the GroqAssured governance layer
Data, GDPR & hosting
A consolidated view of how Groq handles your data.
GDPR overview
GDPR implementation is concrete rather than declarative. Groq publishes a Customer Data Processing Addendum, effective 15 October 2025, that names the GDPR (EU 2016/679) alongside the UK GDPR, CCPA, the Swiss FADP and the Saudi PDPL, and relies on the EU Standard Contractual Clauses plus the UK International Data Transfer Addendum for cross-border transfers. Two Article 27 representatives are named: DP-Dock GmbH, Ballindamm 39, 20095 Hamburg, Germany for the EU, and DP Data Protection Services UK Ltd. in London for the United Kingdom, both reachable at groq@gdpr-rep.com. The privacy policy lists access, rectification, erasure, portability, restriction, objection and consent withdrawal, plus the right to complain to a supervisory authority. Privacy requests go to privacy@groq.com, and the subprocessor list is public.
Who owns the data?
Groq splits the question in two. For website and account information it acts as data controller under its privacy policy. For the inputs you send to the API and the outputs you get back, it acts as a processor: the Services Agreement makes the customer responsible for all inputs and for evaluating every output, while Groq commits to not accessing, using, storing or retaining that content except as needed to run the service, follow your instructions, comply with the law or check policy compliance. Crucially, the same clause states Groq is not permitted to use inputs or outputs to train or fine-tune any model unless you explicitly allow it.
Reuse rights
Customers keep practical control of what they send and receive. The Services Agreement places responsibility for inputs and for the use of outputs on the customer and its authorised users, which means outputs can be reused in your own product without asking Groq for permission, subject to the acceptable use policy and to applicable law. Groq undertakes not to access, use, store or retain inputs and outputs beyond what running the service requires, and is contractually barred from training or fine-tuning models on them without explicit customer permission. Limits do apply: you may not strip provenance metadata, digital signatures or watermarks from generated output, you may not use the service to build a competing offering, and you remain accountable for checking accuracy before acting on anything the models produce.
Data retention & training
Hosting summary
Groq draws a firm line between where inference happens and where data is stored. Compute runs from thirteen data centres on four continents, with named sites in the United States, Canada, Finland, the United Kingdom, Australia and Saudi Arabia. Storage is a different matter: the documentation states that all retained customer data is held in Google Cloud Platform buckets located in the United States, whichever region served the request. Groq itself is a United States company and maintains processing operations across several jurisdictions, so international transfers are expected rather than exceptional. Those transfers are covered by the EU Standard Contractual Clauses and the UK International Data Transfer Addendum, and the Data Processing Addendum anticipates onward transfers to the United States and to other countries where Groq or its subprocessors operate. Access controls and security standards are described in the Groq Trust Center, and the subprocessor list is published publicly.
Things to keep in mind
Risks and trade-offs to weigh before adopting Groq.
- Outputs can be wrong: the acceptable use policy puts the burden of checking accuracy and appropriateness entirely on you, and speed makes it tempting to skip that check
- Automated decisions with material impact on people in employment, healthcare, finance, legal, housing, insurance or social welfare require human oversight, and are otherwise forbidden
- Usage-based billing has no natural ceiling, so a runaway loop or a leaked key can generate real cost; spend limits exist but must be set deliberately
- Protected health information is barred without a signed Business Associate Addendum, and ITAR-controlled material is barred outright
- Preview models can disappear at short notice, which is a genuine dependency risk if you ship on one
- Data residency is not configurable: retained data lives in the United States even when inference ran elsewhere
- Liability for beta and free services is capped at USD 5,000, and repeated payment failures can lead to suspension
Setup & Integrations
Technical difficulty
Low to moderate, for a developer. An account and an API key are enough for a first call, and the documentation opens with working curl, Python and TypeScript examples. Because the API mirrors OpenAI's, an existing application often migrates by changing the base URL. The Playground allows testing with no code at all. That said, this is not a no-code product: you need to be able to program. Moving to production is a bigger step, involving the production readiness checklist, latency tuning, security onboarding, Prometheus metrics, projects and model permissions.
Deployment
Integrations
Supported languages
Behind Groq
Fundraising
Social
Resources
All the official URLs gathered for verification and reference.
Frequently asked questions
What exactly is Groq?
Is there a free plan?
How much does it cost?
Does Groq train its models on my data?
How long is my data kept?
Where is the data hosted?
Is a Data Processing Agreement available?
Who is Groq's GDPR representative?
What is the minimum age to use the service?
Which models can I run?
Should you pick Groq?
Groq occupies a clear and slightly unusual position: it is infrastructure, not an application. You do not use Groq the way you use a chatbot, you build on it. What it sells is speed at the inference layer, backed by hardware the company designs itself, and it is refreshingly specific about that speed, publishing throughput figures model by model rather than hiding behind adjectives.
Two things make it easy to try. The API deliberately mirrors OpenAI's, so an existing integration usually moves across by changing one URL, and the free tier is permanent rather than a countdown. Two things make it easy to trust: retained customer data is minimal by default, and the Services Agreement contains an explicit contractual ban on training models with your inputs and outputs, which is a stronger commitment than the policy language many providers offer.
The reservations are real but manageable. There is no pricing page on the marketing site, and rates are buried in the console documentation as per-million-token unit prices, which makes budgeting an exercise in measurement rather than arithmetic. Several of the larger models are gated behind Contact Sales with no published price. And although inference runs from thirteen data centres across four continents, anything Groq does retain lives in United States storage, which will matter to organisations with European data residency requirements.
For teams whose product lives or dies on response latency, and who are comfortable with usage-based costs and open-weight models, Groq is a serious option with unusually transparent legal and technical documentation. For anyone wanting a packaged assistant, a proprietary frontier model or a predictable monthly bill, it is the wrong shelf.
- Choosing a selection results in a full page refresh.
- Opens in a new window.