Harmonai
Harmonai is a Stability AI lab publishing open-source generative audio tools under the MIT licence. Its Dance Diffusion notebooks let musicians and researchers generate audio samples and fine-tune models on their own sound libraries.
What is Harmonai?
Harmonai is not a product but a research lab. Its homepage says so in a single line: a Stability AI Lab releasing open-source generative audio tools to make music production more accessible and fun for everyone. The rest of the site, a single Squarespace page, carries three promises (express your creativity without limitations, generate your own custom infinite sound libraries, bring the power back to the artists), a scrolling banner reading AI by musicians, for musicians, and a button through to a Discord server. There is no sign-up, no dashboard, no pricing table and no product interface of any kind.
The substance lives on GitHub, under the Harmonai-org organisation, which has 773 followers and three public repositories, all MIT-licensed. sample-generator is the flagship: described as tools to train a generative model on arbitrary audio samples, it carries 1,117 stars, 172 forks, 44 watchers and 140 commits, and ships two Colab notebooks, one to generate with Dance Diffusion and one to fine-tune it on your own sounds, alongside a training script, dataset utilities and visualisation code. oobleck publishes soundstream-ish VAE audio codecs for downstream neural audio synthesis, with continuous, vector-quantised and spherical variants planned and validation designed around MUSHRA listening tests, visqol and si-snr. A third repository forks archinetai's audio-diffusion-pytorch.
Access is direct and free. The code costs nothing, needs no account and no API key, and the quickest route in is to open the Dance Diffusion notebook in Google Colab; running it locally needs Python 3.7 or later and a single pip install. What it does cost the user is compute: a GPU, whether rented from Colab or already owned.
Two facts temper the picture. The community is genuinely alive, the Discord server counting roughly 30,450 members with about 1,467 online, but the code is not moving: sample-generator was last updated on 29 April 2024, oobleck in June 2023 and the fork in September 2023. Harmonai is best read today as a well-starred, freely reusable body of generative audio research rather than a tool under active development.
What it does
- Generate audio samples with the Dance Diffusion diffusion model
- Train a generative model on an arbitrary library of audio samples
- Fine-tune an existing Dance Diffusion model on your own sounds
- Encode and decode audio through VAE codecs for downstream neural synthesis
- Run everything from a browser-based Colab notebook, with nothing to install
- Inspect, modify and redistribute the source code under the MIT licence
- Join the lab's Discord community to get help and follow the work
When to use Harmonai / When not to
A quick filter to help you decide if Harmonai is the right fit.
When to use Harmonai
- Music producers comfortable with Python who want sample libraries nobody else owns
- Sound designers building custom one-shot and texture banks for film, television or games
- Audio machine-learning researchers working on diffusion models and neural audio codecs
- Game developers who need generated loops and one-shots without licensing costs
- Teams wanting to fine-tune a generative audio model on their own private sound archive
When not to use Harmonai
- Musicians looking for a ready-made web or mobile app, since neither exists
- Anyone unwilling to run Python 3.7 or later, or a Colab notebook on a GPU
- Studios needing a contractual SLA or paid support, which the terms explicitly exclude
- Teams requiring an API to embed audio generation inside their own product
- Buyers who need an actively maintained codebase, as no repository has changed since April 2024
How to use Harmonai
A typical end-to-end flow, from setup to results.
- Browse the Harmonai-org organisation on GitHub to see which of the three repositories fits your goal
- For a first run, open the Dance Diffusion notebook in Google Colab from the sample-generator README
- Connect the notebook to a GPU runtime before executing any cell
- Run the cells to generate samples, then download the audio the model produces
- To work locally instead, clone the sample-generator repository
- Check that Python 3.7 or later is available, as the README requires
- Run pip install from the root of the repository to pull in the dependencies
- Prepare your own audio library if you intend to fine-tune rather than simply generate
- Open the fine-tuning notebook and point it at your folder of samples
- Join the Discord server from the homepage button for help, since there is no support desk
Pros & Cons
Pros
- Completely free and open source under the MIT licence, with no account required
- A Colab notebook makes a first experiment possible without installing anything
- Fine-tuning on your own sounds ships as standard rather than as a paid add-on
- Backed by Stability AI rather than being an anonymous side project
- An active Discord community of roughly 30,450 members, about 1,467 of them online
- The code is auditable end to end, so nothing runs as a black box
- 1,117 stars and 172 forks on the main repository point to a real user base
Cons
- Nothing is usable without Python: no web app, no mobile app and no API
- A GPU is needed in practice, which usually means paying for Google Colab
- All three repositories are dormant, the most recent change dating from 29 April 2024
- The public to-do list in the README was never completed
- The terms explicitly exclude any obligation to maintain or support the site
- Documentation stops at the README files, the website itself being a single page
- No hosting jurisdiction, no data processing agreement and no subprocessor list are published
Pricing & Plans
Harmonai is free of charge. The three repositories are published under the MIT licence, and the website offers no paid plan, no subscription and no trial; no pricing page exists on the domain. There is consequently no lowest price point to quote in any currency. The only expenditure falls outside Harmonai's scope: the compute needed to run the models, whether purchased from Google Colab or supplied by the user's own hardware.
Data, GDPR & hosting
A consolidated view of how Harmonai handles your data.
GDPR overview
Implementation is real but minimal. The privacy notice of 6 August 2022 names both the GDPR and the UK GDPR, sets out the legal bases relied upon (consent, legitimate interests, legal obligations and vital interests) and identifies Harmonai as the data controller, explicitly so for United Kingdom residents. Data subjects in the EEA, the United Kingdom and Canada are offered access, rectification, erasure, restriction, portability, objection and withdrawal of consent, together with the right to complain to their local supervisory authority. Requests go to a single email address. Beyond that, nothing: no Article 27 representative is designated, no data protection officer is named, no data processing agreement is offered, no subprocessor list is published, and no certification or standard contractual clause appears anywhere on the site.
Who owns the data?
Under the terms of service dated 8 August 2022, Harmonai Ltd claims no ownership over what users create: contributors keep full ownership of their contributions and of any intellectual property attached to them. The balance shifts for feedback, since any question, comment, suggestion or idea sent about the site becomes the company's sole and exclusive property, usable and distributable without compensation or acknowledgement. The published code is a separate matter again and sits under the MIT licence across all three repositories. Audio generated by running the models in Colab or on a local machine never reaches Harmonai, so no ownership question arises over it at all.
Reuse rights
The three public repositories are released under the MIT licence, so users may run, modify, redistribute and commercialise the code and whatever they generate with it without asking permission, provided the copyright notice travels with it. Audio produced from the models is never transmitted to Harmonai, which places no restriction on what becomes of it. The website itself is treated far more restrictively: the terms forbid systematically retrieving its content to build a collection or database, reverse engineering it, and using the site or its content for any revenue-generating purpose without written permission. Personal data collected on the site may be used by the publisher for marketing, targeted advertising, security and analytics.
Data retention & training
Hosting summary
Harmonai publishes no hosting information at all. Neither the privacy notice nor the terms of service names a country, a region, a data centre or a cloud provider, and no trust or security page exists on the domain. What can be observed rather than read: the website is served by Squarespace and resolves to an anycast address whose node geolocates to New York City in the United States. That describes the marketing site, not a product backend, because there is no product backend. The models run wherever the user runs them, in a Google Colab session or on a local machine, and audio generated that way never passes through Harmonai's infrastructure. For anyone assessing jurisdiction, the relevant published facts are that the terms are governed by the law of England and Wales with exclusive jurisdiction in London, and that Harmonai declares itself the data controller for United Kingdom residents. No transfer mechanism, standard contractual clause or hosting certification is mentioned anywhere.
Things to keep in mind
Risks and trade-offs to weigh before adopting Harmonai.
- No repository has been updated since April 2024, so building a workflow on this code means adopting an unmaintained dependency
- The footer's Privacy Policy and Terms of Service links lead to Stability AI documents that never mention Harmonai once
- The site's own legal texts date from August 2022 and cover the website only, not the models you actually run
- Under the terms, any idea or suggestion you send about the site becomes the publisher's exclusive property
- Liability is capped at the lesser of what you paid and one pound sterling, which for a free tool means effectively nothing
- The privacy notice declares targeted advertising and marketing communications, which is unusual for a research lab's site
- Generating endless samples is easy; letting the model stand in for the ear training and taste that make samples worth using is the quieter risk
Setup & Integrations
Technical difficulty
Two routes, two levels. Opening the Dance Diffusion notebook in Google Colab is easy: nothing to install, no account with Harmonai, connect a GPU runtime and run the cells. Installing locally is intermediate: clone the repository, provide Python 3.7 or later, then run a pip install from its root. oobleck goes a step further, installed straight from git and configured through a .gin file. In every case a GPU is needed in practice and no graphical interface is provided, so comfort with notebooks and a terminal is the real prerequisite.
Integrations
Supported languages
Behind Harmonai
Resources
All the official URLs gathered for verification and reference.
Frequently asked questions
Is Harmonai free to use?
Do I need to create an account?
Is there a web app or a mobile app?
Does Harmonai offer an API?
What do I need technically to run it?
Can I train the model on my own sounds?
Who is behind Harmonai?
Is the project still maintained?
How long is data collected on the website kept?
Where can I ask a question?
Should you pick Harmonai?
Harmonai occupies an unusual place in a directory of AI tools: it is not a service you subscribe to but a body of open-source work you download. Judged as a product it looks thin, offering a single-page website, no interface, no account, no API and nothing to buy. Judged as a research resource it is substantial. Dance Diffusion made diffusion-based audio generation approachable through a Colab notebook, the fine-tuning notebook lets anyone adapt it to a private sound library, and oobleck's VAE codecs tackle the harder problem of representing audio for downstream synthesis. All of it sits under the MIT licence, which means no permission to seek and no cost to absorb beyond compute.
The audience is therefore narrower than the homepage's AI by musicians, for musicians suggests. A producer who writes no Python will find nothing to click; a producer who does will find a route to sample libraries nobody else owns. Researchers and audio engineers get auditable code with 1,117 stars behind it and a Discord community of roughly 30,450 people to ask.
The reservation is dormancy. No repository has changed since 29 April 2024, the README's own to-do list was never finished, and the terms of service disclaim any obligation to maintain or support anything. The legal documents on the domain date from August 2022 and cover only the website, not the models. Anyone building on this code should treat it as a stable snapshot to fork rather than a dependency that will keep pace.
Taken on those terms, Harmonai is worth the download. It costs nothing, commits you to nothing, and hands over genuinely useful generative audio research, provided you accept that maintaining it will be your job rather than theirs.
- Choosing a selection results in a full page refresh.
- Opens in a new window.