Clips AI
Clips AI is a free, open-source Python library that turns long spoken-word videos into short clips. It transcribes the audio, splits it by topic, and reframes the picture from 16:9 to 9:16.
What is Clips AI?
Clips AI is an open-source Python library, published under the MIT licence by Clips AI, Inc., that turns one long video into a set of short ones. Two operations sit at its core: cutting a recording into self-contained clips, and reframing the picture from 16:9 to 9:16 so the result fits a vertical feed. Both are reachable in a handful of lines of code.
The library is deliberately narrow. It targets audio-centric, narrative-based material such as podcasts, interviews, speeches and sermons, because the cutting decision is made on the words rather than the pictures. Three documented modules carry the work. Transcribe wraps WhisperX, itself a wrapper on OpenAI's Whisper, and returns a Transcription object exposing characters, words and sentences, each with its own timestamps, plus the detected language. Clip feeds that transcript to the TextTiling algorithm, first described by Marti A. Hearst in the 1990s and modernised here with BERT embeddings; it looks for shifts in topic rather than topics themselves, segmenting at sentence granularity so clip lengths follow the content. Resize combines three signals, namely speaker diarisation from Pyannote, scene-change detection from PySceneDetect, and face detection from MTCNN and MediaPipe, to follow whoever is talking and return crop coordinates over time.
Everything runs on your own hardware. There is no hosted service, no API key and no upload: install the package, point it at a file path, and the output lands on your disk. That makes it a strong fit for confidential recordings and a poor one for anyone hoping for a web app.
The catch is maintenance. The GitHub repository was created in December 2023 and has received no commit since 17 January 2024, while the last PyPI release, 0.2.1, dates from 15 January 2024. Fifteen issues sit open, and the demo the site still advertises at demo.clipsai.com now redirects to itself in an endless loop. The repository has not been archived, so nothing is officially discontinued, but on a stack built from PyTorch, WhisperX and Pyannote, two years of silence is long enough to expect friction at install time.
What it does
- Transcribe video or audio with word-level and character-level timestamps
- Detect the spoken language automatically, or force it with an ISO 639-1 code
- Split a transcript into coherent clips by detecting shifts in topic
- Return the exact start and end time of every clip found
- Track the active speaker and reframe the picture around them
- Convert a video to 9:16 or to any other aspect ratio you specify
- Cut and re-encode the resulting video files
When to use Clips AI / When not to
A quick filter to help you decide if Clips AI is the right fit.
When to use Clips AI
- Python developers adding automatic video repurposing to their own product
- Machine learning and computer vision engineers who want a documented, self-hosted clipping pipeline
- Podcast and interview producers with in-house engineering support
- Researchers working on transcript segmentation, speaker diarisation or automatic reframing
- Teams handling confidential recordings that must never leave their own infrastructure
When not to use Clips AI
- Creators who want a ready-made web app, since everything here runs through Python code
- Anyone expecting a hosted service or a REST API, as Clips AI runs only on your own machine
- Teams needing vendor support or contractual guarantees, development having stopped in January 2024
- Music videos, gameplay or heavily visual footage, because the algorithm reads the transcript
- Users without a GPU or an appetite for a heavy install (ffmpeg, libmagic, PyTorch, a Hugging Face token)
How to use Clips AI
A typical end-to-end flow, from setup to results.
- Create a Python virtual environment, which the documentation recommends to avoid dependency conflicts
- Install the package with pip install clipsai
- Install WhisperX from its Git repository with pip install whisperx@git+https://github.com/m-bain/whisperx.git
- Install the libmagic system library
- Install ffmpeg
- Create a free Hugging Face access token for Pyannote, needed only for reframing
- Transcribe the file with Transcriber().transcribe(audio_file_path='/abs/path/to/video.mp4')
- Find the clips with ClipFinder().find_clips(transcription=transcription), then read start_time and end_time on each result
- Compute the crop with resize(video_file_path=..., pyannote_auth_token=..., aspect_ratio=(9, 16))
- Apply it with MediaEditor().resize_video(), passing crop_width, crop_height and the segments returned by resize
Pros & Cons
Pros
- Completely free under a permissive MIT licence, commercial use included
- Runs entirely on your own machine, so source videos never leave your infrastructure
- A full pipeline fits in a few lines of Python
- Reference documentation covers every class and parameter, with links to the source code
- The algorithms are named and cited, including a PDF report on the reframing method
- Built on well-established components: Whisper, WhisperX, Pyannote, PySceneDetect, MediaPipe
- No quota, no account and no usage limit of any kind
Cons
- Effectively dormant, with no commit since 17 January 2024 and no release since 15 January 2024
- Fifteen open issues with no maintainer response
- The demo advertised on the site is dead: demo.clipsai.com redirects to itself indefinitely
- No interface and no hosted service, so writing Python is mandatory
- Heavy installation involving WhisperX from Git, libmagic, ffmpeg, PyTorch and around twenty dependencies
- Reframing requires a Hugging Face token, adding an external account to the setup
- No terms of service, no privacy policy, no legal notice and no postal address anywhere
Pricing & Plans
Clips AI is free of charge in full. The library is distributed under the MIT licence and installed with a single pip command; the site publishes no pricing page, no paid tier and no trial, because there is nothing to sell. The documentation also confirms that the Pyannote models used for reframing carry no fee. The only real expense is your own compute, as transcription and face detection are demanding and a GPU is strongly advisable.
Data, GDPR & hosting
A consolidated view of how Clips AI handles your data.
GDPR overview
There is no GDPR mention of any kind on clipsai.com. A search through the archived HTML of all four pages returns no privacy policy, no cookie banner, no data protection officer, no Article 27 representative and no reference to GDPR, CCPA or any other regime. The site publishes no postal address and no contact form, and email enumeration across every page returned nothing at all. That silence is a real gap in the paperwork, but the practical exposure is limited by design: the site is static documentation with no user account and no declared collection, and the library executes locally, so recordings never reach the publisher. Anyone processing personal data in the EU with this library remains the sole controller and receives no assistance whatsoever from the vendor.
Who owns the data?
Clips AI publishes no terms of service and no privacy policy, so no contract governs what the vendor may do with your material. The only legal document is the MIT licence covering the code itself, copyright Clips AI, Inc. 2023. That licence grants broad rights over the software and disclaims all warranty and liability, but it says nothing about video content. In practice the question is largely moot: the library runs on your own machine, and no upload to a Clips AI server is documented anywhere. Your source videos, transcripts and generated clips stay on your infrastructure, under your sole control and ownership.
Reuse rights
The MIT licence lets you reuse, modify, redistribute and commercialise the code without asking permission, on one condition: keep the copyright notice and the licence text. Nothing restricts what you do with the clips and transcripts you generate, and the vendor makes no claim over them in any published document. One caveat matters. Clips AI is a wrapper around third-party models that carry their own terms: Whisper, WhisperX, Pyannote, PySceneDetect, MediaPipe and MTCNN each have separate licences, and the Pyannote models require a Hugging Face account. Check those individually before shipping commercial output, because the permissive licence of Clips AI does not extend to them.
Data retention & training
Hosting summary
No data hosting information is published: there is no trust page, no privacy policy and no terms of service. The architecture largely displaces the question, since the library runs on your own hardware, so videos, transcripts and generated clips are stored wherever you choose to run it, under whatever jurisdiction applies to you. Nothing in the documentation describes an upload to a Clips AI server. The only hosting facts that can be established concern the documentation site itself rather than user data: the domain resolves to 76.76.21.21, an anycast CDN address geolocated in the United States on AS16509 (Amazon.com, Inc.), and the registrar is Cloudflare, Inc., also in the United States. None of this describes where user data lives, because a CDN edge node is a delivery point and not storage. One external dependency deserves a mention: reframing downloads the Pyannote models from Hugging Face using your access token, so that platform sees the token, although inference itself stays local.
Things to keep in mind
Risks and trade-offs to weigh before adopting Clips AI.
- Unmaintained since January 2024, so no security fixes on a large dependency chain including PyTorch and OpenCV
- No terms of service or privacy policy means no contractual commitment from the publisher whatsoever
- The MIT licence supplies the software as is and excludes the authors' liability entirely
- Automatic cutting remains an algorithmic judgement, and a clip lifted out of context can distort what a speaker meant
- Rights to the source material stay your responsibility when reusing third-party podcasts or speeches
- Speaker-following crops can cut out a second participant or an important visual element without warning
- The Hugging Face token appears in plain text in the documented examples and must be handled as a secret
Setup & Integrations
Technical difficulty
High, as this is a developer tool. You need Python 3.9 or later and, ideally, a virtual environment, since the documentation warns about dependency conflicts. WhisperX installs from Git rather than PyPI, and libmagic and ffmpeg must be installed separately at system level. Around twenty Python dependencies follow, including PyTorch, OpenCV, MediaPipe and pyannote.audio. Reframing needs a Hugging Face token, and a GPU is strongly advisable. Add two years without an update and version conflicts become likely, so budget real time for the install.
Deployment
Integrations
Behind Clips AI
Fundraising
Resources
All the official URLs gathered for verification and reference.
Frequently asked questions
Is Clips AI free?
Do I need to know how to code?
What kind of video does it handle best?
How does it decide where to cut?
How does the automatic reframing work?
What do I need to install?
Are my videos uploaded anywhere?
Is the project still maintained?
Is there a demo I can try?
Which languages are supported?
Should you pick Clips AI?
Clips AI does one thing, and its documentation explains exactly how. Turning a two-hour podcast into a handful of vertical clips normally means trusting a black box; here the reasoning is laid out, from TextTiling with BERT embeddings to find topic boundaries through to Pyannote and MediaPipe keeping the speaker in frame, with citations and a PDF report behind it. For a technical audience that transparency is worth a great deal, and because everything runs locally it is one of the few options usable on recordings that must not leave the building.
The reservation is not about quality but about time. The repository has been silent since January 2024. The demo the site still advertises has been broken long enough that nobody noticed, fifteen issues sit unanswered, and the dependency stack of PyTorch, WhisperX and Pyannote has moved a long way in two years. Nothing has been officially discontinued and the package remains installable, but expect to spend an afternoon reconciling versions before anything runs.
Judge it accordingly: not as a product with a roadmap and a support desk, but as a well-documented open-source component you would adopt and maintain yourself. On those terms it is a solid starting point, and the MIT licence grants every right needed to fork it. Teams wanting a supported service should look elsewhere. Teams with a Python engineer and a privacy constraint will find that the two hundred lines they would otherwise have written are already done, explained and free.
- Choosing a selection results in a full page refresh.
- Opens in a new window.