discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens
qdTXTbp/FuFumidi · source
Snapshot 0 · 2026-11-04T12:47:28Z · Python ·
Content fingerprint
71eea23525f705cb41c9e72a88cab0756e24f428bdb5527302941786197924d1GitHub description at capture
Offline MIDI workstation: transcription, piano roll editing, visualization, analysis, score, film scoring. 离线 MIDI 工作站：转录、编辑、可视化、分析、乐谱、影视配乐。
README at capture
# FuFumidi
<p align="center">
<img src="docs/EN.png" alt="FuFumidi project logo" />
</p>
<p align="center"><strong>English</strong> | <a href="README-CN.md">中文</a></p>
[About](#about) | [Features](#features) | [Transcriber](#transcriber) | [Editor](#editor) | [Playback](#playback-and-visualization) | [Analysis](#analysis) | [Score](#score) | [Conversion](#conversion-and-video-export) | [Film Scoring](#film-scoring) | [Offline Runtime](#offline-runtime) | [Architecture](#architecture) | [Layout](#repository-layout) | [Getting Started](#getting-started) | [Building](#building-and-packaging) | [Testing](#testing) | [Plugins](#plugins) | [CI](#continuous-integration) | [Credits](#credits) | [License](#license)
---
## About
FuFumidi is a fully offline desktop workstation for MIDI. It targets musicians, arrangers, transcriptionists and film composers who need a single tool that can go from raw audio to a clean, editable, publishable MIDI/score asset without ever leaving the machine.
The application is packaged as a classic Electron desktop app with a modern Vue 3 + TypeScript renderer, a bundled Python transcriber/runtime, and an optional Rust core. All audio, model weights and inference execute locally; transcription and editing never upload anything. Only when you opt in to cloud sync does your playlist and MIDI library get uploaded to your own cloud account.
Current release line: **6.3.0** ([release notes](https://github.com/qdTXTbp/FuFumidi/releases/tag/v6.3.0), stable channel; test builds are published under the [beta](https://github.com/qdTXTbp/FuFumidi/releases/tag/beta) tag).
What's new in 6.3.0: note audition, multi-note dragging and grid snap, preserved transport/loop after edits, logic-editor and macro undo fixes, a single UI-scale source, a redesigned scrollable song-tab bar (pin/group/overflow), arrangement clips in playback, internal CC7/CC11/CC64 and pitch bend, cross-measure MusicXML ties, a 32MB undo budget with action labels, core keyboard/dialog accessibility, a 596MB installer (down from 712MB), main-process file logs, and empty-catch CI gate. 6.3.0 also includes six surface materials, a third-party-resources disclaimer, and the FuMiVoice download entry. See the [6.3.0 release notes](https://github.com/qdTXTbp/FuFumidi/releases/tag/v6.3.0) for details.
### What it does
- **Audio to MIDI** using three interchangeable local engines (universal, piano-specialized, voice-separated).
- **MIDI editor** with a full piano-roll, CC automation lanes, drums, macros, a logic editor and a list editor.
- **Playback and visualization** with a Synthesia-style waterfall, spectrum, waveform, real-time chord reading and MIDI hardware output.
- **Analysis** that produces key, chord, density and statistical summaries of a song.
- **Score engraving** through Verovio, with automatic clefs and octaves, MusicXML import, and PDF export.
- **Conversion and video export** that renders MIDI back to audio (WAV / MP3 / OGG) or to MP4 with synchronized visualization.
- **Film scoring** support: SMPTE timecode, embedded video track, and two-way clicking between score and piano-roll.
### Highlights
- 100% offline after installation. The Python runtime, core transcription models and Demucs vocal-separation weights ship inside the full installer; optional transcription engines (`YourMT3+`, `MIROS`, `hFT-Transformer`) are one-click downloads from the resource centre.
- Vue 3 + TypeScript + Vite renderer, rebuilt from the original monolithic renderer.
- Five primary areas (Home / Music / Workspace / Create / Resources), hash-routed sub-tabs, four interface languages, 12 built-in themes, custom accent and image themes, `Ctrl+K` command palette, 6-chapter onboarding guide, and a unified static + motion wallpaper library.
- Plugin sandbox that runs third-party code inside its own worker.
- Integrity check and one-click repair for the bundled runtime.
- Optional native Rust core (`rust-core/`) accelerates batch, CPU-heavy library operations.
- GPU acceleration options for CUDA and DirectML, with a graceful fallback to CPU.
### i18n contribution rule
The UI supports four interface languages: **Simplified Chinese / Traditional Chinese / English / Japanese**.
- Every new UI string MUST include an English translation in `frontend/src/core/i18n.js` (`I18N_MAP`); missing English = incomplete.
- Traditional Chinese is auto-derived from the Simplified source via `frontend/src/core/zhHant.js` (no manual per-string translation needed; add to its `PHRASE` map only when a term needs a Traditional-Chinese wording difference).
- The language can be switched anytime in Settings (persisted to `localStorage(fufumidi_lang)` and `settings.lang`). On a fresh install the UI follows the system/browser language once; updates never reset the user's choice.
---
## Features
A short map of the main views. Each view is reachable from the sidebar or via the command palette.
| Area | What it holds |
| --- | --- |
| Home | Continue or start a project, recent items, system status, appearance shortcut. |
| Music | Library/playlists, playback, visualization, score. |
| Workspace | Piano-roll editing, lyrics, analysis, arrangement. |
| Create | Transcription, conversion, score conversion, cover, singing/tuning. |
| Resources | Models, soundfonts, storage, plugins and diagnostics. |
| Appearance (drawer) | Themes, accent colour, wallpapers, font size, density, UI scale, language, reduced motion. |
| Settings | Sound, GPU, features, shortcuts, cloud sync, updates and feedback. |
---
## Transcriber
The transcriber supports three modes and multiple sub-models. Every mode runs against the bundled Python runtime; the core models ship with the installer, while the optional engines (`YourMT3+`, `MIROS`, `hFT-Transformer`) are downloaded once from the resource centre and then run fully offline.
| Mode | Sub-models | What it is good for |
| --- | --- | --- |
| Universal | `basic-pitch` (ONNX int8), `MuScriptor` (small / medium / large), `YourMT3+` (YPTF+Single), `MIROS` (2025 AMT Challenge champion) | Broad polyphonic material, mixing instruments. |
| Piano | `piano_pt`, `aria`, `transkun`, `hFT-Transformer` | Piano recordings, including pedal detection. |
| Separate | `demucs` (htdemucs) | Vocal-first or stem-separated transcription. |
Advanced options exposed by the view include onset/frame thresholds, minimum note length, merge gap, pedal detection, drum extraction, denoising, normalization, automatic tempo detection, stem export (WAV) and a transcription queue with pause, cancel and reordering.
The engine layer is spread across a small family of Python modules so a specific transcriber can be swapped without touching the UI:
- `engine/engine.py` - front door and dispatch.
- `engine/engine_basic.py`, `engine/engine_muscriptor.py`, `engine/engine_aria.py`, `engine/engine_pt.py`, `engine/engine_transkun.py` - per-model inference; `engine/engine_msst.py` - MSST source separation.
- `engine/pause_gate.py` - the file-flag pause/resume gate shared by every streaming transcriber (MuScriptor token stream, basic-pitch window generator, the batched piano forward loop).
- `engine/engine_gpu.py` - CUDA / DirectML selection and fallback.
- `engine/smart_midi.py` - post-transcription correction and smart cleanup.
- `engine/midi_post.py`, `engine/midi_edit.py` - shared MIDI utilities used by the editor and by plugins.
- `engine/deps.py` - dependency check and auto-repair.
- `engine/diag.py` - diagnostic export for support reports.
Transcription results are validated by `engine/tests/`, which exercises the pipeline end-to-end with the fixture files `data/_test_melody.wav` and `data/_test_out.mid`.
---
## DiffSinger Singing Synthesis (Optional Module)
DiffSinger is a **fully optional** AI singing-synthesis module: disabled by default with zero downloads and zero disk footprint; only after enabling does it fetch the two component groups on demand — the Python inference dependencies (`onnxruntime` / `pyyaml`) and the universal NSF-HiFiGAN vocoder (DiffSinger Community Vocoder release by openvpi). **The official training repository is never cloned**: voicebanks ship with ONNX models, and `engine/engine_diffsinger.py` runs inference directly (phonemization → duration allocation → frame-level pitch curve → acoustic → vocoder → WAV), the same way OpenUTAU loads them.
The tuning workbench lets you **pick a MIDI directly from your existing music library**: it parses all non-drum tracks, imports the chosen melody track as the base note sequence, and renders singing after per-note lyric / pitch / duration / vibrato editing — with playback and WAV export. Voicebanks can be imported locally (zip / .oudep) or downloaded from the curated public registry.
---
## Editor
The editor is the largest single surface of the application.
- Piano roll with pencil, select and eraser tools, snap-to-grid from off up to 1/32, click-to-audition and multi-note dragging.
- Undo and redo on full track snapshots with a 32MB/80-step budget, action labels and a size/time history panel.
- Note-edge stretching for independent start and length edits.
- A velocity curve editor with envelope drawing over a selected range.
- CC automation lanes for CC1 (modulation), CC7 (velocity), CC10 (pan), CC11 (expression) and CC64 (sustain), each drawable with a freehand, line or curve brush.
- Drum editor with a 29-instrument by 8-bar grid and per-track visibility.
- Key Switch visualization and mapping, with C-2 to C0 technique names and Spitfire, VSL and EastWest presets.
- Smart quantize with Funk, Jazz, Rock, Latin and custom groove templates, plus an "extract selected as groove" workflow.
- Logic editor for batch rules (target x condition x operation).
- Macro system with built-in macros (project clean-up, batch transpose, velocity normalization) and user-defined macros that are persisted locally.
- Audio alignment tool: load the original audio, see the waveform at the bottom of the piano roll, snap note onsets within a +/-80 ms window, and audition in sync.
- Smart accompaniment that generates three tracks - bass, arpeggiated chords and pad - from a per-bar chord analysis.
- Timbre tools: GM voice picker, apply-to-all-non-drum, and smart timbre selection by range or by name.
- Batch operations: chord selection, delete short notes under 80 ms, loudness +/-10%, transpose +1 / +8 / -1 / -8 semitones.
- Lyrics injection directly from the editor onto selected notes.
- Fullscreen editing, an inspector panel, a help overlay and a `Ctrl+S` export to a native save dialog.
---
## Playback and Visualization
- Web Audio synthesis with js-synthesizer plus FluidSynth and GeneralUser.sf2 for MIDI output through the desktop.
- `navigator.requestMIDIAccess` selection of an external MIDI device; note-on / note-off events are mirrored, and an all-notes-off is sent on stop to prevent stuck notes.
- Tempo control is two-way with BPM. Changing the tempo multiplier updates the BPM field and vice versa.
- Mixer popup for volume, pan, solo and mute across tracks.
- Four visualization styles: Synthesia-style note waterfall, spectrum waterfall, oscilloscope waveform and real-time chord reading.
- Dashboard and waterfall layout toggle, plus fullscreen visualization.
---
## Analysis
The analysis view produces a report on the current song.
- Key detection and chord detection over time.
- Pitch, velocity and duration distributions.
- Timeline density (notes per bar, notes per second).
- Natural-language summary of the material, ready to paste into a brief or a caption.
The analysis engine is split between `frontend/src/core/analysis.js` (report assembly) and `engine/music2midi.py` (feature extraction used by the transcriber).
---
## Score
- Verovio-based engraving with automatic clef, 8va, 8vb and 15ma placement.
- Chord markers, lyrics and section marks.
- Score-to-piano-roll sync: click a note on the score and the playback head and piano-roll cursor move together.
- MusicXML import via `frontend/src/core/musicxml.js`, with support for multiple parts, time signatures, key signatures and ornaments.
- PDF export through `printToPDF`, and paginated SVG-to-PNG preview.
- Split preview of score and piano roll side by side.
---
## Conversion and Video Export
`ViewConvert` renders MIDI back to audio or to video.
Audio export supports WAV, MP3 and OGG, with a choice of 26 timbres (piano, electric piano, organ, strings, brass, flute, guitar, bass, lead, pad, violin, cello, harp, marimba, music box, vibraphone, choir, trumpet, sax, clarinet, oboe, nylon guitar, steel guitar, synth bass, bell, accordion, banjo) plus an auto-select mode that picks a preset per GM program number. Tempo multiplier, sample rate and gain are tunable, and a range selector lets you render the full song or a specific section.
The audio renderer is written as a bucketed offline synthesis (`renderAudioBuffer`), so a single long track does not need a single massive `OfflineAudioContext`.
Video export uses `canvas.captureStream` plus `MediaRecorder` to record the visualization canvas, then runs the captured stream through the bundled ffmpeg via the `video:transcode` bridge to produce an MP4 with the rendered audio. Available controls:
- Aspect ratio: landscape, portrait, subtitle-safe area.
- Resolution: 720p up to 4K.
- Frame rate: 24, 30, 60 fps.
- Quality preset or a custom bitrate.
- Clip length: 15 s, 30 s, 60 s, full song, or a custom range.
- Background: color or image.
- Watermark with adjustable opacity.
- Lyrics subtitles, progress bar and timecode toggles.
The visualization inside the video is one of the four styles above, so the same rendering code is reused for on-screen playback and for offline capture.
---
## Film Scoring
- SMPTE timecode display alongside the transport.
- Embedded video track with synchronized playback; the video is shown in a dock at the top-right of the editing surface.
- Click-to-locate: click a note on the score or the piano roll and the video head jumps to the corresponding timecode.
---
## Offline Runtime
- The full installer bundles the Python runtime, transcriber models and the Demucs weights. After installation the app does not need to reach the network.
- The base installer ships only the basic-pitch model and downloads additional models on first use, so it is much smaller.
- `engine/deps.py` performs a dependency check on startup and can auto-repair a broken environment.
- An integrity check runs in the background at startup and surfaces a warning banner plus a one-click repair action in the Settings view.
- A SQLite database (`main/db.js`) persists settings, playlists and plugin state locally.
- Wallpapers, video wallpapers and soundfonts ship inside the installer.
---
## Architecture
FuFumidi is a three-process application.
- **Main process** (`main.js`, `preload.js`, `main/`)
Electron main process, IPC routing, dialogs, settings, playlist, database, updater, video transcode bridge, wallpaper service and plugin host.
- **Renderer** (`frontend/`)
Vue 3 application built by Vite to `renderer/dist`. State is held in Pinia stores (`stores/app.ts`, `stores/playlist.ts`, `stores/settings.ts`). Views live under `views/`, reusable components under `components/`, and the audio/MIDI/analysis/scoring core lives under `core/`.
- **Python engine** (`engine/`)
The transcriber and audio-tooling layer. Invoked through a JSON protocol so the renderer only needs a single `bridge.convert` / `bridge.engine.run` call.
Optional accelerations:
- **Rust core** (`rust-core/`) - a standalone native CLI (`fufumidi-core`) driven over a JSON subprocess protocol. It handles the heavy, CPU-bound batch work in the library: track-health scanning (`batch-stats`), duplicate detection via SHA-256 content hashing (`hash-batch`), and batch quantization / transposition (`quantize-batch` / `transpose-batch`). It degrades gracefully to JS / Python when the binary is not bundled. (`src-tauri/` is a separate Tauri migration shell, not used by the current Electron runtime.) Built via `npm run build:rust` and validated by `npm run test:rust`.
- **GPU runtime** (`engine/engine_gpu.py`, `main/gpu.js`, `main/gpu-ipc.js`) - selects CUDA or DirectML based on the local driver, and falls back to CPU when neither is available.
- **Plugins** (`plugins/`, `plugin-host.js`, `plugin-worker.js`) - a sandboxed extension system that runs each plugin in a dedicated worker with a scoped API.
---
## Repository Layout
```
FuFumidi/
main.js Electron main entry
preload.js Secure bridge from renderer to main
main/ Main-process modules (engine, gpu, db, plugins, ...)
frontend/ Vue 3 + TypeScript renderer
src/
views/ One component per app view
components/ Reusable UI (PianoRoll, EditorCanvas, PlayerBar, ...)
core/ MIDI, synth, player, analysis, score, viz, i18n
stores/ Pinia stores
bridge/ Electron bridge shims
assets/ Logo and shared images
renderer/ Built renderer output and vendor files (soundfonts, js-synthesizer, abcjs, verovio)
engine/ Python transcriber, models, ffmpeg wrapper, MIDI utilities
tests/ Engine test suite with fixture wav/midi
plugins/ Built-in plugins and the plugin developer guide
src-tauri/ Tauri migration shell (not used by the current Electron runtime)
rust-core/ Native batch-processing CLI (fufumidi-core) over a JSON protocol
build/ Icon, kachina config, installer artwork
scripts/ Build and verification scripts
.github/workflows/ CI pipelines
docs/ Language-specific README assets (EN.png, CN.png)
```
---
## Getting Started
### Prerequisites
- Windows 10 / 11, macOS 12+ or a recent Linux desktop.
- Node.js 20+ and npm.
- Python 3.11 (recommended for the separate mode) or Python 3.13 for a slimmer install. The `engine/deps.py` script can detect and repair what is missing.
- Optional: Rust toolchain (`rustc` / `cargo`) for `npm run build:rust`.
- Optional: CUDA toolkit or WSL + MSVC for GPU builds; MSVC is required for the Rust core.
### Installing
Download the installer from the [releases page](https://github.com/qdTXTbp/FuFumidi/releases/latest):
- **`FuFumidi-Setup-X.Y.Z.exe` (full installer)** - Python runtime, all transcriber models, Demucs weights. Recommended for offline use and new installs.
- **`FuFumidi.Install.exe` (fixed-name incremental package)** - app code only, keeps the existing environment on disk. For in-app auto-update, one-click repair and overwrite installs.
On first launch the app runs an integrity check and prompts you to repair if anything is missing.
### Running from source
```bash
cd frontend
npm install
npm run build
cd ..
npm install
npm start
```
`npm start` launches Electron against the freshly built renderer. The Python engine is resolved through the same `engine/deps.py` logic used in the installer, so point it at a Python 3.11 or 3.13 environment with the transcription dependencies installed.
---
## Building and Packaging
```bash
# Frontend
cd frontend && npm install && npm run build
cd ..
# Desktop runtime
npm install
# Full installer (Python + all models)
npm run dist:win
# Base installer (basic-pitch only)
electron-builder --config electron-builder.base.yml
# Source archive (7z -mx9)
npm run pack:source
# Rust core
npm run build:rust
```
Before running any of the installer commands, place the large binaries in the expected locations. The build scripts assume:
- `resources/` - Python runtime, `elevate.exe`, and any platform-specific helpers.
- `models/` - transcription model weights (basic-pitch, piano transcriber, Demucs).
Both `resources/` and `models/` are listed in `.gitignore` because they are large.
`engine/deps.py` can be run directly to check and repair the Python environment:
```bash
python engine/deps.py
```
---
## Testing
```bash
# Rust core
npm run test:rust
# Frontend unit tests (macro, i18n, util)
npm run test:ui
# Plugin sandbox
npm run test:plugin
# Everything
npm run test
# Frontend type check
npm run typecheck
# Engine tests
cd engine && pytest
```
`engine/tests/` covers audio I/O, engine configuration, MIDI post-processing and presets against the bundled fixtures.
---
## Plugins
FuFumidi supports plugins as third-party extensions that run without modifying the main program. A plugin is a directory with a `plugin.json` manifest and an entry script. The full developer guide is at `plugins/plugin-dev.html` and the concise index at `plugins/README.md`.
Bundled plugins:
- `plugins/auto-key-cmajor/` (**v1.0.0**) - detects the key of a MIDI file and transposes it to C major using a natural-minor mapping, writing the result to a new file (the original is left untouched).
> Policy: `auto-key-cmajor` is the **only** bundled plugin. The earlier demo plugins
> (`example-hello` / `beat-detect` / `midi-stats` / `batch-rename`) have been removed and
> must not be re-added in future releases — `plugins/plugin-dev.html` plus this README are
> enough to write a plugin without shipping an example inside the app.
Each plugin runs in its own worker with a scoped `ctx` object that exposes `commands`, `events`, `engine.run`, `settings`, `ui`, `log` and `app.getSongMeta`. The engine is invoked through the same Python runtime as the built-in transcriber, so a plugin can call any script under `engine/`.
---
## Continuous Integration
- `.github/workflows/ci.yml` - the gate that runs on `master` / `main`: `npm ci`, build the frontend,
run the UI tests and the plugin-sandbox tests, then pack and verify the asar.
- `.github/workflows/build.yml` - manual (`workflow_dispatch`): installs the Python and Node
dependencies, runs the engine probe and packs the asar on Windows, macOS and Linux.
- `.github/workflows/build-installers.yml` - manual: produces Linux and macOS installers.
- `.github/workflows/test-installers.yml` - manual: downloads a release and smoke-tests
install / launch / uninstall. It takes a `tag` input and needs that release to actually carry
Linux or macOS assets.
Releases currently ship **Windows installers only**. `FuFumidi-Setup-<version>.exe` is built on the
machine that holds the `resources/` tree (Python runtime, models) - it is not in this repository, so
the release job cannot run on a hosted runner. The three `workflow_dispatch` workflows are
cross-platform verification paths, not release jobs.
Coding conventions are documented in `.github/CODING_GUIDELINES.md`; repository hygiene findings
and their dispositions in `docs/HYGIENE.md`.
## Acknowledgements
FuFumidi is built on a lot of other people's work. Below is what ships in, or is downloaded by, this
application, grouped by what it does. Licences are those of the upstream projects — the authoritative
texts for the bundled components are next to the files they cover and are listed in
[LICENSE](LICENSE); versions and SHA-256 hashes of the vendored binaries live in
[docs/VENDOR.md](docs/VENDOR.md).
### Application shell and interface
- [Electron](https://github.com/electron/electron) + Chromium — the desktop shell
- [Vue 3](https://github.com/vuejs/core), [Vite](https://github.com/vitejs/vite),
[Pinia](https://github.com/vuejs/pinia), [Vue Router](https://github.com/vuejs/router) — the renderer
- [pdf.js](https://github.com/mozilla/pdf.js) — PDF rendering for the score importer
### Sound and soundfonts
- [js-synthesizer](https://github.com/surikov/js-synthesizer) and
[FluidSynth](https://github.com/FluidSynth/fluidsynth) (`libfluidsynth`) — SoundFont playback
- [GeneralUser GS](https://schristiancollins.com/generaluser.php) by S. Christian Collins — the bundled default soundfont (CC BY 3.0)
- [FluidR3_GM](https://member.keymusician.com/Member/FluidR3_GM/index.html) (MIT),
[Salamander Grand Piano](https://sfzinstruments.github.io/pianos/salamander) by Alexander Holm (CC BY 3.0),
[Arachno SoundFont](https://www.arachnosoft.com/main/soundfont.php),
[Aspirin-DX Soundbank](https://github.com/NeoSoundFonts/Aspirin-DX-Soundbank) (NeoSoundFonts),
SGM-V2.01, Timbres of Heaven, FM/GM and GIGA FM — the additional soundfonts in the workshop
### Scores and optical recognition
- [Verovio](https://github.com/rism-digital/verovio) — MusicXML engraving (staff / tablature views)
- [Audiveris](https://github.com/Audiveris/audiveris) — the optional OMR backend, downloaded on demand
### Transcription, separation and analysis models
- [basic-pitch](https://github.com/spotify/basic-pitch) (Spotify) — general audio-to-MIDI, bundled ONNX model
- [piano-transcription-inference](https://github.com/qiuqiangkong/piano_transcription_inference) / the CRNN model
published on [Zenodo](https://zenodo.org/record/4034264) — the piano-specific transcriber
- [MuScriptor](https://github.com/muscriptor/muscriptor) — the MuScriptor model family
- [Aria-AMT](https://github.com/EleutherAI/aria-amt) (EleutherAI) and `aria-utils` — the Aria chain
- [Transkun](https://github.com/Yujia-Yan/TransKun) — the transkun transcriber
- [Demucs](https://github.com/facebookresearch/demucs) (Meta) — vocal / accompaniment separation
- [MSST](https://github.com/ZFTurbo/Music-Source-Separation-Training) via [pymss](https://pypi.org/project/pymss/) —
the extra separation models (BS-RoFormer, Mel-Band-RoFormer, HTDemucs, MDX23C, SCNet, Bandit, Swin-UperNet configs)
- [Beat This!](https://github.com/CPJKU/beat_this) — beat grid detection
### Singing synthesis
- [OpenUtau](https://github.com/stakira/OpenUtau) (MIT) — the UTAU-compatible engine this project ports,
together with the WORLD / `worldline` native runtimes it ships
- [DiffSinger](https://github.com/openvpi/DiffSinger) ecosystem (openvpi) and the
[NSF-HiFiGAN vocoder](https://github.com/openvpi/vocoders) — AI singing synthesis
- [pypinyin](https://github.com/mozillazg/python-pinyin) — Chinese lyrics to phonemes
- Community voicebank and model releases linked from the in-app catalogue (for example
[Ria](https://github.com/RibosomeK/RiaDiffSinger), [utsu](https://github.com/titinko/utsu),
[HowHow-UTAU](https://github.com/EarlySpringCommitee/HowHow-UTAU)) — downloaded by the user, not bundled
### Python runtime and scientific stack
- CPython, and [NumPy](https://github.com/numpy/numpy), [SciPy](https://github.com/scipy/scipy),
[librosa](https://github.com/librosa/librosa), [soundfile](https://github.com/bastibe/python-soundfile),
[pretty_midi](https://github.com/craffel/pretty-midi), [mido](https://github.com/mido/mido),
[PyYAML](https://github.com/yaml/pyyaml), [Pillow](https://github.com/python-pillow/Pillow),
[tqdm](https://github.com/tqdm/tqdm), [einops](https://github.com/arogozhnikov/einops),
[OmegaConf](https://github.com/omry/omegaconf), [ml_collections](https://github.com/google/ml_collections),
[beartype](https://github.com/beartype/beartype), [pydub](https://github.com/jiaaro/pydub),
[mir_eval](https://github.com/craffel/mir_eval), [resampy](https://github.com/bmcfee/resampy)
- [ONNX Runtime](https://github.com/microsoft/onnxruntime) and [ONNX](https://github.com/onnx/onnx) — model inference
- [PyTorch](https://github.com/pytorch/pytorch) + torchaudio — used by the GPU acceleration packs
- [FFmpeg](https://ffmpeg.org/) (via [imageio-ffmpeg](https://github.com/imageio/imageio-ffmpeg)) — audio / video encoding
### Storage, packaging and hosting
- [SQLite](https://sqlite.org/) via [sql.js](https://github.com/sql-js/sql.js) — the local library database;
[adm-zip](https://github.com/cthackers/adm-zip) — archive handling
- [electron-builder](https://github.com/electron-userland/electron-builder) and [NSIS](https://nsis.sourceforge.io/) — installers
- [GitHub Actions](https://github.com/features/actions) — CI
- [Cloudflare Workers](https://workers.cloudflare.com/), D1, R2 and Turnstile — the optional cloud sync service
(`cloud-sync/`, which you deploy yourself)
If you believe something used here is missing from this list or attributed incorrectly, please open an issue —
we would rather fix the credit than keep an incomplete list.
## Credits
Thanks to OrbisAI Security for providing automated security auditing.
## License
MIT