International Audio Laboratories Erlangen

PULSE-IT: Lightweight and Expressive Synthesis of Wind Instrument Playing

Welcome to the accompanying website for the PULSE-IT late-breaking demo. This is an accompanying website to the following publication:

  1. Christian Dittmar, Johannes Zeitler, Stefan Balke, Simon Schwär, and Meinard Müller
    PULSE-IT: Lightweight and Expressive Synthesis of Wind Instrument Playing
    In Late-Breaking Demos of the International Society for Music Information Retrieval Conference (ISMIR), 2025. PDF 
    @inproceedings{DittmarZBSM25_PulseIt_ISMIR-LBD,
author      = {Christian Dittmar and Johannes Zeitler and Stefan Balke and Simon Schw{\"a}r and Meinard M{\"u}ller},
title       = {PULSE-IT: Lightweight and Expressive Synthesis of Wind Instrument Playing},
booktitle   = {Late-Breaking Demos of the International Society for Music Information Retrieval Conference ({ISMIR})},
address     = {Daejeon, Republic of Korea},
year        = {2025},
url-pdf    = {https://audiolabs-erlangen.de/resources/MIR/2025_DittmarZBSM_WindInstrumentSynth_ISMIR-LBD/2025_DittmarZBSM_WindInstrumentSynth_ISMIR-LBD.pdf}
}

Abstract

Wind instruments enable highly expressive performances through playing techniques such as vibrato, slurs, and growl. In this demo, we introduce PULSE-IT, a lightweight synthesis method that combines simple signal processing components with data-driven control signals. Based on a pulsetable oscillator and time-variant filtering, PULSE-IT supports both re-synthesis and cross-synthesis (timbre transfer) of expressive wind instrument playing at low computational cost. Despite its simplicity, the method yields convincing results and provides a practical alternative to more complex neural approaches.

PULSE-IT System Overview

System Overview

Cross-Synthesis Listening Examples

In this example, we perform cross-synthesis with a short excerpt of a jazz trumpet solo recorded in the course of the Jazzomat Project. We extract the spectral centroid as described in eq. (1) [5]. For synthesis, we use pulsetables representing trumpet, clarinet, bassoon and tuba, adjusting the fundamental frequency octave to the target instrument range. For clarinet, bassoon and tuba, we apply smoothing to the control signals to simulate higher inertia.

hfm_original_trumpet

Synthesis from Symbolic MIDI Data

In this example, we synthetically generate micro-modulations instead of extracting them from the original instrument recording. Besides the MIDI pitch, the symbolic data comprises note onsets, note durations, and velocity. Per convention, overlapping notes mark legato playing. This example does not strive at a completely realistic re-creation of the original trumpet solo, but shall illustrate how strongly the micro-modulations influence the plausibility of the synthesized phrases. Most of the variations are coming from a random number generator with fixed seed. In future work this could be done better by generative neural sequence models. For instruments like clarinet, bassoon and tuba, the attack phases have been extended to simulate higher inertia.

hfm_original_trumpet

Cross-Synthesis in ChoraleBricks

Cross-Synthesis in URMP

Cross-Synthesis in Bach10

Additional Remarks

Other Resources

Already in the mid 1970s, electronic wind instruments such as the Martinetta and Variophon were using principles of pulse forming synthesis [9] to emulate tones of real acoustic wind instruments. Despite being implemented with analog circuits, the sound comes relatively close to the actual instruments, thanks to micro-modulations captured with a wind controller.


Today, much more elaborate instrument models can be implemented in digital synthesizer software. The patches by Joel Blanco Berg achieve quite some realistic tones. Here, no neural audio models used, but rather a refined implementation of wavetables plus time-variant filter, similar to the technique by Horner and Beauchamp [5].

Acknowledgments

DFG Logo

This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Grant No. 500643750 (MU 2686/15-1) and under Grant No. 555525568 (MU 2686/18-1). The International Audio Laboratories Erlangen are a joint institution of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) and Fraunhofer Institute for Integrated Circuits IIS.

References

  1. Xavier Rodet
    Time-Domain Formant-Wave-Function Synthesis
    Computer Music Journal, 8(3): 9–14, 1984. 
    @article{Rodet84_FormantWaveSynthesis_CMJ,
author = {Xavier Rodet},
journal = {Computer Music Journal},
number = {3},
pages = {9--14},
publisher = {JSTOR},
title = {Time-Domain Formant-Wave-Function Synthesis},
volume = {8},
year = {1984}
}
  2. Ron W. Berry
    Experiments in Computer Controlled Acoustic Modelling (A Step Backwards??)
    In Proceedings of the International Computer Music Conference, (ICMC), 1988. 
    @inproceedings{Berry88_AcousticModeling_ICMC,
author       = {Ron W. Berry},
title        = {Experiments in Computer Controlled Acoustic Modelling {(A} Step Backwards??)},
booktitle    = {Proceedings of the International Computer Music Conference, ({ICMC})},
address      = {Cologne, Germany},
month        = {September},
publisher    = {Michigan Publishing},
year         = {1988},
}
  3. Douglas H. Keefe
    Physical Modeling of Wind Instruments
    Computer Music Journal, 16(4): 57–73, 1992. 
    @article{Keefe92_PhysicalModeling_CMJ,
author = {Douglas H. Keefe},
journal = {Computer Music Journal},
number = {4},
pages = {57--73},
publisher = {JSTOR},
title = {Physical Modeling of Wind Instruments},
volume = {16},
year = {1992}
}
  4. Julius O. Smith
    Physical modeling using digital waveguides
    Computer Music Journal, 16(4): 74–91, 1992. 
    @article{Smith91_PhysicalModeling_CMJ,
title={Physical modeling using digital waveguides},
author={Julius O. Smith},
journal={Computer Music Journal},
volume={16},
number={4},
pages={74--91},
year={1992},
publisher={JSTOR}
}
  5. Andrew Horner and James Beauchamp
    Synthesis of trumpet tones using a wavetable and a dynamic filter
    Journal of the Audio Engineering Society, 43(10): 799–812, 1995. 
    @article{HornerB95_TrumpetSynthesis_JAES,
title={Synthesis of trumpet tones using a wavetable and a dynamic filter},
author={Andrew Horner and James Beauchamp},
journal={Journal of the Audio Engineering Society},
volume={43},
number={10},
pages={799--812},
year={1995},
publisher={Audio Engineering Society}
}
  6. Istvan Derenyi and Roger B. Dannenberg
    Synthesizing Trumpet Performances
    In Proceedings of the International Computer Music Conference (ICMC), 1998. 
    @inproceedings{DerenyiD98_TrumpetSynthesis_ICMC,
author       = {Istvan Derenyi and Roger B. Dannenberg},
title        = {Synthesizing Trumpet Performances},
booktitle    = {Proceedings of the International Computer Music Conference ({ICMC})},
address      = {Ann Arbor, Michigan, USA},
month        = {October},
publisher    = {Michigan Publishing},
year         = {1998},
}
  7. Curtis Roads
    Introduction to pulsar synthesis
    The Journal of the Acoustical Society of America, 109(5 _Supplement): 2401–2401, 2001. 
    @article{Roads01_PulsarSynthesis_JASA,
title={Introduction to pulsar synthesis},
author={Curtis Roads},
journal={The Journal of the Acoustical Society of America},
volume={109},
number={5\_Supplement},
pages={2401--2401},
year={2001},
publisher={Acoustical Society of America}
}
  8. Wim D’haes and Xavier Rodet
    A new estimation technique for determining the control parameters of a physical model of a trumpet
    In Proceedings of the International Conference on Digital Audio Effects (DAFx), 2003. 
    @inproceedings{DhaesR03_TrumpetPhysModControl_DAFx,
title={A new estimation technique for determining the control parameters of a physical model of a trumpet},
author={Wim D’haes and Xavier Rodet},
booktitle={Proceedings of the International Conference on Digital Audio Effects {(DAFx)}},
address = {London, UK},
year={2003}
}
  9. Michael Oehler
    Die digitale Impulsformung als Werkzeug für die Analyse und Synthese von Blasinstrumentenklängen
    PhD Thesis, Hochschule für Musik und Theater Hannover, 2007. 
    @phdthesis{Oehler08_ImpulseFormation_Dissertation,
author       = {Michael Oehler},
title        = {Die digitale {I}mpulsformung als {W}erkzeug für die {A}nalyse und {S}ynthese von {B}lasinstrumentenkl\"{a}ngen},
school       = {Hochschule f\"{u}r Musik und Theater Hannover},
year         = {2007},
type         = {Doctoral Dissertation},
}
  10. Esteban Maestre, Rafael Ram ' irez, Stefan Kersten, and Xavier Serra
    Expressive Concatenative Synthesis by Reusing Samples from Real Performance Recordings
    Computer Music Journal, 33(4): 23–42, 2009. 
    @article{MaestreRKS09_ConcatSynth_CMJ,
author       = {Esteban Maestre and Rafael Ram{\'{\i}}rez and Stefan Kersten and Xavier Serra},
title        = {Expressive Concatenative Synthesis by Reusing Samples from Real Performance Recordings},
journal      = {Computer Music Journal},
volume       = {33},
number       = {4},
pages        = {23--42},
year         = {2009},
}
  11. Martín Rocamora, Ernesto López, and Luis Jure
    Wind instruments synthesis toolbox for generation of music audio signals with labeled partials
    In Proceedings of the Brazilian Symposium on Computer Music (BSCM): 1–12, 2009. 
    @InProceedings{RocamoraLJ09_WIST_BSCM,
author       = {Mart\'{i}n Rocamora and Ernesto L\'{o}pez and Luis Jure},
title        = {Wind instruments synthesis toolbox for generation of music audio signals with labeled partials},
booktitle    = {Proceedings of the Brazilian Symposium on Computer Music ({BSCM})},
address      = {Recife, Pernambuco, Brasil},
pages        = {1--12},
month        = {September},
year         = {2009},
}
  12. Zhiyao Duan, Bryan Pardo, and Changshui Zhang
    Multiple Fundamental Frequency Estimation by Modeling Spectral Peaks and Non-Peak Regions
    IEEE/ACM Transactions on Audio, Speech, and Language Processing, 18(8): 2121–2133, 2010. 
    @article{DuanPZ10_Bach10_TASLP,
author       = {Zhiyao Duan and Bryan Pardo and Changshui Zhang},
title        = {Multiple Fundamental Frequency Estimation by Modeling Spectral Peaks and Non-Peak Regions},
journal      = {{IEEE}/{ACM} Transactions on Audio, Speech, and Language Processing},
volume       = {18},
number       = {8},
pages        = {2121--2133},
year         = {2010},
}
  13. Ning Hu
    Automatic Construction of Synthetic Musical Instruments and Performers
    PhD Thesis, Carnegie Mellon University, 2018. 
    @phdthesis{Hu18_InstrumentSynthesis_Dissertation,
author = {Ning Hu},
title = {Automatic Construction of Synthetic Musical Instruments and Performers},
year = {2018},
school = {Carnegie Mellon University},
type         = {Doctoral Dissertation},
}
  14. Esther Fee Feichtner and Bernd Edler
    Description of the Single Note Database SNDB
    In Proceedings of the Audio Engineering Society Convention, 2018. 
    @inproceedings{FeichtnerE18_SNDB_AES,
author={Esther Fee Feichtner and Bernd Edler},
title={Description of the Single Note Database {SNDB}},
booktitle={Proceedings of the Audio Engineering Society Convention},
year={2018},
address = {New York City, USA},
month = {October},
organization={Audio Engineering Society}
}
  15. Bochen Li, Xinzhao Liu, Karthik Dinesh, Zhiyao Duan, and Gaurav Sharma
    Creating a Multitrack Classical Music Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applications
    IEEE Transactions on Multimedia, 21(2): 522–535, 2019. 
    @article{LiEtAl19_URMP_TMM,
author={Bochen Li and Xinzhao Liu and Karthik Dinesh and Zhiyao Duan and Gaurav Sharma},
journal={{IEEE} Transactions on Multimedia},
title={Creating a Multitrack Classical Music Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applications},
year={2019},
volume={21},
number={2},
pages={522--535},
}
  16. Jesse H. Engel, Lamtharn (Hanoi) Hantrakul, Chenjie Gu, and Adam Roberts
    DDSP: Differentiable Digital Signal Processing
    In Proceedings of the International Conference on Learning Representations (ICLR), 2020. 
    @inproceedings{EngelHGR20_DDSP_ICLR,
title={{DDSP}: Differentiable Digital Signal Processing},
author={Jesse H. Engel and Lamtharn (Hanoi) Hantrakul and Chenjie Gu and Adam Roberts},
booktitle={Proceedings of the International Conference on Learning Representations ({ICLR})},
year={2020},
}
  17. Murray Campbell, Joël Gilbert, and Arnold Myers
    The science of brass instruments
    Springer, 2021. 
    @book{CampbellGM21_ScienceBrassInstr_book,
title={The science of brass instruments},
author={Murray Campbell and Jo{\"e}l Gilbert and Arnold Myers},
volume={436},
year={2021},
publisher={Springer}
}
  18. Erica Cooper, Xin Wang, and Junichi Yamagishi
    Text-to-Speech Synthesis Techniques for MIDI-to-Audio Synthesis
    In Proceedings of the ISCA Speech Synthesis Workshop (SSW): 130–135, 2021. 
    @inproceedings{CooperWY21_PianoSynthesis_SSW,
author       = {Erica Cooper and Xin Wang and Junichi Yamagishi},
title        = {Text-to-Speech Synthesis Techniques for MIDI-to-Audio Synthesis},
booktitle    = {Proceedings of the {ISCA} Speech Synthesis Workshop ({SSW})},
address      = {Budapest, Hungary},
month        = {August},
pages        = {130--135},
year         = {2021},
}
  19. Yusong Wu, Ethan Manilow, Yi Deng, Rigel Swavely, Kyle Kastner, Tim Cooijmans, Aaron C. Courville, Cheng-Zhi Anna Huang, and Jesse H. Engel
    MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modeling
    In Proceedings of the International Conference on Learning Representations (ICLR), 2022. 
    @inproceedings{WuEtAl22_MidiDDSP_ICLR,
author       = {Yusong Wu and Ethan Manilow and Yi Deng and Rigel Swavely and Kyle Kastner and Tim Cooijmans and Aaron C. Courville and Cheng{-}Zhi Anna Huang and Jesse H. Engel},
title        = {{MIDI-DDSP:} Detailed Control of Musical Performance via Hierarchical
Modeling},
booktitle    = {Proceedings of the International Conference on Learning Representations ({ICLR})},
month        = {April},
year         = {2022},
}
  20. Ben Maman, Johannes Zeitler, Meinard Müller, and Amit H. Bermano
    Multi-Aspect Conditioning for Diffusion-Based Music Synthesis: Enhancing Realism and Acoustic Control
    IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024. 
    @article{MamanZMB24_MusicSynthesis_TASLP,
title={Multi-Aspect Conditioning for Diffusion-Based Music Synthesis: Enhancing Realism and Acoustic Control},
author={Ben Maman and Johannes Zeitler and Meinard M{\"u}ller and Amit H. Bermano},
journal={{IEEE}/{ACM} Transactions on Audio, Speech, and Language Processing},
year={2024},
publisher={IEEE}
}
  21. Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre D 'efossez
    Simple and Controllable Music Generation
    In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), 2023. 
    @inproceedings{CopetEtAl24_MusicGen_NeurIPS,
author       = {Jade Copet and Felix Kreuk and Itai Gat and Tal Remez and David Kant and Gabriel Synnaeve and Yossi Adi and Alexandre D{\'{e}}fossez},
title        = {Simple and Controllable Music Generation},
booktitle    = {Proceedings of the Annual Conference on Neural Information Processing Systems ({NeurIPS})},
address      = {New Orleans, LA, USA},
month        = {December},
year         = {2023},
}
  22. Stefan Balke, Axel Berndt, and Meinard Müller
    ChoraleBricks: A Modular Multitrack Dataset for Wind Music Research
    Transaction of the International Society for Music Information Retrieval (TISMIR), 8(1): 39–54, 2025. PDF Details Demo DOI 
    @article{BalkeBM25_ChoraleBricks_TISMIR,
author = {Stefan Balke and Axel Berndt and Meinard M{\"u}ller},
title = {{ChoraleBricks}: {A} Modular Multitrack Dataset for Wind Music Research},
journal = {Transaction of the International Society for Music Information Retrieval ({TISMIR})},
volume = {8},
number = {1},
pages = {39--54},
year = {2025},
doi = {10.5334/tismir.252},
url-pdf   = {2025_BalkeBM_ChoraleBricks_TISMIR_ePrint.pdf},
url-demo = {https://www.audiolabs-erlangen.de/resources/MIR/2025-ChoraleBricks},
url-details={https://transactions.ismir.net/articles/10.5334/tismir.252}
}