AI_PhD@Lille 2021-2024 – Modeling symphonic writing
- Fully funded PhD 2021-2024 in computer music and AI / machine learning, program AI_PhD@Lille
- In Lille, France (CRIStAL, CNRS, Université de Lille)
- Supervisors: Mathieu Giraud (CNRS, CRIStAL, Université de Lille), Florence Levé (MIS, Université de Picardie Jules-Verne, Amiens), Louis Bigo (CRIStAL, Université de Lille).
- Deadline for applications: 8 march 2021 (CV and letter, by mail)
- Links: http://www.algomus.fr/jobs/phd-sympho-en
The Algomus computer music team, from the CRIStAL lab (CNRS, Université de Lille), in collaboration with the MIS lab (UPJV, Amiens) works in Music Information Retrieval (MIR) and in Computational Music Analysis (CMA).
Orchestration is the art of writing music for several instruments. It goes much further than an instrumentation that would be a simple distribution of voices. Whether in so-called “classical” music or in other repertoires such as film music, the orchestration is nourished by the musical possibilities of each instrument, by those of their instrument group, and finally by the “sound” of the entire orchestra . Writing orchestral contrasts concerns both the instruments used and what they each play.
In computer music, there were recent works towards automatic orchestration  and the reconstruction of audio spectra, but few of them focused on the characteristics of orchestral writing. One of the MIR (and music) challenges of the analysis and generation of orchestral writing comes from the richness of the combinations of the possible instruments. Understanding and modeling orchestral scores require significant knowledge and corpora. Is it possible to envisage new tools that encourage the co-creativity between the composer and the computer, and even that may help the casual listener, when listening to, analyzing or composing symphonic music?
This thesis will propose computer methods – both from machine learning and discrete algorithmics – to systematically analyze orchestral scores. The goal is to illustrate and complement the knowledge on orchestration and its link to tonal harmony and form, and to design tools for the analysis and co-creation of orchestral music.
Concretely, the PhD student will study a bibliography on orchestration, computer music, and deep learning models for music, especially around auto-encoders and latent spaces [2,4]. She/he will address the following points:
Identify, gather, and improve corpora of tonal orchestral scores, whether classical, (post-)romantic or contemporary;
Model annotations and analyses of tonal orchestral scores, by extending computer representations of harmony, texture  and by modeling orchestration according to form;
Design, implement and evaluate learned and/or algorithmic computer models applied to orchestration analysis and co-creation of symphonic music.
The learning models will aim at explicability and interpretability for the users. This thesis will thus be done in collaboration with music theorists. Collaborations will also be sought with living composers, including film music composers and/or orchestration classes.
MSc in computer science, with skills in AI/data science, ideally with a first experience in MIR. Music practice and music theory skills highly desired (analysis, harmony and ideally orchestration). We particularly welcome applications from women and under-represented groups in music and computer science research.
Schedule, publications, and collaborations
The Algomus team is used to publish at major conferences in the field, including ISMIR. The PhD student will be guided in writing and submitting papers to these conferences, whether on his/her own work or on collaborative work inside or outside the team. The PhD student will regularly participate at conferences and other events in the field, and will be encouraged to collaborate with other scientists and artists. Each PhD student in the Algomus team also undertakes an international research stay during the course of his or her thesis.
The schedule is individualized for each student, but could be as follows:
- T0 to T0+6 months: bibliography
- T0+6 to T0+16: first models for orchestration analysis, corpus modeling and annotation, first paper submission
- During the second year: 2-months stay in a foreign partner university to consolidate the work and open up to new projects
- T0+18 to T0+26: model validation, co-creative generative models, submissions on the personal work as well on collaborative work
- T0+26 to T0+34: thesis writing, possibly new paper submission
- T0+35 and T0+36: PhD defense, new projects and collaborations
Links with the AI_PhD@Lille program
This subject of analysis and musical generation, in a perspective of human-machine co-creation, is fully integrated in the AI_PhD@Lille program.
Fundamentally, processing musical data by AI poses challenges of musical representation and architecture of networks to manage both the pitch and the time axes. Orchestral writing particularly manifests these challenges.
The explicability and steerability of AI are essential for artistic co-creativity. The thesis has thus a strong focus on analysis, where the aim is to explain the strata of orchestral writing. When it comes to generating music, the goal here will not to provide a black box, but rather to draw on the musical experience of the artist or the music lover and interact with him or her. Models based on auto-encoders with latent spaces are particularly relevant for this.
Hosting team and supervisors
The Algomus team is an ideal place for conducting innovative research in AI and computer music, both fundamental and applied. In 2020, the team was ranked 4th in the European AI Song Contest: http://www.algomus.fr/i-keep-counting. The team is in contact with artists, teachers, college classes and local and global companies (Arobas Music in Lille, TikTok/ByteDance in London). The PhD student will benefit from this environment.
The three supervisors have been collaborating for several years (more than 10 years of collaboration between FL (Amiens) and MG (Lille)) and have collectively about 40 publications on the field in recent years. They are attentive to the proper conduct of the thesis as well as to the personal development of everyone.
-  S. Adler, The Study of Orchestration, 2016 (4th ed)
-  J. P. Briot, G. Hadjeres, F. Pachet, Deep Learning Techniques for Music Generation - A Survey, 2019, https://arxiv.org/abs/1709.01620
-  L. Crestel et al., A database linking piano and orchestral MIDI scores with application to automatic projective orchestration, 2018, https://hal.archives-ouvertes.fr/hal-01578292
-  P. Dhariwal et al., Jukebox: A Generative Model for Music, 2020, https://arxiv.org/abs/2005.00341
-  M. Giraud, F. Levé et al., Towards Modeling Texture in Symbolic Data, ISMIR 2014, https://hal.archives-ouvertes.fr/hal-01057017