Behind the Beat: How AI is Changing Music w/Maya Ackerman PhD

AI can expand the musician’s option space

Maya Ackerman’s path into AI music began with a technical foundation in machine learning and a personal relationship with music. She studied foundational machine learning, took voice lessons from an opera singer, and began performing and writing songs. Those experiences helped her see the gap between having an idea and having every musical skill needed to execute it.

In 2014, she discovered a small research field focused on computational creativity. Researchers were making machines that produced music, art, poetry, dance choreography, and recipes long before the current wave of consumer tools. The technology was not replacing the artist’s purpose. It was opening a space of possibilities for people who wanted to explore an idea.

Match the tool to the creative task

The practical question is what kind of help a musician needs. A lyric-focused tool can help someone develop words and concepts. A melody tool can let a singer or producer explore different top-line melodies over an existing beat. The right starting point depends on whether the bottleneck is lyrics, melody, arrangement, or production.

That is a useful way for marketers to evaluate creative AI too. Do not ask whether the tool can make a song, image, or video in the abstract. Ask which part of the process it improves and what expertise still has to come from the human. A musician with a strong idea may use AI to test variations without surrendering the taste that decides which version is worth keeping.

Preserve authorship and judgment

AI can make experimentation cheaper, but it also raises questions about originality, rights, and the relationship between a creator and a tool. Those questions deserve more care than a novelty demo. The practical path is to start with a contained project, understand the tool’s terms, and keep track of what the human contributed.

The opportunity is collaboration. AI can fill a gap, suggest a direction, or help a creator move past a technical bottleneck. It cannot decide what the song means to the person making it. The best result combines expanded possibilities with a human reason for choosing one.

welcome back to AI microskills where we're trying to empower marketers to master AI in revolutionize their practices today I'm excited because we're venturing a little bit off the beaten path of marketing to talk about music with Dr Maya Ackerman who's the CEO and co-founder of wave AI around how ai's transforming the music space so Maya thank you so much for joining me on the show it's a real pleasure thank you for having me so I'd love to dive into like your backstory like how this even became a thing for you I didn't even realize I was in music until a few weeks ago in a mutual friend of ours introduced me to it and I was like oh my gosh like of course of course it is because I mean for lyric writing but then even

music and being able to just improve The Melody of the beat or whatever is going on I'm like it just makes a lot of sense a it just seems like AI is creeping into everything you know and it's helping everything but how did it get started for you when inspired you to mix these two things together yeah it was really quite an unusual story I come from an academic background I got my PhD at University of Waterloo focusing on foundational machine learning and then continue down the academic path in my post office called Tech then you see San Diego and in 2014 discovered a little known research field known as competition all creativity group of a couple of 100 shares mostly out of Europe

who are making machines that make made music and art and poetry and dance choreography and recipes since about the 1980s even dating as far as in 1950s it was just a really really small number of people who are focusing on it and there was a really wasn't any kind of Spotlight on it that's before IBM before Microsoft before Google before any of these big companies even started dabbling in it what helped me recognize the opportunity specifically music was that during my PhD I took voice lessons from an opera singer and started performing semi professionally within a year and that's what led me to try to write my own songs and you know the frustrations that come with not being able to

play the songs that I feel in my heart and it was just you know sometimes your life experiences prepare you to see something that's there but that other people just haven't had those experiences yet to be able to recognize the opportunity and so knowing a little bit about computational creativity and the possibilities there and knowing what I knew about songwriting and the struggles I was having it was right there it's right there so funny hearing that because I came I came into marketing through a design background and obviously like most people get into design because they were a visual artist first you know we all start with like pen and paper and drawing but what you just said I'm like oh that map so well to like where I mean with a visual artist you like you have this Grand Vision of like what something

yet your ability and skill to be able to bring it to life is sometimes it's just not there it takes more practice more reps and I have been finding with AI on a visual side that it's giving me tools to be able to unlock my creativity and maybe even from lack lack of skill of being able to get it there or just time fiddling with something I'm being able to find and make things faster and actually get to where I'd kind of dream to something might be sooner because I can go to Mid journey and create something quickly find and select something so it's interesting he says it sounds like it's kind of taking place in a music place as well and its own way it's a little what we're doing is quite different from the work of midi I love my journey I think

so fun I'm not official artist you know I dabbled in it as a kid that does not count so I'm okay to some degree to have something that driving the process for me in art even though there too I get frustrated sometimes that I can't really convey all of my vision it understands what it understands and it fills it in how to fill it in it's marvelous it's magical but I'm not really driving everything that my company does in music wave AI the user is not only getting help but they're never getting hindered that's just as important you think about a human collaborator if I'm helping you with something it's important that at the same time I don't also hinder you just as much right so that's

very different perspective on what the role of AI is well let's dive into it how do the lyrics to do your Melody Studio work and what like what makes them unique like how is it guiding the process so our products are made entirely for musicians and people who want to be musicians they're not for people who want music there for people who want to make music and that attitude is kind of in the in the DNA of the company would build our machine learning models Our Generation models completely from scratch completely from scratch so that respect for artists that need for the model to lean into the artist Style

need to be infinitely creative to the same suggestions are never given to the two different users all these ideas that make that make Rye really helpful to the artist is built completely from scratch completely from scratch we don't rely on child support or any of the other large language models we don't rely on the large music models with build things from scratch so that it really helps you express your vision rather than kind of replacing you or like doing so much work for you that there's no opportunity for you to express yourself or no opportunity for you to grow as an artist so it's really the difference between the AI being the star and the AI being background

actor and for us it's really about that support role so that our users should really be the only stars in the show so is it something where you kind of have to get it started where do you bring it a rough draft first and then it helps you fill in the gaps or is it coming in and like maybe helping you master something it's well it's specifically for lyrics and Melodies and chords so it helps you create those lead sheets that Foundation element of music but it can work in many different ways and that's part of the non-invasive of the process that you can you can get started in a lot of different ways so for example somebody brand new can come in and pick a few topics and just start clicking on the suggestions and quickly build something

on the other hand we've had an artist who had a song laying in her drawers for 3 years she pulled out the song typed it into lyrics studio and just wanted help with this second verse and it helped her with that second verse and then she was done right so help how she needs help I use it a little differently I sometimes have one line idea and I generate a lot of thoughts from from it and it helps me lay out my song it helps me kind of come up with the emphasis for each verse I don't know if anyone else uses it like this that's how I that's one of

and then there's a whole bunch of different stuff for Melody Studio that's just as flexible we have one person who like real time generates the lyrics and sing them in real time as they get generated and it's beautiful it's just like really fits his workflow and there's so many more examples with how Melody Studio gets used as well what are some of the processes you've trained the AI on in order for it to be better at coming up with lyrics than something like a Chad GPT well you know child should be tea is I'm in such a of these large models I really really am I don't want anybody to think otherwise I think things are marvelous I think they're brilliant obviously have problems as well that's not what I think you know the research on the space hasn't been

so it really is a brilliant of openai was that they decided to make a chatbot so it's either generative models which until judge Vita called it a chatbot until opening I called decided to call it a chatbot people thought about it as a generative model which is really conceptually quite different when we think of a chatbot we think of something that should have a clue about what it's talking about for example you could ask it some simple questions you know some really basic math or maybe questions about when different countries were founded and you expect answers that are correct and maybe this seems trivial but it's really not because when you take a generative

inherently imaginative and you wanted to start giving you the right answer to some stuff you have to make a lot of decisions in the construction of the model to end up in that space we are not constrained by that lyrics Studio does not need to know anything about you know how to cook what ingredients go well together what the weather is like in different parts of the world it just needs to know how to write lyrics and we give it permission to be as imaginative as possible because it's not constrained again there's no notion of right or wrong and that's one of the core kind of foundational principles that make our models really orders of magnitude more creative than anyone ever could be that's interesting so because hallucinations

is there a benefit in your case because the way it can be creative if it comes up with a weird way of saying something you're kind of like actually I kind of works because lyrics kind of always wear that weighs are in the DNA of generative AI it's actually kind of funny the word choice is funny he listens are associated with something negative right like when I clearly imaginative oh no that's positive imaginative creative right so we have chosen and negative word to talk about the capabilities of these systems because we are thinking about because we want the machine to be convergent we want the machine to do more things and be right

then suddenly the fact that they're hallucinating is obviously an advantage that's an interesting way to frame it what word do you use to describe it we've been I've been in the field for 10 years before the rise of generative Ai and these are creative machines that's a word I use their creative and maybe he's not creative enough for some people but my models definitely are created on the Melanie's Melody Studio side how is it helping musicians craft better music so the core there is to help create a lead sheet

I'm going to talk first about our most recent use case because I'm so excited about it so we just added the capability of our musician users uploading a backing track so any beats that they already have they get to upload it to the system and then the system can help them right lyrics as well or they can bring their own and then the system comes up with Top Line Melodies with the vocal Melodies that fit the lyrics and fit the backing track wow that's such a game changer I'm so excited about this so that's already out it's already rolled out people are already using it we're just so so thrilled to be able to offer that the default use case used to be just the same thing but without the backing tracks so that's when musician wants to build the song from

make instrumental music because for me it's really not about the new replacing the old I'm an opera singer in a piano player right like so that's census like when people use our AI tools to go back to instrumental music which is what I do very often I think that's really cool you know in addition to our very many users who essentially use our tools to create modern music that includes digital on your work stations in some part of the process can you discuss any challenges or limitations musicians face when using AI in general or are tools specifically

we can start in general and then and zoom into your tools there's a lot of it's a lot of challenges the more I'm on the musician side partially because I am a musician and partially because they're right they have a lot of concerns a lot of concerns how can they not have a lot of concerns like look what streaming did to them it used to be if you were you know you could be a small scale musician and make a living it's almost impossible these days to make a living through your music because you can't sell CDs right and streaming just pay so little to what city is used to pay it's really the Gap is wild and this all happened sort of just without

operation agreement or anything of musician that just sort of happened to them as opposed to even you know not even with them and I don't want the same thing to happen with AI they don't want the same thing to happen with AI I don't want the same thing to happen with AI you know and sometimes when a company say we're helping artists you go and you try their tool and it's just not true it's just not true you pick up your parameters and it writes the whole song for you in what universe is that helping that's replacing right and musicians are smart people they can tell the difference between helping and replacing so and sometimes you know I should give credit where credit is due you know you have very very creative people who take these tools that are not really designed to work with them and they still find ways to use these kind of

play some tools inside of their creative process because they're creative people sometimes I find a way anyways but they're also feeling that you know the threat as a result and so I actually feel a lot of personal responsibility here in in being on the right side of History so in a universe where automated music generation gets progressively better so music that really doesn't need a person very much or doesn't need a musician you know World words this kind of music exists we definitely need the best possible tools to help artists so that's that's where my it's fascinating so I don't know that I definitely thought for a while on the visual side that these tools are going to replace them the more I play with it

as good as they are I'm like you know what this is only going to be enhancing what they do like that I guess you could go in the far future and just be like write me a hundred songs and it makes 100 songs and you pick pick a few that works better and then submit them to the streaming services or whatever is available then but the same time musicians are going to be able to do it better than everybody else because they just have the ear for what is good or what should be or what could be I mean it's a problem with journey for most people is they don't have the vocabulary even to describe what they want they're very true it's an interesting but it's not entirely clear yet where it's going to land I think like yourself I tend to be optimistic I tend to believe that humans creativity

remain dominant for a variety of reasons but at the same time I like to just if I'm being really sincere I think there is some threat and I think artists and musicians don't want any threat to their livelihood so they're not okay with like a small percentage of it being carved out because there's already not enough there's already so many talented people that can't make a living on their art and craft and musicianship so is some damage going to happen that's possible and that's I need to be need to be responsible or best and just take their concerns seriously that's my take can you share a success story of somebody using your tool in order to make

make a masterpiece that maybe they would have had a hard time doing or taking a much longer time or have been impossible before there's so much of that oh my goodness that's why we do what we do right so I mean one of my favorite stories is Curtis King about a year ago now he made an album called DIY too he used to 8 in his little and it ended up being number one on the iTunes hip hop charts while so that was that was just so awesome to hear just totally made my week but then there's so much more there are just beautiful piece of music that we get sent sometimes

able to write faster and more which in the result they make more money which is excellent but then like for me it doesn't have to be like a monetary or in a word for it to be a success story the words this one like several like really incredible songs that I heard like wow this is just so good and we only hear the songs on our users to send them to us otherwise it's completely privacy so in case of you know those few songs that were shared with us

the people who made them they said that they wouldn't have been able to make them without our tools they just they were missing certain skills like lyrics writing or Melody creation it's like wow this song would not have existed without our tools and this person would not have been able to express it and sometimes it's such a personal stories and get expressed in these in the song so really it would help millions of people write music but you know even one song that we helped get out there and help the artist Cher what's in their hearts I think I think it's kind of Priceless for me absolutely and hopefully these these stories that you're sharing now convinced more people but if for those who are skeptical for those who are and many musicians are right like musicians are kind of a skeptical Bunch when it comes to

sometimes technology or business or a lot of other things what advice would you give for those who are kind of on the edge and not sure they want to try out an AI tool as part of their workflow what advice would you give to them I think the first thing that it says that I get it given the tools that are popular nowadays I can understand why people are you know some people find it hard to believe that there are tools that are genuinely helpful for artists what I can say is that within the first few minutes musicians get our tools like it's not a long process they try they're like oh oh that is actually helping me and then get really excited so I do hope your check it out but you know we are here to be part of your creative Journey if this is something that you'd like and I'm perfectly happy to respect

if musicians you know you know some people are curious in different ways and that's that's also good if some people feel strongly about not using AI I can respect that as well and I hope there will be some people who never do you know and go all analog everything because there's something beautiful about that too right where is right kind of we really need artists of all kinds and it's I really don't think there's any tool that's for everybody and I think that's okay I think you know as people who make music tools I think we need to be comfortable with that and give people the invitation you know one way that we give people the invitation is that you can try our tools for free right and I think that's in part because we want to respect that it's a very new tool so people should be able to play around with it

it's not credit card back you just go in and play with it a little bit to see if you like it before deciding if you want to add it to your repertoire and then also like most people don't tell anybody that they use our tools so it's just and there's no way to tell you'll never be able to tell it's not like church of it or it's detectable yeah yeah so for those listening and want to get started with it where's the best place to start would you start with just one or two or the other or would you kind of dive into both a lot of wrappers for example who just use lyrics Studio because that's kind of a perfect very very simple interface just focusing on lyrics but if if you think you could if you're curious how AI could be a little bit helpful to brainstorm melodies for your top

if you are a singer and you know you have your Beats that you got some more of your producer you create your own beats with Melody Studio you could see kind of like explore the space of possibilities for what type of Top Line Melodies can work on top of these beads that you have yes it depends on who you are you could test out one of the other products fantastic thank you so much for joining me on the show and helping us understand where AI can really help musicians succeed and fill in some gaps or just compliment their current skill sets my absolute pleasure thanks so much for having me