26 Jul 2024
When you keep moving around and you don’t have a studio (or most of it is locked in storage for over a year), it can be quite difficult to record an album, let alone keep momentum high and deliver it in a reasonable timeframe.
This is the backstory for FRUTAL, coming out on the 9th of August.
In early July 2022, Pyrathee — who’s been a friend of mine for over 20 years — had me over to his house. As I was getting ready to leave, he told me, “Dude… we don’t play music anymore. I’ve got all this kit and yet we never play. We used to play all the time.”
This really got me thinking. We had a band for about 5 years until everyone moved to different countries, and eight-or-so years later, here I was, just 20 minutes away by bike, with no excuses.
I was planning on moving to Ireland a few months later and was going to start selling a lot of my equipment in September. So, I thought — why not get together, write a few songs, make a small EP, and wrap it up before I move away?
We agreed to meet a week or so later. We moved his drum set, tube amp, and guitar from the house to the garage outside, and we jammed for a bit.
Not content with just jamming, we started trying to find some structure, some riffs that worked, and soon we were pitching random ideas: What about metal with odd time signatures? What about Rammstein-inspired heavy riffs? What about double kick drums with weird patterns? What about fruits? Yeah, what about fruits?
He dived into a story about how he got stopped at the airport in the Dominican Republic trying to take a few dragon fruits through security. They had to scarf them down (with plastic knives and forks) to try to make it past security in time for their flight.
I was like, “Man! That’s amazing, let’s write something about that.” So, we started coming up with ideas, which I scribbled down on a notepad. We also recorded a few drafts on my phone just to remember them afterward.
After this initial session, I went back home and finished writing the other parts of the songs. I wrote down some lyrics, and we had a chat about making it a four-week project. Two sessions to practice, two sessions to record. Four songs in total: Cherries, Dragon Fruits, Strawberries, and Guanabanas.
We recorded them a few weeks later at a house I was renting. It was three stories high, and on the top floor, I could usually play without bothering the neighbors too much, as they seemed to use their third floor for storage. These are Belgian houses, so very thin and tall, with somewhat scary steps to climb down.
The session went well, so I fiddled around a bit with quantization, trying to get the drum sound right, trying to get the guitar sound right, and so on. I recorded the guitars on a small Fender amp, which didn’t really have the tone I was looking for. So, I left the songs incomplete for months, partly working on them every now and then but never really satisfied.
After moving to Ireland towards the end of the year, I bought a new guitar. Shortly after, I found a DSP VST for guitars that had a tone I fell in love with: Ampbox by Mercuriall, using the U530 Preamp. When I compared it with what I recorded using my Fender amp through a (probably badly placed) Shure SM57, the tone was miles away. This sounded professional, wide, grungy, and powerful, exactly what we needed.
So, I re-tracked all the guitars, added a few solos, and started double-tracking the drums.
But once you get started, it is so easy to get sidetracked. I thought maybe I could add a random spacer between each song, maybe hire voice-over artists on Fiverr to narrate the description of fruits and place a weird glitchy track in the background, or mess around with the voices, making them stutter or glitch out.
And soon after, I realized maybe I could add a few calmer tracks, because guttural screaming isn’t for everyone. Maybe the sound of the songs could be reflected in the type of fruit, or maybe sometimes the inverse to switch things up and make it unpredictable.
It’s funny it took so long. You can write a song in a few hours, but sometimes life gets in the way. When you pick the project back up, the scope increases, and you realize you want to make something much larger than what you originally intended.
Over time, I started adding more and more fruit songs. At some point, I became dead-set on the idea of making a song about every fruit. But I’m glad I backed away from that… it turns out there are a lot of fruits! But hopefully, the series of albums will be interesting, even if the idea behind them is a bit silly.
A year later, I had over 10 tracks down, including the Fiverr voiceover ones, and then I moved back to Belgium… To a flat, which made it a bit harder to record. Now I really had to pay attention to avoid bothering the neighbors. Slowly but steadily, over the next year, I wrote another 12+ tracks, and I reached the point where I noticed it was going to take me a long time to write a song about every fruit.
I’d take all the fruits I had, plug them into ChatGPT, ask which ones I was missing, and it would always come back with a massive new list. So during a run on a random weekday, I had this sudden stroke of inspiration: Let’s split the album and cover some interesting fruits, but not all of them… maybe 2 or 3 albums of 15 tracks each.
Finally a plan.
So, I chose the 15 songs for the first album and tried to mirror them with the second so that the same style of song lands in the same place. Whether it is a description, a calmer song, or a heavier song, the first two volumes would be mirrored in a way.
The mixing and mastering process during the following month was hell. A lot of listening to reference songs, then listening to the album and noticing I was missing lower end, then going out for a walk and listening on earphones instead of monitors or headphones and noticing it sounded weak. Then adding a lot of compression and noticing it sounded oversaturated. Then diving into LUFS and hearing about headroom and trying to lower things a bit.
Finally, I found a sweet spot that I liked. It is still very, very loud. I used as much of the frequency spectrum as I could. I love things sounding heavy, with boomy lower ends, but also very fresh and bright on the top end. I think I’d get a slap from any mix/mastering engineer, but I went my own way for this one. Maybe someday I’ll learn.
Why is the album called FRUTAL? The idea behind the name was because we wanted to make something heavy, brutal, like very intense. So brutal, but about fruits… How about FRUTAL? And the name just stuck.
I’m really looking forward to putting this out and listening to your feedback. So stay tuned for more updates, you can visit lemiffe.com/subscribe where you can subscribe to the newsletter if you’d like to receive
an email the moment it drops. Otherwise I’ll likely be posting videos and promo material on YouTube, X and Threads along the way.
29 Mar 2024
This is a list of tools, VSTs, services, websites and research on AI music generation. Last updated: 30/Mar/2024.
I’ve divided the list into four areas: Research, Websites & Services, AI vocalists, VSTs & Plugins for DAWs. There is quite
a varied use of AI models across the board, the first section primarily focuses on prompt to music, aka “ChatGPT for music”,
but there are also tools and VSTs that use RNNs, deep neural nets, diffusion models, etc.
I’ve also provided my opinion/thoughts on some of these tools, I didn’t have a chance to try out every website/service, nor
did I purchase every VST as some are a bit expensive, but I did go through example videos, and sometimes comparison videos
with other tools/services, to get a high-level overview on each tool/service/plugin. Hopefully it is useful to you as a reference list.
State of the art research in text to music prompts and audio effect generators
- Meta
- Google
- Stability AI
- Stable Audio: Diffusion-based generative models for latent audio diffusion
- Had a few demos here that while they were impressive, I would put them on par with Google and Meta’s efforts so far. Check the official website here.
- Another demo that seems to be more recent (and better!)
- HN comment about Ed Newton-Rex who built stable audio quit after release re training data - he went on to found https://www.fairlytrained.org/
- Generative Music in Games
- Research paper on the topic
- Could be the most interesting area of research: on-the-fly generation of music that adapts based on the character, story arc, etc. It could potentially work incredibly well for those games that have a bunch of different endings where every choice matters in the decision tree. This way the musicians and audio direction can focus on the theme and primary music while leaving atmosphere and subtle background music to embedded AI music generators.
Websites and services to generate music and sound effects with AI
- Audiogen.co: Doesn’t mention what model they are using but it seems very likely they’ll be using one of the research ones covered above
- Lalal.ai: Extracts or isolate stems in music - so you can, for example, remove the drums and overdub with your own, or use the vocal takes in a remix
- AI Mastering Services: Quite a few options available here, more popping up all the time. AI mastering has already been a thing for a while, but certain tools/VSTs and websites are making it much easier to use and afford. This article from Ars Technica includes a very good breakdown and comparison amongst popular services such as LANDr, Ozone, etc.
- SoundDraw: Interesting one as it doesn’t work with the usual
prompt to music principle, it uses segments built of samples submitted by the devs, and you can turn segments on and off and rearrange them, like in a rudimentary DAW, as well as generate new segments as well. A big pro is that you can try it out without making an account; but I’m unsure about the licensing aspects (there is a FAQ that dives into this however)
- Beatoven.ai: At a glance seems similar to the previous one
- Fast registration (requires email confirmation)
- Generation took less than a minute, with 4 alternate tracks
- Quality was poor, and the default length was very small (plans start at $6)
- AIVA: AI music generation assistant
- Login / account creation was broken when I tried it out
- Loudly: Also similar to previous websites
- Quick registration process
- Instead of just being prompt based, it offers parameters, where you select duration, instruments, genre, etc.
- Generation is FAST
- Provides 3 options per generation, liked some of the options but I’d like to see how it generates more variance with longer tracks
- Limited to 31 second output for free, you have to upgrade to get longer tracks
- Suno: Also similar to the previous websites.
- BEST choice out there right now in my opinion
- Very simple interface, got 50 credits for free upon logging in the first time.
- V3 of their engine allows you to make a track up to 2 minutes long, you can also toggle to generate instrumental-only songs.
- It took a couple of minutes to finish, and it generated 2 options for 5 credits per track, which means you can try 5 prompts for free
- It allows you to continue from the end of a previously generated track, which would allow you to “stich the results” together to create longer tracks
- The results were quite interesting, I asked for
A droning techno with a slow buildup towards an epic theme and then with a decay back into techno… The first option delivered just this, albeit a bit lackluster, but the second option went from absolute rubbish into amazing bright vivid super uplifting music which quite surprised me
- Login only works with Discord, Google or Microsoft? (why no email option?)
- Boomy: Untested. Con: Need to make an account
- Musicfy AI: Similar to previous but with AI vocals as well?
- Soundful
- Pro: License covers everything from videos, websites, corporate use, games, etc.
- Pricing is per year, and is less than half the price that Epidemic Sound costs per year. Or you can pay a bit more to obtain full copyright for all creations and not just a license for usage.
- First one billed as “royalty free music for creators” — I think this is what a lot of us thought when AI music generation started becoming a viable thing… that its primary use-case would be to generate background music for videos and live streaming on Twitch and YouTube, not to create new content to flood streaming services like Spotify with auto-generated songs… yet a few of the last 6 or 7 services seem aimed towards people without music or singing knowledge to create tracks and assist with publishing them on streaming services
There’s this section in Colin and Samir’s Jacob Collier Interview where they dive into the topic of AI and music and I really loved the positive attitude, specially where he talks about crafting new sounds and using it as a tool to unlock or further creativity.
As a music world builder (as Jacob Collier talks about himself) it can be instrumental to find sounds that can change the way we dream up and work with compositions, venturing into unexplored audio territory, kind of what analog and modular synths have done to open up a new world of possibilities over the past 50 or 60 years, or granular synthesis in the past 20 to 30 years.
It seems anyone with a say on AI and music generation take one of three stances:
- Hate it, afraid of the implications for copyright, don’t like the potential for saturation with “fake work”, the dilution of quality music in a sea of artificial sounds.
- Afraid, unsure about what it means, how much impact will it cause, will it be used as a tool or will people use it to create full works, will it take away jobs?
- Positive but slightly weary, knowing there is potential for wrong-doing but the toolset it gives us outweighs the cons, such as crafting new soundscapes, imagining sounds that we could have never thought possible otherwise, helping create drafts of soundscapes which we can then build upon, or helping unblock continuation of songs for artists struggling with a creative block
AI singers and artist imitation
AI virtual music instruments and VSTs
- Synplant 2: Lets you craft sounds and branches - this is an EPIC plugin!
- iZotope Elements: RX, Neutron, Ozone and Nectar plugins
- Guitar Rig 7 Pro: Guitar and Bass Amp Simulators. Uses AI (with Convolution?) to craft epic guitar tones
- For example you feed it the tone of an amp or a sound you are looking for and it adjusts its parameters to allow you to reach that sound without having to own expensive amps / preamps. It is expensive though.
- Great example video
- Focusrite’s FAST bundle: Makes mixing much easier
- Magenta Studio (for Ableton)
- Covered 1.0 but now 2.0 is out and it has been redesigned to be much simpler to use. Only works in session view but you can drag and drop the clips into arrangement view. Very easy plugin to make drum loops sound a bit more natural, or to extend parts of a song.
- Powered by RNNs
- Synthesizer V - Voice synthesizer
- From lyrics and midi to vocals, without having to open your mouth! I’m still conflicted about this area of plugins; it does open doors and reduces barrier to entry, but on the other hand live music is amazing and vocals are often the centrepiece and I’m not sure I want to live in a world where even more emphasis is placed on the DJ, specially when no vocalist is required anymore, everything can be virtual. Using it in combination with real lyrics, or in a few tracks where it can be used stylistically, hell yes! But as a full replacement? I’m not sure. I’m still on the fence.
- The price? 89 dollars, sounds amazing - scarily amazing - supports 3 languages as of February 2024
- Research (don’t include in credits, mentioned below)
- Check out the website for videos/examples
- Deep neural network-based
- Vocaloid 6 (by Yamaha) - Voice synthesizer
- Another vocal synth, a bit less impressive than than Synthesizer V in my opinion, with less voices and quite a bit more expensive out-of-the-box (although you can pay per voice pack with Synthesizer V). Also supports 3 languages. IMO sounds a bit more robotic, yet for some styles this might be the right fit!
- Research (same link as Synthesizer V)
- Check out this example video from their website
Continuation of my previous rant:
Even though the AI singers imitating known artists generated a lot of drama, in my opinion it pales in comparison to the voice synthesis… I mean, I think a lot of people were frustrated with autotune, it just means you don’t even need to know how to sing, YouTubers such as Roomie have generated many videos of autotune vs real vocals. Some artists would use it covertly to fix mistakes, but then it spawned a use-case where it was used as a texture, the whole point was cranking it to 10 making it a stylistic choice…
And I just think in a way AI voice synthesis for singing means that some people who make a living singing on tracks will maybe make less if producers don’t have to find the right talent anymore, not even just someone with a voice to then auto-tune, but now they can skip the vocalist completely. How will that work in a live setting?
These are just some of my raw thoughts as all of this is still quite new and I don’t know how it will pay out in the long run, but the pace of innovation in the AI music generation and voice synthesis in general seems to be accelerating, and I’m very curious what’s going to come out next, and how these tools will be used once the dust has settled.
Other Tools and Services:
There are a bunch of lists of top AI tools including quite a few I didn’t cover like Orb Producer Suite 3, Playbeat sequencer, Atlas 2, Spark Chords and others… This website covers some of them if you are interested in more overviews and links to the plugins
Thanks for reading!
