Lemiffe Music, Software, Stories & AI

FRUTAL Volume 1 is out now!

FRUTAL Album Cover

Click here to listen to the album!

If you’d like to read the story behind the album you can click here.

Thanks for listening and I hope you enjoy it!

Sign up to the newsletter to receive new album updates:

Email: 
Name: 

And feel free to follow @lemiffe if you enjoy the album:

 

Spotify

The Making of FRUTAL

When you keep moving around and you don’t have a studio (or most of it is locked in storage for over a year), it can be quite difficult to record an album, let alone keep momentum high and deliver it in a reasonable timeframe.

This is the backstory for FRUTAL, coming out on the 9th of August.

In early July 2022, Pyrathee — who’s been a friend of mine for over 20 years — had me over to his house. As I was getting ready to leave, he told me, “Dude… we don’t play music anymore. I’ve got all this kit and yet we never play. We used to play all the time.”

This really got me thinking. We had a band for about 5 years until everyone moved to different countries, and eight-or-so years later, here I was, just 20 minutes away by bike, with no excuses.

I was planning on moving to Ireland a few months later and was going to start selling a lot of my equipment in September. So, I thought — why not get together, write a few songs, make a small EP, and wrap it up before I move away?

We agreed to meet a week or so later. We moved his drum set, tube amp, and guitar from the house to the garage outside, and we jammed for a bit.

Not content with just jamming, we started trying to find some structure, some riffs that worked, and soon we were pitching random ideas: What about metal with odd time signatures? What about Rammstein-inspired heavy riffs? What about double kick drums with weird patterns? What about fruits? Yeah, what about fruits?

He dived into a story about how he got stopped at the airport in the Dominican Republic trying to take a few dragon fruits through security. They had to scarf them down (with plastic knives and forks) to try to make it past security in time for their flight.

I was like, “Man! That’s amazing, let’s write something about that.” So, we started coming up with ideas, which I scribbled down on a notepad. We also recorded a few drafts on my phone just to remember them afterward.

After this initial session, I went back home and finished writing the other parts of the songs. I wrote down some lyrics, and we had a chat about making it a four-week project. Two sessions to practice, two sessions to record. Four songs in total: Cherries, Dragon Fruits, Strawberries, and Guanabanas.

We recorded them a few weeks later at a house I was renting. It was three stories high, and on the top floor, I could usually play without bothering the neighbors too much, as they seemed to use their third floor for storage. These are Belgian houses, so very thin and tall, with somewhat scary steps to climb down.

The session went well, so I fiddled around a bit with quantization, trying to get the drum sound right, trying to get the guitar sound right, and so on. I recorded the guitars on a small Fender amp, which didn’t really have the tone I was looking for. So, I left the songs incomplete for months, partly working on them every now and then but never really satisfied.

After moving to Ireland towards the end of the year, I bought a new guitar. Shortly after, I found a DSP VST for guitars that had a tone I fell in love with: Ampbox by Mercuriall, using the U530 Preamp. When I compared it with what I recorded using my Fender amp through a (probably badly placed) Shure SM57, the tone was miles away. This sounded professional, wide, grungy, and powerful, exactly what we needed.

So, I re-tracked all the guitars, added a few solos, and started double-tracking the drums.

But once you get started, it is so easy to get sidetracked. I thought maybe I could add a random spacer between each song, maybe hire voice-over artists on Fiverr to narrate the description of fruits and place a weird glitchy track in the background, or mess around with the voices, making them stutter or glitch out.

And soon after, I realized maybe I could add a few calmer tracks, because guttural screaming isn’t for everyone. Maybe the sound of the songs could be reflected in the type of fruit, or maybe sometimes the inverse to switch things up and make it unpredictable.

It’s funny it took so long. You can write a song in a few hours, but sometimes life gets in the way. When you pick the project back up, the scope increases, and you realize you want to make something much larger than what you originally intended.

Over time, I started adding more and more fruit songs. At some point, I became dead-set on the idea of making a song about every fruit. But I’m glad I backed away from that… it turns out there are a lot of fruits! But hopefully, the series of albums will be interesting, even if the idea behind them is a bit silly.

A year later, I had over 10 tracks down, including the Fiverr voiceover ones, and then I moved back to Belgium… To a flat, which made it a bit harder to record. Now I really had to pay attention to avoid bothering the neighbors. Slowly but steadily, over the next year, I wrote another 12+ tracks, and I reached the point where I noticed it was going to take me a long time to write a song about every fruit.

I’d take all the fruits I had, plug them into ChatGPT, ask which ones I was missing, and it would always come back with a massive new list. So during a run on a random weekday, I had this sudden stroke of inspiration: Let’s split the album and cover some interesting fruits, but not all of them… maybe 2 or 3 albums of 15 tracks each.

Finally a plan.

So, I chose the 15 songs for the first album and tried to mirror them with the second so that the same style of song lands in the same place. Whether it is a description, a calmer song, or a heavier song, the first two volumes would be mirrored in a way.

The mixing and mastering process during the following month was hell. A lot of listening to reference songs, then listening to the album and noticing I was missing lower end, then going out for a walk and listening on earphones instead of monitors or headphones and noticing it sounded weak. Then adding a lot of compression and noticing it sounded oversaturated. Then diving into LUFS and hearing about headroom and trying to lower things a bit.

Finally, I found a sweet spot that I liked. It is still very, very loud. I used as much of the frequency spectrum as I could. I love things sounding heavy, with boomy lower ends, but also very fresh and bright on the top end. I think I’d get a slap from any mix/mastering engineer, but I went my own way for this one. Maybe someday I’ll learn.

Why is the album called FRUTAL? The idea behind the name was because we wanted to make something heavy, brutal, like very intense. So brutal, but about fruits… How about FRUTAL? And the name just stuck.

I’m really looking forward to putting this out and listening to your feedback. So stay tuned for more updates, you can visit lemiffe.com/subscribe where you can subscribe to the newsletter if you’d like to receive an email the moment it drops. Otherwise I’ll likely be posting videos and promo material on YouTube, X and Threads along the way.

The State of AI Music Generation

This is a list of tools, VSTs, services, websites and research on AI music generation. Last updated: 30/Mar/2024.

I’ve divided the list into four areas: Research, Websites & Services, AI vocalists, VSTs & Plugins for DAWs. There is quite a varied use of AI models across the board, the first section primarily focuses on prompt to music, aka “ChatGPT for music”, but there are also tools and VSTs that use RNNs, deep neural nets, diffusion models, etc.

I’ve also provided my opinion/thoughts on some of these tools, I didn’t have a chance to try out every website/service, nor did I purchase every VST as some are a bit expensive, but I did go through example videos, and sometimes comparison videos with other tools/services, to get a high-level overview on each tool/service/plugin. Hopefully it is useful to you as a reference list.

State of the art research in text to music prompts and audio effect generators

  • Meta
  • Google
  • Stability AI
    • Stable Audio: Diffusion-based generative models for latent audio diffusion
    • Had a few demos here that while they were impressive, I would put them on par with Google and Meta’s efforts so far. Check the official website here.
    • Another demo that seems to be more recent (and better!)
    • HN comment about Ed Newton-Rex who built stable audio quit after release re training data - he went on to found https://www.fairlytrained.org/
  • Generative Music in Games
    • Research paper on the topic
    • Could be the most interesting area of research: on-the-fly generation of music that adapts based on the character, story arc, etc. It could potentially work incredibly well for those games that have a bunch of different endings where every choice matters in the decision tree. This way the musicians and audio direction can focus on the theme and primary music while leaving atmosphere and subtle background music to embedded AI music generators.

Websites and services to generate music and sound effects with AI

  • Audiogen.co: Doesn’t mention what model they are using but it seems very likely they’ll be using one of the research ones covered above
  • Lalal.ai: Extracts or isolate stems in music - so you can, for example, remove the drums and overdub with your own, or use the vocal takes in a remix
  • AI Mastering Services: Quite a few options available here, more popping up all the time. AI mastering has already been a thing for a while, but certain tools/VSTs and websites are making it much easier to use and afford. This article from Ars Technica includes a very good breakdown and comparison amongst popular services such as LANDr, Ozone, etc.
  • SoundDraw: Interesting one as it doesn’t work with the usual prompt to music principle, it uses segments built of samples submitted by the devs, and you can turn segments on and off and rearrange them, like in a rudimentary DAW, as well as generate new segments as well. A big pro is that you can try it out without making an account; but I’m unsure about the licensing aspects (there is a FAQ that dives into this however)
  • Beatoven.ai: At a glance seems similar to the previous one
    • Fast registration (requires email confirmation)
    • Generation took less than a minute, with 4 alternate tracks
    • Quality was poor, and the default length was very small (plans start at $6)
  • AIVA: AI music generation assistant
    • Login / account creation was broken when I tried it out
  • Loudly: Also similar to previous websites
    • Quick registration process
    • Instead of just being prompt based, it offers parameters, where you select duration, instruments, genre, etc.
    • Generation is FAST
    • Provides 3 options per generation, liked some of the options but I’d like to see how it generates more variance with longer tracks
    • Limited to 31 second output for free, you have to upgrade to get longer tracks
  • Suno: Also similar to the previous websites.
    • BEST choice out there right now in my opinion
    • Very simple interface, got 50 credits for free upon logging in the first time.
    • V3 of their engine allows you to make a track up to 2 minutes long, you can also toggle to generate instrumental-only songs.
    • It took a couple of minutes to finish, and it generated 2 options for 5 credits per track, which means you can try 5 prompts for free
    • It allows you to continue from the end of a previously generated track, which would allow you to “stich the results” together to create longer tracks
    • The results were quite interesting, I asked for A droning techno with a slow buildup towards an epic theme and then with a decay back into techno… The first option delivered just this, albeit a bit lackluster, but the second option went from absolute rubbish into amazing bright vivid super uplifting music which quite surprised me
    • Login only works with Discord, Google or Microsoft? (why no email option?)
  • Boomy: Untested. Con: Need to make an account
  • Musicfy AI: Similar to previous but with AI vocals as well?
  • Soundful
    • Pro: License covers everything from videos, websites, corporate use, games, etc.
    • Pricing is per year, and is less than half the price that Epidemic Sound costs per year. Or you can pay a bit more to obtain full copyright for all creations and not just a license for usage.
    • First one billed as “royalty free music for creators” — I think this is what a lot of us thought when AI music generation started becoming a viable thing… that its primary use-case would be to generate background music for videos and live streaming on Twitch and YouTube, not to create new content to flood streaming services like Spotify with auto-generated songs… yet a few of the last 6 or 7 services seem aimed towards people without music or singing knowledge to create tracks and assist with publishing them on streaming services

There’s this section in Colin and Samir’s Jacob Collier Interview where they dive into the topic of AI and music and I really loved the positive attitude, specially where he talks about crafting new sounds and using it as a tool to unlock or further creativity.

As a music world builder (as Jacob Collier talks about himself) it can be instrumental to find sounds that can change the way we dream up and work with compositions, venturing into unexplored audio territory, kind of what analog and modular synths have done to open up a new world of possibilities over the past 50 or 60 years, or granular synthesis in the past 20 to 30 years.

It seems anyone with a say on AI and music generation take one of three stances:

  • Hate it, afraid of the implications for copyright, don’t like the potential for saturation with “fake work”, the dilution of quality music in a sea of artificial sounds.
  • Afraid, unsure about what it means, how much impact will it cause, will it be used as a tool or will people use it to create full works, will it take away jobs?
  • Positive but slightly weary, knowing there is potential for wrong-doing but the toolset it gives us outweighs the cons, such as crafting new soundscapes, imagining sounds that we could have never thought possible otherwise, helping create drafts of soundscapes which we can then build upon, or helping unblock continuation of songs for artists struggling with a creative block

AI singers and artist imitation

AI virtual music instruments and VSTs

  • Synplant 2: Lets you craft sounds and branches - this is an EPIC plugin!
  • iZotope Elements: RX, Neutron, Ozone and Nectar plugins
  • Guitar Rig 7 Pro: Guitar and Bass Amp Simulators. Uses AI (with Convolution?) to craft epic guitar tones
    • For example you feed it the tone of an amp or a sound you are looking for and it adjusts its parameters to allow you to reach that sound without having to own expensive amps / preamps. It is expensive though.
    • Great example video
  • Focusrite’s FAST bundle: Makes mixing much easier
  • Magenta Studio (for Ableton)
    • Covered 1.0 but now 2.0 is out and it has been redesigned to be much simpler to use. Only works in session view but you can drag and drop the clips into arrangement view. Very easy plugin to make drum loops sound a bit more natural, or to extend parts of a song.
    • Powered by RNNs
  • Synthesizer V - Voice synthesizer
    • From lyrics and midi to vocals, without having to open your mouth! I’m still conflicted about this area of plugins; it does open doors and reduces barrier to entry, but on the other hand live music is amazing and vocals are often the centrepiece and I’m not sure I want to live in a world where even more emphasis is placed on the DJ, specially when no vocalist is required anymore, everything can be virtual. Using it in combination with real lyrics, or in a few tracks where it can be used stylistically, hell yes! But as a full replacement? I’m not sure. I’m still on the fence.
    • The price? 89 dollars, sounds amazing - scarily amazing - supports 3 languages as of February 2024
    • Research (don’t include in credits, mentioned below)
    • Check out the website for videos/examples
    • Deep neural network-based
  • Vocaloid 6 (by Yamaha) - Voice synthesizer
    • Another vocal synth, a bit less impressive than than Synthesizer V in my opinion, with less voices and quite a bit more expensive out-of-the-box (although you can pay per voice pack with Synthesizer V). Also supports 3 languages. IMO sounds a bit more robotic, yet for some styles this might be the right fit!
    • Research (same link as Synthesizer V)
    • Check out this example video from their website

Continuation of my previous rant:

Even though the AI singers imitating known artists generated a lot of drama, in my opinion it pales in comparison to the voice synthesis… I mean, I think a lot of people were frustrated with autotune, it just means you don’t even need to know how to sing, YouTubers such as Roomie have generated many videos of autotune vs real vocals. Some artists would use it covertly to fix mistakes, but then it spawned a use-case where it was used as a texture, the whole point was cranking it to 10 making it a stylistic choice…

And I just think in a way AI voice synthesis for singing means that some people who make a living singing on tracks will maybe make less if producers don’t have to find the right talent anymore, not even just someone with a voice to then auto-tune, but now they can skip the vocalist completely. How will that work in a live setting?

These are just some of my raw thoughts as all of this is still quite new and I don’t know how it will pay out in the long run, but the pace of innovation in the AI music generation and voice synthesis in general seems to be accelerating, and I’m very curious what’s going to come out next, and how these tools will be used once the dust has settled.

Other Tools and Services:

There are a bunch of lists of top AI tools including quite a few I didn’t cover like Orb Producer Suite 3, Playbeat sequencer, Atlas 2, Spark Chords and others… This website covers some of them if you are interested in more overviews and links to the plugins

Thanks for reading!

On FRUTAL's Development and Scope Creep

It’s funny how sometimes you’ll have a spark of an idea, to create something small like a sketch, or a little art project, or a small video series, or an EP, or a web app. Something not too big, yet large enough to convey a message, to craft a little world, to tell a story with changing themes, to hold and express sentiment like a roller-coaster at sunset with a glistening bay in view, going up and down as the light dims revealing another world.

And then the output of the spark that ignited the creation spirals out of control and the original work no longer suffices, so you place the first creations on the side while you work on a different set of related ideas, elevated to a higher level, and then - months later - you rediscover the original work and it not only holds sentimental value but you see how you can bring it back into the narrative, so you reintegrate them, and work on them anew, yet after more months of work you find that they are too distinct, as if two bodies of work loosely connected by an idea, but instead of separating them into two separate projects you start drawing bridges, and you try to connect the work together with more work, shorter work, like commas in a long unruly sentence.

And suddenly the small project has turned out to be an expedition lasting close to a couple of years, and the work reaching an end no longer looks like the spark of the original idea, yet you are sure there is some merit, as it is its own world now, and you have to set it free, but you are unsure if it will meet expectations.

And then you remember: There are no expectations.

And thus I’ve been working on FRUTAL for a while; an album I started working with my friend Pyrathee before I set off to Ireland. We decided to hang out for a series of 4-5 sessions to work on some material, where he’d play drums and I’d compose, play guitar and sing. The result was 4 songs related to fruits in one way or another, some based on real stories, and some fantasy. The intention was to craft them with a heavy grungy sound, hence the name of the album would be a wordplay on “Brutal”.

Yet 4 songs weren’t enough, so I paid a few voice actors and narrators on Fiverr to read out fruit names and definitions, and then these had to be woven into little musical worlds or motifs. But now we had 6 bridges and 4 songs, not enough songs for an album, but wait, this wasn’t supposed to be an album! Yet there I was, writing more songs, now in a different style, and then I had to construct more bridges, toppling some of the older ones down in the process, and more songs were crafted, about plums, about apples, about mangoes.

And the “Cantaloupes” barbershop quartet no longer matched the theme and had to be re-written thrice. And then I was satisfied, knowing the work consisted of three themes, that reference each other, that play together, that flow, that come back in one way or another throughout the album, a few themes to carry different feelings, different weights, some frutal, some brutal. In a way it resembles my personality, and life changes over the past few years, a story. Rock, indie, electronic, and bridges in between.

The work is nearly done, I hope you enjoy it and that it generates the feeling of going on a journey in you, one full of fruits with a touch of madness. Follow the blog, or subscribe to the newsletter to be notified when it drops.

And remember: Eat your Daily Mangoes.

The Discomfort of Evening (by Marieke Lucas Rijneveld)

As the weeks pass by after finishing this book, I realise more and more how perturbing the experience was. It’s like a lingering nightmare from nights gone by which looms every night, threatening to return while you sleep, to terrorise your dreams with a complex distorted reality of cold, coats, toads, rabbits, cows and “the other side”.

The Discomfort of Evening, known in Dutch as De Avond is Ongemak written by Marieke Lucas Rijneveld, is a whirlwind to say the least.

It is an easy read which is a pro, full of ample descriptions of this world which is described which is so visual yet so drab and gray, I can imagine the featureless landscape, as I write this looking out the gray skies of Belgium on a winter morning. I can picture the mud, the fields, the cows, as when I run long distances I often stumble upon many farms, with the cows grazing, often looking up at me as I run by and then returning to their rumination. I always wondered what kind of people live in these farms, how do they live their lives, and this book gives a portrayal of a glimpse into this life, albeit heavily distorted by the family’s circumstances.

Speaking of rumination, I still can’t digest this book. It is an endurance run of pain or a marathon of grimness, every page gray and drab, a bitter or metallic taste in the mouth, like blood. This is not to say it is badly written, as the language and the story carry beautifully from page to page, similar to how the monotonous voice of Charlie (aka MoistCr1TiKaL) on his YouTube channel delivers his videos, starkly honestly without a facade, direct, blunt, piercing. In this same light the book reads like an honest down-to-earth journal, narrating the smallest yet nastiest intricacies of the daily life of a troubled family in a run-down town in the middle of nowhere in the Netherlands.

I have mixed feelings after reading this book. Marieke crafted a world so unique that if it wasn’t for the mention of a few real world characters I swear it could have been about a run-down farm in Russia or Thailand, or Mars for all I care. It transports you to this place and puts you in those muddy boots and featureless landscape, and it drags you through terrible situation after terrible situation, screaming and kicking.

At times I’d be reading on the train, bus, or plane, hiding the pages from people, afraid they’d catch a glimpse of the perturbing sentences, page after page. It was a diarrhea of all possible intrusive thoughts, everything weird you might have thought of doing, to yourself, or to others, growing up and exploring the world, wondering “why not?”. All of those crazy disgusting weird thoughts, turned into a reality.

I don’t know whether I’d recommend this book. I’m still perturbed by it, so I guess it did its job. 7/10