Spotify may start using Sliding Doors-style ‘what if?’ AI to suggest your new music

Not content with the recent and highly specific niche mix rollout, Spotify? Think there must be a better way to understand our deepest unspoken musical wants and desires? 

Personally, I’d just love for you to make good on your Spotify HiFi promise, (and I can take or leave the TikTok-style revamp, thank you) but it seems you have other plans.

The concept I’m about to share is a cerebral one and something most of us likely have a tenuous grasp of, at best. It’s all about counterfactuals (opens in new tab), causation, and causal inference in machine-learning mathematics (so, if something didn’t happen, but what if it had happened? Now add that to math!) and Spotify is all set to use it to help the streaming giant suggest your next favorite album or playlist. 

Think of the movie Sliding Doors (opens in new tab) but with your music collection, and you’ve got the basic idea – but the concept is a complex one and no mistake. 

What actually is it though? As reported by the bimonthly Massachusetts-based tech publication, MIT Technology Review (opens in new tab), a team of researchers at Spotify just built a new kind of machine-learning model which, for the first time, captures the complex math behind counterfactual analysis. And apparently, it can be used to identify the causes of past events – and predict the effects of future ones.

How does that translate to the 90s mix you streamed from 1 AM ’til 4 AM last night after sharing a bottle of wine with a friend… and possibly texting your ex? Well, if Spotify knows about it, it could improve the accuracy of automated decision-making, especially personalized recommendations.

Analysis: Music of the Mind? Or Spotify’s road to Virtual Insanity?

Well done if you noticed I just put two Jamiroquai songs in a sub-head – we’ll probably get similar acid-jazz playlists beamed to us later. High five to you!

Now let’s go up a level. Stay with me and we can relax with a nice chill mix later. The whole counterfactual math approach is inspired by Einstein’s light cones. Think of a ‘futures cone’ where the narrowest circle, created by the thinnest cone, is ‘probable’, then ‘plausible’ going out and out as the cone gets wider towards ‘possible’ and finally ‘preposterous’ – ie. highly unlikely. 

Got that? Good. Now, the basic theory behind counterfactuals is to ask what would have happened in a situation had certain things been different – if Helen (aka Gwyneth Paltrow) had neither made the train nor missed the train in Sliding Doors in 1998, but had instead decided to go see a movie and ended up spilling popcorn all over the true love of her life… By altering the correct variables, it should be possible to distinguish genuine causation from association and coincidence. So we narrow the cone! 

All sound a bit pie in the sky? I agree. I, for example, might have crammed the same Falco songs into my brain on repeat for two months in 2009 because I was dancing to them, for work. What does that say about me, Spotify? Perhaps it was not my music of choice… perhaps I also listened to Ella Fitzgerald on vinyl to chill out. Did you know about that? What of it? 

But see, where it gets truly mind-melting is that counterfactual AI works on the outcome of an event even if the event never actually happened.

The Spotify team has tested the model out using real-world case studies, including one looking at the health implications of a water source in Kenya. And it really is the future. Spotify is not the only tech giant galloping towards machine-learning models with the ability to perform cause-and-effect reasoning; Meta, LinkedIn, Amazon, and TikTok’s owner ByteDance have also been working on similar technology. Meta is apparently using causal inference in a machine-learning tool to manage how many notifications (and what kind) Instagram can send its users and still keep them hooked. 

Will it be any better than “You like Fontaines D.C. so have a bang on Yard Act” when Spotify implements it? Impossible to say. Do I want it? No, no – but then, I look upon Spotify as an old friend from simpler times, rather than one of the best music streaming services of 2023. Also, I’ve been known to listen to the same 1984 XTC song on loop for 90 minutes (Wake Up, if you’re wondering) just to appreciate the backing vocals… I do not care to know what Spotify thinks of that. 

Becky is a senior staff writer at TechRadar (which she has been assured refers to expertise rather than age) focusing on all things audio. Before joining the team, she spent three years at What Hi-Fi? testing and reviewing everything from wallet-friendly wireless earbuds to huge high-end sound systems. Prior to gaining her MA in Journalism in 2018, Becky freelanced as an arts critic alongside a 22-year career as a professional dancer and aerialist – any love of dance starts with a love of music. Becky has previously contributed to Stuff, FourFourTwo and The Stage. When not writing, she can still be found throwing shapes in a dance studio, these days with varying degrees of success.