{"id":625418,"date":"2023-04-04T09:50:00","date_gmt":"2023-04-04T14:50:00","guid":{"rendered":"https:\/\/news.sellorbuyhomefast.com\/index.php\/2023\/04\/04\/the-complex-math-of-counterfactuals-could-help-spotify-pick-your-next-favorite-song\/"},"modified":"2023-04-04T09:50:00","modified_gmt":"2023-04-04T14:50:00","slug":"the-complex-math-of-counterfactuals-could-help-spotify-pick-your-next-favorite-song","status":"publish","type":"post","link":"https:\/\/newsycanuse.com\/index.php\/2023\/04\/04\/the-complex-math-of-counterfactuals-could-help-spotify-pick-your-next-favorite-song\/","title":{"rendered":"The complex math of counterfactuals could help Spotify pick your next favorite song"},"content":{"rendered":"
\n
\n

The basic idea behind counterfactuals is to ask what would have happened in a situation had certain things been different. It\u2019s like rewinding the world, changing a few crucial details, and then hitting play to see what happens. By tweaking the right things, it\u2019s possible to separate true causation from correlation and coincidence.<\/p>\n

\u201cUnderstanding cause and effect is super important for decision making,\u201d says Ciaran Gilligan-Lee, leader of the Causal Inference Research Lab at Spotify, who co-developed the model. \u201cYou want to understand what impact a choice you take now will have on the future.\u201d<\/p>\n

In Spotify\u2019s case, that might mean choosing what songs to show you or when artists should drop a new album. Spotify isn\u2019t yet using counterfactuals, says Gilligan-Lee. \u201cBut they could help answer questions that we deal with every day.\u201d<\/p>\n

Counterfactuals are intuitive. People often make sense of the world by imagining how things would have played out if this<\/em> had happened instead of that<\/em>. But they are monstrous put into math.<\/p>\n

\u201cCounterfactuals are very strange-looking statistical objects,\u201d says Gilligan-Lee. \u201cThey\u2019re weird things to contemplate. You\u2019re asking the likelihood of something occurring given that it didn\u2019t occur.\u201d<\/p>\n

Gilligan-Lee and his coauthors started working together after reading about each other\u2019s work in a MIT Technology Review story<\/a>. They based their model on a theoretical framework for counterfactuals called twin networks.<\/p>\n

Twin networks were invented in the 1990s by the computer scientists Andrew Balke and Judea Pearl. In 2011, Pearl won the Turing Award\u2014computer science\u2019s Nobel Prize\u2014for his work on causal reasoning and artificial intelligence.<\/p>\n

Pearl and Balke used twin networks to work through a handful of simple examples, says Gilligan-Lee. But applying the mathematical framework to larger and more complicated real-world cases by hand is hard.<\/p>\n<\/p><\/div>\n

\n

That\u2019s where machine learning comes in. Twin networks treat counterfactuals as a pair of probabilistic models: one representing the actual world, the other representing the fictional one. The models are linked in such a way that the model of the actual world constrains the model of the fictional one, keeping it the same in every way except for the facts you want to change.\u00a0\u00a0<\/p>\n

Gilligan-Lee and his colleagues used the framework of twin networks as a blueprint for a neural network and then trained it to make predictions about how events would play out in the fictional world. The result is a general-purpose computer program for doing counterfactual reasoning. \u201cIt lets you answer any counterfactual question about a scenario that you want,\u201d says Gilligan-Lee.<\/p>\n

Dirty water<\/h3>\n

The Spotify team tested their model using several real-world case studies, including one looking at credit approval in Germany, one looking at an international clinical trial for stroke medication, and another looking at the safety of the water supply in Kenya.<\/p>\n

In 2020 researchers investigated whether installing pipes and concrete containers to protect springs from bacterial contamination in a region of Kenya would reduce levels of childhood diarrhea. They found a positive effect. But you need to be sure what caused it, says Gilligan-Lee. Before installing concrete walls around wells across the country, you need to be sure that the drop in sickness was in fact caused by that intervention and not a side effect of it.<\/p>\n

It\u2019s possible that when researchers came in to do the study and install concrete walls around the wells, it made people more aware of the risks of contaminated water and they started boiling it at home. In that case, \u201ceducation would be a cheaper way to scale up the intervention,\u201d says Gilligan-Lee.<\/p>\n<\/p><\/div>\n

\n

Gilligan-Lee and his colleagues ran this scenario through their model, asking whether children who got sick after drinking from an unprotected well in the actual world also got sick after drinking from a protected well in the fictional world. They found that changing just the detail of where the child drank and maintaining other conditions, such as how the water was treated at home, did not have a significant impact on the outcome, suggesting that the reduced levels of childhood diarrhea were not (directly) caused by installing pipes and concrete containers.\u00a0\u00a0<\/p>\n

This replicates the result of the 2020 study, which also used counterfactual reasoning. But those researchers built a bespoke statistical model by hand just to ask that one question, says Gilligan-Lee. In contrast, the Spotify team\u2019s machine-learning model is general purpose and can be used to ask multiple counterfactual questions about many different scenarios.<\/p>\n

Spotify is not the only tech company racing to build machine-learning models that can reason about cause and effect. In the last few years, firms such as Meta, Amazon, LinkedIn, and TikTok\u2019s owner ByteDance have also begun to develop the technology.<\/p>\n<\/p><\/div>\n

\n

\u201cCausal reasoning is critical for machine learning,\u201d says Nailong Zhang, a software engineer at Meta. Meta is using causal inference in a machine-learning model that manages how many and what kinds of notifications Instagram should send its users<\/a> to keep them coming back.\u00a0<\/p>\n

Romila Pradhan, a data scientist at Purdue University in Indiana, is using counterfactuals to make automated decision making more transparent. Organizations now use machine-learning models to choose who gets credit<\/a>, jobs<\/a>, parole<\/a>, even housing<\/a> (and who doesn\u2019t). Regulators have started to require organizations to explain the outcome of many of these decisions to those affected<\/a> by them. But reconstructing the steps made by a complex algorithm is hard.\u00a0<\/p>\n

Pradhan thinks counterfactuals can help. Let\u2019s say a bank\u2019s machine-learning model rejects your loan application<\/a> and you want to know why. One way to answer that question is with counterfactuals. Given that the application was rejected in the actual world, would it have been rejected in a fictional world in which your credit history was different? What about if you had a different zip code, job, income, and so on? Building the ability to answer such questions into future loan approval programs, Pradhan says, would give banks a way to offer customers reasons rather than just a yes or no.\u00a0\u00a0\u00a0\u00a0<\/p>\n

Counterfactuals are important because it\u2019s how people think about different outcomes, says Pradhan: \u201cThey are a good way to capture explanations.\u201d<\/p>\n

They can also help companies predict people\u2019s behavior. Because counterfactuals make it possible to infer what might happen in a particular situation, not just on average, tech platforms can use it to pigeonhole people with more precision than ever.\u00a0<\/p>\n

The same logic that can disentangle the effects of dirty water or lending decisions can be used to hone the impact of Spotify playlists, Instagram notifications, and ad targeting. If we play this song, will that user listen for longer? If we show this picture, will that person keep scrolling? \u201cCompanies want to understand how to give recommendations to specific users rather than the average user,\u201d says Gilligan-Lee.<\/svg> <\/p>\n<\/div>\n<\/div>\n

Read More<\/a>
\n Will Douglas Heaven<\/p>\n","protected":false},"excerpt":{"rendered":"

The basic idea behind counterfactuals is to ask what would have happened in a situation had certain things been different. It\u2019s like rewinding the world, changing a few crucial details, and then hitting play to see what happens. By tweaking the right things, it\u2019s possible to separate true causation from correlation and coincidence. \u201cUnderstanding cause<\/p>\n","protected":false},"author":1,"featured_media":625419,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[22166,121108,46],"tags":[],"class_list":{"0":"post-625418","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-complex","8":"category-counterfactuals","9":"category-technology"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/625418","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/comments?post=625418"}],"version-history":[{"count":0,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/625418\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media\/625419"}],"wp:attachment":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media?parent=625418"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/categories?post=625418"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/tags?post=625418"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}