![\"causaldiscoveryalgo\"](\"https://user-images.githubusercontent.com/33357428/219991483-bd875dbc-d04d-4bca-a1db-1b553162f199.png\")
Explainable AI refers to models that can easily be interpreted by humans. Bayesian inferencing is a natural choice to obtain explanations of trained deep learning models.
","url":"http://localhost:2368/explaining-bayesian-neural-networks/","canonical_url":null,"uuid":"6bef7972-5e59-4ea0-8ccd-99292bb992ab","codeinjection_foot":null,"codeinjection_head":null,"codeinjection_styles":null,"comment_id":"63eeea8172a3c427182edcc0","reading_time":0,"send_email_when_published":null,"email_subject":null,"childHtmlRehype":{"html":"Explainable AI refers to models that can easily be interpreted by humans. Bayesian inferencing is a natural choice to obtain explanations of trained deep learning models.
","htmlAst":{"type":"root","children":[{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Explainable AI refers to models that can easily be interpreted by humans. Bayesian inferencing is a natural choice to obtain explanations of trained deep learning models."}]}],"data":{"quirksMode":false}},"tableOfContents":[]},"featureImageSharp":{"base":"photo-1646583288948-24548aedffd8.jpg","publicURL":"/static/e6860d2b93d53230bfd7e05394764440/photo-1646583288948-24548aedffd8.jpg","imageMeta":{"width":2000,"height":1333},"childImageSharp":{"fluid":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAANABQDASIAAhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAAAAIDAQT/xAAVAQEBAAAAAAAAAAAAAAAAAAAAAf/aAAwDAQACEAMQAAABTnWKSMD/xAAYEAEBAQEBAAAAAAAAAAAAAAABAAIDEP/aAAgBAQABBQJbrlyS2lfP/8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAwEBPwE//8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAgEBPwE//8QAGBAAAgMAAAAAAAAAAAAAAAAAAAERIEH/2gAIAQEABj8CJe1//8QAGRAAAgMBAAAAAAAAAAAAAAAAAAEQETEh/9oACAEBAAE/IeYkoD0ZRqOP/9oADAMBAAIAAwAAABDPP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQMBAT8QP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8QP//EABwQAQEBAAEFAAAAAAAAAAAAAAERABAhMUFRYf/aAAgBAQABPxAaswSgie+peDzmEip9blb33//Z","aspectRatio":1.4957264957264957,"src":"/static/e6860d2b93d53230bfd7e05394764440/ea4ab/photo-1646583288948-24548aedffd8.jpg","srcSet":"/static/e6860d2b93d53230bfd7e05394764440/477ba/photo-1646583288948-24548aedffd8.jpg 175w,\n/static/e6860d2b93d53230bfd7e05394764440/06776/photo-1646583288948-24548aedffd8.jpg 350w,\n/static/e6860d2b93d53230bfd7e05394764440/ea4ab/photo-1646583288948-24548aedffd8.jpg 700w,\n/static/e6860d2b93d53230bfd7e05394764440/3055e/photo-1646583288948-24548aedffd8.jpg 1050w,\n/static/e6860d2b93d53230bfd7e05394764440/eff08/photo-1646583288948-24548aedffd8.jpg 1400w,\n/static/e6860d2b93d53230bfd7e05394764440/4e5f3/photo-1646583288948-24548aedffd8.jpg 2000w","srcWebp":"/static/e6860d2b93d53230bfd7e05394764440/89afa/photo-1646583288948-24548aedffd8.webp","srcSetWebp":"/static/e6860d2b93d53230bfd7e05394764440/9fca7/photo-1646583288948-24548aedffd8.webp 175w,\n/static/e6860d2b93d53230bfd7e05394764440/37a4e/photo-1646583288948-24548aedffd8.webp 350w,\n/static/e6860d2b93d53230bfd7e05394764440/89afa/photo-1646583288948-24548aedffd8.webp 700w,\n/static/e6860d2b93d53230bfd7e05394764440/78e7a/photo-1646583288948-24548aedffd8.webp 1050w,\n/static/e6860d2b93d53230bfd7e05394764440/03d34/photo-1646583288948-24548aedffd8.webp 1400w,\n/static/e6860d2b93d53230bfd7e05394764440/49d6b/photo-1646583288948-24548aedffd8.webp 2000w","sizes":"(max-width: 700px) 100vw, 700px"}}}}},{"node":{"id":"Ghost__Post__63b58208bdc6867fe35526ff","title":"Causal Machine Learning - Part 5","slug":"causal-machine-learning-part-5","featured":false,"feature_image":"https://images.unsplash.com/photo-1648007547791-404a2abfdc82?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDY5Mnx8cm9ib3R8ZW58MHx8fHwxNjcyODM5Njc3&ixlib=rb-4.0.3&q=80&w=2000","excerpt":"This post is the fifth post of the series on Causal Machine Learning. This blog post is based on the work of Judea Pearl. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !","custom_excerpt":"This post is the fifth post of the series on Causal Machine Learning. This blog post is based on the work of Judea Pearl. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !","visibility":"public","created_at_pretty":"4 Jan 2023","published_at_pretty":"4 Jan 2023","updated_at_pretty":"20 Feb 2023","created_at":"2023-01-04T19:11:28.000+05:30","published_at":"2023-01-04T19:12:15.000+05:30","updated_at":"2023-02-20T09:33:55.000+05:30","meta_title":null,"meta_description":null,"og_description":null,"og_image":null,"og_title":null,"twitter_description":null,"twitter_image":null,"twitter_title":null,"authors":[{"slug":"amaljith","url":"http://localhost:2368/author/amaljith/","name":"Amaljith","bio":"Research Scholar @ IIT Kharagpur","cover_image":null,"profile_image":"http://localhost:2368/content/images/2022/09/Screenshot-from-2022-09-07-18-00-00.png","location":null,"website":null,"twitter":null,"facebook":null,"meta_title":null,"meta_description":null,"coverImageSharp":null,"profileImageSharp":null}],"primary_author":{"slug":"amaljith","url":"http://localhost:2368/author/amaljith/","name":"Amaljith","bio":"Research Scholar @ IIT Kharagpur","cover_image":null,"profile_image":"http://localhost:2368/content/images/2022/09/Screenshot-from-2022-09-07-18-00-00.png","location":null,"website":null,"twitter":null,"facebook":null,"meta_title":null,"meta_description":null,"coverImageSharp":null,"profileImageSharp":{"base":"Screenshot-from-2022-09-07-18-00-00.png","publicURL":"/static/28e31bfedd96b4afe90237d2c1f700c3/Screenshot-from-2022-09-07-18-00-00.png","imageMeta":{"width":316,"height":237},"childImageSharp":{"fluid":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAPCAYAAADkmO9VAAAACXBIWXMAAAsTAAALEwEAmpwYAAAEJElEQVQ4yyXO21fTBQDA8d9fUh5LQAW1eEImbIPJbSB3kZtijjSBxNvAGSAqlJZ64GAHIQ0VjkKICCiCQ8cYsYFjYwhsYxeGjJsopmEnH76dYw+f94/w2jmKfaSX9wtTrC1a+bBk5d83dlbnzFhHehnq62TBZeLjGyfvFqf4a9nGstuAx/oM96Qai76NieFu5u0jLLrNCCsOI47nat56J1hbtLH60kTfg5tUV6hQFeSSFLGTgtwcBtUteGc0zLoGmXNpmXU8Yd6txTOlxvq8hzmr/hNhyTaC06hmya3nn6VxHKMPKS08RFF+HgWK/aiO5JP3TQ4xMgn3mqt599bC8pyBRY+OxRkdXkc/rrE+POZ+nMY+hFXHKPPWP5m3a/i4Yqax9kfCREEkyKPJSk3mQFYmOelpRISGEBMuwahv4/WC4dNuwa3l1Us906O9jGs7sWg7EHQdv6NurWVmvJv33iHKTuSyft06tm3xJygwEIlIxDb/zUhFIvw2+HDrxkX+XjUx5x7A6+pn3qXBZnyIoacFTftNhKarlTxprsMz3sPbWR2jmiYiJSICt27lqwB/tmza+L8Af5KkYsYNHXhcvbidahy2x0ya2jFqm7lxqYxrF04jvLQMYB16jKazHrOulRWHjiJFOus/+5yAjX5s8vNhs58vfj5foooO43LJMaovncI1/ZhpWxcTlk6mRjrpa7uOrqsJYcaiY25imKftv6F52IBx4D61Z46y6Yv1+PpsYLOvD36+G0gJDuRcrIS0nTvY8XUAl66osDq60A3e5sHtXzCoW3CY+hHeeKdZdL5gytDDgsOAe1zL9KiawowEsoK2URi+HWWMmKvZuzibKuNcqoTiWBEXShToLHfo7K6h+9413DYzS/MehJnJYZxjOowDHdjHnjE53M2kUU1/zVmu742lOlNOkyKRmtzd1KkUtObFcacgmUetVTT0/kTDvQpemPoxDWmoralG8NrNvJqZwmbS8Xp5FrfNhMtuxtNei75Kia3xAmtdVZgaf6axvprKvXH8kB5FXd1lFMcyuVp7Hnd9E0+bb5Cdvgfhw4oXPq5hGR5kzGzkUVcPVeWlPK8tx/BrCfcrjnI+bx+KPSmcPJxP2QklFUolRfmHyf/uEEXK49y9XMndaxdJjpQirCy4WLCPoe38A4Nez8XySjISU5CFhCIO3k5wUDASkZiUuHj2Z2ahyN5HvuIgRUeOcUZZTOlJJeUlKgbU7Txta0AYbLnFqewMrpSepvhIITm700nblYg8IpJIWRjyiAjiI6JJjJKTGBtHcnwCqYlJpCelkJOWwfe533Jw337Kik9g1N5HqC9RcfyAgvTUPcRHyYmWypCFiJGKQggNDkYsEiEJEhEmCiE8REyYWIJMKiVCEoY8fCcJUTHEx8QgCw0l/8Be/gOTiTiUD46AHAAAAABJRU5ErkJggg==","aspectRatio":1.3333333333333333,"src":"/static/28e31bfedd96b4afe90237d2c1f700c3/6ccb0/Screenshot-from-2022-09-07-18-00-00.png","srcSet":"/static/28e31bfedd96b4afe90237d2c1f700c3/7d89d/Screenshot-from-2022-09-07-18-00-00.png 28w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/f4091/Screenshot-from-2022-09-07-18-00-00.png 55w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/6ccb0/Screenshot-from-2022-09-07-18-00-00.png 110w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/30481/Screenshot-from-2022-09-07-18-00-00.png 165w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/33bd6/Screenshot-from-2022-09-07-18-00-00.png 220w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/d398b/Screenshot-from-2022-09-07-18-00-00.png 316w","srcWebp":"/static/28e31bfedd96b4afe90237d2c1f700c3/8678c/Screenshot-from-2022-09-07-18-00-00.webp","srcSetWebp":"/static/28e31bfedd96b4afe90237d2c1f700c3/59cda/Screenshot-from-2022-09-07-18-00-00.webp 28w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/7da75/Screenshot-from-2022-09-07-18-00-00.webp 55w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/8678c/Screenshot-from-2022-09-07-18-00-00.webp 110w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/f282e/Screenshot-from-2022-09-07-18-00-00.webp 165w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/a7b21/Screenshot-from-2022-09-07-18-00-00.webp 220w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/fb2b8/Screenshot-from-2022-09-07-18-00-00.webp 316w","sizes":"(max-width: 110px) 100vw, 110px"}}}},"primary_tag":{"slug":"machine-learning","url":"http://localhost:2368/tag/machine-learning/","name":"Machine Learning","visibility":"public","feature_image":null,"description":null,"meta_title":null,"meta_description":null,"featureImageSharp":null},"tags":[{"slug":"machine-learning","url":"http://localhost:2368/tag/machine-learning/","name":"Machine Learning","visibility":"public","feature_image":null,"description":null,"meta_title":null,"meta_description":null,"featureImageSharp":null},{"slug":"artificial-intelligence","url":"http://localhost:2368/tag/artificial-intelligence/","name":"Artificial Intelligence","visibility":"public","feature_image":null,"description":null,"meta_title":null,"meta_description":null,"featureImageSharp":null},{"slug":"causal-machine-learning","url":"http://localhost:2368/tag/causal-machine-learning/","name":"Causal Machine Learning","visibility":"public","feature_image":null,"description":null,"meta_title":null,"meta_description":null,"featureImageSharp":null}],"plaintext":"This post is the fifth post of the series on Causal Machine Learning. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful. In this post Causal Discovery will be discussed in detail. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !\n\n\nWhat is Causal Discovery?\n\n\nCausal inference focuses on estimating the causal effect of a specific intervention or exposure on an outcome.\n\n\n\nCausal discovery focuses on identifying the underlying causal relationships between variables in a system.\n\n\n\nCausal inference, which aims to answer questions involving cause and effect. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful.\n\nCausal discovery aims to infer causal structure from data. In other words, given a dataset, derive a causal model that describes it.\n\nFinding causal relationships is one of the fundamental tasks in science. A widely used approach is randomized experiments. For example, to examine whether a recently developed medicine is useful for cancer treatment, researchers recruit subjects and randomly divide subjects into two groups. One is the control group, where the subjects are given placebo, and the other is the treatment group, where the subjects are given the newly developed drug. The reason of randomization is to remove possible effects from confounders. For example, age can be one of the possible confounders which affects both taking the drug or not and the treatment effect. Thus, in practical experiments, we should keep the distribution of ages in the two groups almost the same.\n\n\nHow Does It Work ?\n\n\nHowever, in many cases, randomized experiments are very expensive and hard to implement, and sometimes it may even involve ethical issues. In recent decades, inferring causal relations from purely observational data, known as the task of causal discovery, has drawn much attention in machine learning, philosophy, statistics, and computer science.\n\nCausal discovery is an example of an inverse problem. This is like predicting the shape of an ice cube based on the puddle it left on the kitchen counter. Clearly, this is a hard problem, since any number of shapes could generate the same puddle. Connecting this to causality, the puddle of water is like statistical associations embedded in data, and the ice cube is the like underlying causal model.\n\n\nCausal Discovery Assumptions & Properties\n\n\nThe usual approach to solving inverse problems is to make assumptions about what you are trying to uncover. This narrows down the possible solutions and hopefully makes the problem solvable. There are four common assumptions made across causal discovery algorithms.\n\n👉 Acyclicity — Causal structure can be represented by DAG (G)\n\n\n👀 Markov Property — All nodes are independent of their non-descendants when conditioned on their parents\n\n\n🙃 Faithfulness — All conditional independences in true underlying distribution p are represented in G\n\n\n👍 Sufficiency — Any pair of nodes in G has no common external cause\n\n\nAlthough these assumptions help narrow down the number of possible models, they do not fully solve the problem. This is where a few tricks/tests for causal discovery are helpful. There is no single method for causal discovery that dominates all others. Although most methods use the assumptions above (perhaps even more), the details of different algorithms can vary tremendously. A taxonomy of algorithms based on the following tricks is given in the figure below.\n\n\n\n\nConditional Independence Testing\n\n\nOne of these earliest causal discovery algorithms is the PC algorithm named after its authors Peter Spirtes and Clark Glymour. This algorithm (and others like it) use the idea that two statistically independent variables are not causally linked. The PC algorithm is illustrative of this first trick. An outline of the algorithm is given in the figure below.\n\n\n\n\nThe first step is to form a fully connected, undirected graph using every variable in the dataset. Next, edges are deleted if the corresponding variables are independent. Then, connected edges undergo conditional independence testing e.g. independence test of the bottom and far right node conditioned on the middle node in the figure above (step 2).\n\nIf conditioning on a variable kills the dependence, that variable is added to the Separation set for those two variables. Depending on the size of the graph, conditional independence testing will continue (i.e. condition on more variables) until there are no more candidates for testing.\n\nNext, colliders (i.e. X → Y ← Z) are oriented based on the Separation set of node pairs. Finally, the remaining edges are directed based on 2 constraints, 1) no new v-structures, and 2) no directed cycles can be formed.\n\nGreedy Search of Graph Space\n\n\nA greedy search is a way to navigate a space such that you always move in a direction that seems most beneficial based on the local surroundings. Although greedy searches cannot guarantee an optimal solution, for most problems the space of possible DAGs is so big that finding a true optimal solution is intractable. The Greedy Equivalence Search (GES) algorithm uses this trick. GES starts with an empty graph and iteratively adds directed edges such that the improvement in a model fitness measure (i.e. score) is maximized. An example score is the Bayesian Information Criterion (BIC)\n\nExploiting Asymmetries\n\n\n\n\n\nA fundamental property of causality is asymmetry. A could cause B, but B may not cause A. There is a large space of algorithms that leverage this idea to select between causal model candidates.\n\n\n\nFunctional asymmetry assumes models that better fit a relationship are better candidates. For example, given two variables X and Y, the nonlinear additive noise model (NANM) performs a nonlinear regression between X and Y, e.g. y = f(x) + n, where n = noise/residual, in both directions. The model (i.e. causation) is then accepted if the potential cause (e.g. x) is independent of the noise term (e.g. n).\n\n\nConclusion\n\n\nThere is no way I could fit a comprehensive review of causal discovery in a short blog post. Despite being young, causal discovery is a promising field that may help bridge the gap between machine and human knowledge.\n\n\n\n\nReferences\n\n\nCausal Deep Learning\n","html":"This post is the fifth post of the series on Causal Machine Learning. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful. In this post Causal Discovery will be discussed in detail. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
Causal inference focuses on estimating the causal effect of a specific intervention or exposure on an outcome.\nCausal discovery focuses on identifying the underlying causal relationships between variables in a system.\nCausal inference, which aims to answer questions involving cause and effect. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful.
Causal discovery aims to infer causal structure from data. In other words, given a dataset, derive a causal model that describes it.
Finding causal relationships is one of the fundamental tasks in science. A widely used approach is randomized experiments. For example, to examine whether a recently developed medicine is useful for cancer treatment, researchers recruit subjects and randomly divide subjects into two groups. One is the control group, where the subjects are given placebo, and the other is the treatment group, where the subjects are given the newly developed drug. The reason of randomization is to remove possible effects from confounders. For example, age can be one of the possible confounders which affects both taking the drug or not and the treatment effect. Thus, in practical experiments, we should keep the distribution of ages in the two groups almost the same.
However, in many cases, randomized experiments are very expensive and hard to implement, and sometimes it may even involve ethical issues. In recent decades, inferring causal relations from purely observational data, known as the task of causal discovery, has drawn much attention in machine learning, philosophy, statistics, and computer science.
Causal discovery is an example of an inverse problem. This is like predicting the shape of an ice cube based on the puddle it left on the kitchen counter. Clearly, this is a hard problem, since any number of shapes could generate the same puddle. Connecting this to causality, the puddle of water is like statistical associations embedded in data, and the ice cube is the like underlying causal model.
The usual approach to solving inverse problems is to make assumptions about what you are trying to uncover. This narrows down the possible solutions and hopefully makes the problem solvable. There are four common assumptions made across causal discovery algorithms.
👉 Acyclicity — Causal structure can be represented by DAG (G)
\n👀 Markov Property — All nodes are independent of their non-descendants when conditioned on their parents
\n🙃 Faithfulness — All conditional independences in true underlying distribution p are represented in G
\n👍 Sufficiency — Any pair of nodes in G has no common external cause
\nAlthough these assumptions help narrow down the number of possible models, they do not fully solve the problem. This is where a few tricks/tests for causal discovery are helpful. There is no single method for causal discovery that dominates all others. Although most methods use the assumptions above (perhaps even more), the details of different algorithms can vary tremendously. A taxonomy of algorithms based on the following tricks is given in the figure below.
One of these earliest causal discovery algorithms is the PC algorithm named after its authors Peter Spirtes and Clark Glymour. This algorithm (and others like it) use the idea that two statistically independent variables are not causally linked. The PC algorithm is illustrative of this first trick. An outline of the algorithm is given in the figure below.
![\"pcalgo\"](\"https://user-images.githubusercontent.com/33357428/219992079-8d7af535-e4bd-48e0-93a6-a993f65b84cb.png\")
The first step is to form a fully connected, undirected graph using every variable in the dataset. Next, edges are deleted if the corresponding variables are independent. Then, connected edges undergo conditional independence testing e.g. independence test of the bottom and far right node conditioned on the middle node in the figure above (step 2).
If conditioning on a variable kills the dependence, that variable is added to the Separation set for those two variables. Depending on the size of the graph, conditional independence testing will continue (i.e. condition on more variables) until there are no more candidates for testing.
Next, colliders (i.e. X → Y ← Z) are oriented based on the Separation set of node pairs. Finally, the remaining edges are directed based on 2 constraints, 1) no new v-structures, and 2) no directed cycles can be formed.
A greedy search is a way to navigate a space such that you always move in a direction that seems most beneficial based on the local surroundings. Although greedy searches cannot guarantee an optimal solution, for most problems the space of possible DAGs is so big that finding a true optimal solution is intractable. The Greedy Equivalence Search (GES) algorithm uses this trick. GES starts with an empty graph and iteratively adds directed edges such that the improvement in a model fitness measure (i.e. score) is maximized. An example score is the Bayesian Information Criterion (BIC)
\n\nA fundamental property of causality is asymmetry. A could cause B, but B may not cause A. There is a large space of algorithms that leverage this idea to select between causal model candidates.
\n
Functional asymmetry assumes models that better fit a relationship are better candidates. For example, given two variables X and Y, the nonlinear additive noise model (NANM) performs a nonlinear regression between X and Y, e.g. y = f(x) + n, where n = noise/residual, in both directions. The model (i.e. causation) is then accepted if the potential cause (e.g. x) is independent of the noise term (e.g. n).
There is no way I could fit a comprehensive review of causal discovery in a short blog post. Despite being young, causal discovery is a promising field that may help bridge the gap between machine and human knowledge.
This post is the fifth post of the series on Causal Machine Learning. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful. In this post Causal Discovery will be discussed in detail. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
Causal inference focuses on estimating the causal effect of a specific intervention or exposure on an outcome.\nCausal discovery focuses on identifying the underlying causal relationships between variables in a system.\nCausal inference, which aims to answer questions involving cause and effect. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful.
Causal discovery aims to infer causal structure from data. In other words, given a dataset, derive a causal model that describes it.
Finding causal relationships is one of the fundamental tasks in science. A widely used approach is randomized experiments. For example, to examine whether a recently developed medicine is useful for cancer treatment, researchers recruit subjects and randomly divide subjects into two groups. One is the control group, where the subjects are given placebo, and the other is the treatment group, where the subjects are given the newly developed drug. The reason of randomization is to remove possible effects from confounders. For example, age can be one of the possible confounders which affects both taking the drug or not and the treatment effect. Thus, in practical experiments, we should keep the distribution of ages in the two groups almost the same.
However, in many cases, randomized experiments are very expensive and hard to implement, and sometimes it may even involve ethical issues. In recent decades, inferring causal relations from purely observational data, known as the task of causal discovery, has drawn much attention in machine learning, philosophy, statistics, and computer science.
Causal discovery is an example of an inverse problem. This is like predicting the shape of an ice cube based on the puddle it left on the kitchen counter. Clearly, this is a hard problem, since any number of shapes could generate the same puddle. Connecting this to causality, the puddle of water is like statistical associations embedded in data, and the ice cube is the like underlying causal model.
The usual approach to solving inverse problems is to make assumptions about what you are trying to uncover. This narrows down the possible solutions and hopefully makes the problem solvable. There are four common assumptions made across causal discovery algorithms.
👉 Acyclicity — Causal structure can be represented by DAG (G)
\n👀 Markov Property — All nodes are independent of their non-descendants when conditioned on their parents
\n🙃 Faithfulness — All conditional independences in true underlying distribution p are represented in G
\n👍 Sufficiency — Any pair of nodes in G has no common external cause
\nAlthough these assumptions help narrow down the number of possible models, they do not fully solve the problem. This is where a few tricks/tests for causal discovery are helpful. There is no single method for causal discovery that dominates all others. Although most methods use the assumptions above (perhaps even more), the details of different algorithms can vary tremendously. A taxonomy of algorithms based on the following tricks is given in the figure below.
One of these earliest causal discovery algorithms is the PC algorithm named after its authors Peter Spirtes and Clark Glymour. This algorithm (and others like it) use the idea that two statistically independent variables are not causally linked. The PC algorithm is illustrative of this first trick. An outline of the algorithm is given in the figure below.
The first step is to form a fully connected, undirected graph using every variable in the dataset. Next, edges are deleted if the corresponding variables are independent. Then, connected edges undergo conditional independence testing e.g. independence test of the bottom and far right node conditioned on the middle node in the figure above (step 2).
If conditioning on a variable kills the dependence, that variable is added to the Separation set for those two variables. Depending on the size of the graph, conditional independence testing will continue (i.e. condition on more variables) until there are no more candidates for testing.
Next, colliders (i.e. X → Y ← Z) are oriented based on the Separation set of node pairs. Finally, the remaining edges are directed based on 2 constraints, 1) no new v-structures, and 2) no directed cycles can be formed.
A greedy search is a way to navigate a space such that you always move in a direction that seems most beneficial based on the local surroundings. Although greedy searches cannot guarantee an optimal solution, for most problems the space of possible DAGs is so big that finding a true optimal solution is intractable. The Greedy Equivalence Search (GES) algorithm uses this trick. GES starts with an empty graph and iteratively adds directed edges such that the improvement in a model fitness measure (i.e. score) is maximized. An example score is the Bayesian Information Criterion (BIC)
\n\nA fundamental property of causality is asymmetry. A could cause B, but B may not cause A. There is a large space of algorithms that leverage this idea to select between causal model candidates.
\n
Functional asymmetry assumes models that better fit a relationship are better candidates. For example, given two variables X and Y, the nonlinear additive noise model (NANM) performs a nonlinear regression between X and Y, e.g. y = f(x) + n, where n = noise/residual, in both directions. The model (i.e. causation) is then accepted if the potential cause (e.g. x) is independent of the noise term (e.g. n).
There is no way I could fit a comprehensive review of causal discovery in a short blog post. Despite being young, causal discovery is a promising field that may help bridge the gap between machine and human knowledge.
This post is the fourth post of the series on Causal Machine Learning. This blog post is based on the work of Judea Pearl. We have so far discussed about Causal AI , How to represent causality using SCMs , How naive statistics can fail (Spurious Correlations , Simpson Paradox & Asymmetry In Causal Inference) , Pearl's Causal Hierarchy and gave flavors of Causal Inferences & Causal Discovery etc. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
At first glance there, there could be some ambiguities you feel while trying to explore Bayesian Networks and Causal Networks. This is completely normal. Lets try to uncover the differences before going ahead!.
Bayesian networks and Causal AI models appear similar. But Causal AI models capture underlying causal relationships; Bayesian networks just describe patterns of correlations.
You remember the example we discussed earlier about the data describing people’s sleeping habits. We found that there’s a strong correlation between falling asleep with shoes on and waking up with a headache.
![\"Headache](\"https://user-images.githubusercontent.com/33357428/211162715-e934cca9-50aa-4d08-a259-c04c4112fa11.png\")
\nA Bayesian network representing this is given below.
![\"Bayesian](\"https://user-images.githubusercontent.com/33357428/211162842-a197dc73-3cd8-4597-a12b-d93ad49fad82.png\")
\nThe BN tells us that both sleeping with shoes on and waking up with a migraine are correlated with drinking the night before, since there is a path between the variables. It also says that conditional on us knowing that someone was drinking the night before, knowing that they slept with their shoes on tells us absolutely nothing extra about whether they have a headache the next morning (this is called “conditional independence”). We can read off the conditional independence relationship by noticing that drinking alcohol “blocks” the pathway from shoe- sleeping to headache. BNs help to draw conclusions when more data becomes available (via “Bayes’ theorem”)
BNs sound useful! What’s the catch? The core problem is that Bayesian networks are blind to causality. This key, missing ingredient makes BNs very limited when it comes to more sophisticated reasoning and decision-making challenges.
Causal AI is a new category of machine intelligence. Causal AI builds models that are able to capture cause- effect relationships while also retaining the benefits of BNs.
![\"Bayesian](\"https://user-images.githubusercontent.com/33357428/211164237-265f8d20-5b8c-4a31-95cc-897989888817.png\")
\n![\"Bayesian](\"https://user-images.githubusercontent.com/33357428/211164446-257a87fc-975c-4cef-accc-1fac10ba489f.png\")
\nMany BNs are all statistically compatible with the data, but only one BN corresponds to the genuine causal relationships in the system. As a result, it’s always left ambiguous whether your BN is a good causal model or not. And the overwhelming chances are that your BN is not a good causal model. The number of possible BNs grows exponentially as the number of features increases. With, say, 20 variables in your data, there’s effectively zero chance of randomly stumbling across the true causal model. This means you’re using a BN that’s making bad modeling decisions.
Hope this clarifies the differences and ambiguities between Bayesian and Causal Networks.
Causal inference refers to the process of drawing conclusions about the causal relationships between variables. In other words, it involves making judgments about whether changes in one variable are responsible for changes in another variable.
Here are some examples of causal inference questions:
These are just a few examples, but causal inference questions can be asked in many different fields and contexts. The key is that they are trying to understand whether a particular intervention or exposure causes a change in some outcome or dependent variable.
Before going ahead , lets again define Causality in terms of interventions.
In the context of interventions, causality refers to the relationship between an intervention (also known as a treatment or exposure) and the resulting effect on an outcome or dependent variable. A causal relationship between an intervention and an outcome means that the intervention is responsible for the observed change in the outcome. In other words, if we change the intervention, we expect to see a corresponding change in the outcome.
For example, if a medication is found to reduce blood pressure in people with hypertension, we can say that there is a causal relationship between taking the medication and lowering blood pressure. This is because we expect that if we give the medication to a group of people with hypertension, their blood pressure will decrease as a result of the intervention. However, it is important to note that there may be other factors that could have influenced the relationship between the intervention and the outcome.
In the context of causal AI, do-calculus can be used to reason about the effects of interventions on a causal model. In causal AI, a causal model represents the relationships between different variables in a system, and the do-calculus can be used to reason about how changing the value of one variable (the \"intervention\") will affect the values of other variables in the system. This can be useful for understanding the potential consequences of interventions in a real-world system, or for identifying the most effective intervention to achieve a particular outcome.
Here is Judea Pearl’s canonical primer on do-calculus—a short PDF with lots of math and proofs (Pearl 2012).
However, how does that fit into causality’s mathematical representation?
The do-operator is a mathematical representation of a physical intervention.
If we start with the model Z → X → Y, we can simulate an intervention in X by deleting all the incoming arrows to X, and manually setting X to some value x0
\nBelow is the illustration how do-operator works.
![\"Illustration](\"https://user-images.githubusercontent.com/33357428/211196318-0f07a704-ff54-4eff-82a7-d2b31913406c.png\")
![\"Do](\"https://user-images.githubusercontent.com/33357428/211195804-e99bff5b-b622-4184-9b21-99e96e9c9495.png\")
Beneath this scary math, each rule has specific intuition and purpose behind it! Here’s what each rule actually does:
Rule 1: Decide if we can ignore an observation\nRule 2: Decide if we can treat an intervention as an observation\nRule 3: Decide if we can ignore an intervention\nWhoa! That’s exceptionally logical. Each rule is designed to help simplify and reduce nodes in a DAG by either ignoring them (Rules 1 and 3) or making it so interventions like do(⋅)do(⋅) can be treated like observations instead (Rule 2).
![\"Do](\"https://user-images.githubusercontent.com/33357428/213162724-cbc40a1b-71eb-42c0-bb7e-c6f0f4f5dc27.jpg\")
This post is the fourth post of the series on Causal Machine Learning. This blog post is based on the work of Judea Pearl. We have so far discussed about Causal AI , How to represent causality using SCMs , How naive statistics can fail (Spurious Correlations , Simpson Paradox & Asymmetry In Causal Inference) , Pearl's Causal Hierarchy and gave flavors of Causal Inferences & Causal Discovery etc. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
At first glance there, there could be some ambiguities you feel while trying to explore Bayesian Networks and Causal Networks. This is completely normal. Lets try to uncover the differences before going ahead!.
Bayesian networks and Causal AI models appear similar. But Causal AI models capture underlying causal relationships; Bayesian networks just describe patterns of correlations.
You remember the example we discussed earlier about the data describing people’s sleeping habits. We found that there’s a strong correlation between falling asleep with shoes on and waking up with a headache.
\nA Bayesian network representing this is given below.
\nThe BN tells us that both sleeping with shoes on and waking up with a migraine are correlated with drinking the night before, since there is a path between the variables. It also says that conditional on us knowing that someone was drinking the night before, knowing that they slept with their shoes on tells us absolutely nothing extra about whether they have a headache the next morning (this is called “conditional independence”). We can read off the conditional independence relationship by noticing that drinking alcohol “blocks” the pathway from shoe- sleeping to headache. BNs help to draw conclusions when more data becomes available (via “Bayes’ theorem”)
BNs sound useful! What’s the catch? The core problem is that Bayesian networks are blind to causality. This key, missing ingredient makes BNs very limited when it comes to more sophisticated reasoning and decision-making challenges.
Causal AI is a new category of machine intelligence. Causal AI builds models that are able to capture cause- effect relationships while also retaining the benefits of BNs.
\n\nMany BNs are all statistically compatible with the data, but only one BN corresponds to the genuine causal relationships in the system. As a result, it’s always left ambiguous whether your BN is a good causal model or not. And the overwhelming chances are that your BN is not a good causal model. The number of possible BNs grows exponentially as the number of features increases. With, say, 20 variables in your data, there’s effectively zero chance of randomly stumbling across the true causal model. This means you’re using a BN that’s making bad modeling decisions.
Hope this clarifies the differences and ambiguities between Bayesian and Causal Networks.
Causal inference refers to the process of drawing conclusions about the causal relationships between variables. In other words, it involves making judgments about whether changes in one variable are responsible for changes in another variable.
Here are some examples of causal inference questions:
These are just a few examples, but causal inference questions can be asked in many different fields and contexts. The key is that they are trying to understand whether a particular intervention or exposure causes a change in some outcome or dependent variable.
Before going ahead , lets again define Causality in terms of interventions.
In the context of interventions, causality refers to the relationship between an intervention (also known as a treatment or exposure) and the resulting effect on an outcome or dependent variable. A causal relationship between an intervention and an outcome means that the intervention is responsible for the observed change in the outcome. In other words, if we change the intervention, we expect to see a corresponding change in the outcome.
For example, if a medication is found to reduce blood pressure in people with hypertension, we can say that there is a causal relationship between taking the medication and lowering blood pressure. This is because we expect that if we give the medication to a group of people with hypertension, their blood pressure will decrease as a result of the intervention. However, it is important to note that there may be other factors that could have influenced the relationship between the intervention and the outcome.
In the context of causal AI, do-calculus can be used to reason about the effects of interventions on a causal model. In causal AI, a causal model represents the relationships between different variables in a system, and the do-calculus can be used to reason about how changing the value of one variable (the \"intervention\") will affect the values of other variables in the system. This can be useful for understanding the potential consequences of interventions in a real-world system, or for identifying the most effective intervention to achieve a particular outcome.
Here is Judea Pearl’s canonical primer on do-calculus—a short PDF with lots of math and proofs (Pearl 2012).
However, how does that fit into causality’s mathematical representation?
The do-operator is a mathematical representation of a physical intervention.
If we start with the model Z → X → Y, we can simulate an intervention in X by deleting all the incoming arrows to X, and manually setting X to some value x0
\nBelow is the illustration how do-operator works.
Beneath this scary math, each rule has specific intuition and purpose behind it! Here’s what each rule actually does:
Rule 1: Decide if we can ignore an observation\nRule 2: Decide if we can treat an intervention as an observation\nRule 3: Decide if we can ignore an intervention\nWhoa! That’s exceptionally logical. Each rule is designed to help simplify and reduce nodes in a DAG by either ignoring them (Rules 1 and 3) or making it so interventions like do(⋅)do(⋅) can be treated like observations instead (Rule 2).