Bayesian Networks

Bayesian Networks are probabilistic graphical models that offer a convenient, compact way of representing joint probability distribution. A Bayesian Network consists of a Directed Acyclic Graph (DAG) that connects different parameters (node), each edge indicating a dependence (an edge from node A to node B if the variable A helps explain B). Each node (random variable) is associated a distribution in the form of a Conditional Probability Distribution (CPD), the condition being on all the parent of that node. By definition, Bayesian Networks do not contain cycles. Which is not the case of Markov Random Fields. For that reason, Bayesian Networks are most often used when one tries to understand a causal relationship between the variables.

The construction of a Bayesian Networks involve at least 2 steps:

1. Generating the structure of the DAG (i.e., what nodes are connected and in what direction). That is what the DAGs with NO TEARS algorithm does, in an efficient wayi (along with code).
2. Estimating the CPDs. This can be done by MLE or Bayesian estimation.

The website for the Quantum Black library causalnex contains a brief introduction to Bayesian Networks. A longer, mode in-depth explanation can be found in this Stanford class on Probabilistic Graphical Models. An in-between solution might be to look at the slides for these two presentations 1 and 2. For sequential or temporal models, Dynamic Bayesian Networks were developped. Kevin Muprhy has a tutorial on his webpage.