In this post on the last day of the year, I thought I will share a map of mathematical structures that are useful for thinking about knowledge representation and reasoning (KRR) issues in Artificial Intelligence and Machine Learning. It is built on top of the diagram shown in Max Tegmark’s paper Is “the theory of everything” merely the ultimate ensemble theory? and extended with my own understanding of historical and recent work across quite a few different fields of AI. As such, it is necessary biased towards my own personal experience and taste.

Generally speaking, each arrow involves the addition of some new symbols and the axioms that provide their definitions and / or properties. Some boxes have multiple incoming arrows; these are systems constructed from the union of multiple sets of new symbols and axioms. Note also that the relationships represented by the arrows are, in general, transitive.

Essentially all the systems have a syntax, a semantics in the style of Tarski, and an inference engine. The expressiveness of the different systems are also tightly connected. For example, everything that extends first-order logic (predicate calculus) in the diagram can be formalised in higher-order logic (type theory). Also, I have shown in the diagram how higher-order logic can be extended to modal higher-order logic and probabilistic higher-order logic; first-order logic can be extended in a similar way although this is not explicitly reflected in the diagram.

There are also different boxes that provide equivalent ways of defining computability, including logic, lambda calculus, and Turing machines. Naturally, there is a connection between some of the boxes to actual programming languages, for example SQL (relational algebra), Prolog (predicate calculus), OWL (description logic), functional languages like ML, Haskell (higher-order logic), and imperative languages like C++, Java (von Neumann machines, which extend universal Turing machines).

Some of the most exciting AI work happen in the intersection of probabilistic graphical models (in particular POMDPs), logic (first-order and higher-order logics), and causal inference (Do Calculus). This is the area I am actively working on.

Enjoy.

### Like this:

Like Loading...

*Related*

Hi, nice article! I was wondering what you used to generate the diagram? I’ve been looking into different approaches but your diagram is cleaner than others I could find. Any tips would be great 🙂

LikeLike

Thanks. The diagram is generated using tikz in latex. Hope that helps.

LikeLike